Physics of Test Driven Development

Test Driven Development is a challenging practice. Why should you bother to learn it? You should learn it because it is a productive and predictable way to develop software.

Let’s compare TDD to the most popular way of programming, something I call Debug Later Programming. In DLP, code is considered “done” after it is designed and written. After the code is “done” it is debugged. Hmmm. Interesting definition of done isn’t it? The definition fails to include about half the effort.

It’s well known that developers make mistakes during design and code. If developers did not make mistakes, they would have no need to test their code. In DLP, testing is an individualized activity that may include some of the following activities. Some single step through their code to make sure it does the right thing. Others might write a small test driver and stub out some dependencies to test their code. And still others pepper their code with logging statements and inspect the log output. The problem with these techniques is that they are inherently non-repeatable and labor intensive.

When the future changes roll in, and they will roll in, there is a big problem. Changes can have unintended consequences, a.k.a. bugs. A manual test process is very unlikely to catch all these bugs. Also, the longer the bugs lay hidden the harder it is to exterminate them.

Physics of Debug Later Programming

When the time to discover a bug (Td) increases the time to find a defect’s root cause (Tfind) also increases, often dramatically. For some bugs, the time to fix the bug (Tfix) is not impacted by Td. But if the mistake is compounded by code building on top of a wrong assumption, Tfix may increase dramatically as well.

The problem with Debug Later Programming is that the feedback, revealing programming mistakes, may take days, weeks or months to get back to the developer. By then so many changes have been made that there is often no clear root cause. The programmer’s only recourse is to start debugging. This inherently unpredictable activity can destroy the most carefully crafted plans.

In comparison, with TDD there is immediate feedback. When the changes roll in, and you have the comprehensive tests that are a product of TDD, your tests immediately tell you when you make a mistake. This immediate notification helps prevent bugs. Not all bugs, of course, but most of them.

Preventing bugs is a big deal. Not only does the delivered quality improve, but less time is spent (a.k.a. wasted) in the activity called debugging.

Physics of Test Driven Development

When the time to discover a bug (Td) approaches zero the time to find the bug’s root cause (Tfind) also approaches zero. Why? Because the problem, just introduced, is often obvious. When the cause is not obvious, the developer is only a few UNDOs away from the prior all-tests-passing state. The time to fix the bug (Tfix) is as low as it can get given that things can only get worse as time clouds the programmer’s memory, and as more code is built on top of the shaky foundation.

Why Use TDD?

Writing tests takes time. If people did not make mistakes, writing test would be a waste of time. But we do make mistakes. The time to write a test is predictable. The time to find and fix a bug is not.

How long does it take to fix a bug? I’ve heard an industry average of eight hours. That includes those one minute fixes as well as the bugs that take months to find. As a schedule owner, I would want to improve predictability, and TDD can help. I think the physics of TDD make it a very powerful technique.

23 thoughts on “Physics of Test Driven Development

  1. I like the formulation here. I’d previously presented it as a generality: mean time to repair (MTTR) *generally* increases as the time increases between introduction of a defect and its discovery. Are you aware of any studies or statistics that back (or refute) up the hypothesis? I figure some have to exist.

  2. Looks like a good model.
    I would however like to bring up a third alternative in the comparison (besides comparing TDD and “Debug later programming”).
    I’d call it “Test before check-in” (unit tests are written shortly after the code is written to test the code before it is checked-in).

    I bring this third alternative up because the challenge with getting TDD adopted is in my experience not only to argue for it in relation to debug-later programming but also against those that in fact are quite good at writing unit tests for their code although they don’t do it TDD style.

    If you can consider that alternative in you models somehow that would be interesting. The challenge with doing this however is that the arguments are not that easy to put a value on as you do in the existing model.
    Some advantages I see with TDD in relation to test before check-in:
    *Lower coverage becuase you tend to writ less unit tests when you write them after the code (so you still get some “debug later” effects)
    *Feedback loops are slightly longer (from a few minutes to a few hours, what is the additional cost of that?)
    *You loose the too me important effect of written tests that first are expected to fail but due to an error in the test don’t (not noticed with test last unit tests).
    *Impact on design (not so simple and “testable”)

  3. Lars,

    Test before check-in model would certainly be a big improvement over debug later programming. I suspect that the longer feedback loops would impact the time to find the bug, but I would expect that code written and tested in the same day would be significant improvement over the debug later approach. I agree with you assessment of TDD advantages. I might add the TBCI might suffer from some testability issues as a result of test being a second separate task. I expect that some individuals would get very good at this.


  4. Pingback: James Grenning’s Blog » Blog Archive » Agile? How do I learn more?

  5. Pingback: James Grenning’s Blog » Blog Archive » Don’t Let Embedded Tool Chain Slow You Down.

  6. Pingback: Exercise in TDD - DerekHammer.Com

  7. Pingback: James Grenning’s Blog » Blog Archive » Extra! Extra! TDD Doubles LOC and No one Cares!

  8. Pingback: Test Driven Development (TDD) Is Not About Bugs » The Cutting Ledge

  9. I like your statement that “The time to write a test is predictable. The time to find and fix a bug is not.” That’s probably the biggest argument for TDD in my opinion, as long as the programmers can write tests quickly, and the code doesn’t have too many dependencies.

  10. Wow, interesting this post is still getting comments since 2008. As if the insights and arguments presented in this post wasn’t reason enough to encourage one to consider TDD’s benefits and use, then add to it the benefits of increased understanding, knowledge creation, documentation (in the form of executable requirements), and the positive effects on code design (echoing comments too from the “Cutting Ledge” above, why so many refer to TDD as “test driven design” as well as “test driven development”).

  11. Pingback: Experiences for a Kanban trainer in a Scrum Gathering | Yeret on Agile/Kanban

  12. Pingback: “My Code is Exceptional, I Don’t Need TDD” | James Grenning’s Blog

  13. Pingback: Can TDD Help Reduce Integration Time? | James Grenning’s Blog

  14. Pingback: Thai translation of this article.

  15. Pingback: Agile Quick Links #29 | Agile Pain Relief

  16. Pingback: The End of “Test-Driven” Development | This code was generated by a tool

  17. Jeff L asked about studies backing up the idea that it takes longer to fix a bug the longer you wait. Jeff Sutherland cited such a study, in his new book “Scrum: The art of doing twice the work in half the time”. At Palm, they found that if a bug was addressed the day it was created it took an hour to fix. If it was addressed 3 weeks later it took 24 hours to fix. 24 times longer!

  18. Pingback: Why do engineers dislike Agile? | James Grenning’s Blog

  19. Hi James,

    Can we use TDD only for new project that we code from scratch?
    How can we use TDD for enhancements after the final code has been released?
    How to maintain the tests for each module?

Leave a Reply

Your email address will not be published. Required fields are marked *

Be gone spammers * Time limit is exhausted. Please reload the CAPTCHA.