Fake News: Test-Driven Development has no Design!

Even if this was true, Test-Driven Development would be useful for realizing an up-front design. Humans make so many mistakes coding, the feedback loop of TDD helps you find those mistakes in the most efficient way I’ve ever seen (but I’ve only seen what I have seen). Certainly Test-First Programming (TFP) more effective than predominant programming practice, Debug Later Programming. TDD is much more than TFP. Let me debunk this fake news.

fake-news-banner

My First 2 decades of development

For about the first twenty years of my programing career, I pursued big design up-front (BDUF). Teams I led got pretty good with with BDUF. We would collect requirements for months on end. Get customers approval, then hide and do our design work, implement the system and go to the lab. We had specs, diagrams, text, pseudo code, and reviewed it all. It was an improvement over the chaos approach before BDUF. Our cycle time was an impressive 6 to 12 months (ahem). Sometimes we even made the deadline.

I worked mostly in embedded systems. When we finally got hardware, with anticipation we fired up the system somehow expecting it to work. Guess what? The system crashed. Now, late in the development cycle we had to deal with the mistakes, wrong requirements, untried hardware, unconfirmed design ideas and assumptions made to get to this point.

With reality setting in, we were forced to the iterative and incremental process we all know too well, debugging.

A Chage of Course in 1999

Thankfully my path changed in 1999. I attended the first Extreme Programming Immersion in the suburbs of Chicago at Object Mentor, my employer at the time. Kent and the crew described problems with development that I was all too familiar with: Late delivery, no warning of coming problems, defects delivered to customers, deteriorating design, and on top of that the burned out developers suffering through a death-march.

Kents 12 practices of XP seemed to address many of the problems. The techniques did not require a long explanations and a long time to payback. The logic-chain from problem to solution was made of only a few links. The discrete practices supported each other. We could try the ideas pretty easily and find out for ourselves.

I’d say at the time, XP was light on design advice. That was OK, because many of us early learners had already invested a lot in learning design. Robert Martin’s Object Mentor taught design and coined the term SOLID. We knew our way around upfront design.

XP does not prescribe how much design upfront is needed because each system has different needs. A small team needs less formal design than someone on a large multi-team effort. It is easy and ego building to over-engineer and design up-front. It is harder to do just enough design, just in time. We have to take a more humble attitude, we can’t get it all right up-front. Iterate to good design was a major insight by Kent, Ward, Ron, Martin and others that I am grateful to have heard about in 1999.

TDD Ecosystem

What experienced Test-Driven Developers simply call TDD today is more of an ecosystem than simply the application of one XP practice with a new name (the old name was TFP). Kent Beck originally used the term metaphor to describe design. It was confusing to me, but included having a system of names and conventions to navigate the design.

Recalling what Ron Jeffries told me back in 1999… design is so important, we do it everyday. When we discover something new about the need, the environment, or the existing system, consider the design impact. Design is not an isolated activity we can do once. Do design every day. Little things matter, and they scale. Ron said “Modularity works!”. TDD encourages modularity. TDD provides subtle pressure on a design to improve cohesion and coupling. These responsibility assignment choices happen at many levels everyday.

For me, design is creating a vision of the overall structure of the system. Design is never done, at least not until you stop supporting the system. Imagine a map of the United States, it’s made up of individual states connected through an interstate highway system. The states are also grouped into regions with different cultures, like Midwest, New England, the Southeast…. The country is sliced north to south into timezones. To navigate this system, diagrams, patterns and naming conventions really help. I want that help.

The TDD ecosystem includes applying design principles and having a vision of how the pieces are connected, how they are grouped and how they interact with each other. How formal does the design need to be? That depends on your situation.

Considering the needs continue to evolve, it is silly to think design happens only in the beginning. Nothing new here. We discovered this while we tried to do BDUF/waterfall in the late 80s. Did we give up on BDUF and waterfall? No. We started a number of little waterfalls. This helped, but we never would have had the insight to get to something like TDD and XP’s iterative and incremental engineering techniques.

Design is hard, learn from those before us

To be good at software development you need a lot of design insight and tools at your disposal. TDD is one of those tools. TDD helps you see your poor design decisions. Added complexity in tests, hard to use interfaces, complex initializations are some of the early warnings.

What are some of the other design principles and concepts the test-driven developer needs to understand and apply:

  • Principle of Least Knowledge.
  • Separation of Concerns (SoC)
  • SOLID
  • DRY
  • DAMP
  • GRASP
  • ZOMBIES
  • FIRST
  • DDD
  • BDD
  • Design Patterns
  • Hexagonal architecture (ports and adaptors)
  • Recognizing code smells
  • Envisioning a better design
  • Refactorings and incremental improvement
  • Law of Demeter (LoD)

Conclusion

There is plenty of design in the TDD ecosystem. Design is so important, do it all the time. BTW: you don’t have to learn TDD. Find another fast-feedback way to work. I would love to hear about it.

Here are a couple of other articles, from 2012 ,on the topic of TDD has no design.

“TDD ignores the design” “No it doesn’t” “Yes it does” – Part 1
“TDD ignores the design” “No it doesn’t” “Yes it does” – Part 2

Self-Paced Training: TDD for Embedded C/C++

Hello Friends

I am developing a series of self-paced training modules for Test-Driven Development for Embedded C and C++. The training modules will have exercises to cement the learning. I’ll offer them individually or in a couple bundles. Each module will take six or more hours to complete. The modules follow the flow of my on-site and live-via-the-web formats found on wingman-sw.
Continue reading

Wrestle Legacy C and C++ into a Test Harness – Linker Errors

Getting started with TDD in C and C++ is challenging. On top of that, you have your whole product’s code base to start adding tests to. You don’t have time to stop all development and add tests to your code, so you need a pragmatic approach. As you drag your legacy code, kicking and screaming into a test harness, take your time and solve one problem at a time. It is the fast way.

This article is about getting past a boat-load of C and C++ linker errors. I’ve got a method and a tool to help get through that challenging step. You can find the tool, gen-xfakes, here on my GitHub account.

Continue reading

Responsibilities of the Test-Driven Developer in a Legacy Code Base

This is a follow-up question from an attendee of a recent TDD for Embedded C++ training course.

What exactly is a person’s responsibility for unit testing when they go into existing untested code?

Like all good questions in software development, the answer is “it depends”.

Continue reading

In the beginning, there was no code…

…and it was good.

Why is it that code starts out nice and deteriorates over time?

What happened happened to make the code badness grow? Probably something joyous like the first order for a new product or adding people to the team. Or maybe something sad like losing a key employee.

I see a lot of legacy C code. How does a function get to be 1000 lines long? How did that file get to be 40 KLOC or 100 KLOC? A friend showed me a 27 page C function in some telecom code. Amazingly enough, it worked! How does code get that way?

One Copy/Paste at a time!

Continue reading

Is SAFe safe for your organization? I don’t know.

If your definition of safe, is like “no one has even been fired for choosing IBM”, SAFe may be safe. Though, you really need to consider, will it help your organization improve? I hoped that Scrum would help the world discover iterative and incremental engineering practices and principles. It seems to me, it has failed. Maybe SAFe can be different; I don’t know.

I have not spent any time looking into SAFe. I cannot make an informed opinion. I would make my opinion based on the message of SAFe and how it is being adopted. Generally, I am critical of what most Agile adoptions have become, Agile in name only (AINO). AINO adoptions leave developers feeling like they are being micromanaged and pressured to do poor work.

Continue reading

Thoughts on TDD (After almost 20 years)

I used to think I was good at programming (1976–1999), then I started doing Test-Driven Development. Since then I discovered that I make mistakes regularly. TDD puts your mistakes in your face, immediately. With this kind of feedback and a focus on small steps you can prevent many defects in you code from getting past you to your co-workers, QA or your customer. — For me, it means very little time hunting down mysterious behavior, a.k.a. bugs!

The best you, a programmer, can do with TDD is get code to do what you think the code should do. This simple concept, a confirmation that your code is doing what you think it should do, is very powerful. Your system works on purpose. Have you ever fixed a bug and broke something else? That other thing only worked by accident (offsetting defects allowed the other thing to work). In a way you are blindfolded and have no idea that there’s a problem.

(Photo by Andrew Bossi (Own work) [CC-BY-SA-2.5 (www.creativecommons.org/licenses/by-sa/2.5)], via Wikimedia Commons)

I like dealing with code with my eyes open.

Defect prevention is only the most obvious benefit. The next benefit is that as you change your code (either because you have a better design idea, or your customer asks for a change, or you discover how you were wrong in the first place) you have a regression test suite that holds your code steady, notifying you of any unintended consequences. TDD helps you when you have to make changes, and you will have to make changes.

TDD by itself does not mean good design. I wish it were so, though it can lead down a path to better design. If your code is hard to test, it’s not a test problem; usually it is a design problem. TDD provides early warning of design problems as well as a subtle pressure on you to make the code better. TDD works best with cohesive modules that are loosely coupled. This is not automatic. You need to understand the subtleties of SOLID design, DRY (don’t repeat yourself) and DAMP (descriptive and meaningful phrases) and to be able to envision better code constructs.

Here is a partial list of TDD benefits:

  • TDD helps prevent initial defects.
  • TDD gives fast feedback on what your code is doing.
  • TDD produces a set of regression tests (almost for free, once you learn TDD).
  • TDD supports you when you change your code, letting you know when unintended things happen.
  • The tests are the detailed documentation for the code under test — this documentation does not lie. When it deviates from the code it tells you.
  • The tests act as the detailed long term memory of what the code is supposed to do.
  • Oh yeah! TDD is fun. The feedback is fast. You get satisfaction every few minutes as you get your code to dance.

Now for the downside…

  • TDD takes time to learn.
  • You’ll probably make a mess in your code and tests.
  • Tests have to be maintained — (Though it is worth it. Not having the tests makes the product harder to maintain.)
  • Good tests and good design take time to learn.
  • Applying TDD takes judgement. Not every line of code is a candidate for test driving. (Not really a downside, but you do have to think. So if thinking is something you want to avoid, you might want to go into a different field.)

    Most cited TDD misconception

  • TDD takes too long — Editorial comment: Compared to what? It may take some time to learn, but I’ve found that debugging takes a lot longer.
  • My life as a programmer is much better with TDD.

    Unfortunately, I cannot convince you about the value of TDD. Though, you can convince yourself. The only way I know for you to convince yourself is by trying it. Take the time to experience TDD. Pair with an experienced test-driven developer. Take a training course. Find a friend to experiment with.

    There are many seasoned TDD practitioners that started as skeptics. Actually I do not know of any that did not.

    This blog originated as an answer to this Quora question: What are your thoughts on Test-driven Development? What are the pros and cons?

    Linker Substitution in C – Limitations and Workarounds

    Let’s say you have code that in some test cases you want the real code and other test cases you want a test double. In C, using link-time binding, you only get to have a single function with a specific name.

    I’d suggest that most the time this comes up you want to directly test some module (or function) and you want a substitute when testing code that depends on the module’s interface.

    What options do you have?

    • Create multiple test runners, where one test runner would use the real function and the other test runner would use the test double.
    • Function pointer substitution.
    • #include the .c file in a namespace in the test file.
    • Linker wrapping.
    • Weak linking.

    Multiple test runners

    This might be a decent solution. Though it can lead to needing many test runners.

    Function pointer substitution

    Changing a direct function call to an indirect function call is a small change, and the compiler and linker will make sure you get it right. There are a couple examples in my book, in chapter nine. Here’s one of them.

    // RandomMinute.h -- original
    int RandomMinute_Get(void);
    
    // RandomMinute.c -- original
    
    int RandomMinute_Get(void)
    {
        return bound - rand() % (bound * 2 + 1);
    }
    
    // RandomMinute.h - revised
    extern int (*RandomMinute_Get)(void);
    
    // RandomMinute.c - revised
    
    static int __randomMinute_Get(void)
    {
        return bound - rand() % (bound * 2 + 1);
    }
    
    int (*RandomMinute_Get)(void) = __randomMinute_Get;
    

    That refactor is pretty easy, but you must make sure all users of RandomMinute_Get() include the header file. If you happen to be lazy about calling functions without having them advertised in a header file, the caller will think this is a direct function call. Bad things will happen.

    In the test group file, write FakeRandomMinute_Get().

    //Test file for SomethingThatUses_RandomMinute_Get
    
    extern "C"
    {
        static randomMinuteValue;
        static randomMinuteIncrement;
        static void FakeRandomMinute_Init(int first, int increment)
        {
            randomMinuteValue = first;
            randomMinuteIncrement = increment;
        }
    
        static int FakeRandomMinute_Get(void)
        {
            int value = randomMinuteValue;
            randomMinuteValue += randomMinuteIncrement;
            return value
        }
    }
    

    in setup() override RandomMinute_Get() with FakeRandomMinute_Get().

    //Test file for SomethingThatUses_RandomMinute_Get - Continued
    
    TEST_GROUP(SomethingThatUses_RandomMinute_Get)
    {
        void setup()
        {
            //other setup
            FakeRandomMinute_Init(-30, 3);
            UT_PTR_SET(RandomMinute_Get, FakeRandomMinute_Get);
        }
    }
        void teardown()
        {
        }
    }
    

    UT_PTR_SET overrides the function pointer and then restores its original value after teardown(). Your code under test now has predcitable random minutes.

    In another test group, the production RamdomMinute_Get() function can be tested without overriding the function pointer.

    // RandomMinuteTest.cpp
    
    TEST_GROUP(RandomMinute) {}
    
    TEST(RandomMinute, GetIsInRange)
    {
        for (int i = 0; i < 100; i++)
        {
            minute = RandomMinute_Get();
            AssertMinuteIsInRange();
        }
    }
    

    But maybe you don’t want to use a function pointer. You might prefer this next solution.

    #include the .c file in a C++ namespace

    In this solution, the test-double’s function name in FakeRandomMinute.c is the same as the function being faked, business as usual for link-time substitution.

    // FakeRandomMinute.h
    #include "RandomMinute.h"
    void FakeRandomMinute_Init(int first, int increment);
    
    // FakeRandomMinute.c
    #include "FakeRandomMinute.h"
    
    void FakeRandomMinute_Init(int first, int increment)
    {
        randomMinuteValue = first;
        randomMinuteIncrement = increment;
    }
    
    int RandomMinute_Get(void)
    {
        int value = randomMinuteValue;
        randomMinuteValue += randomMinuteIncrement;
        return value
    }
    

    To test the production implementation, the the RandomMinute.c file can be #included in the test file. In addition, the #include needs to be wrapped in a C++ namespace to keep the name out of the global namespace.

    // in your test file
    
    namespace Test
    {
        #include "RandomMinute.c"
    }
    

    In the test cases, make calls to Test::RandomMinute_Get().

    TEST(RandomMinute, GetIsInRange)
    {
        for (int i = 0; i < 100; i++)
        {
            minute = Test::RandomMinute_Get();
            AssertMinuteIsInRange();
        }
    }
    

    Linker Wrapping

    The linker tricks are not generally available except from gcc. I’ve experimented with wrapping in gcc., but could not find it in visual studio or IAR (at the time I looked).

    In gcc, wrap RamdomMinute_Get() with these command line options:

    gcc -Wl,--wrap=RamdomMinute_Get ...
    

    The linker will redirect calls to RamdomMinute_Get() to __wrap_RamdomMinute_Get(). You can get at there production implementation by calling __real_RamdomMinute_Get(). You need to provide __wrap_RamdomMinute_Get() to hold your fake implimentation and to keep the linker happy.

    int __wrap_RamdomMinute_Get()
    {
        // some fake implementation
    }
    

    Maybe some test cases want the real RamdomMinute_Get(), you could make the wrapped function default to the real with function.

    static int (*customRamdomMinute_Get)() = 0;
    int __wrap_RamdomMinute_Get()
    {
        if (customRamdomMinute_Get)
                return customRamdomMinute_Get();
        return __real_RamdomMinute_Get();
    }
    

    I chose to use a function pointer for test specific behavior. This lets you have a generic fake implemention that is customized close to your test case.

    There are some limitations so you may want to look at this Stack Overflow article for linker wrapping

    Weak Linking

    I’d say stay away from weak linking as it means adding non-standard stuff to your producion C code.

    Finally

    You can mix and match these techniques. Let me know how this works for you or if you know some other solutions to this problem.

    Damn, I used to think I was good at programming

    I used to think I was good at programming. Since I started doing TDD in 1999, I’ve come to discover that I am not so good. I make a lot of little mistakes. Copy-paste is a specifically error-prone activity for me. When I copy and paste and N changes are needed, on a good day I can usually get N-1 on the first try.

    When I’m almost done, I get a little careless. I’m not the only one! Take a look at this analysis of the Last Line Effect. He show lots of copy-paste bug examples from real code from products you know. In the article the author also mentions this may not be just programming.

    mountain-climbers often fall off in the last few dozen meters of their ascent. Not because they are tired; they are simply too joyful about almost reaching the top – they anticipate the sweet taste of victory, get less attentive, and make some fatal mistake. I guess something similar happens to programmers.

    I’ve watched a lot of programmers program in training context. They are not so good at programming either. Little mistakes are the norm, not the exception. But programmers generally don’t know how error prone their work is, because the feedback loop is too long pre-TDD. TDD puts the mistakes in your face so you can find them before they hatch into bugs.

    With the mistakes in your face, you can certainly see them and fix them. You might, though awareness, start to do them less as you work on your weakness.

    Cause and Effect – Foundation of TDD

    Virtually every line of code is written in response to some failure. It often is a test failure, but might be a compilation failure, or a web page that does not exist.

    Through my 16 years of practicing TDD, I have come to discover that establishing cause and effect is critical to getting every line of code right. I’ve done most my TDD in embedded C and C++.

    Over the last couple years, with the help of a couple friends, I built my ruby on rails website. I did not know ruby (or rails obviously). I know it a little bit now. I used to follow the several step procedures to bring up a new feature on my website. I would do the 3–6 steps outlined, and my new page would not come up. I’d spend considerable time trying to find the mistake and eventually get the thing to work. It would have involved several changes. The next day I would likely run into another problem because one of the changes was not really needed and I broke something else. In hindsight, I made changes without establishing cause and effect. My new feature started working often due to offsetting defects.

    I started treating each step in the procedure as a test. For example If I tried to bring up a web page, that did not exist, what error would I get? I would look for a way to make one of the changes in the several step procedure and get a different error message. Getting new errors was not, but was it the right error? I learned to get the right error that moved my code in the right direction. I establish cause and effect. Cause and effect?! I am programming as an engineer. What is the problem? Can I impact that problem positively, and move my code in the right direction. Sometimes this cause and effect involves TDD tests, other times it is wiring code that once I get it right, I leave it alone

    I just found this definition of engineering “skillful or artful contrivance; maneuvering.” I’d say that is the ideal I strive for in building software and a business. Yes, TDD affects how I run my business. TDD is built on cause and effect. Engineering! I doubt I would appreciate the cause and effect approach to programming (and life) if it were not for TDD.