Creating automated tests can be very difficult, especially when the code has gotten long in the tooth and was not created with automated tests to begin with. Many product development teams don’t invest in automated tests. They think they cannot afford them. They think their product is different and can’t be tested automatically. This thinking is flawed.
Back in the products younger days, manual test was not too time consuming. But slowly that changed. The system grows, the manual test effort grows. Eventually, it seems that no amount of manual test effort finds all the problems.
In this article I show a simple model that illustrates why manual test is unsustainable and that a sustainable software product development effort must include considerable test automation.
I have a few courses that share materials and ideas. I used to use the age-old practice of cut and paste to share the slides. As handy as it is, cut and paste causes trouble. If I’m teaching TDD in C and make a change to a discussion that applies also to C++ or or some other language, I’ve got a problem. I’ll never remember to put the change in the other places where the change is needed. This means that later I find myself in front of a group of C++ programmers about to make a big point, and find that the dramatic conclusion added to the C version I made is not in the current slide deck.
As with code, duplication in presentations is a liability. So I needed to refactor my training materials, and need a practical way to deliver them and provide a PDF handout of the presentation.
After the Agile2011 conference, my wife and I took some great hikes in the mountains of Utah. We were careful as we navigated the slot canyons, rushing streams, and falling water. We were careful, we did not want a mistake to cause an injury; but we have no fear of height and love adventure. After a few days of hiking, our skill grew and we worked the terrain more quickly and safely.
I’ve described TDD using the analogy of stepping stones to cross a river for years. Here’s a Utah stream coming from a slot canyon.
You can see there are a few ways across. It’s not a straight path, but with a few careful steps you get to the other side, keeping your shoes dry.
Over several days of hiking, I discovered another metaphor for TDD, often thought of as a very careful way to program. The careful way is the fast way.
My last article featured a hand crafted a spy to monitor
asm directives. Now let’s use CppUMock (the mock support companion CppUTest) to create a mock version of
Sometimes embedded developers have to use inline assembler instructions to get better control of the processor, or to improve performance. How should we deal with those when we’re doing TDD and testing off the target?
What’s the problem? The embedded
asm statements cause compilation errors if the assembler instructions are not part of the off-target test platform instruction set. Also some of the instructions might not be legal in the test environment. This article shows how to insert a test double for the
asm directives with gcc and CppUTest.
In Jeff Langr’s blog, Jeff responded to an assertion (from someone Jeff calls Schmoo) that writing tests after developing a unit of production code takes less time than using TDD to create production code and its tests. For starters, I am happy the discussion is about when to write the unit tests and not if.
I think a model would help us talk about this issue. It would be great to have some real numbers in the model; that will be hard. But for starters let’s look at a model. Maybe then someone can figure out how to put some numbers to the model.
Here is a legacy code change policy for a team adopting TDD that has a legacy code base:
- Test-drive new code
- Add tests to legacy code before modification
- Test-drive changes to legacy code
Refactoring without tests is dangerous; with all the details we must keep straight, a mistake is easy to make. How many code reviews have you been in where the recommended design changes are not made because “we already tested it”? You avoid the change because it’s dangerous to change code without tests. So, the Boy Scout adds tests too. For more on Boy Scouts, see previous post.
The Boy Scouts have a rule: leave the camp cleaner than you found it. This does not mean that all the trash has to be cleaned up now, but you can’t let it get worse, and it must get at least a little better. In Bob Martin’s book, Clean Code, he asks, “What if code got a little better every time you change it?” I’ll answer it: the industry would not find itself in the mess it’s in. The industry norm is for code to incrementally worsen with each change.
Much of the time, following the Boy Scout Rule won’t be hard. It’s an incremental strategy. It’s easy to start and easy to sustain. Here are some typical challenges and ideas on how to be a Boy Scout.
At the start of a new development effort, there is considerable uncertainty. There are unknowns in hardware, software, product goals and requirements. How can we get started with all this uncertainty? Isn’t better to wait? If you wait, there really is no end to the waiting, so its better to get started sooner even though there will be some things you decide early that get changed later.
One important realization on the journey from a BDUF approach to an iterative and agile approach is that design is never done. Designs evolve. The waterfall emphasis has been to unnaturally try to control software physics by imposing requirements freezes and burdensome change control. The process of developing software is part science and part creative. You are applying science toward the invention of something. Design is capturing knowledge both about what the end user need is, and one solution to that need.