I spend the day updating the Fake Function Framework, Mike Long’s (@meekrosoft) creation. With my client last week, one of the engineers had some problems with Visual Studio (don’t we all) and the Fake Function Framework (a.k.a FFF). I promised to integrate the changes into the FFF. The FFF is created with a ruby script, but you don’t have to care about that unless you have functions to fake that have more than 10 parameters. But anyway, I spent the day updating and reviewing the ruby script that generates the FFF.
As programmers adopt test-driven development, they are going to prevent many defects that would have escaped to testers. Programmers that write unit tests just after programming are going to find many of the defects that would have escaped. Keep in mind that the programmer tests only check what the programmer thinks the code is supposed to do. The tests do not assure the code meets the customers’ needs. But making sure the code is doing what the programmer thinks it is supposed to do is needed for high quality systems. They work on purpose, rather than by the accident of offsetting defects.
Testers, and the whole team, are responsible for making sure the product does what they think the customer needs. These are different tests, they cover more code per test as well as checking the interactions between modules. As testers adopt automated testing, they need different skills than the manual tester that is following a script or exploring. Some adopt Acceptance Test Driven Development. For testers in this environment, the world is changing. Changing for the better with the intellectual challenges of automation, keeping the cost of retest low, and reducing the repetitive, error prone and boring manual tests. Not all manual tests go away, but we work to automate what is repetitive. In addition ATDD moves testers upstream from reacting to the code written to specifying system behavior by example.
All that said, most of the world of software development still practices Debug Later Programming and manual test, the same practices I used in my first job as a programmer. Congratulations for using 1979 programming techniques! We’ve learned a lot since then.
For some adopting TDD and ATDD is changing their world. For most it is not. There is so much legacy code out there, as well as people and organizations that do not know how or why to automate their tests, that the status quo of testing will likely have a long life.
Kent Beck told me years ago, if the code does not need to work, then there is no need to test it. He continued and observed, why bother writing it if it does not need to work. Since hearing that and discovering how frequently I make coding mistakes, I want thorough tests.
Maybe you are asking yourself “I’ve got integration and system tests, why do I need unit tests?”. The answer is simple, simple math.
Here is a good question, and my reply, from a recent attendee of my Test-Driven Development for Embedded C training.
As I work more with TDD, one of the concepts I am still struggling to grasp is how to test “leaf” components that touch real hardware. For example, I am trying to write a UART driver. How do I test that using TDD? It seems like to develop/write the tests, I will need to write a fake UART driver that doesn’t touch any hardware. Let’s say I do that. Now I have a really nice TDD test suite for UART drivers. However, I still need to write a real UART driver…and I can’t even run the TDD tests I created for it on the hardware. What value am I getting from taking the TDD approach here?
One of the biggest challenges for someone experiencing TDD the first time is working in the small steps. The small steps of TDD appear to be an independent discovery of John Gall’s observation described in the Systems Bible:
Maybe you read Part 1 of this article. If you did you’ll know it concerns adding tests to legacy code (legacy code is code without tests). You will also know that the code has file scope functions and data that we want to test directly.
My opinion on accessing private parts of well designed code, is that you do not need to. You can test well design code through its public interface. Take it as a sign that the design is deteriorating when you cannot find a way to fully test a module through its public interface.
Part 1 showed how to
#include the code under test in the test file to gain access to the private parts, a pragmatic thing to do when wrestling untested code into a test harness. This article shows another technique that may have an advantage for you over the technique shown in Part 1. Including the code under test in a test case can only be done once in a test build. What if you need access to the hidden parts in two test cases? You can’t. That causes multiple definition errors at link time.
This article shows how to create a test access adapter to overcome that problem.
And a Happy Leap Year Bug
It’s a new year; last year was a leap year; so the quadrennial reports of leap year bugs are coming in. Apologies are in the press from Apple, TomTom, and Microsoft. Trains we stopped from running in China. Somehow calling them glitches seems to make it someone else’s fault, something out of their control. How long have leap years been around? Julius Caesar introduced Leap Years in the Roman empire over 2000 years ago. The Gregorian calendar has been around since 1682. This is not a new idea, or a new bug.
I’m going to try to take one excuse away from the programmers that create these bugs by answering a question that comes up all the time, “How do I test static functions in my C code?”
TDD ignores design; that is a frequently stated misconception. Many people get this idea from the code focus of TDD. TDD does not call for the creation of any non-executable design documentation. So the questioning developer gets the idea that there is no design. But I say, “yes there is”.
A recent TDD for Embedded C attendee asks me “TDD does not help reduce the time I spend in the lab during system integration testing”
(This is asked in the context of embedded development. But the answer is half applicable to any development where there is integration.)
TDD should definitely help reduce the time you spend in integration. How does it do that? It helps you eliminate the non-integration problems before you get to the integration lab. An honest appraisal of what you do in integration may reveal that during integration you are finding problems you could have discovered before integration. It would help reduced integration time if you find fewer problems during integration. Don’t you think?
This cause/effect diagram helps to visualize the relationship of TDD to time spend in integration.