Preventing Brittle Tests (and Production Code)

Have you written unit tests only later to find they slow you down because your changes cause a lot of test breakage? You think, these tests are not living up to the promises I heard. So you toss the tests and go back to business as usual.

Hey, not so fast. Maybe its not test in general, but your tests or your code. The first project I used TDD on in 1999/2000 we ran into this problem. Some were ready to give up. But I wanted the tests to work and looked for what was wrong. In our excitement we kept copying, pasteing and tweaking the test cases. They were an ugly (in hindsite) mass of duplication. Small changes made for ripple effects through the tests. But I could see, it was our fault.

Someone on Quora asked me “How do you make unit tests less brittle”. Here is a more complete answer based on having written my own bad tests and seen many learners of TDD and unit testing go the wrong direction with their designs and tests.
Continue reading

Accessing static Data and Functions in Legacy C — Part 2

Maybe you read Part 1 of this article. If you did you’ll know it concerns adding tests to legacy code (legacy code is code without tests). You will also know that the code has file scope functions and data that we want to test directly.

My opinion on accessing private parts of well designed code, is that you do not need to. You can test well design code through its public interface. Take it as a sign that the design is deteriorating when you cannot find a way to fully test a module through its public interface.

Part 1 showed how to #include the code under test in the test file to gain access to the private parts, a pragmatic thing to do when wrestling untested code into a test harness. This article shows another technique that may have an advantage for you over the technique shown in Part 1. Including the code under test in a test case can only be done once in a test build. What if you need access to the hidden parts in two test cases? You can’t. That causes multiple definition errors at link time.

This article shows how to create a test access adapter to overcome that problem.
Continue reading

Accessing static Data and Functions in Legacy C — Part 1

And a Happy Leap Year Bug

It’s a new year; last year was a leap year; so the quadrennial reports of leap year bugs are coming in. Apologies are in the press from Apple, TomTom, and Microsoft. Trains we stopped from running in China. Somehow calling them glitches seems to make it someone else’s fault, something out of their control. How long have leap years been around? Julius Caesar introduced Leap Years in the Roman empire over 2000 years ago. The Gregorian calendar has been around since 1682. This is not a new idea, or a new bug.

I’m going to try to take one excuse away from the programmers that create these bugs by answering a question that comes up all the time, “How do I test static functions in my C code?”
Continue reading

Unit testing RTOS dependent code – RTOS Test-Double – Part 3

I’m going to bend the example a little, and look at how to write tests that interact with an interrupt service routine and again with a semaphore. I’ll also get some help with the tedious parts of building fakes by using Mike Long’s Fake Function Framework. (BTW: I am using a beta version, that has changes the released version does not yet have.)

This figure shows the concurrent runtime relationship between the MessageProcessor and its SerialDriver.

Continue reading

Unit testing RTOS dependent code – RTOS Test-Double

When you’ve got legacy code that depends on the Real-time Operating System, you have a challenge to get your code off the target for unit testing. If you want to test with the concurrency provided by the RTOS, these are not really unit tests, and you won’t be able to write thorough tests, and you need unit tests to be through.

You’re going to need to make a test-double so you can get your code off the RTOS and into your development system for unit testing. In this article we’ll go through the steps to get started.

Continue reading

Hiding Non-standard C Keywords for Off-Target Testing

Some silicon vendors extend the C language so the programmers can easily interact with the silicon. Using these extensions tie production code to the silicon vendors compiler and consequently the code can only run on the target system. This is not a problem during production, but is a problem for off-target unit testing.

The good news is that we may be able to get around this problem without having to change production code, one of our goals when adding tests to legacy code.
Continue reading

#include Test Double

It’s day one of adding tests to your legacy C code. You get stopped dead when the compiler announces that the code you are coaxing into the test harness can’t be compiled on this machine. You are stuck on the Make it compile step of Crash to Pass.

Moving your embedded legacy C code (embedded C code without tests) into a test harness can be a challenge. The legacy C code is likely to be tightly bound to the target processor. This might not be a problem for production, but for off-target unit testing, it is a big problem.

For C we have a limited mechanisms for breaking dependencies. In my book, I describe at length link-time and function pointer substitutions, but only touch on preprocessor stubbing.

In this article we’ll look at #include Test-Double as a way to break dependencies on a problem #include file.

Continue reading

Spying on Embedded ‘asm’ directives

Sometimes embedded developers have to use inline assembler instructions to get better control of the processor, or to improve performance. How should we deal with those when we’re doing TDD and testing off the target?

What’s the problem? The embedded asm statements cause compilation errors if the assembler instructions are not part of the off-target test platform instruction set. Also some of the instructions might not be legal in the test environment. This article shows how to insert a test double for the asm directives with gcc and CppUTest.

Continue reading

Legacy Code Change – a Boy Scout Adds Tests

Here is a legacy code change policy for a team adopting TDD that has a legacy code base:

  • Test-drive new code
  • Add tests to legacy code before modification
  • Test-drive changes to legacy code

Refactoring without tests is dangerous; with all the details we must keep straight, a mistake is easy to make. How many code reviews have you been in where the recommended design changes are not made because “we already tested it”? You avoid the change because it’s dangerous to change code without tests. So, the Boy Scout adds tests too. For more on Boy Scouts, see previous post.

Continue reading