Unit testing RTOS dependent code – RTOS Test-Double – Part 1

Posted on by

When you’ve got legacy code that depends on the Real-time Operating System, you have a challenge to get your code off the target for unit testing. If you want to test with the concurrency provided by the RTOS, these are not really unit tests, and you won’t be able to write thorough tests, and you need unit tests to be through.

You’re going to need to make a test-double so you can get your code off the RTOS and into your development system for unit testing. In this article we’ll go through the steps to get started.


Let’s consider this scenario: Our legacy code waits on a semaphore for some condition to occur. In a concurrent application, one thread waits while an interrupt routine or another thread satisfies the need. For example, a message processor waits for a line of text to be entered. Under the hood, there is a UART serviced by an interrupt. The interrupt saves characters until the line terminator is detected and then posts to the semaphore, waking the message processor.

To unit test the message processor, we can introduce a test-double for the semaphore pend operation using the linker. In the test-double we can put the message string into the buffer that we want the message processor to process.

So, from the message processor point of view, nothing is changed. It waits on the semaphore and processes the message when it wakes up.

To make this vision come true, we need an RTOS test-double. So let’s test drive that using the Crash to Pass algorithm. The tests will guide us, and document the test-double.

First, we have to create a test case for the test-double, that calls some RTOS function. I am going to use the Micrium uCos-II RTOS in my example. The fist test-double we need is OSSemPend(), that is defined in the ucos_ii.h. This test drives me to discover the include dependencies that come with ucos_ii.h.

#include "CppUTest/TestHarness.h"
 
extern "C"
{
#include "ucos_ii.h"
}
 
TEST_GROUP(uCosIITestDouble)
{
    void setup() { }
    void teardown() { }
};
 
TEST(uCosIITestDouble, OSSemPend)
{
    FAIL("tilt");
}

This fails because the compiler can’t find ucos_ii.h. Now I go to the command line an look for this file in RTOS directory using the unix find command.

$ find . -name "ucos_ii.h"
./uCOS-II/Micrium/uCOS-II/Source/ucos_ii.h
$

I can grab the output line and paste it right into your makefile or IDE include path. Here is what I put in my makefile:

    $(MICRIUM_HOME)/uCOS-II/Micrium/uCOS-II/Source/

MICRIUM_HOME points to the base Micrium directory. The next build causes a bunch more failures. Don’t be fooled, only look at the first error or errors, which in this case is other include files that cannot be found (app_cfg.h, os_cfg.h). So we search again like this:

$ find . -name "app_cfg.h"
./App/app_cfg.h
$ find . -name "os_cfg.h"
./App/os_cfg.h

Add $(MICRIUM_HOME)/App to the include path. Now os_cpu.h can’t be found.

If you are compiling with gcc, it helps to set these preprocessor flags:

    -Wfatal-errors

That makes the compiler stop complaining after the first compilation error.

$ find . -name "os_cfg.h"
./uCOS-II/Micrium/uCOS-II/Ports/ARM-Cortex-M3/Generic/GNU/os_cpu.h
./uCOS-II/os_cpu.h

The last search yields two directories. I’ll pick the generic one, rather than the one for the ARM. With $(MICRIUM_HOME)/uCOS-II added to the include path we get a clean build. My test reports “tilt”.

We’ve gotten the header to compile, now let’s try to get the a call to OSSemPend to compile in the test case.

Here is the prototype for OSSemPend():

void  OSSemPend   (OS_EVENT *pevent,
                  INT32U           timeout,
                  INT8U           *perr);

Add a call to OSSemPend to the TEST.

TEST(uCosIITestDouble, OSSemPend)
{
    OS_EVENT event;
    INT8U error = -1;
    OSSemPend(&event, 0, &error);
}

Miraculously, this compiles, though it won’t link, as we can only build with uCos-II on the target. We need a link time stub. Add a stub implementation in a new file mocks/uCosIITestDouble.c. The initial implementation looks like this:

// uCosIITestDouble.c
#include "ucos_ii.h"
 
void  OSSemPend(OS_EVENT *event, INT32U timeout, INT8U *error)
{
}

Now we can teach this OSSemPend test-double to satisfy the condition that the production code was looking for, a message to process. inputqueue.h defines a FIFO to hold the characters. The production message processor gets characters from the input queue; the interrupt routine puts characters into the queue. So, here’s a test-double implementation:

// uCosIITestDouble.c
#include "ucos_ii.h"
#include "inputqueue.h"
 
void  OSSemPend2(OS_EVENT *event, INT32U timeout, INT8U *error)
{
    const char * input = "help";
    while (*input)
    {
      InputQueue_Put(*input);
      input++;
    }
}

This will help us get through the first test for the first client of OSSemPend(), but we’ll have to evolve it for other tests and other users of OSSemPend(). That is the topic of a coming article. If you want to see it, tweet this article or post a reply.

17 thoughts on “Unit testing RTOS dependent code – RTOS Test-Double – Part 1”

  3. I have a similar problem, but so far I was using a different approach, maybe more complicated, but gives me the opportunity to run the tests on the same platform.
    Not sure if this is an advantage 🙂
    Basically I create a test-double library with the needed external APIs implemented in it. I keep all the “original” headers. And just for the test executable link module under test with the test-double lib.
    It takes more time, and the TDD is still under PC, but once I have the unit tests implemented I can run them on the platform. And so far I have so some strange behaviors of target device compilers. As I said not sure if that is worthed yet 🙂

  4. I’d sure like to see the next article. I’m full of questions, but since I’ve typed and then deleted them at least 3 times, I clearly can’t articulate them.

    I think my confusion is around the uCosIITestDouble TEST_GROUP. Does it exist solely to test your test doubles? And if so, wouldn’t you want to start with a TEST_GROUP(MessageProcessor) to get off the ground?

    “The less committed TDDers might get discouraged by this process…” I think I’m starting to see what you meant here.

  5. @Chris

    Yes, the uCosIITestDouble TEST_GROUP is there to define and build the test double. You are also right that I should have one or more MessageProcessor tests written or in mind to drive me to create this test-double. I had it in mind, but what I wanted to write about is the RTOS test-double 🙂

    I probably would have started with a simple case that just needed a stub version of the OSSemPend() that returned some fixed value. Eventually I would need the more sophisticated test-double.

    James

  6. @drifter

    I encourage people to get their code off the target for TDD and unit testing. Off-target testing has plenty of other advantages too (like guiding you to a more modular and portable design).

    My preference is to use the production code headers, but sometimes they need to be substituted (as described in an earlier blog post).

    Running the tests on the target is great too. It can reveal portability problems, library and compiler bugs. I first described an approach like yours in Progress Before Hardware. Sounds like you are discovering the same thing.

    Many embedded developers adopting TDD never get around to running tests in the target. They find plenty of value in just the off-target testing. I encourage people to get a continuous integration server going that can do the off-target builds, then cross-compile for the target, and even push the test build to a simulator or some real hardware.

    James

  7. Pingback: Unit testing RTOS dependent code – RTOS Test-Double – Part 2 « James Grenning’s Blog

  8. Test double have the advantage to don’t require the embedde hardware this is the true advantage and it is also possible to perform tests in shorter time. Testing on the production hardware it is also a must because is the system that will work on the field.

    So I think we need to discover a newer testing approach that may work well in both situations.

    I personally use BDD to drive my developments and so mocks are a must and helps a lot to find the more complex modules. It is a top-down approach and so in a shorter time it is possible to have the application skeleton ready and then introduce the modules to substitute the mokcs and refine the design.

    Sometimes I use the test double approach but as I am able to perform continuous integration and test on the embedded system I prefer to use it.

  9. The way I think of it, the unit tests have a goal (and there are others) of making sure the code does what the programmer expects. TDD off-target lets me do that more quickly. As you correctly point out, this does not mean the code will actually work. I figure it has a chance of working for the right reasons (because the code is doing what I intended, and my intentions happened to also be correct.)

    Automated unit tests like I am showing also make it easy to cause situations that the code must handle, but are difficult to set up in the real execution environment. A couple examples are a hardware fault, or a lost message in a communication sequence.

    I like the CI approach too. A check in will cause several builds. Off-target unit tests, on-target unit tests, off and on-target acceptance tests. I want my off-target CI build to deply the test loads to a real or evaluation hardware.

  10. Interesting. Need to have a closer look on it.
    We also prefer to test off the target and many times, this is efficient and enough.
    But we still face such situations, where the target fails on successfully tested code.

    Thanks for sharing these ideas.
    Waiting for the next post.

  11. @James: yes both are not optimal solutions. An intermediate solution that works pretty well is to use a board/microcontroller simulator. The problem is to find the simulator for the microcontroller you need. About test automation with the real hardware when I can I use it, now I have a plan to find a more general solution, it should work for little and medium complexity projects.

  12. These unit tests do not mean on-target tests are not needed. They are. Both tests have their place in my opinion.

    I am not sure what your point of the top down is in this context. It’s one of the approaches I use as well.

  13. Hi Massimo

    I prefer instantaneous feedback that I can get form off-target (this includes simulator) test execution. It’s best if you can also run the tests in the target, though not always practical. Most the people I know doing this find the biggest value in getting their code off-target so they can test independently. There are some less tangible portability benefits that may also some in handy one day.

  14. I added a link to a C stubbing script that can be useful for doing RTOS fakes:

    Please join the Embedded Agile group.

  15. Pingback: Designing and testing embedded systems | jonaslinde.se

Leave a Reply

Your email address will not be published. Required fields are marked *

Be gone spammers * Time limit is exhausted. Please reload the CAPTCHA.