Last weekend was the Deep Agile Embedded Conference that I participated in. In this article I’ll answer some of the panel questions related to concurrent hardware development. There seems to be a theme here, because the hardware is involved, an embedded development team really can’t be agile. That’s not my point of view, or my experience. I am not a hardware engineer, but I have worked near them. Let’s see some of the questions and answers.
With agile iterations only a few weeks long, do agile teams really expect to modify hardware that frequently?
It would be better to ask a hardware engineer this question. But anyway, hardware engineers I worked with did not get their complex designs right the first time. They would do some design, then build a prototype, then test the design. It’s an incremental process. Hardware, as I’ve watched its design, starts with a high level block diagram, then some specific area it taken to more detail. Circuits, like code, are incrementally designed and refined based on continual learning.
I know teams that do overnight board turns. Its expensive, so the don’t do it every night, but have that as an option, and find it a valuable capability.
Because hardware lead-times are long, don’t you have to design most of the system up-front? Isn’t this against what agile calls for?
In embedded systems development hardware, software and mechanics are built concurrently. That means a BDUF is needed. Comments?
Hardware lead times are long, and granted, HW is not as easy to change as software. Big design up front might seem like the answer. If that is the question, you might hear an agile wise guy ask “how’s that BDUF working for you?” After the laughter dies down (admitting that there is a problem is the first step to recovery) we can get real. BDUF has problems; there is still rework; there is still incremental learning and progress.
At an early stage of development we need to understand the HW/SW boundary in concept, but maybe not in detail. The actual boundary will be fuzzy in the beginning while engineering is still learning; it solidifies over time. Design is a creative learning process no matter how much we want to be able to finish it right the first time.
The fuzzy boundary does not have to halt progress. We can make good progress on both sides of the fuzzy boundary while learning the issues. An agile team will get good at keeping options open while making concrete progress toward the product vision. If you commit too early, you can end up with significant rework, the very thing that concerns you with agile.
The waterfall approach encourages committing before there is enough information to make a good decision on all the technical issues. I am not suggesting that we digress into analysis paralysis. The contrary, get good at keeping options open, make progress concretely by building something and testing it. As the lean folk say: commit at the last responsible moment. It’s a judgement call, but that guidance helps us keep design options open longer and helps avoid designing ourselves into a corner.
Design, be it hardware or software, is an incremental process. Code gets written one line at a time. Boards are designed one circuit at a time, mechanics are… well I don’t know much about that, but it is a creative process. Don’t do to much work on a speculative basis without trying the ideas out. The feedback and course corrections will lead to a better designed product with visible progress.
Hardware is often not ready until late in the product cycle, how can testing be continuous from the beginning without hardware?
From a software perspective, developers must decouple the core application code from the hardware. This allows code to be tested both on and off the target, called dual targeting. Where progress can be made early and often, code is first tested in the development system. Periodically code is compiled and run in the target. The target may not be available early in the development process. So dual targeting is critical for early software progress. That progress is measured in passing automated tests rather than document weight.
To pull this off, embedded software developers must manage their dependencies on the hardware and operating system. A thin layer should separate the actual hardware from the core system behaviors.
With the ability to run some tests on the development environment and some in target, the next thing to do is to get a continuous integration server going. There are a number of tools out there. I have found hudson a breeze to set up. What a CI server does is monitor your source repository and initiate a build when there are changes. First the code is automatically built and automated tests are run on the development system. Next the CI script should build for target and automatically deploy the tests executable to the target, or target compatible execution environment. If there are failures, all contributors to the build are notified so the build can be fixed immediately. Many integration problems are uncovered automatically with this approach.
The advice to abstract the hardware is nothing unique to agile. Think of your own recent history, if you had abstracted the hardware, would that major redesign have been needed when moving your software to the new hardware platform. Instead, you would be readapting your hardware abstraction layer to new HW. The TDD approach encourages the creation of these test seams in your code.
See Progress Before Hardware for a in-depth look at this topic.