Tips for software integration

I have been involved a couple of times in software development efforts where my small module (~20 KLOC) had to be integrated into a larger module (~1 MLOC).

Complicating factors:

• It was not possible to install the larger module on my development environment. There was no offline version.
• Interface Control Document (ICD) did not exist or was out of date the minute it was written because the larger module was also in development.

Under these circumstances it took a lot of effort to make sure that my module worked ok with the larger one. I once measured the amount of time required to correct a simple bug during integration testing and imagined how much it would take me to do the same thing on my own development environment. The factor was about 10, i.e. it takes 10 times more effort to fix something in integration because:

• You don't have the tools you are used to.
• You won't get detailed error reports. What you get is usually the equivalent of the blue screen of death.

The lessons I learned the hard way are:

• If a formal ICD does not exist, write an informal one yourself.
• For the parts of your software that are independent of the larger module, write unit tests.
• Use defensive programming. All your interface functions should have sanity checks (are the values within a sensible boundary etc.). Use assertions or exceptions to stop right where the error happened and not deep down in your module where you wonder why a value is NaN and try to work your way back from that hole.
• For functionality that is not unit testable, write a separate test module which has the same interfaces of the original but only performs automated testing of interface functions when run. Let's say you use a calcRange() method from the larger module which takes two points and returns the distance. Your interface test module should on start up exercise calcRange() with different point combinations and log the results. You should first run this interface test module before integrating the real one. If all tests pass, you will be sure that the interfaces work as assumed. I have seen over and over again where an interface worked a month ago and was broken the next time we tried to integrate.
• If you develop on Windows 7 but the deployment machine has Windows XP, test your code on all versions of Windows XP. I have seen cases where my module worked fine on XP service Pack 1 but failed miserably on Service Pack 2. It cost me 6 months.
• And lastly, it will take way more time than you think. If this is the first time you are about do integration, multiply your effort estimates by 100. I am not kidding! I know that you can never tell this to upper management (read Waltzing with Bears). But at least YOU should know it. I prefer reality to fairy tales and in the long run it reduces my stress when I know what to expect.