- Carry out different tests
- Perform batch runs (do not stop when there is an exception in a particular test)
- Compare results of test with baseline and report the outcome
- Avoid cluttering of main source code with test scripts. Test tool should be decoupled from source as much as possible.
- Be able to do low level test (i.e. test individual functions with customized inputs)
- Record exceptions, errors, execution time
- Support VC++ 7, Ada 2005
The first way of satisfying these requirements is to write your own autotest tool. It means a major software development undertaking with all it's design, documentation and tests. On the bright side, you have all the flexibility you want Flexibility is a good thing only if you are an expert. Otherwise you come up with sloppy designs and painful reworking (which might cost more than you saved on automating the tests).
The second approach is to use already developed tools like TestComplete, NUnit.
Standard wisdom about software development says that if you can buy it, then buy it! TestComplete seems promising (Eric Sink advises it). It is relatively cheap (500$ per licence). My main concerns:
- You can't acess non-visual classes directly. You have to provide wrappers in visual classes (e.g. TForm). That is a big turn off because a) we usually write non-visual code b)it means we have to modify source which is close to blashpemy ;)
- You have to learn how to use it (interface, scripting etc)
- Seems to be especially beneficial for user interface testing since it can simulate user actions easly and provides recording so that you dont have to write all the scripts manually
- Does not support Ada
- Government purchase procedures are complicated
Verdict: Look at available test tools for inspiration, write your own test driver for the application at hand.
Below is a test case where I prepared an application in Delphi (it calculates sqrt) and tested it with TestComplete 4 Trial version:
No comments:
Post a Comment