Wednesday, April 30, 2008

Testing by JExample

The back end of our software - the part which allows us to scale to millions of users - has some very long tests involving networks and databases. They take long enough to run (30+ minutes) that, when run over our VPN, the connection is sometimes dropped before the tests can conclude. While infrequent it is a cause of great frustration when it happens.

Unit tests need isolation from potential side-effects. Providing that isolation requires potentially extensive and / or time-consuming setup and sometimes tear-down code.

A very new testing framework based on JUnit is called JExample. Unlike JUnit, tests can depend one upon another. Since the tests have passed, the environment is in a known, tested as correct, state and can be relied upon as the basis of subsequent tests.If test A passes, then test B can rely upon the result and context of A. When A is a long test, or requires considerable setup or teardown, this can result in substantial time savings.

A dependency graph of tests is built and used to order the tests.

The main results are less elapsed time for the tests and test failures which are more likely to pin-[point the exact place where a problem is occurring.

The work is based on the paper by  Kuhn, Van Rompaey, Haesenbarger, Nierstrasz, Demeyer, Gaelli and Van Leemput. There is an introduction and Eclipse integration. Java 6 is required.

1 comment:

Bart said...

We lowered the Java requirement to Java 5, as it appeared to be only a small API issue.