Documentation Driven Testing
Although a lot of tools have been created to improve your code quality (checkstyle, code coverage, sonar metrics...), being able to ensure that your software has a good functional quality (i.e. fulfills the functional requirements) is still very hard, because of a clear lack of tooling.
The Eclipse Modeling Project, associated with tools like Mylyn Intent, leads us to the establishment of testing techniques which could be described as Documentation-Driven Testing.
The more a project grows, the more complicated managing of all of its testing assets becomes. When a test fails, you can easily have technical information about this failure (through the failure message, comments in the tests, the code...). However, quickly determining which functionnal requirements have been broken by a test failure is a burden: you'll have to browse manually through your documentation or your bugzilla issues, hoping that you'll be able to find some sentence explaining clearly what the application is supposed to do.
A new solution can now be proposed based on two axis:
- A model-driven testing framework, introducing a better level of abstraction in the tests. Using a modeling system for the scenarii, this framework details precisely the sequence of tests and allows other modeling tools to easily interact with it.
- A bridge between this framework and Mylyn Intent. By interpreting a document describing the intent and conduct of the tests, Mylyn Intent can infer the test scenarii which will be executed by the test framework.
This approach brings 2 main advantages:
- when a test fails, you immediatly get the documentation part that specifies how your software should react. You can then decide if you really made a regression or if the specifications should evolve.
- when you want to make a change in your specifications, you can easilly get all the tests to modify, and vice-versa. Therefore your software and your functionnal requirements gain in Agility.
During this session, we will present the concepts outlined above and show you how we used Documentation-Driven Testing to test EEF 2.0.