Project : triskell
Section: New Results
Automatic Test Cases Optimization: a Bacteriologic Algorithm
The level of confidence in a software component is often linked to the quality of its test cases. This quality can in turn be evaluated with mutation analysis: faults are injected into the software component (making mutants out of it) to check the proportion of mutants detected by the test cases (in the mutation terminology, the mutant is said "killed" by the test case). But while the generation of basic test cases set is easy, improving its quality may require prohibitive effort. This work focuses on the issue of automating the test optimization. The application of genetic algorithms looks like an interesting way to solve it. The optimization problem is modeled as follows from an evolutionary point of view: a test case can be considered as a predator while a mutant program is analogous to a prey. The aim of the selection process is to generate test cases able to kill as many mutants as possible, starting from an initial set of predators, which is the test cases set provided by the programmer. To overcome disappointing experimentation results, on .Net components and Eiffel classes, a slight variation on this idea is studied, no longer at the "animal" level (lions killing zebras) but at the bacteriological level. The bacteriological level indeed better reflects the test case optimization issue: it mainly differs from the genetic one by the introduction of a memorization function and the suppression of the cross over operator.  explains how the genetic algorithms have been adapted to fit with the issue of test optimization.  shows the properties of the resulting algorithm which differs so much from genetic algorithms that it has been given another name: bacteriological algorithm.
Measuring Design Testability of a UML Class Diagram
Design-for-testability is a very important issue in software engineering. It becomes crucial in the case of OO designs where control flows are generally not hierarchical, but are diffuse and distributed over the whole architecture. We concentrate on detecting , pinpointing and suppressing potential testability weaknesses of a UML class diagram (). The attribute significant from design testability is called ``class interaction'' and is generalized in the notion of testability anti-pattern: it appears when potentially concurrent client/supplier relationships between classes exist in the system. These interactions point out parts of the design that need to be improved, driving structural modifications or constraints specifications, to reduce the final testing effort. The testability measurement we propose counts the number and the complexity of interactions that must be covered during testing. The approach is illustrated on application examples, taken from varous application domains (a distributed chat software, a compiler, all the catalogue of design patterns from the GoF).
 synthesizes our research efforts in the field of object-oriented design testability measurement, that has also been done in collaboration with Gerson Sunyé from LINA lab. These efforts have two different goals. First, we identify recurrent design structures (or testability anti-patterns) that worsen software testability. Second, we use the UML extension mechanisms to better specify design information that can make implementation more testable. Although detecting testability anti-patterns during software design is a crucial task, one cannot expect from a non-specialist to make the right improvements, without guidance or automation. To overcome this limitation, each definition of an anti-pattern is associated with an alternative design solution.
From Testing to Diagnosis: An Automated Approach
The need for testing-for-diagnosis strategies has been identified for a long time, but the explicit link from testing to diagnosis is rare. Here (see ), we start with the study of an algorithm for fault localization that consists of cross-checking information collected from test cases execution traces. Analyzing the type of information needed for an efficient localization, we identify the attribute (called Dynamic Basic Block) that restricts the accuracy of a diagnosis algorithm. Based on this attribute, a test criterion is proposed and validated through rigorous case studies: it shows that test cases can be completed to reach a high level of diagnosis accuracy. So, the dilemma between a reduced testing effort (with as few test cases as possible) and the diagnosis accuracy (that needs as much test cases as possible to get more information) is partly solved by selecting only test cases relevant for diagnosis.
System Testing of Product Families: from Requirements to Test Cases
In large software system, it is mandatory to address the question of continuity in the refinement process from high level textual requirements to analysis, design and test. We focus on functional requirements, expressing the expected system services. The main constraint we seek to solve here is to remain both domain-independent (so portable for other application contexts), and to let the possibility for a company to create its own requirement assets. In this domain, our contribution is two-fold: a method , supported by a tool , to disambiguate incrementally textual requirements and create requirement assets (called ``interpretation patterns''), and a new to use the underlying formal model as a first basis to ensure refinement continuity, by requirements simulation, and automated derivation of early analysis (use cases) and design models, expressed within the UML . The approach has been applied on two components of last generation combat aircrafts developed by THALES Airborne System.
Contract-Based Testing: from Objects to Components and from contracts to diagnosis probes
Contracts on classes have been first developed as an OO software design approach. They were also quickly used for supporting class testing, providing a form of design for testability. We identify the tracks to extend the contract-based built-in test technique to hierarchical components.
To do that, we build on our previous work on stclass, a framework supporting Design by Contract and built-in test for Java, and on confract, a contracting system for the fractal component platform. Tests are embedded in the components and are generated with respect to a category of contract (library, interface, composition). In collaboration with D. Deveaux, P. Collet and R. Rousseau , we studied how this approach, firstly dedicated to objects, can be valuable for components testability. As a result of the test process, the embedded contracts are more robust and offer an original way to improve the observability of the component-based system. Contracts make it aware of its execution, and thus able to detect erroneous behaviors at runtime.
In addition to detection of errors related to design, coding or deployment of an application, the diagnosis is a well-known technique for understanding the behaviour of a software system and an absolute requirement for its improvement. Unfortunately, applications become more difficult to diagnosis as functionalities provided become complex.
In collaboration with the KEREVAL company, we explore the use of dynamic probes (sensors) that will be injected into running system to collect various information. The innovative part of this approach is the use of generic probes to develop diagnosis framework .