Section: New Results
Test Generation on Enumerative and Symbolic Models
Integrating formal verification and conformance testing for reactive systems
In this work we describe a methodology integrating verification and conformance testing. A specification of a system - an extended input-output automaton, which may be infinite-state - and a set of safety properties (“nothing bad ever happens”) and possibility properties (“something good may happen”) are assumed. The properties are first tentatively verified on the specification using automatic techniques based on approximated state-space exploration, which are sound, but, as a price to pay for automation, are not complete for the given class of properties. Because of this incompleteness and of state-space explosion, the verification may not succeed in proving or disproving the properties. However, even if verification did not succeed, the testing phase can proceed and provide useful information about the implementation. Test cases are automatically and symbolically generated from the specification and the properties, and are executed on a black-box implementation of the system. The test execution may detect violations of conformance between implementation and specification; in addition, it may detect violation/satisfaction of the properties by the implementation and by the specification. In this sense, testing completes verification. The approach is illustrated on simple examples and on a Bounded Retransmission Protocol [Oops!]
Automatic test generation from interprocedural specifications
This work is done in collaboration with Bertrand Jeannet (Inria Rhône-Alpes) and partly supported by France Telecom R & D. It adresses the generation of test cases for testing the conformance of a reactive black-box implementation with respect to its specification. We aim at extending the principles and algorithms of model-based testing for recursive interprocedural specifications that can be modeled by Push-Down Systems (PDS). Such specifications may be more compact than non-recursive ones and are more expressive. The generated test cases are selected according to a test purpose, a (set of) scenario of interest that one wants to observe during test execution. The test generation method we propose in this paper is based on program transformations and a coreachability analysis, which allows to decide whether and how the test purpose can still be satisfied. However, despite the possibility to perform an exact analysis, the inability of test cases to inspect their own stack prevents it from using fully the coreachability information. We discuss this partial observation problem, its consequences, and how to minimize its impact [Oops!] .
Application to security
While a lot of work has been done on formal verification of security, in particular for cryptographic protocols, very little has been done on formal security testing. As a consequence, testing security often resort on the expert knwoledge and leads to ad hoc solutions. The general challenge is to study how formalization of security policies and information systems can help in automatically (or systematically) performing security testing. Several approaches are already investigated. In the context of ACI Potestat and RNRT Politess, we study how test generation techniques, and in particular test generation from safety properties [Oops!] , can be used for the automatic generation of possible attacks, attacks which should then be tested on the real system, due to the abstraction used in modelling and generation.
Finally, during our collaboration with University of Nijmegen we studied the combinaison of verification, testing and learning. The verification of cryptographic protocol specifications is an active research topic and has received much attention from the formal verification community. By contrast, the black-box testing of actual implementations of protocols, which is, arguably, as important as verification for ensuring the correct functioning of protocols in the “real” world, is not much studied. We propose an approach for checking secrecy and authenticity properties not only on protocol specifications, but also on black-box implementations. The approach is compositional and integrates ideas from verification, testing, and learning. It is illustrated on the Basic Access Control protocol implemented in biometric passports [Oops!] .