xmlmate

Teaser video

http://youtu.be/-yKom5mbft0

Abstract

Generating system inputs satisfying complex constraints is still a challenge for modern test generators. We present XMLMATE, a search-based test generator specially aimed at XML-based systems. XMLMATE leverages program structure, existing XML schemas, and XML inputs to generate, mutate, recombine, and evolve valid XML inputs.

XMLMATE is a search-based test generator. As sketched in Figure 2, it either takes a population of sample XML inputs or randomly generates one, and then systematically evolves this population over several generations to reach the desired coverage goal. To evolve inputs it applies mutation and crossover operators to individuals and favors the procreation and survival of the fittest according to a fitness function geared towards obtaining coverage and triggering exceptions. Measuring the fitness of an input requires executing the program under test; therefore, the runtime of the approach scales linearly with the runtime of the program.

XMLMATE is a search-based test generator. As sketched in Figure 2, it either takes a population of sample XML inputs or randomly generates one, and then systematically evolves this population over several generations to reach the desired coverage goal. To evolve inputs it applies mutation and crossover operators to individuals and favors the procreation and survival of the fittest according to a fitness function geared towards obtaining coverage and triggering exceptions. Measuring the fitness of an input requires executing the program under test; therefore, the runtime of the approach scales linearly with the runtime of the program.

Over a set of seven XML-based systems, XMLMATE detected 31 new unique failures in production code, all triggered by system inputs and thus true alarms.