Best Paper Presentation at ICTSS 2024 in London
Luiza Corpaci, representing SBA’s CORE and CALGO, joined the 36th International Conference on Testing Software and Systems (ICTSS), held from October 30 to November 1 in London, UK.
In the very first session of the conference, Luiza presented the best paper award-winning contribution Estimating Combinatorial t-way Coverage based on Matrix Complexity Metrics, a joint work with Michael Wagner, Sebastian Raubitzek, Ludwig Kampel, Kevin Mallinger and Dimitris E. Simos.
© SBA Research
In their work, the authors devise and evaluate machine learning algorithms that use matrix complexity metrics as features for estimating the t-way coverage of randomly generated software test sets. This new approach provides a good estimation while being faster and more scalable than a state of the art tool for the exact computation of combinatorial t-way coverage. Further, the experiments show a connection between the spectrum of singular values of a random test set and its t-way coverage.
The work and the presentation was very well received by the audience and generated a lot of attention. The result were vivid discussions Luiza and other participants, starting directly after the talk and continued in the coffee break following.
Abstract
Efficiently estimating combinatorial t-way coverage in software testing remains a significant challenge due to the high-dimensional complexity of modern software systems and the inherent complexity of the problem. This article explores a novel method that uses matrix complexity metrics as features in machine learning algorithms to estimate the t-way coverage of random test sets.
Based on an input parameter model of the system under test (SUT) we derive complexity metrics from the singular value decomposition of the matrix representation of the test set. This makes our approach independent of the SUT’s input space dimension and the size of the test set. Our approach provides a good estimation of the combinatorial t-way coverage while being faster and more scalable than a state of the art tool for the exact computation of combinatorial t-way coverage.
Moreover, our experiments show a connection between the spectrum of singular values of a random test set and its t-way coverage. The use of complexity metrics as predictors in our machine learning pipeline adds a new dimension to the combinatorial testing domain, offering an additional tool for improving software testing processes.