MATRIS @The International Workshop on Combinatorial Testing
On Sunday, April 16, 2023, the 12th International Workshop on Combinatorial Testing (IWCT 2023) was held co-located with the 16th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2023 in Dublin, Ireland. Angelo Gargantini (University of Bergamo) and Bernhard Garn (MATRIS Research Group, SBA Research) served as Co-Program Committee Chairs and Dimitris Simos (Group lead of MATRIS Research Group) served as General Chair of the workshop.
IWCT is one of the prime scientific venues focusing exclusively on combinatorial testing (CT), where theorists and practitioners of CT come together to present and discuss their newest results and the latest developments of interest in the wider software testing community. After three years of virtual editions, this year’s physical event was particularly enjoyed by the workshop participants.
This year, a total of 13 publications were accepted, seven of which in the category full paper, two in the category short paper and four in the category journal-first paper.
The workshop started with a keynote given by Rachel Tzoref-Brill titled “Combinatorial Testing: From Theory to Practice” describing different real-world challenges in the application of CT in practice as well as corresponding solutions for overcoming these challenges. Solving these challenges led to wide use of CT across IBM testing services. Rachel’s keynote was closely followed by the participants and her expert view enriched the program of IWCT considerably.
MATRIS was not only involved in the program organization of the workshop, but also contributed several submissions to IWCT that were subsequently accepted and presented at the workshop. From the MATRIS Research Group, Bernhard Garn, Ludwig Kampel, Manuel Leithner, Michael Wagner and Dimitris Simos joined the workshop in Dublin.
After the scheduled talks, Manuel Leithner, Michael Wagner and Andrea Bombarda from the University of Bergamo presented the results of this year’s CT tool competition, which was hosted for the second time as part of IWCT. Six different combinatorial test generation tools were tested on 240 benchmark instances, split up into 10 different tracks, and evaluated based on their execution time and the size of the generated test sets. The data of the competition has been make publicly available at ZENODO. Just like last year, the tool CAgen by the MATRIS research group ended up performing the best out of all evaluated tools, followed by ACTS and caopt on ranks 2 and 3, respectively.
Our congratulations go to the authors of the original ACTS paper (Linbin Yu and Yu Lei from the University of Texas at Arlington; Raghu Kacker and Richard Kuhn from US NIST), which was published in ICST ten years ago, for their win in this year’s most influential paper prize this year. It is great to see their influential work being acknowledged by the wider software testing community and we are proud of our collaboration with these well-known scientists from US NIST and University of Texas at Arlington.