MATRIS Members @IWCT
On Sunday, April 16, 2023, the 12th International Workshop on Combinatorial Testing (IWCT 2023) was held co-located with the 16th IEEE International Conference on Software Testing, Verification and Validation (ICST) 2023 in Dublin, Ireland. Angelo Gargantini (University of Bergamo) and Bernhard Garn (MATRIS Research Group, SBA Research) served as Co-Program Committee Chairs and Dimitris Simos (Group lead of MATRIS Research Group) served as General Chair of the workshop.
IWCT is one of the prime scientific venues focusing exclusively on combinatorial testing (CT), where theorists and practitioners of CT come together to present and discuss their newest results and the latest developments of interest in the wider software testing community. After three years of virtual editions, this year’s physical event was particularly enjoyed by the workshop participants.
This year, a total of 13 publications were accepted, seven of which in the category full paper, two in the category short paper and four in the category journal-first paper. MATRIS members contributed to the workshop through six talks by Ludwig Kampel, Manuel Leithner, Michael Wagner.
The first talk of Ludwig Kampel was an automotive case study on Combinatorial Testing Fault-localization (CT-FLA) methods for Automatic Emergency Braking functions, a collaborative work in the scope of the EU Ecsel project between MATRIS (Dimitris Simos, Michael Wagner, and Ludwig Kampel), AVL List GmbH (Mihai Nica, Dino Dodig, and David Kaufmann) and TU Graz (Prof. Franz Wotawa). In this talk, Ludwig Kampel showcased their pioneering work that utilizes CT-FLA methods to screen parameter settings in virtual driving scenarios focusing on identifying the scenario specifications that lead to critical scenarios, such as crashes. While in his second talk, Ludwig Kampel presented a Summary of Locating Hardware Trojans using Combinatorial Testing for Cryptographic Circuits, an extensive case study of a published journal article in IEEE Access. Ludwig explained how to locate hardware Trojans with trigger patterns of lengths up to eight by inserting them in an FPGA board while implementing an AES (Advanced Encryption Standard) algorithm with a 128-bit key length.
Moreover, a Journal-First paper titled “Summary of Combinatorial Methods for Testing Internet of Things Smart-home Systems” was presented. The presentation displayed a detailed creation of a dedicated input parameter model (IPM) for such systems which was then utilized via combinatorial test case generation strategies and also showed the developed automated test execution framework containing two test oracles. Additionally, the presentation highlighted the comparison between findings obtained by combinatorial test sets to those acquired by a random testing approach. While the combinatorial approach offers time-efficient results, all considered approaches performed nearly equally well in pointing to multiple errors and observations in the tested real-world IoT system.
Manuel Leithner presented two further works, “Combinatorial Methods for HTML Sanitizer Security Testing” and “Summary of Combinatorial Methods for Dynamic Gray-Box SQL Injection Testing”.
In the first presentation, Manuel displayed a combinatorial security testing (CST) based approach for detecting bypasses in HTML sanitizers and addressed how to utilize them to filter malicious user-provided HTML input based on a use case-specific configuration that allows or disallows specific HTML elements and attributes. He also showed an example from a real-world medical scheduling application that was utilized as the system under test in cooperation with Mobimed Software GmbH.
In his second talk, Manuel provided a Journal-First summary of a work originally published in Software: Testing, Verification, and Reliability.
Manuel also presented the results that show that our evaluation detects more vulnerable endpoints and produces a greater ratio of executed malicious inputs compared to state-of-the-art testing tools such as w3af, sqlmap, and wapiti.
Later, The original journal article [CPHF] was published in 2022 in the journal “Applied Mathematics and Computation”, where the authors designed an efficient algorithm to construct Covering Perfect Hash Families, which can be considered a compact representation of certain families of Covering Arrays. In an extensive set of experiments, they managed to construct the smallest known Covering Arrays in the literature for many different instances.
Overall, the workshop was a great scientific success for the MATRIS Research group, as their talks and presentations were not only well received by the audience but also generated considerable interest from the participants of IWCT and the wider ICST conference. At the end of the workshop day, Dimitris Simos gave positive concluding remarks and shared his view on the future direction of IWCT as part of the Steering Committee Meeting of IWCT.
Links
– Locating Hardware Trojans using Combinatorial Testing for Cryptographic Circuits (Ludwig Kampel; Paris Kitsos; Dimitris E. Simos): https://ieeexplore.ieee.org/document/9713898/authors#authors
– Combinatorial methods for testing Internet of Things smart home systems (Bernhard Garn, Dominik-Philip Schreiber, Dimitris E. Simos, Rick Kuhn, Jeff Voas, Raghu Kacker): https://onlinelibrary.wiley.com/doi/abs/10.1002/stvr.1805
– Combinatorial methods for dynamic gray-box SQL injection testing (Bernhard Garn, Jovan Zivanovic, Manuel Leithner, Dimitris E. Simos): https://onlinelibrary.wiley.com/doi/abs/10.1002/stvr.1826
– In-Parameter-Order strategies for covering perfect hash families (Michael Wagner, Charles J. Colbourn, Dimitris E. Simos): https://www.sciencedirect.com/science/article/abs/pii/S0096300322000388