Combinatorial security testing tutorial @ DATAWorks 2019, Springfield, VA, USA
Dimitris Simos (SBA Research, MaTRIS research group) and Rick Kuhn (NIST, Computer Security Division) were invited to jointly present a tutorial on combinatorial security testing at the Defense and Aerospace Test and Analysis workshop (DataWorks 2019) on April 11th, 2019.
The event takes place at Springfield, VA, USA and is organized by NASA, Institute for Defense Analyses (IDA) and the Office of the Secretary of Defense.
Tutorial: Combinatorial Methods for Testing and Analysis of Critical Software and Security Systems
Abstract: Combinatorial methods have attracted attention as a means of providing strong assurance at reduced cost, but when are these methods practical and cost-effective? This tutorial includes two sections on the basis and application of combinatorial test methods: The first section explains the background, process, and tools available for combinatorial testing, with illustrations from industry experience with the method. The focus is on practical applications, including an industrial example of testing to meet FAA-required standards for life-critical software for commercial aviation. Other example applications include modeling and simulation, mobile devices, network configuration, and testing for a NASA spacecraft. The discussion will also include examples of measured resource and cost reduction in case studies from a variety of application domains.
The second part explains combinatorial testing-based techniques for effective security testing of software components and large-scale software systems. It will develop quality assurance and effective re-verification for security testing of web applications and testing of operating systems. It will further address how combinatorial testing can be applied to ensure proper error-handling of network security protocols and provide the theoretical guarantees for detecting Trojans injected in cryptographic hardware. Procedures and techniques, as well as workarounds will be presented and captured as guidelines for a broader audience.
The tutorial can be download from here.