News
Talk by Marcel Böhme Monday 10:00 "Software Testing as Species Discovery"
Written on 04.02.2020 13:29 by Andreas Zeller
Dear all,
as promised, here is the first of two talks highly recommended for fuzzing enthusiasts:
Next Monday at 10:00, Marcel Böhme (Monash U) will give a talk on "Software Testing as Species Discovery" (MPI-SWS E1 5 room 029). This is actually the very Marcel Böhme who wrote the chapter on "When to stop fuzzing", so here's your opportunity to ask him about anything! I have attended several of Marcel's talks, and his vision is mind-blowing. If you can, join us!
Looking forward to see you,
Andreas
- - cut here - -
Dear all,
You are cordially invited to an institute colloquium on:
Software Testing as Species Discovery
by Marcel Böhme from Monash University, Australia.
hosted by Catalin Hritcu
on Monday the 10th of February, 2020 in SB E 1 5 room 029, videocast to KL room 111 (VMR6312).
Abstract:
A fundamental challenge of software testing is the statistically well-grounded extrapolation from program behaviors observed during testing. For instance, a security researcher who has run the fuzzer for a week has currently no means (i) to estimate the total number of feasible program branches, given that only a fraction has been covered so far, (ii) to estimate the additional time required to cover 10% more branches (or to estimate the coverage achieved in one more day, resp.), or (iii) to assess the residual risk that a vulnerability exists when no vulnerability has been discovered. Failing to discover a vulnerability, does not mean that none exists—even if the fuzzer was run for a week (or a year). Hence, testing provides no formal correctness guarantees.
In this talk, I establish an unexpected connection with the otherwise unrelated scientific field of ecology, and introduce a statistical framework that models Software Testing and Analysis as Discovery of Species (STADS). For instance, in order to study the species diversity of arthropods in a tropical rain forest, ecologists would first sample a large number of individuals from that forest, determine their species, and extrapolate from the properties observed in the sample to properties of the whole forest. The estimation (i) of the total number of species, (ii) of the additional sampling effort required to discover 10% more species, or (iii) of the probability to discover a new species are classical problems in ecology. The STADS framework draws from over three decades of research in ecological biostatistics to address the fundamental extrapolation challenge for automated test generation. Our preliminary empirical study demonstrates a good estimator performance even for a fuzzer with adaptive sampling bias—AFL, a state-of-the-art vulnerability detection tool. The STADS framework provides statistical correctness guarantees with quantifiable accuracy.
Bio:
Marcel Böhme is 2019 ARC DECRA Fellow and Lecturer (Asst Prof) at the Faculty of IT at Monash University, Australia. He completed his PhD at the National University of Singapore advised by Prof Abhik Roychoudhury in 2014. It followed a postdoctoral stint at the CISPA-Helmholtz Zentrum Saarbrücken with Prof. Andreas Zeller and a role as senior research fellow at the TSUNAMi Security Research Centre in Singapore. Marcel's research is focused on automated vulnerability discovery, program analysis, testing, debugging, and repair of large software systems, where he investigates practical topics such as efficiency, scalability, and reliability of automated techniques via theoretical and empirical analysis. His high-performance fuzzers have discovered 100+ bugs in widely used software systems, more than 60 of which are security-critical vulnerabilities that are registered as CVEs at the US National Vulnerability Database.