Security Testing

Security Testing

[Preliminary info from 2021; may change for 2023/24]

Software has bugs, and catching bugs can involve lots of effort. This course addresses this problem by automating software testing, specifically by generating tests automatically. Recent years have seen the development of novel techniques that lead to dramatic improvements in test generation and software testing.  In this course, we explore these techniques – in theory and in code.

Course Organization

Every week, you will be provided with Jupyter Notebooks that teach a particular topic and illustrate it using plenty of runnable Python code.  These notebooks come from The Fuzzing Book, a textbook on how to generate software tests written by yours truly.

In the notebook, you can edit the code as you like, run your own experiments, and re-use and extend the code to your liking.  Your task will be to use these techniques (and their code) to build a series of fuzzers (i.e. test generators) that find bugs in a number of challenging settings.

Meetings

This course uses the "inverted classroom" principle – you learn at home, and discuss issues with your instructor.  In our weekly meeting, we use the gathering in the lecture hall to

• discuss the assignment of last week
• discuss the assignment of next week
• discuss ongoing projects as well as general questions.

These meetings come with live coding, so we can explore ideas right on the go.

Projects

During this course, you apply the techniques learned in weekly exercises and two projects which form your coursework. Projects are graded for effectiveness, efficiency, elegance, and creativity. Projects offer special challenges which allow you to gain bonus points.

Exercises

Every week, you get a simple exercise assignment covering the material of the last lecture. Performance in these exercises will make 33% of the final grade.  Note that there is no final exam.

Requirements

Advanced programming skills (such as obtained after two years of successfully studying CS) are required. Knowledge in Python is useful, but can easily be acquired along the course.

Passing Criteria and Grading

To pass this course, you need to have

• at least 50% of the points in each of the projects, and 
• at least 50% of the total exercise points.

Lecture Plan

The course is organized as "inverted classroom": Every week, we discuss a chapter of the book, which will be supplied with an introduction video; we meet once a week to discuss the material, the associated exercises, and the ongoing projects.

The sequence of chapters is different from the book; in order to synchronize with the projects, we first discuss black-box techniques, then white-box techniques, and then domain-specific approaches.

2021-10-19: Introduction to the course; Introduction to Software Testing
2021-10-26: Introduction to Fuzzing
2021-11-02: Fuzzing with Grammars
2021-11-09: Efficient Grammar Fuzzing • Grammar Coverage
2021-11-16: Probabilistic Grammar Fuzzing (makes use of Parsing Inputs) • Fuzzing with Generators
2021-11-23: Code Coverage
2021-11-30: Mining Input Grammars (guest lecture)
2021-12-07: Mutation-Based Fuzzing • Greybox Fuzzing
2021-12-14: Greybox Fuzzing With Grammars 
2021-01-04: Tracking Information Flow • Concolic Fuzzing
2021-01-11: Symbolic Fuzzing
2021-01-18: Fuzzing APIs • Fuzzing Configurations
2021-01-25: Testing Web Applications • Testing Graphical User Interfaces
2021-02-01: Reducing Failure-Inducing Inputs
2021-02-08: When to Stop Fuzzing (guest lecture) • Current Trends in Fuzzing Research

The lecture plan may be subject to changes; these will be announced in time.

Date, Time, Location

• 15 lectures
• 6 Credit Points