# Selected Topics in Automated Testing and Debugging Rahul Gopinath + Rafael Dutra + Andreas Zeller

## News

29.09.2021

#### Final Presentation Tomorrow 10:00

Dear students,

We're scheduling the remaining final presentation for tomorrow, 30 September at 10 am.

https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09

Vinay Tilwani will present Smart Greybox Fuzzing (AFLSmart).

Dear students,

We're scheduling the remaining final presentation for tomorrow, 30 September at 10 am.

https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09

Vinay Tilwani will present Smart Greybox Fuzzing (AFLSmart).

Attendance is not mandatory, but you are encouraged to participate if you are available.

The final grades for the other students should already be posted on LSF.

Cheers,

Rafael+Rahul+Andreas.

16.09.2021

#### Help us tailoring automated debugging to your needs

Dear seminar participants,

Thank you for your interest in software testing and debugging! We're developing Alhazen, a tool that automatically finds out when and why your program fails. We are heavily relying on your opinion as professional developers. Please take... Read more

Dear seminar participants,

Thank you for your interest in software testing and debugging! We're developing Alhazen, a tool that automatically finds out when and why your program fails. We are heavily relying on your opinion as professional developers. Please take part in our user study to help us tie our approach to your needs:

Thanks a lot!

Andreas Zeller + Team

07.09.2021

#### Zoom link for final presentations

Hi,

The old link doesn't seem to work. We're using this new link for the final presentations:

https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09

See you there!

Rafael+Rahul+Andreas

30.08.2021

#### Final Presentations (7 and 8 September)

Dear students,

As announced earlier, the remaining final presentation will start next week. We'll have 2 presentation dates, with up to 4 presenters on each date.

Tuesday 7 September 9am-11am

Florian Bauckholt: A review of reverse debugging... Read more

Dear students,

As announced earlier, the remaining final presentation will start next week. We'll have 2 presentation dates, with up to 4 presenters on each date.

Tuesday 7 September 9am-11am

Florian Bauckholt: A review of reverse debugging (Time-travel Debugging)

Vikram Vashishth: Compiler Validation via Equivalence Modulo Inputs (EMI)

Reza Zamiri: FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)

Wednesday 8 September 2pm-4pm

Benjamin Peters: Delta Debugging

Ali Alhasani: Evocative Patterns for Behavioral Abstraction

Birk Blechschmidt: Mining Dynamic Invariants

Ruiyi Zhang: MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE)

Please time your presentation to a maximum of 20 minutes (including possibly a demo), which will be followed by 10 minutes discussion.

Thank you,

Rafael+Rahul+Andreas

09.07.2021

#### Abstract Submissions for FormatFuzzer

Dear Students,

Hope to see you soon at 10am for the first round of final presentations.

As we mentioned during the previous meeting, the next seminar will be on FormatFuzzer.

*  FormatFuzzer: Effective Fuzzing of Binary File Formats

Dear Students,

Hope to see you soon at 10am for the first round of final presentations.

As we mentioned during the previous meeting, the next seminar will be on FormatFuzzer.

*  FormatFuzzer: Effective Fuzzing of Binary File Formats
Dutra, Gopinath, Zeller
https://dl.cispa.de/s/3q2PyqP7rqZzrNn

We expect a short abstract, no more than one page, capturing the essential ideas. Also think about new directions to use/extend the FormatFuzzer framework.

As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for FormatFuzzer is here: https://cms.cispa.saarland/testdebug/submissions/view/10

Regards,

Rafael

07.07.2021

#### Seminar evaluation – please fill out form by July 14

Dear all,

as usual for all courses at Saarland University, you have the opportunity to evaluate the seminar and provide feedback – all anonymously, of course. Please do us a favor and fill out the form at

Dear all,

as usual for all courses at Saarland University, you have the opportunity to evaluate the seminar and provide feedback – all anonymously, of course. Please do us a favor and fill out the form at

Filling out the form takes only five minutes. Please provide your feedback by July 14!

Best wishes,

Rahul + Rafael + Andreas

02.07.2021

#### Abstract Submissions for AFLSmart

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Smart Greybox Fuzzing (AFLSmart)".

*  Smart Greybox Fuzzing (AFLSmart)
Pham, Böhme, Santosa, Caciulescu, Roychoudhury
https://arxiv.org/pdf/1811.09447.pdf

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Smart Greybox Fuzzing (AFLSmart)".

*  Smart Greybox Fuzzing (AFLSmart)
Pham, Böhme, Santosa, Caciulescu, Roychoudhury
https://arxiv.org/pdf/1811.09447.pdf

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for AFLSmart is here: https://cms.cispa.saarland/testdebug/submissions/view/9

Regards,

Rafael

30.06.2021

#### Final Presentations (9 July, 7 September, 8 September)

Dear students,

We have assigned the final topics as follows:

Benjamin Peters
1) Delta Debugging

Ali Alhasani
2) Evocative Patterns for Behavioral Abstraction

Osama Altamar
3) Language of Program Behaviors

Birk Blechschmidt

Dear students,

We have assigned the final topics as follows:

Benjamin Peters
1) Delta Debugging

Ali Alhasani
2) Evocative Patterns for Behavioral Abstraction

Osama Altamar
3) Language of Program Behaviors

Birk Blechschmidt
4) Mining Dynamic Invariants

Tim Speicher
5) Mining Program Input Grammars

Florian Bauckholt
6) A review of reverse debugging (Time-travel Debugging)

Vikram Vashishth
7) Compiler Validation via Equivalence Modulo Inputs (EMI)

Ruiyi Zhang
8) MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE)

Vinay Tilwani
9) Smart Greybox Fuzzing (AFLSmart)

Reza Zamiri
10) FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)

The first presentation day will be Friday 9 July in the morning at 10:00.

Tim and Osama are scheduled to present on 9 July.

There will be two additional presentation days in September: Tuesday 7 September in the morning at 9:00 and Wednesday 8 September in the afternoon at 14:00.

Birk, you can choose to present either in July or in September, as you prefer.

Cheers,

Rafael + Rahul + Zeller

28.06.2021

#### Choosing Dates for Final Presentation

Dear students,

Please select the dates you are available to present on this link. You can mark options with "yes" (preferred) or "yes, if need be" (not ideal, but still available).

Dear students,

Please select the dates you are available to present on this link. You can mark options with "yes" (preferred) or "yes, if need be" (not ideal, but still available).

Please complete the poll by tomorrow (29 June).

We may have more than one presentation day (for example, if some students want to present in July and some in September).

Of course, any students presenting in July will not have as a final presentation topic one of the two last papers (AFLSmart and FormatFuzzer).

Thank you,

Rafael + Rahul + Zeller

25.06.2021

#### Abstract Submissions for MultiSE

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "MultiSE: Multi-Path Symbolic Execution using Value Summaries".

*  MultiSE: Multi-Path Symbolic Execution using Value Summaries
Sen, Necula, Gong, Choi

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "MultiSE: Multi-Path Symbolic Execution using Value Summaries".

*  MultiSE: Multi-Path Symbolic Execution using Value Summaries
Sen, Necula, Gong, Choi
https://people.eecs.berkeley.edu/~ksen/papers/multise.pdf

This is an alternative way of implementing symbolic execution. In particular, you should discuss how it compares with the traditional symbolic and concolic execution techniques presented, for example, in the fuzzinbook.

Symbolic Execution: https://www.fuzzingbook.org/html/SymbolicFuzzer.html

Concolic Execution: https://www.fuzzingbook.org/html/ConcolicFuzzer.html

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/8

Regards,

Rafael

18.06.2021

#### Choosing Topic for Final Presentation

Dear Students,

In order to assign the topics for final presentations, please send us (rafael.dutra@cispa.de, rahul.gopinath@cispa.de) a ranking of your 3 preferred topics (with 1 being the most preferred).

1)

2)

3)

The possible topics are

Dear Students,

In order to assign the topics for final presentations, please send us (rafael.dutra@cispa.de, rahul.gopinath@cispa.de) a ranking of your 3 preferred topics (with 1 being the most preferred).

1)

2)

3)

The possible topics are

1) Delta Debugging
* Simplifying and isolating failure-inducing input
Zeller, Hildebrandt
https://www.st.cs.uni-saarland.de/papers/tse2002/tse2002.pdf
* Reducing Failure-Inducing Inputs
https://www.fuzzingbook.org/html/Reducer.html

2) Evocative Patterns for Behavioral Abstraction
* Abstracting Failure-Inducing Inputs
Gopinath, Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3103/7/issta2020-language-of-failure.pdf

* When does my Program do this? Learning Circumstances of Software Behavior
Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3107/7/fse2020-alhazen.pdf

3) Language of Program Behaviors
* Input Algebras
Gopinath, Nemati, Zeller
https://publications.cispa.saarland/3208/7/gopinath2021input.pdf
*  Learning the Language of Error
Chapman, Chockler, Kesseli, Kroening, Strichman, Tautschnig http://www.cprover.org/learning-errors/learning-the-language-of-error-including-a-proof-supplement.pdf

4) Mining Dynamic Invariants
* Dynamically discovering likely program invariants to support program evolution
Ernst, Cockrell, Griswold, Notkin
https://homes.cs.washington.edu/~mernst/pubs/invariants-tse2001.pdf

* Mining Function Specifications
https://www.fuzzingbook.org/html/DynamicInvariants.html

5) Mining Program Input Grammars
*  Mining Input Grammars from Dynamic Control Flow
Gopinath, Mathis, Zeller
https://publications.cispa.saarland/3101/1/fse2020-mimid.pdf

* Mining Input Grammars from Dynamic Taints
Höschele, Zeller
https://www.fuzzingbook.org/html/GrammarMiner.html

6) A review of reverse debugging (Time-travel Debugging)
Engblom

7) Compiler Validation via Equivalence Modulo Inputs (EMI)
Le, Afshari, Su
https://www.cs.ucdavis.edu/~su/publications/emi.pdf

8) MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE)
Sen, Necula, Gong, Choi
https://people.eecs.berkeley.edu/~ksen/papers/multise.pdf

9) Smart Greybox Fuzzing (AFLSmart)
Pham, Böhme, Santosa, Caciulescu, Roychoudhury
https://arxiv.org/pdf/1811.09447.pdf

10) FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)
Dutra, Gopinath, Zeller
https://dl.cispa.de/s/3q2PyqP7rqZzrNn

Thank you,

Rafael + Rahul + Zeller

18.06.2021

#### Abstract Submissions for Compiler Validation via Equivalence Modulo Inputs (EMI)

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Compiler Validation via Equivalence Modulo Inputs (EMI)".

*  Compiler Validation via Equivalence Modulo Inputs
Le, Afshari, Su

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Compiler Validation via Equivalence Modulo Inputs (EMI)".

*  Compiler Validation via Equivalence Modulo Inputs
Le, Afshari, Su
https://www.cs.ucdavis.edu/~su/publications/emi.pdf

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/7

Regards,

Rafael

08.06.2021

#### Abstract Submissions for Reverse Debugging (Time-travel Debugging)

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Reverse Debugging (Time-travel Debugging)". The following publication provides the basic background.

*  A review of reverse debugging (Time-travel Debugging)

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Reverse Debugging (Time-travel Debugging)". The following publication provides the basic background.

*  A review of reverse debugging (Time-travel Debugging)
Engblom

Students are also encouraged to play with actual implementations of reverse debugging, for example, in gdb:

https://www.gnu.org/software/gdb/news/reversible.html

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/6

Regards,

Rafael
06.06.2021

#### Abstract Submissions on Mining Program Input Grammars

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Mining Program Input Grammars". The following publication and fuzzing book chapter provide the basic background.

*  Mining Input Grammars from Dynamic Control Flow

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Mining Program Input Grammars". The following publication and fuzzing book chapter provide the basic background.

*  Mining Input Grammars from Dynamic Control Flow
Gopinath, Mathis, Zeller
https://publications.cispa.saarland/3101/1/fse2020-mimid.pdf

* Mining Input Grammars from Dynamic Taints
Höschele, Zeller
https://www.fuzzingbook.org/html/GrammarMiner.html

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for dynamic invaraint abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/5

Regards,

Rahul

27.05.2021

#### Abstract Submissions for Mining Dynamic Invariants

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Mining Dynamic Invariants". The following publication and fuzzing book chapter provide the basic background.

* Dynamically discovering likely program invariants to... Read more

Dear Students,

As we mentioned during the previous meeting, the next seminar will be on "Mining Dynamic Invariants". The following publication and fuzzing book chapter provide the basic background.

* Dynamically discovering likely program invariants to support program evolution
Ernst, Cockrell, Griswold, Notkin
https://homes.cs.washington.edu/~mernst/pubs/invariants-tse2001.pdf

* Mining Function Specifications
https://www.fuzzingbook.org/html/DynamicInvariants.html

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for dynamic invaraint abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/4

Regards,

Rahul

14.05.2021

#### Abstract Submissions for Language of Program Behaviors

Dear Students,

As we mentioned during the previous week, the next seminar will be on "Language of Program Behaviors". The following publications provide the basic background.

* Input Algebras
Gopinath, Nemati, Zeller

Dear Students,

As we mentioned during the previous week, the next seminar will be on "Language of Program Behaviors". The following publications provide the basic background.

* Input Algebras
Gopinath, Nemati, Zeller
https://publications.cispa.saarland/3208/7/gopinath2021input.pdf
Talk: Osama Altamar

*  Learning the Language of Error
Chapman, Chockler, Kesseli, Kroening, Strichman, Tautschnig http://www.cprover.org/learning-errors/learning-the-language-of-error-including-a-proof-supplement.pdf
Talk: Reza Zamiri

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for evocative patterns abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/3

Regards,

Rahul

04.05.2021

#### Abstract Submissions for Evocative Patterns for Behavioral Abstraction

Dear Students,

As we mentioned during the previous week, the next seminar will be on evocative or failure inducing patterns for behavioral abstraction. The following publications provide the basic background.

* Abstracting Failure-Inducing Inputs

Dear Students,

As we mentioned during the previous week, the next seminar will be on evocative or failure inducing patterns for behavioral abstraction. The following publications provide the basic background.

* Abstracting Failure-Inducing Inputs
Gopinath, Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3103/7/issta2020-language-of-failure.pdf

Talk: Benjamin Peters

(You are encouraged to read the Debugging book chapter on this subject: https://www.debuggingbook.org/beta/html/DDSetDebugger.html which details a simpler algorithm than that given in the paper, and understand the trade offs.)

* When does my Program do this? Learning Circumstances of Software Behavior

Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3107/7/fse2020-alhazen.pdf

Talk: Ali Alhasani

We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for evocative patterns abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/2

Regards,

Rahul

02.05.2021

#### Abstract Submissions for Delta Debugging

Dear Students,

As we mentioned during the previous week, the first seminar will be on Delta Debugging. The following publications provide the basic background.
* Simplifying and isolating failure-inducing input
Zeller, Hildebrandt

Dear Students,

As we mentioned during the previous week, the first seminar will be on Delta Debugging. The following publications provide the basic background.
* Simplifying and isolating failure-inducing input
Zeller, Hildebrandt
https://www.st.cs.uni-saarland.de/papers/tse2002/tse2002.pdf
* Reducing Failure-Inducing Inputs
https://www.fuzzingbook.org/html/Reducer.html

We expect a short abstract, no more than one page, capturing the essential ideas of the delta-debugging. You can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for delta-debugging abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/1

Regards,

Rahul

28.04.2021

#### Please register in LSF for the Seminar before May 10

Dear all,

The CS examination office tells us that all students are asked to register for the exams for the summer semester 2021. For (pro)seminars, registration or cancellation is possible up to three weeks after the topic assignment or kick-off meeting. In our... Read more

Dear all,

The CS examination office tells us that all students are asked to register for the exams for the summer semester 2021. For (pro)seminars, registration or cancellation is possible up to three weeks after the topic assignment or kick-off meeting. In our case, this was April 19, 2021, which means that you have to register in LSF by May 10 at the latest.

In case of problems wih the exam registration, students of the MI faculty should contact: studium@cs.uni-saarland.de. Other students have to contact their examination office.

If you are not a student of Computer Science or Cybersecurity, please read on:

These students are not able to register in HISPOS:

• VSIMint
• Master IT und Recht
• Bachelor-Plus MINT
• Erasmusstudents
• guest students

Currently the following students of the Computer Science department cannot register in HISPOS:

• Bachelor of Computer Science, StO 2020
• Bachelor Cyber Security, StO 2020
• Bachelor Media Informatics, StO 2020
• Master Media Informatics, StO 2020
• Bachelor Mathematics and Computer Science, StO 2020
• Teacher of computer science, StO 2020

If you fall into any of these categories, let us know, and we will forward your names and grades to the examination office after the seminar. We hope that the courses will be implemented in HISPOS by the start of the exam period and that exam registration will be possible then.

All the best,

Andreas Zeller

26.04.2021

#### Slides of "How to give a good research talk" now available...

... at https://dl.cispa.de/s/q9EfYZ5AqMrmSc7. Enjoy!

Andreas Zeller

22.04.2021

#### Details of presentations and grading

Dear Students,
As we discussed in our meeting on Monday, our seminar will continue to take place regularly on Monday 17:00 hours. The seminar web page is at: https://cms.cispa.saarland/testdebug/

Dear Students,
As we discussed in our meeting on Monday, our seminar will continue to take place regularly on Monday 17:00 hours. The seminar web page is at: https://cms.cispa.saarland/testdebug/

We will use the following zoom link: https://cispa-de.zoom.us/j/99579531702?pwd=eStVWTZXbkNDc2RnVHByZkNZcDUwZz09
(This is the same zoom link we used in our kickoff meeting)

We will start our seminar with a talk by Prof. Dr. Zeller on "How to Give a Talk". This talk will help you organize your presentations (especially the last one). So, don't miss it.

From the following week onward, we will have regular recurring sessions. We will have topics along with relevant publications assigned for reading each week. The format is for two students to volunteer for a short (5 minutes), ungraded introductory talk on the subject and publications, followed by discussions on the given publications possible limitations, published advancements, possible avenues of improvement, and possible applications other than what is envisioned in the paper. The other students will be required to read the publications, and prepare abstracts of the publications. These abstracts will be graded (pass/fail).

At the end of the course, each student will pick a topic, and provide a detailed (20 minutes) presentation. This presentation can include not only the publication, but also interesting advancements, avenues of improvement etc. The discussion on the paper by the class can be used for material in the final presentation. New related material can also be added. You will be graded on your final presentation.

Most papers are associated with artifacts/tools that you could run. Students are encouraged to play with those tools to form their own opinions on them (especially for the last presentation).

0) "How to give a talk": Prof. Dr. Andreas Zeller

1) Delta Debugging
* Simplifying and isolating failure-inducing input
Zeller, Hildebrandt
https://www.st.cs.uni-saarland.de/papers/tse2002/tse2002.pdf
* Reducing Failure-Inducing Inputs
https://www.fuzzingbook.org/html/Reducer.html

2) Evocative Patterns for Behavioral Abstraction
* Abstracting Failure-Inducing Inputs
Gopinath, Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3103/7/issta2020-language-of-failure.pdf

* When does my Program do this? Learning Circumstances of Software Behavior
Kampmann, Havrikov, Soremekun, Zeller
https://publications.cispa.saarland/3107/7/fse2020-alhazen.pdf

3) Language of Program Behaviors
* Input Algebras
Gopinath, Nemati, Zeller
https://publications.cispa.saarland/3208/7/gopinath2021input.pdf
*  Learning the Language of Error
Chapman, Chockler, Kesseli, Kroening, Strichman, Tautschnig http://www.cprover.org/learning-errors/learning-the-language-of-error-including-a-proof-supplement.pdf

4) Mining Dynamic Invariants
* Dynamically discovering likely program invariants to support program evolution
Ernst, Cockrell, Griswold, Notkin
https://homes.cs.washington.edu/~mernst/pubs/invariants-tse2001.pdf

* Mining Function Specifications
https://www.fuzzingbook.org/html/DynamicInvariants.html

5) Mining Program Input Grammars
*  Mining Input Grammars from Dynamic Control Flow
Gopinath, Mathis, Zeller
https://publications.cispa.saarland/3101/1/fse2020-mimid.pdf

* Mining Input Grammars from Dynamic Taints
Höschele, Zeller
https://www.fuzzingbook.org/html/GrammarMiner.html

6) A review of reverse debugging (Time-travel Debugging)
Engblom

7) Compiler Validation via Equivalence Modulo Inputs (EMI)
Le, Afshari, Su
https://www.cs.ucdavis.edu/~su/publications/emi.pdf

8) MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE)
Sen, Necula, Gong, Choi
https://people.eecs.berkeley.edu/~ksen/papers/multise.pdf

9) Smart Greybox Fuzzing (AFLSmart)
Pham, Böhme, Santosa, Caciulescu, Roychoudhury
https://arxiv.org/pdf/1811.09447.pdf

10) FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)
Dutra, Gopinath, Zeller
https://dl.cispa.de/s/3q2PyqP7rqZzrNn

We expect a short abstract, no more than one page, capturing the essential ideas of the topic. You can use the following latex snippet to determine what fits in a single page and what doesn't.


\documentclass[12pt]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\title{My Title}
\begin{document}
\section{Abstract}
\blindtext
\blindtext

\blindtext
\blindtext
\end{document}


Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot will open at least a day before the talk. If you face problems, please let us know.

The topics would be assigned to earliest volunteers of that topic. So, please send rahul.gopinath@cispa.de a mail regarding your choices as early as possible.

Regards,
Rahul + Rafael + Zeller

12.04.2021

#### Kickoff Meeting on Monday, April 19, 17:00

Welcome to the seminar on "Selected Topics in Automated Testing and Debugging”! We’re happy you’re with us, and we look forward to an exciting and inspiring seminar.

We have a kick-off meeting on Monday, April 19, 17:00in which we will discuss details of the... Read more

Welcome to the seminar on "Selected Topics in Automated Testing and Debugging”! We’re happy you’re with us, and we look forward to an exciting and inspiring seminar.

We have a kick-off meeting on Monday, April 19, 17:00in which we will discuss details of the seminar. This will also likely be the recurrent meeting time of the seminar, so try to keep this space free of other events. Details of the meeting are sent to you by mail.

Looking forward to see you, and best wishes,

Rahul + Rafael + Andreas

## Selected Topics in Automated Testing and Debugging

Description: Wouldn't it be cool if one could leave the task of testing and debugging code to a computer? In this seminar, we will discuss current results and new problems in the domain of _automated testing and debugging_ based on relevant scientific papers. The focus will be on techniques available in the Fuzzing Book (https://www.fuzzingbook.org) and the Debugging Book (https://beta.debuggingbook.org), as well as recent advances.

The general process will be as follows: Each week, you get 1-2 reading assignments and write an abstract about them. You may also be asked to give an (ungraded) five-minute mini presentation to kick off the discussion and improve your presentation skills. At the end of the seminar, you give a 15-20 minute presentation on one of the techniques, preferably including small experiments or demonstrations on how well they work; these will then be graded.

Requirements: Prior knowledge in automated testing and debugging techniques will be beneficial. For experiments and evaluations, programming knowledge will be helpful, too. The Fuzzing Book and Debugging Book infrastructures are implemented in Python, and we recommend doing experiments and evaluations in Jupyter Notebooks – so don't be afraid of these two.