News
Final Presentation Tomorrow 10:00Written on 29.09.21 by Rafael Dutra Dear students, We're scheduling the remaining final presentation for tomorrow, 30 September at 10 am. https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09 Vinay Tilwani will present Smart Greybox Fuzzing (AFLSmart). Attendance is not mandatory, but you are encouraged… Read more Dear students, We're scheduling the remaining final presentation for tomorrow, 30 September at 10 am. https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09 Vinay Tilwani will present Smart Greybox Fuzzing (AFLSmart). Attendance is not mandatory, but you are encouraged to participate if you are available. The final grades for the other students should already be posted on LSF. Cheers, Rafael+Rahul+Andreas. |
Help us tailoring automated debugging to your needsWritten on 16.09.21 by Andreas Zeller Dear seminar participants, Thank you for your interest in software testing and debugging! We're developing Alhazen, a tool that automatically finds out when and why your program fails. We are heavily relying on your opinion as professional developers. Please take part in our user study to help us… Read more Dear seminar participants, Thank you for your interest in software testing and debugging! We're developing Alhazen, a tool that automatically finds out when and why your program fails. We are heavily relying on your opinion as professional developers. Please take part in our user study to help us tie our approach to your needs: https://tinyurl.com/debuggingstudy Thanks a lot! Andreas Zeller + Team |
Zoom link for final presentationsWritten on 07.09.21 by Rafael Dutra Hi, The old link doesn't seem to work. We're using this new link for the final presentations: https://cispa-de.zoom.us/j/92117483665?pwd=bHUyWWtvMkh2a0o4NHVLdTJYOWxEdz09 See you there! Rafael+Rahul+Andreas |
Final Presentations (7 and 8 September)Written on 30.08.21 (last change on 31.08.21) by Rafael Dutra Dear students, As announced earlier, the remaining final presentation will start next week. We'll have 2 presentation dates, with up to 4 presenters on each date.
Tuesday 7 September 9am-11am Florian Bauckholt: A review of reverse debugging (Time-travel Debugging) Vikram Vashishth:… Read more Dear students, As announced earlier, the remaining final presentation will start next week. We'll have 2 presentation dates, with up to 4 presenters on each date.
Tuesday 7 September 9am-11am Florian Bauckholt: A review of reverse debugging (Time-travel Debugging) Vikram Vashishth: Compiler Validation via Equivalence Modulo Inputs (EMI) Reza Zamiri: FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)
Wednesday 8 September 2pm-4pm Benjamin Peters: Delta Debugging Ali Alhasani: Evocative Patterns for Behavioral Abstraction Birk Blechschmidt: Mining Dynamic Invariants Ruiyi Zhang: MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE)
Please time your presentation to a maximum of 20 minutes (including possibly a demo), which will be followed by 10 minutes discussion. Thank you, Rafael+Rahul+Andreas |
Abstract Submissions for FormatFuzzerWritten on 09.07.21 by Rafael Dutra Dear Students, Hope to see you soon at 10am for the first round of final presentations. As we mentioned during the previous meeting, the next seminar will be on FormatFuzzer. * FormatFuzzer: Effective Fuzzing of Binary File Formats Dear Students, Hope to see you soon at 10am for the first round of final presentations. As we mentioned during the previous meeting, the next seminar will be on FormatFuzzer. * FormatFuzzer: Effective Fuzzing of Binary File Formats We expect a short abstract, no more than one page, capturing the essential ideas. Also think about new directions to use/extend the FormatFuzzer framework. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for FormatFuzzer is here: https://cms.cispa.saarland/testdebug/submissions/view/10 Please submit your abstracts before the class. Regards, Rafael |
Seminar evaluation – please fill out form by July 14Written on 07.07.21 by Andreas Zeller Dear all, as usual for all courses at Saarland University, you have the opportunity to evaluate the seminar and provide feedback – all anonymously, of course. Please do us a favor and fill out the form at https://qualis.uni-saarland.de/eva/?l=130529&p=57j3r6 Filling out the form takes… Read more Dear all, as usual for all courses at Saarland University, you have the opportunity to evaluate the seminar and provide feedback – all anonymously, of course. Please do us a favor and fill out the form at https://qualis.uni-saarland.de/eva/?l=130529&p=57j3r6 Filling out the form takes only five minutes. Please provide your feedback by July 14! Best wishes, Rahul + Rafael + Andreas |
Abstract Submissions for AFLSmartWritten on 02.07.21 by Rafael Dutra Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Smart Greybox Fuzzing (AFLSmart)". * Smart Greybox Fuzzing (AFLSmart) We expect a short abstract, no more… Read more Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Smart Greybox Fuzzing (AFLSmart)". * Smart Greybox Fuzzing (AFLSmart) We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for AFLSmart is here: https://cms.cispa.saarland/testdebug/submissions/view/9 Please submit your abstracts before the class. Regards, Rafael |
Final Presentations (9 July, 7 September, 8 September)Written on 30.06.21 (last change on 19.08.21) by Rafael Dutra Dear students, We have assigned the final topics as follows:
Benjamin Peters
Ali Alhasani
Osama Altamar
Birk Blechschmidt
Tim… Read more Dear students, We have assigned the final topics as follows:
Benjamin Peters
Ali Alhasani
Osama Altamar
Birk Blechschmidt
Tim Speicher
Florian Bauckholt
Vikram Vashishth
Ruiyi Zhang
Vinay Tilwani
Reza Zamiri
The first presentation day will be Friday 9 July in the morning at 10:00. Tim and Osama are scheduled to present on 9 July.
There will be two additional presentation days in September: Tuesday 7 September in the morning at 9:00 and Wednesday 8 September in the afternoon at 14:00. Birk, you can choose to present either in July or in September, as you prefer.
Cheers, Rafael + Rahul + Zeller |
Choosing Dates for Final PresentationWritten on 28.06.21 by Rafael Dutra Dear students, Please select the dates you are available to present on this link. You can mark options with "yes" (preferred) or "yes, if need be" (not ideal, but still available). Please complete the poll by tomorrow (29… Read more Dear students, Please select the dates you are available to present on this link. You can mark options with "yes" (preferred) or "yes, if need be" (not ideal, but still available). Please complete the poll by tomorrow (29 June). https://doodle.com/poll/gvwmkxe4xnrc4pu4?utm_source=poll&utm_medium=link
We may have more than one presentation day (for example, if some students want to present in July and some in September). Of course, any students presenting in July will not have as a final presentation topic one of the two last papers (AFLSmart and FormatFuzzer).
Thank you, Rafael + Rahul + Zeller |
Abstract Submissions for MultiSEWritten on 25.06.21 by Rafael Dutra Dear Students, As we mentioned during the previous meeting, the next seminar will be on "MultiSE: Multi-Path Symbolic Execution using Value Summaries". * MultiSE: Multi-Path Symbolic Execution using Value Summaries Dear Students, As we mentioned during the previous meeting, the next seminar will be on "MultiSE: Multi-Path Symbolic Execution using Value Summaries". * MultiSE: Multi-Path Symbolic Execution using Value Summaries This is an alternative way of implementing symbolic execution. In particular, you should discuss how it compares with the traditional symbolic and concolic execution techniques presented, for example, in the fuzzinbook. Symbolic Execution: https://www.fuzzingbook.org/html/SymbolicFuzzer.html Concolic Execution: https://www.fuzzingbook.org/html/ConcolicFuzzer.html We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/8 Please submit your abstracts before the class. Regards, Rafael |
Choosing Topic for Final PresentationWritten on 18.06.21 by Rafael Dutra Dear Students, In order to assign the topics for final presentations, please send us (rafael.dutra@cispa.de, rahul.gopinath@cispa.de) a ranking of your 3 preferred topics (with 1 being the most preferred). 1) 2) 3) The possible topics are 1) Delta Debugging Dear Students, In order to assign the topics for final presentations, please send us (rafael.dutra@cispa.de, rahul.gopinath@cispa.de) a ranking of your 3 preferred topics (with 1 being the most preferred). 1) 2) 3) The possible topics are 1) Delta Debugging 3) Language of Program Behaviors 4) Mining Dynamic Invariants 5) Mining Program Input Grammars 7) Compiler Validation via Equivalence Modulo Inputs (EMI) 8) MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE) 9) Smart Greybox Fuzzing (AFLSmart) 10) FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer)
Thank you, Rafael + Rahul + Zeller |
Abstract Submissions for Compiler Validation via Equivalence Modulo Inputs (EMI)Written on 18.06.21 by Rafael Dutra Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Compiler Validation via Equivalence Modulo Inputs (EMI)". * Compiler Validation via Equivalence Modulo Inputs We expect a short… Read more Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Compiler Validation via Equivalence Modulo Inputs (EMI)". * Compiler Validation via Equivalence Modulo Inputs We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/7 Please submit your abstracts before the class. Regards, Rafael |
Abstract Submissions for Reverse Debugging (Time-travel Debugging)Written on 08.06.21 by Rafael Dutra Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Reverse Debugging (Time-travel Debugging)". The following publication provides the basic background. * A review of reverse debugging (Time-travel Debugging) Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Reverse Debugging (Time-travel Debugging)". The following publication provides the basic background. * A review of reverse debugging (Time-travel Debugging) Students are also encouraged to play with actual implementations of reverse debugging, for example, in gdb: https://www.gnu.org/software/gdb/news/reversible.html We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for reverse debugging is here: https://cms.cispa.saarland/testdebug/submissions/view/6 Please submit your abstracts before the class. Regards, Rafael |
Abstract Submissions on Mining Program Input GrammarsWritten on 06.06.21 by Rahul Gopinath Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Mining Program Input Grammars". The following publication and fuzzing book chapter provide the basic background. * Mining Input Grammars from Dynamic Control Flow Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Mining Program Input Grammars". The following publication and fuzzing book chapter provide the basic background. * Mining Input Grammars from Dynamic Control Flow We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for dynamic invaraint abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/5 Please submit your abstracts before the class. Regards, Rahul |
Abstract Submissions for Mining Dynamic InvariantsWritten on 27.05.21 by Rahul Gopinath Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Mining Dynamic Invariants". The following publication and fuzzing book chapter provide the basic background. * Dynamically discovering likely program invariants to support program evolution Dear Students, As we mentioned during the previous meeting, the next seminar will be on "Mining Dynamic Invariants". The following publication and fuzzing book chapter provide the basic background. * Dynamically discovering likely program invariants to support program evolution We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for dynamic invaraint abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/4 Please submit your abstracts before the class. Regards, Rahul |
Abstract Submissions for Language of Program BehaviorsWritten on 14.05.21 by Rahul Gopinath Dear Students, As we mentioned during the previous week, the next seminar will be on "Language of Program Behaviors". The following publications provide the basic background. * Input Algebras Dear Students, As we mentioned during the previous week, the next seminar will be on "Language of Program Behaviors". The following publications provide the basic background. * Input Algebras We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for evocative patterns abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/3 Please submit your abstracts before the class. Regards, Rahul |
Abstract Submissions for Evocative Patterns for Behavioral AbstractionWritten on 04.05.21 (last change on 04.05.21) by Rahul Gopinath Dear Students, As we mentioned during the previous week, the next seminar will be on evocative or failure inducing patterns for behavioral abstraction. The following publications provide the basic background.
Dear Students, As we mentioned during the previous week, the next seminar will be on evocative or failure inducing patterns for behavioral abstraction. The following publications provide the basic background.
Talk: Benjamin Peters (You are encouraged to read the Debugging book chapter on this subject: https://www.debuggingbook.org/beta/html/DDSetDebugger.html which details a simpler algorithm than that given in the paper, and understand the trade offs.)
Talk: Ali Alhasani We expect a short abstract, no more than one page, capturing the essential ideas. As before you can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for evocative patterns abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/2 Please submit your abstracts before the class. Regards, Rahul |
Abstract Submissions for Delta DebuggingWritten on 02.05.21 by Rahul Gopinath Dear Students, As we mentioned during the previous week, the first seminar will be on Delta Debugging. The following publications provide the basic background. Dear Students, As we mentioned during the previous week, the first seminar will be on Delta Debugging. The following publications provide the basic background. We expect a short abstract, no more than one page, capturing the essential ideas of the delta-debugging. You can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot for delta-debugging abstracts is here: https://cms.cispa.saarland/testdebug/submissions/view/1 Please submit your abstracts before the class. Regards, Rahul |
Please register in LSF for the Seminar before May 10Written on 28.04.21 by Andreas Zeller Dear all, The CS examination office tells us that all students are asked to register for the exams for the summer semester 2021. For (pro)seminars, registration or cancellation is possible up to three weeks after the topic assignment or kick-off meeting. In our case, this was April 19, 2021, which… Read more Dear all, The CS examination office tells us that all students are asked to register for the exams for the summer semester 2021. For (pro)seminars, registration or cancellation is possible up to three weeks after the topic assignment or kick-off meeting. In our case, this was April 19, 2021, which means that you have to register in LSF by May 10 at the latest.
In case of problems wih the exam registration, students of the MI faculty should contact: studium@cs.uni-saarland.de. Other students have to contact their examination office. If you are not a student of Computer Science or Cybersecurity, please read on: These students are not able to register in HISPOS:
Currently the following students of the Computer Science department cannot register in HISPOS:
If you fall into any of these categories, let us know, and we will forward your names and grades to the examination office after the seminar. We hope that the courses will be implemented in HISPOS by the start of the exam period and that exam registration will be possible then. All the best, Andreas Zeller |
Slides of "How to give a good research talk" now available...Written on 26.04.21 by Andreas Zeller ... at https://dl.cispa.de/s/q9EfYZ5AqMrmSc7. Enjoy! Andreas Zeller |
Details of presentations and gradingWritten on 22.04.21 (last change on 08.06.21) by Rahul Gopinath Dear Students, Dear Students, We will start our seminar with a talk by Prof. Dr. Zeller on "How to Give a Talk". This talk will help you organize your presentations (especially the last one). So, don't miss it. From the following week onward, we will have regular recurring sessions. We will have topics along with relevant publications assigned for reading each week. The format is for two students to volunteer for a short (5 minutes), ungraded introductory talk on the subject and publications, followed by discussions on the given publications possible limitations, published advancements, possible avenues of improvement, and possible applications other than what is envisioned in the paper. The other students will be required to read the publications, and prepare abstracts of the publications. These abstracts will be graded (pass/fail). At the end of the course, each student will pick a topic, and provide a detailed (20 minutes) presentation. This presentation can include not only the publication, but also interesting advancements, avenues of improvement etc. The discussion on the paper by the class can be used for material in the final presentation. New related material can also be added. You will be graded on your final presentation. Most papers are associated with artifacts/tools that you could run. Students are encouraged to play with those tools to form their own opinions on them (especially for the last presentation). 0) "How to give a talk": Prof. Dr. Andreas Zeller 1) Delta Debugging 3) Language of Program Behaviors 4) Mining Dynamic Invariants 5) Mining Program Input Grammars 7) Compiler Validation via Equivalence Modulo Inputs (EMI) 8) MultiSE: Multi-Path Symbolic Execution using Value Summaries (MultiSE) 9) Smart Greybox Fuzzing (AFLSmart) 10) FormatFuzzer: Effective Fuzzing of Binary File Formats (FormatFuzzer) We expect a short abstract, no more than one page, capturing the essential ideas of the topic. You can use the following latex snippet to determine what fits in a single page and what doesn't. ``` \title{My Title} \blindtext Please upload your summary as a PDF to the CMS submission slot (you can use the above latex code to generate the PDF). The submission slot will open at least a day before the talk. If you face problems, please let us know. The topics would be assigned to earliest volunteers of that topic. So, please send rahul.gopinath@cispa.de a mail regarding your choices as early as possible. Regards, |
Kickoff Meeting on Monday, April 19, 17:00Written on 12.04.21 by Andreas Zeller Welcome to the seminar on "Selected Topics in Automated Testing and Debugging”! We’re happy you’re with us, and we look forward to an exciting and inspiring seminar. We have a kick-off meeting on Monday, April 19, 17:00, in which we will discuss details of the seminar. This will also likely be the… Read more Welcome to the seminar on "Selected Topics in Automated Testing and Debugging”! We’re happy you’re with us, and we look forward to an exciting and inspiring seminar. We have a kick-off meeting on Monday, April 19, 17:00, in which we will discuss details of the seminar. This will also likely be the recurrent meeting time of the seminar, so try to keep this space free of other events. Details of the meeting are sent to you by mail. Looking forward to see you, and best wishes, Rahul + Rafael + Andreas |
Selected Topics in Automated Testing and Debugging
Description: Wouldn't it be cool if one could leave the task of testing and debugging code to a computer? In this seminar, we will discuss current results and new problems in the domain of _automated testing and debugging_ based on relevant scientific papers. The focus will be on techniques available in the Fuzzing Book (https://www.fuzzingbook.org) and the Debugging Book (https://beta.debuggingbook.org), as well as recent advances.
The general process will be as follows: Each week, you get 1-2 reading assignments and write an abstract about them. You may also be asked to give an (ungraded) five-minute mini presentation to kick off the discussion and improve your presentation skills. At the end of the seminar, you give a 15-20 minute presentation on one of the techniques, preferably including small experiments or demonstrations on how well they work; these will then be graded.
Requirements: Prior knowledge in automated testing and debugging techniques will be beneficial. For experiments and evaluations, programming knowledge will be helpful, too. The Fuzzing Book and Debugging Book infrastructures are implemented in Python, and we recommend doing experiments and evaluations in Jupyter Notebooks – so don't be afraid of these two.