News
Final GradesWritten on 03.04.23 by Marius Smytzek Dear students, We have published the final grades. Please find them on your Personal Status page. Statistics are available at https://cms.cispa.saarland/debug_2223/gradescales/view/1. Congratulations on your achievement! |
Project 02 ResultsWritten on 23.03.23 by Paul Zhu Dear Students, You will find your score for Project 02 on your Personal Status page. We included feedback (download the .txt file) which reports the passed and failed tests (hidden tests have name `test_hidden_*`; unlike exercises, the hidden tests of projects will not be published). The grading… Read more Dear Students, You will find your score for Project 02 on your Personal Status page. We included feedback (download the .txt file) which reports the passed and failed tests (hidden tests have name `test_hidden_*`; unlike exercises, the hidden tests of projects will not be published). The grading regulations were mentioned in the sheet (check it again when in doubt). If you have any problems regarding your score, please contact us soon. |
Project 01 ResultsWritten on 10.03.23 by Marius Smytzek Dear Students, You can find your points for Project 01 on your Personal Status page. We included feedback (download the .txt file) which reports the passed and failed tests. The points for Task 1 were given as follows: - If you pass all public tests, you receive 50 points and pass the… Read more Dear Students, You can find your points for Project 01 on your Personal Status page. We included feedback (download the .txt file) which reports the passed and failed tests. The points for Task 1 were given as follows: - If you pass all public tests, you receive 50 points and pass the project. - The 21 secret tests linearly map to the other 10 points, e.g., if you pass 15 tests, you receive seven points; if you pass eight tests, you receive 3.5 points.
The points for Task 2 were given based on how well you identified the faulty location for a subject (the higher the index in the rank, the better) and how unique you identified the location based on how many locations have the same suspiciousness and how many locations are ranked higher. For the index, we also considered the suspiciousness (if available); if two locations have the same suspicion, they have the same index. We calculated a score for 11 subjects (identification x uniqueness) and mapped the average for all subjects linearly to 40 points.
Contact us if there is something wrong with the reported errors so we will double-check them. |
Exercise 09 SolutionWritten on 21.02.23 by Paul Zhu Dear Students, We have uploaded a sample solution for Exercise 09, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the… Read more Dear Students, We have uploaded a sample solution for Exercise 09, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the .txt file) which reports the errors. When in doubt, run your implementation on the test case included in the solution sheet (see the comments for the names of the test cases) and compare your outputs with the expected outputs produced by us. Contact us if there is something wrong in the reported errors so we will double check them. |
Exercise 08 SolutionWritten on 16.02.23 by Paul Zhu Dear Students, We have uploaded a sample solution for Exercise 08, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the… Read more Dear Students, We have uploaded a sample solution for Exercise 08, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the .txt file) which reports the errors. When in doubt, run your implementation on the test case included in the solution sheet (see the comments for the names of the test cases) and compare your outputs with the expected outputs produced by us. Contact us if there is something wrong in the reported errors so we will double check them. |
Exercise 10Written on 14.02.23 by Marius Smytzek Dear Students, We have published Exercise 10. You can find it under Information > Material. Please read the chapters Tracking Bugs and Where the Bugs are for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to… Read more Dear Students, We have published Exercise 10. You can find it under Information > Material. Please read the chapters Tracking Bugs and Where the Bugs are for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 26. Feb 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 07 SolutionWritten on 13.02.23 by Paul Zhu Dear Students, We have uploaded a sample solution for Exercise 07, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the… Read more Dear Students, We have uploaded a sample solution for Exercise 07, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the .txt file) which reports the errors. When in doubt, run your implementation on the test case included in the solution sheet (see the comments for the names of the test cases) and compare your outputs with the expected outputs produced by us. Contact us if there is something wrong in the reported errors so we will double check them. |
Exercise 06 SolutionWritten on 13.02.23 (last change on 13.02.23) by Paul Zhu Dear Students, We have uploaded a sample solution for Exercise 06, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the… Read more Dear Students, We have uploaded a sample solution for Exercise 06, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. If you didn't receive 10 points (full mark), you will find a feedback (download the .txt file) which reports the errors. When in doubt, run your implementation on the test case included in the solution sheet (see the comments for the names of the test cases) and compare your outputs with the expected outputs produced by us. Contact us if there is something wrong in the reported errors so we will double check them. |
Project 02Written on 08.02.23 by Paul Zhu Dear Students, We have published Project 02. You can find it under Information > Material. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You… Read more Dear Students, We have published Project 02. You can find it under Information > Material. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 21. March 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 09Written on 01.02.23 by Paul Zhu Dear Students, We have published Exercise 09. You can find it under Information > Material. Please read the chapter Repairing Code Automatically for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you… Read more Dear Students, We have published Exercise 09. You can find it under Information > Material. Please read the chapter Repairing Code Automatically for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 17. Feb 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 02 SolutionWritten on 24.01.23 by Paul Zhu Dear Students, We have uploaded a sample solution for Exercise 02, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page.
The solution package includes: 1. A reference implementation (idb.py) 2.… Read more Dear Students, We have uploaded a sample solution for Exercise 02, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page.
The solution package includes: 1. A reference implementation (idb.py) 2. The test function (evaluate_02_func.py) 3. The testcases and some explanations on how we grade your code (testcases.py)
If you didn't receive 10 points (full mark), you will find a feedback (download the .txt file) which reports the errors. When in doubt, test your implementation against the test commands (provided in the solution package) and compare that with the expected outputs produced by the reference implementation. Contact us if there is something wrong in the reported errors so we will double check them. |
Exercise 08Written on 23.01.23 (last change on 23.01.23) by Paul Zhu Dear Students, We have published Exercise 08. You can find it under Information > Material. Please read the chapter Debugging Performance Issues for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you… Read more Dear Students, We have published Exercise 08. You can find it under Information > Material. Please read the chapter Debugging Performance Issues for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 03. Feb 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Course Evaluation "Automated Debugging"Written on 16.01.23 by Andreas Zeller Dear all, At CISPA and Saarland University, we constantly aim to improve our courses – including "Automated Debugging". Now it's time for you to provide feedback! Please use the following links to provide your course assessment: Dear all, At CISPA and Saarland University, we constantly aim to improve our courses – including "Automated Debugging". Now it's time for you to provide feedback! Please use the following links to provide your course assessment: The evaluation is only open for a few days, so do not delay. Looking forward to hearing from you, Andreas + Marius + Paul |
Exercise 07Written on 16.01.23 by Paul Zhu Dear Students, We have published Exercise 07. You can find it under Information > Material. Please review and read the chapters Reducing Failure-Inducing Inputs and Generalizing Failure Circumstances for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf… Read more Dear Students, We have published Exercise 07. You can find it under Information > Material. Please review and read the chapters Reducing Failure-Inducing Inputs and Generalizing Failure Circumstances for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 27. January 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Project 01 UpdateWritten on 10.01.23 by Marius Smytzek Dear Students, We have updated the tests for the first task of Project 01. Please download the new revision, update the test file, or visit our Mattermost channel to get the new test file. |
Exercise 06Written on 09.01.23 by Marius Smytzek Dear Students, We have published Exercise 06. You can find it under Information > Material. Please read the chapter on Mining Function Specifications for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to… Read more Dear Students, We have published Exercise 06. You can find it under Information > Material. Please read the chapter on Mining Function Specifications for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 18. January 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 05 SolutionWritten on 09.01.23 by Marius Smytzek Dear Students, We have uploaded a sample solution for Exercise 05, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. |
Exercise 04 SolutionWritten on 05.01.23 by Marius Smytzek Dear Students, We have uploaded a sample solution for Exercise 04, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. |
Exercise 03 SolutionWritten on 05.01.23 by Marius Smytzek Dear Students, We have uploaded a sample solution for Exercise 03, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. |
Lecture on January 3Written on 02.01.23 by Andreas Zeller Happy new year, everyone! This is to remind you that on Tuesday, January 3 (= tomorrow)
The latter might be particularly useful, as most other courses are online-only this… Read more Happy new year, everyone! This is to remind you that on Tuesday, January 3 (= tomorrow)
The latter might be particularly useful, as most other courses are online-only this week, and you may prefer attending from home. To participate via Zoom, go to the course Web site https://cms.cispa.saarland/debug_2223/ and choose Information -> Zoom from the top-level menu. Looking forward to seeing you (in real life or virtual), Andreas Zeller and the Automated Debugging Team |
Project 01 UpdateWritten on 18.12.22 by Marius Smytzek Dear Students, There was a problem with the public tests of Project 01 that did not allow semantically correct solutions that added empty lines during the instrumentation. We have resolved this issue in a new revision you should use from now on. |
Project 01Written on 16.12.22 by Marius Smytzek Dear Students, We have published Project 01. You can find it under Information > Material. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status… Read more Dear Students, We have published Project 01. You can find it under Information > Material. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 29. January 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Upcoming LecturesWritten on 16.12.22 by Marius Smytzek Dear Students Here is some information about the upcoming lectures:
The Automated Debugging Team – Andreas, Marius, and Paul |
Exercise 05Written on 13.12.22 (last change on 09.01.23) by Marius Smytzek Dear Students, We have published Exercise 05. You can find it under Information > Material. Please read the chapter on Statistical Debugging for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask… Read more Dear Students, We have published Exercise 05. You can find it under Information > Material. Please read the chapter on Statistical Debugging for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 04. January 2023 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 04Written on 08.12.22 by Marius Smytzek Dear Students, We have published Exercise 04. You can find it under Information > Material. Please read the chapter on Tracking Failure Origins for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask… Read more Dear Students, We have published Exercise 04. You can find it under Information > Material. Please read the chapter on Tracking Failure Origins for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 21. December 2022 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Today's lecture; upcoming exercise sheetsWritten on 06.12.22 by Andreas Zeller Hello everyone, Professor Zeller has fallen sick and will be absent for the week.
Hello everyone, Professor Zeller has fallen sick and will be absent for the week.
We sincerely apologize for the inconvenience. Stay healthy, stay safe, and see you soon! The Automated Debugging Team – Andreas, Marius, and Paul
|
Exercise 03Written on 28.11.22 (last change on 08.12.22) by Marius Smytzek Dear Students, We have published Exercise 03. You can find it under Information > Material. Please read the chapter on Asserting Expectations for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you… Read more Dear Students, We have published Exercise 03. You can find it under Information > Material. Please read the chapter on Asserting Expectations for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 07. December 2022 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
Exercise 01 SolutionWritten on 28.11.22 by Marius Smytzek Dear Students, We have uploaded a sample solution for Exercise 01, you can find it in the category Solutions under Information > Material. You can find your points for this exercise on your Personal Status page. |
Exercise 02Written on 17.11.22 by Paul Zhu Dear Students, We have published Exercise 02. You can find it under Information > Material. Please read the chapter on How Debuggers Work for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to… Read more Dear Students, We have published Exercise 02. You can find it under Information > Material. Please read the chapter on How Debuggers Work for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 30. November 2022 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
LSF RegistrationWritten on 09.11.22 (last change on 09.11.22) by Marius Smytzek The LSF registration is now open until the 30th of November. Please make sure you register within time. If you encounter problems during the registration, don't hesitate to contact Marius Smytzek or Paul Zhu. Best, |
Exercise 01Written on 08.11.22 by Marius Smytzek Dear Students, We have published Exercise 01. You can find it under Information > Material. Please read the chapter on Tracing Executions for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask… Read more Dear Students, We have published Exercise 01. You can find it under Information > Material. Please read the chapter on Tracing Executions for this exercise. The Zip file contains the required files for this exercise. The sheet.pdf includes the tasks you should try to solve. We ask you to submit your solutions via the CMS on your Personal Status page. You have time until 20. November 2022 23:59 to upload your solutions as a Zip file. Note that we cannot evaluate delayed submissions. |
MattermostWritten on 30.10.22 by Marius Smytzek Dear Students, We have set up a Mattermost channel to discuss and ask questions with rapid responses. Please join this channel under Information > Mattermost. Best, |
Exercise 00Written on 30.10.22 by Marius Smytzek Dear Students, We have published Exercise 00. You can find it under Information > Materials. This exercise introduces the stack we use during the course. The Zip file contains the required files for this exercise. The sheet.pdf contains the tasks you should try to solve. |
Automated Debugging
The Course. Ask yourself: How many hours have you spent chasing bugs? So, wouldn't it be cool if the computer could take care of debugging your program? In this course, we discuss automated debugging and testing techniques, such as
- The Debugging Process
- Observing Executions
- Asserting Expectations
- Correlating Failures
- Simplifying Failures
- Abstracting Failures
- Tracking Origins
- Reproducing Failures
- Repairing Failures (automatically!)
- Learning From Mistakes
Course Material. The course material comes as a collection of Jupyter Notebooks, in which you can study how the individual techniques work – and even do your own experiments and create new combinations. Every week, you will be getting 1–2 new chapters (notebooks) on a new topic, which we will then discuss the next week in the classroom. All chapters are available at
https://www.debuggingbook.org/
The menu in the top left shows the individual chapters (as they are being written); the "Resources" menu allows you to work with the examples or download code or notebooks. Under "Help", you will find tutorials for Python and Jupyter.
Attending. The lectures for this course take place as a hybrid lecture and Zoom webinar, in which our lecturer (Andreas Zeller) will introduce you to the chapters to be read in the upcoming week and answer questions live and as posed in the chat. The hybrid lecture runs every Tuesday from 16:15–17:45 in person and on Zoom. Note that the lectures are not recorded, so attendance is useful. To study offline, the book offers tutorial lectures for each chapter.
Weekly exercises. Every week, you will get an exercise sheet with exercises relating to the current chapter. Your solutions are due after one week and will be graded.
Projects. During the course, you will run two projects in which you will build your own automated debugging tools. Past project topics included:
- An Interactive Debugger for Python
- Automatically Simplifying Python Programs (like CReduce)
- Automatically Repairing Python Programs
- A Tool of your Design
You will implement projects in Python and use Jupyter Notebooks to document design choices and introduce your readers to the included code. We plan to allow the last project to be conducted in groups of two as long as individual contributions are clearly marked.
Grading. Grading will be based on
- points achieved in weekly exercises (33%)
- points achieved in Project one (33%)
- points achieved in Project two (33%)
In order to pass, you need to achieve 50% of points in each category as well as overall.
The Prerequisites. We expect programming skills at the level of "Programming 2". Python knowledge and knowledge of program analysis and instrumentation can be acquired on the go. We use some statistics, logic, and machine learning, but nothing too exotic.
Questions and Answers. We will set up a Mattermost channel for questions and answers. Or ask questions during the lecture and get immediate answers.
Date and Time. Every Tuesday, 16:15–17:45 in CISPA C0 (Stuhlsatzenhausweg), Lecture Hall 0.05, or via Zoom (see "Information → Zoom" in the menu above for the link). In case the seats in 0.05 run out, we have an overflow space with live video transmission in seminar room 0.07 nearby.
The course starts on Tuesday, October 25. There will be no lectures on November 1, December 20, and December 27; the last lecture takes place on February 7.
Enjoy! – Andreas + Marius + Paul