Formal Methods for AI Safety Rayna Dimitrova


Currently, no news are available

The rapid progress in artificial intelligence and machine learning has lead to the deployment of AI-based systems in a number of areas of modern life, such as manufacturing, transportation, and healthcare. However, serious concerns about the safety and trustworthiness of such systems still remain, due to the lack of assurance regarding their behavior. To address this problem, significant efforts in the area of formal methods in recent years have been dedicated to the development of rigorous techniques for the design of safe AI-based systems. 

In this seminar, we will read and discuss research papers that present the latest results in this area. We will cover a range of topics, including the formal specification and verification of correctness properties of AI components of autonomous systems, and the design of reinforcement learning agents that respect safety constraints.

Each participant will give a presentation of an assigned paper, followed by a group discussion. All students are expected to read each paper carefully and to actively participate in the discussions. 



Time: TDB

Venue: TBD

Participation in all meetings is mandatory (exceptions require an official document, such as a doctor's certificate).



  • 3 short reviews plus questions to the speaker: For 3 of the papers that will be discussed during the seminar (different from the paper that you will present) you will write a short (maximum 1 page) review that addresses the following questions:
  1. What is the problem addressed by the paper?
  2. What was done before, and how does the paper improve on previous work?
  3. What are the strengths and the limitations of the techniques in the paper?
  4. What part of the paper was difficult to understand?
  5. What are possible improvements or extensions of the techniques in the paper?

Each review is to be submitted before the meeting at which the paper will be presented and discussed.                      

In addition to your review you will have to submit 2 questions that you will ask to the presenter of the paper.

Each of the three reviews contributes 10% of your final grade.

  • Presentation: You will prepare and deliver a 30 min presentation of the paper assigned to you. You will have the possibility to get feedback on your slides before the presentation.

The slides and presentation make up 40% of the final grade.

  • Summary: After your presentation you will write a summary of the paper that you have presented, including a general overview of the topic and reflecting the group discussion.

The summary will make up 30% of the final grade.






Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators