Formal Methods for AI Safety Rayna Dimitrova

News

09.11.2020

Paper assignment published

The paper assignment has been published on the main page.

06.11.2020

Poll for weekly meeting time

Dear Students,

Please complete this poll by Monday, 9. 11, 11:00 to indicate your availability for the time of the seminar's weekly meeting.

Have a nice weekend!

Rayna

05.11.2020

List of papers and paper preferences

Dear Students,

thank you all for attending the kick-off meeting today. I have added the list of papers, and you can find the slides under Information->Materials.

Please send me by email your 5 preferred papers (1: Highest preference, 5: lowest preference) by... Read more

Dear Students,

thank you all for attending the kick-off meeting today. I have added the list of papers, and you can find the slides under Information->Materials.

Please send me by email your 5 preferred papers (1: Highest preference, 5: lowest preference) by Sunday, 8th November, 23:59.

If you have any questions, please do not hesitate to get in touch.

Best wishes,

Rayna

 

 

03.11.2020

Time of Kick-off meeting changed

Dear Students,

Thank you all very much for completing the poll. The results have spoken: the only time that works for everyone is Thursday at 8:15, so the kick-off meeting will take place 8:15 to 9:45 on Thursday, 5.11.

The details for the Zoom meeting have... Read more

Dear Students,

Thank you all very much for completing the poll. The results have spoken: the only time that works for everyone is Thursday at 8:15, so the kick-off meeting will take place 8:15 to 9:45 on Thursday, 5.11.

The details for the Zoom meeting have been sent by email.

Best wishes,

Rayna

02.11.2020

Kick-off meeting time this week

Hello everyone,

As some of you already indicated a conflict with the planned time of the kick-off meeting this week, I would like to try and find a time this week that works for everyone, if possible.

Therefore, please complete this poll by Tuesday, 3.11,... Read more

Hello everyone,

As some of you already indicated a conflict with the planned time of the kick-off meeting this week, I would like to try and find a time this week that works for everyone, if possible.

Therefore, please complete this poll by Tuesday, 3.11, 16:00 to indicate your availability. 

Note that this is to choose a time slot only for the kick-off meeting this week. For the subsequent weekly meetings I will set up another poll, as I appreciate your constraints might change as you decide which lectures to attend.

Thank you very much!

Best wishes,

Rayna

 

 

 

The rapid progress in artificial intelligence and machine learning has lead to the deployment of AI-based systems in a number of areas of modern life, such as manufacturing, transportation, and healthcare. However, serious concerns about the safety and trustworthiness of such systems still remain, due to the lack of assurance regarding their behavior. To address this problem, significant efforts in the area of formal methods in recent years have been dedicated to the development of rigorous techniques for the design of safe AI-based systems. 

In this seminar, we will read and discuss research papers that present the latest results in this area. We will cover a range of topics, including the formal specification and verification of correctness properties of AI components of autonomous systems, and the design of reinforcement learning agents that respect safety constraints.

Each participant will give a presentation of an assigned paper, followed by a group discussion. All students are expected to read each paper carefully and to actively participate in the discussions. 

 

Logistics

Time: Thursday, 8:15-9:45

Venue: All meetings will be virtual, via Zoom (Disclaimer).

               The link to the recurring meeting has been communicated by email.

Participation in all meetings is mandatory (exceptions require an official document, such as a doctor's certificate).

 

Important dates

5 November, 8:15 Kick-off meeting (participation is strictly required)

November, 23:59 Deadline for paper preference registration

9 November, 18:00 Release of paper assignment

5 March 2021, 18:00 Deadline for submission of seminar report

 

Paper Assignment and Schedule

Date Presenter Paper Short Reviews
19.11 Saif Ali Khan Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

Sukrut Rao,

Maurice Vincon

26.11

postponed,

no meeting

Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness

Muhammad Hassan Rashid,

Maurice Vincon

03.12 Sukrut Rao Verification of Deep Convolutional Neural Networks Using ImageStars
 

Md Jonybul Islam,  

Muhammad Hassan Rashid,

Rati Devidze

10.12 cancelled Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Saif Ali Khan,

Markus Bever,

Rati Devidze

17.12 Maksim Moor Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Components

Kai Greshake,

Md Jonybul Islam

07.01 Markus Bever Verisig: verifying safety properties of hybrid systems with neural network controllers

Sukrut Rao,  

Saif Ali Khan,

Muhammad Hassan Rashid

14.01 Maurice Vincon Cautious Reinforcement Learning with Logical Constraints

Rati Devidze,

Maksim Moor,

Kai Greshake

21.01 Rati Devidze A Composable Specification Language for Reinforcement Learning Tasks

Md Jonybul Islam,

Maurice Vincon

28.01 Muhammad Hassan Rashid Abduction-Based Explanations for Machine Learning Models

Kai Greshake,

Sukrut Rao,

Ashish Khullar

28.01 Md Jonybul Islam Synthesizing Action Sequences for Modifying Model Decisions

Saif Ali Khan,

Ashish Khullar,

Maksim Moor,

Markus Bever

04.02 Kai Greshake FairSquare: Probabilistic Verification of Program Fairness

Markus Bever,  

Maksim Moor,

Ashish Khullar

Deliverables

  • 3 short reviews: For 3 of the papers that will be discussed during the seminar (different from the paper that you will present) you will write a short (maximum 1 page) review that addresses the following questions:
  1. What is the problem addressed by the paper?
  2. What was done before, and how does the paper improve on previous work?
  3. What are the strengths and the limitations of the techniques in the paper?
  4. What part of the paper was difficult to understand?
  5. What are possible improvements or extensions of the techniques in the paper?

Each review is to be submitted before the meeting at which the paper will be presented and discussed.

Each of the three reviews contributes 10% of your final grade.

  • Presentation: You will prepare and deliver a 45 min presentation of the paper assigned to you. You will have the possibility to get feedback on your slides before the presentation.

The slides and presentation make up 40% of the final grade.

  • Summary: After your presentation you will write a summary of the paper that you have presented, including a general overview of the topic and reflecting the group discussion.

The summary will make up 30% of the final grade.

Papers

Verification of Neural Networks

Cyber-Physical Systems with ML Components

Safe Reinforcement Learning

Interpretability

Verification of Fairness Properties

 

 

 

 

 



Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators