Topics in Adversarial Machine Learning

(Pro-)Seminar: Topics in Adversarial Machine Learning

Course Objective: In this hybrid seminar, we will focus on understanding the security threats adversaries pose to machine learning systems and the recent algorithmic advancements in building more robust machine learning systems to mitigate those threats. We will particularly look into several theoretical works on understanding and characterizing the fundamental limits of adversarial machine learning. Registration for the seminar is not possible directly. Please use the CS department assignment system to indicate your interest and to register your bid.

Expected Background: Previous background in statistics and machine learning would be beneficial but optional as long as you are motivated and able to learn relevant fundamentals. To self-assess whether this is the right course, you can read the following papers to check how much you can understand these papers and whether you are interested in the topics or not:

• Intriguing Properties of Neural Networks
• Explaining and Harnessing Adversarial Examples
• Poisoning Attacks against Support Vector Machines

Instructor: Xiao Zhang (xiao.zhang@cispa.de). 

Meeting Time: from 12:15 to 13:45 on each Thursday

Meeting Location: Room 0.02, CISPA Main Building, Stuhlsatzenhaus 5

Course Expectations

Students enrolled in this seminar are expected to:

• Lead discussions on assigned topics. Students will be assigned to teams of 2-3 students. Each week, a team of students will prepare and present two assigned papers on a research topic in adversarial machine learning, then lead the discussion and answer questions from the audience. A different team of students will be responsible for preparing challenging questions for the presenting team and writing a one-page summary to document the in-class activities. The presenting team should deeply understand the research papers to deliver a well-structured presentation.
• Participate actively in class meetings. All other students should also read the assigned papers on the presenting topic each week and contribute to Q&A sessions. Each student is expected to write reviews on two of the assigned research papers during the semester.
• Seminar paper. Each team will hand in a survey paper (for seminar students) or a systematization of knowledge (SoK) paper (for proseminar students). I will explain this in the kick-off meeting. Also, you may want to discuss your team project with the instructor early in the semester.

Deliverables

Review (20%). Write reviews for assigned research papers. Throughout the semester, you should expect to write two reviews (each consisting of 10% of your final grades). The review should aim to address the following questions:

• What is the problem addressed by the paper?
• What was done before, and how does the paper improve prior works?
• What are the strengths and the weaknesses of the paper?
• What part of the paper was difficult to understand?
• What are possible improvements or further implications of the paper?

The review should be, at most, two pages long, using the NeurIPS LaTeX template. You may want to read the ICML 2023 Reviewer Tutorial for instructions on how to write a good review.

Presentation and Summary (40%). You will form a team of 2-3 students and deliver a 45-minute presentation followed by a 30-minute Q&A session on the topics assigned to you in an early class. A different team of students is responsible for preparing (at least three) challenging questions and summarizing the in-class activities (presentation and Q&A sessions).

Seminar Paper (40%). Write a seminar paper on a research topic of adversarial machine learning that is aligned with your interest. For seminar students, you are supposed to write a survey paper. For proseminar students, you are supposed to write a systematization of knowledge (SoK) paper. The seminar paper must use the NeurIPS LaTeX template and not be over six pages, not counting references and appendices. Papers can be shorter, but generally, the provided page limit indicates how long a typical paper should be.

Important Details

1. Kick-off meeting in the first week of the semester
   • Time: from 12:15 to 13:45 on Thursday, 02.05.2024
   • Location: Room 0.02, CISPA Main Building, Stuhlsatzenhaus 5
2. We will assign topics and form teams based on interests. 
3. The summary of in-class activities is due one week after each presentation day.
4. We plan to have in-person meetings as long as possible and switch to online if needed. Attendance and contributions to discussions in all class meetings are mandatory.
5. You may want to discuss the topic of your seminar paper with the instructor earlier in the semester (by appointment).
6. The initial submission of seminar papers is due on 04.07. Based on your submission, you will receive feedback within a week and will have until 18.07 to improve your paper. Note that the first submission must already be good enough for the instructor to review. Otherwise, you will not receive full credits.

List of Topics

We plan to include the following research topics in adversarial machine learning:

1. Adversarial Examples & Robustness Evaluation
2. Adversarial Training & Robust Generalization
3. Robustness Certification Methods
4. Intrinsic Limits on Adversarial Robustness
5. Data Poisoning & Backdoor Attacks 
6. Privacy-related Attacks
7. Adversarial ML in Natural Language Processing
8. Adversarial ML in Large Language Models

Note that the above list of topics may be subject to change.

Honor and Responsibilities

We believe in the value of a community of trust and expect all students in this class to contribute to strengthening that community. The course will be better for everyone if everyone can assume everyone else is trustworthy. The course instructor starts with the assumption that all students deserve to be trusted. In this course, we will be learning about and exploring some vulnerabilities that could be used to compromise deployed systems. You are trusted to behave responsibly and ethically. You may not attack any system without the permission of its owners and may not use anything you learn in this class for evil. If you have any doubts about whether or not something you want to do is ethical and legal, you should check with the course instructor before proceeding.