Deep Generative Models

[Important!] You must register for the examination of this course on LSF by the end of November 16, 2025, in order to receive credits & grades!

Course Description: Deep generative models have been adopted in many AI and ML applications, such as computer vision, natural language processing, and scientific discovery. Recent advances in parametrizing these models using deep neural networks and training them using stochastic optimization methods have enabled scalable modeling of complex, high-dimensional data. In this proseminar, we will look into foundational frameworks and research frontiers of deep generative models. We will cover topics such as variational autoencoders, generative adversarial nets, diffusion models, flow-based models, and autoregressive models.

Each student will be assigned a topic and will present two papers related to the topic throughout the course. The initial presentation will be followed by a Q&A and discussion session, during which you will receive constructive feedback on how to improve your presentation. Your final grade will be largely determined by your performance on the second presentation and the Q&A. At the end of the semester, each student will need to submit a two-page summary of the presented topic.

Course Instructor: Xiao Zhang (xiao.zhang@cispa.de)

Teaching Assistant: Santanu Rathod (santanu.rathod@cispa.de)

Meeting Time: 16:15 - 17:45 weekly on Mondays

Meeting Location: Room 0.07, CISPA C0 Building at Stuhlsatzenhaus 5, 66123 Saarbrücken

Requirements: The required language for the presentations is English. There are no formal requirements for this proseminar, but having taken graduate-level machine learning or optimization courses will be helpful. You are particularly welcome to the course if you are interested in doing research on deep generative models.

---

Covered Topics

1. Variational Autoencoders
2. Generative Adversarial Networks
3. Normalizing Flows
4. Autoregressive Models
5. Continuous Normalizing Flows
6. Score-based Generative Models
7. Diffusion Models
8. Guided Diffusion
9. Flow Matching
10. Consistency Models
11. Masked Diffusion Models
12. Discrete Flow Matching

---

(Tentative) Course Schedule

• Oct 27: Proseminar kickoff – course introduction & overview of the topics
• Nov 03: Tutorial on how to read & present a research paper
• Nov 10: Initial presentation – topics 1 & 2 
• Nov 17: Initial presentation – topics 3 & 4
• Nov 24: Initial presentation – topics 5 & 6
• Dec 01: Initial presentation – topics 7 & 8
• Dec 08: Initial presentation – topics 9 & 10
• Dec 15: Initial presentation – topics 11 & 12
• Jan 05: Final presentation
• Jan 12: Final presentation
• Jan 19: Final presentation

---

Grading Rubrics

In-class Participation (15%). Students are expected to attend the weekly meeting and actively participate in the Q&A sessions. Each student is expected to give a total of five inputs (questions or constructive feedback) during the Q&A and discussion sessions throughout the seminar. The instructor and the TA will keep track of the number of questions you have asked and/or comments you have provided throughout the semester, which will count toward your final grade.

Paper Presentation (70%). Each student will be assigned a topic and two papers related to that topic. During the proseminar kickoff, the instructor will give an introduction and all the topics. You will get access to all the prepared research papers during the kickoff. You are allowed to choose one of the topics (first-come, first-served) by the end of October 27. I strongly suggest you make the selection during the seminar kickoff. If you do not choose any topic, you will be randomly assigned a topic the following day.

• Initial presentation (20%). You will present the first research paper, usually a seminal one regarding the topic, during one of the class meetings in November and December. You have 20 minutes to present the paper, followed by a 25-minute Q&A and discussion session. The instructor, the TA, and other participants will raise questions. You will receive constructive feedback on how to improve your presentation, which should be taken into account in your final presentation.
• Final presentation (50%). You will present the second research paper, typically a recent one on the topic, during one of the three class meetings in January. Again, you have 20 minutes to present the paper, followed by a 10-minute Q&A session. Most of the questions will be asked by the instructor and the TA.

Written Summary (15%). At the end of the semester, you will need to submit a two-page summary, summarizing the topics you have presented, what you have learned throughout the semester, your reflections, etc.

Bonus: TBD