Topics in Optimization for Machine Learning

Seminar: Topics in Optimization for Machine Learning

Optimization lies at the heart of many machine learning algorithms. This seminar teaches an overview of modern mathematical optimization methods for machine learning and data science applications. In particular, we will discuss the theoretical basics of stochastic optimization, the scalability of algorithms to large datasets, and challenges in distributed optimization, such as decentralized or federated machine learning. We will cover a set of foundational papers and a selection of recent publications. 

Previous attendance at the summer term lecture "Optimization for Machine Learning" is recommended but not required (if you know the basic principles of optimization).

Organization

In this seminar, students will learn to present, discuss, and summarize papers in different areas of optimization for machine learning. Specifically, each student will get a topic assigned to them, consisting of two papers (a lead and follow-up paper). Each student will

• prepare a presentation on the topic assigned to them and present the paper to the other students;
• write a short seminar paper on the topic assigned to them, for which the two papers on the topic serve as the starting point;
• write 2-3 short paper reviews (and prepare questions about these papers that can be asked during the discussion) 

We will have a few Zoom meetings during the semester where the writing exercises will be explained, discussed, and evaluated. The main presentations will occur towards the end of the semester (TBA) - participation is mandatory.

The final grade will depend on the presentation, the written deliverables, and the active participation in the discussions.

Important Dates (TBA)

• Kick-off meeting TBA (to be held online, via zoom).
• Voting for topics deadline: TBA
• The presentations will take place between end of November - January (hybrid session format). Participation in the presentations is mandatory.
• Mandatory feedback round/practice talk at least two weeks before the presentation (arrange exact time with the tutor/supervisor).
• Hand-in of report: TBA.

Deliverables

TBD