Registration for this course is open until Tuesday, 30.04.2024 23:59.


Currently, no news are available

Optimization for Machine Learning

This course teaches an overview of modern mathematical optimization methods for machine learning and data science applications. In particular, the scalability of algorithms to large datasets will be discussed in theory and in implementation.

This advanced lecture aims to prepare students to research this topic. An interest in and the ability to understand and apply mathematical proofs are essential.

The first lecture will take place on April 16 (hybrid), building E9 1 (CISPA), room 0.01 ("presentation"). The zoom link will be sent by email to registered participants.


Learning Prerequisites

  • Previous coursework in calculus, linear algebra, and probability is required.
  • Familiarity with optimization and/or machine learning is beneficial.

Students are recommended to register for this course only as master's students, but attendance is also possible for bachelor students in their last semester. There are no strict rules or regulations, but the students must acquire (missing) fundamentals independently. Please note that this course will be offered regularly (yearly).


Course Information 2024

Please note that the information below has not yet been updated. The page will be updated before the start of the semester.

  • The 2024 version of the course will cover similar materials with slight changes in organizational aspects.
  • The lectures will take place on Tuesday 4-6pm, with exercise sessions just before (2-4pm).


Program (2023)

Materials will be posted under the materials tab.

Preliminary schedule (subject to change - materials will be updated under program):

  • April 17, L1, Introduction, Convexity
  • April 24, L2, Gradient Descent I
  • May 1, holiday, no lecture
  • May 8 (only online!), L3, Gradient Descent II
  • May 15, L4, Stochastic Gradient Descent, Non-Convex Optimization
  • May 22 (only online!), L5, Non-Convex Optimization
  • May 29, holiday, no lecture
  • June 5, L6, Coordinate Descent & Accelerated Gradient Descent
  • June 12 (only online!), L7, Adaptive Gradient Methods
  • June 19 (only online!), L8, Distributed Optimization I
  • June 26 (at CISPA), L9, Distributed Optimization II
  • July 3 (at CISPA), L10, Variance Reduction
  • July 10 (only online!), L11, Distributed Optimization III
  • July 17 (at CISPA), L12, Communication compression & Exam Q&A Session
  • August 2 (at CISPA, room 3.21, 3pm), Additional Tutorial/Exam Q&A (with Xiaowen)

Exam: written exam on August 8, 2023. 

Note that only one exam will take place this year (re-exam possible after the next reading in 2024).


Registration (2023)

  • Registration opens on March 28.
  • Students are required to register for the lecture and exam on LSF (the registration deadline is a few weeks after the first lecture, and will be visible on LSF). No assistance can be provided if you miss this deadline - no exceptions.


Organization (2023)

There will be one lecture per week, taking place on Mondays, 16:15 - 17:45. The first lecture is on April 17.

There will be a Q&A session with the teaching assistant (Xiaowen Jiang), on Mondays 15:00 - 16:00, room 3.21. First office hour: April 24.

We might also offer office hours on zoom, if requested.

We will use a hybrid format. All lectures can be attended online, some lectures will be held in building E9 1 (CISPA), room 0.01 (presentation room, ground floor).


Grading and Exam (2023)

The assignments (exercises) will in not be graded. Nevertheless it is strongly recommended to do the exercises every week!

To encourage engagement with the lecture material, you are asked to submit a short quiz every week! Submission is mandatory, but not graded. You will get an automatic feedback on your submitted answers.

A research project (theoretical or practical) will focus on either practical implementation and the real-world performance of one of the studied optimization algorithms or variants, or a theoretical investigation with a small extension (with proof) of one of these schemes. The project is mandatory and done in groups of 2-3 students. The projects will be graded in scale of Fail, Pass, Good (top 30%, 0.3 bonus), Excellent (top 10%, 0.6 bonus). You are required to pass the project to take part in the exam. Failed projects can be resubmitted within 2 weeks.

The final exam will be (oral or written, TBA), and will cover all the material discussed in the lectures and the topics from the assignments/exercises (but not the research projects). If you pass the exam, eventual bonus points from the project will be considered to determine the final grade.


Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators.