Optimization for Machine Learning

Optimization for Machine Learning

This course teaches an overview of modern mathematical optimization methods for machine learning and data science applications. In particular, the scalability of algorithms to large datasets will be discussed in theory and in implementation.

This advanced lecture aims to prepare students to research this topic. An interest in and the ability to understand and apply mathematical proofs are essential.

The first lecture will take place on April 16 (hybrid), building E9 1 (CISPA), room 0.01 ("presentation"). The zoom link will be sent by email to registered participants.

 

Learning Prerequisites

• Previous coursework in calculus, linear algebra, and probability is required.
• Familiarity with optimization and/or machine learning is beneficial.

Students are recommended to register for this course only as master's students, but attendance is also possible for bachelor students in their last semester. There are no strict rules or regulations, but the students must acquire (missing) fundamentals independently. Please note that this course will be offered regularly (yearly).

 

Course Information 2024

• The lectures will take place on Tuesday 4-6pm, with exercise sessions just before (2-4pm).

 

Program

Course materials will be posted under the materials tab, the schedule and suggested reading materials will be updated under the contents tab.

Preliminary schedule (subject to change):

• April 16, first lecture
• June 4, midterm exam
• July 16, project presentation
• July 23, no lecture
• TBD (summer break), Additional Tutorial/Exam Q&A 
• TBD: written exam

Note that only one exam will take place this year (re-exam possible after the next reading in 2025).

 

Registration

• Students are required to register for the lecture and exam on LSF (the registration deadline is a few weeks after the first lecture and will be visible on LSF). No assistance can be provided if you miss this deadline.

 

Organization

• There will be one lecture per week on Tuesdays, 16:15 - 17:45. The first lecture is on April 16.
• There will be a Q&A session with the teaching assistants on Tuesdays, 15:00 - 16:00. First Q&A session is on April 23.
  • Attendance of the tutorials/Q&A session is not mandatory, but strongly recommended.
• Office hours: before/after each lecture.
• We will use a hybrid format. All lectures can be attended online, some lectures will be held in building E9 1 (CISPA), room 0.01 (presentation room, ground floor).

 

Grading and Exam

• (requirement) A research project (theoretical or practical) will focus on either practical implementation and the real-world performance of one of the studied optimization algorithms or variants or a theoretical investigation with a small extension of one of these schemes. The project is mandatory and done in groups of 2-3 students. The projects will be graded as: Fail, Pass, Above Average (top 30%, 0.3 bonus), or Excellent (top 10%, 0.6 bonus). You are required to pass the project to take part in the exam. Failed projects can be resubmitted within 2 weeks.
• (25%) Points from the midterm exam will count for 25% of the final mark.
• (75%) Points from the final exam will count for 75% of the final mark.
• The exams will be written and cover all the material discussed in the lectures and the topics from the assignments/exercises. 
• If you pass the course (at least 4.0), eventual bonus points from the project will be considered to determine the final grade.