Hauptseite

News

Exam scores

Written on 05.10.22 by Sebastian Stich

The scores should be visible in LSF, and solution and grade scale are available under the 'materials' tab.

For the exam review, there is the option that I send you a scanned copy on request, or otherwise please schedule an appointment (preferred date: Oct 17/18).

registration deadine for re-exam today!

Written on 27.09.22 by Sebastian Stich

Just a friendly reminder to register on LSF today.

The exam will take place in GHH (the same room as for the exam in August) on October 4, 13:15-15:45h.

Exam review & Registration for Re-Exam Open

Written on 01.09.22 by Sebastian Stich

Dear students,

If you want to review your exam in person, please write me an email to schedule an appointment next week (Sep 5-8). There is also still the option that I send you a scanned copy.

Remarks

Written on 12.08.22 by Sebastian Stich

The course evaluation is now available under 'materials'. Thanks to all that provided feedback that will help to improve the course next year.

I further uploaded the exam, solutions, and a grade scale.

I adjusted the scale to account for the questions that have not been solved by anybody.

The… Read more

The course evaluation is now available under 'materials'. Thanks to all that provided feedback that will help to improve the course next year.

I further uploaded the exam, solutions, and a grade scale.

I adjusted the scale to account for the questions that have not been solved by anybody.

The re-exam will be on October 4, 13.15 - 15.45 in GHH.

Exam (August 4, 9.30-12h)

Written on 23.07.22 by Sebastian Stich

The exam will take place on

August 4, 9.30h - 12h, Günter-Hotz-Hörsaal (GHH)

please arrive at 9.20h, so that we can start on time
closed book exam, no electronic devices allowed
you can bring one sheet of A4 paper with your own notes (handwritten, or latex, font >= 10pt)

Q&A… Read more

The exam will take place on

August 4, 9.30h - 12h, Günter-Hotz-Hörsaal (GHH)

please arrive at 9.20h, so that we can start on time
closed book exam, no electronic devices allowed
you can bring one sheet of A4 paper with your own notes (handwritten, or latex, font >= 10pt)

Q&A Session:

I offer an additional Q&A session on Monday, July 25, 14-15h (Q&A zoom link posted under materials)

Exam registration:

please do not forget to register for the exam latest a week before the exam
students that can register on LSF must register there (the link on cms will re-direct you there), for other students registration on cms is enabled
(still a few project re-submissions are pending, due date: today)

News

Written on 03.07.22 by Sebastian Stich

Several news regarding the lecture:

1) due to low demand, the morning (11-11.30) Q&A sessions are cancelled for the remaining weeks. I will offer alternative Q&A slots closer to the exam.

2) the exam format will be written. I will give more information on the exam format and structure in one of… Read more

Several news regarding the lecture:

1) due to low demand, the morning (11-11.30) Q&A sessions are cancelled for the remaining weeks. I will offer alternative Q&A slots closer to the exam.

2) the exam format will be written. I will give more information on the exam format and structure in one of the last two lectures.

3) groups that failed the project will be notified by July 5 (other groups receive feedback later) and have the possibility to resubmit by July 23 (see project description). I will also ask groups that did not use the correct template or went over the page limit to resubmit a correctly formatted version by this date.

4) course evaluation: I posted the link to the course evaluation form under 'course materials'. Please fill out this form until July 14 (you will be given some time for this during tomorrows lecture).

Upcoming Talks on Distributed Optimization

Written on 07.06.22 by Sebastian Stich

Dear all,

There will be a few optimization related talks @ CISPA in the coming weeks: talk announcements. Feel free to drop by & ask questions to the speakers.

This Thursday, Kshitij Patel (TTIC) will talk about the Communication Complexity in Distributed Non-Convex Optimization, in two… Read more

Dear all,

There will be a few optimization related talks @ CISPA in the coming weeks: talk announcements. Feel free to drop by & ask questions to the speakers.

This Thursday, Kshitij Patel (TTIC) will talk about the Communication Complexity in Distributed Non-Convex Optimization, in two weeks, Grigory Malinovsky (KAUST) about communication in Federated Learning and Anastasia Koloskova (EPFL) on Asynchronous SGD.

https://docs.google.com/document/d/e/2PACX-1vQ2CpCAD1egpT3iYAI4eefQlAIx3yW0yBT6mI-PjPwvvuzaUr9csUrbZnho76pO98ZLei4Z5rfg0LKd/pub

Next Lecture & Project Teams

Written on 22.05.22 by Sebastian Stich

Dear all,

Just a small note that tomorrow's lecture will be hybrid (on-site attendance possible).

If you are not yet in a team for the project, please check again the forum (there are still a few open requests), or reach out to me by tomorrow.

Q&A Session

Written on 29.04.22 (last change on 17.05.22) by Sebastian Stich

Poll Results: The most preferred slots were just before the lecture and in the morning. So let's try with these slots for the next weeks (starting May 9). We can adjust based on demand and participation.

11.00-11.30h

13.30-13.55h

---

Last lecture I was informed that the proposed slot for… Read more

11.00-11.30h

13.30-13.55h

---

Last lecture I was informed that the proposed slot for the Q&A session is in conflict with the ML course.

I created a quick poll to find an alternative slot. Please participate if you intend to attend the sessions regularly.

For next Monday, let's try with 11h-12h (on zoom only, as I did not book a room yet) and I will let you know during the lecture - based on attendance and poll results - how we proceed. Note that you can always also talk to me just after the lecture (also if you attend on zoom).

First Lecture

Written on 21.04.22 by Sebastian Stich

Dear all,

Thanks for signing up to the Opt4ML lecture.

The first lecture will take place next Monday, April 25, at 14:15. Attendance is possible online (a zoom link will be posted under materials) or in person in the building E9 1 (CISPA), room 0.05 (lecture hall ground floor). Note that a mask… Read more

Dear all,

Thanks for signing up to the Opt4ML lecture.

In order to estimate the capacity needed for the exam and to learn something about your background and expectations for the course, I would like to ask you to fill out this anonymous survey (https://forms.gle/emA3jUhCQF7VfYV2A) by Sunday, April 24.

Show all

Optimization for Machine Learning

This course teaches an overview of modern mathematical optimization methods, for applications in machine learning and data science. In particular, scalability of algorithms to large datasets will be discussed in theory and in implementation.

The first lecture will take place on April 25 (hybrid).

Learning Prerequisites

Previous coursework in calculus, linear algebra, and probability is required.
Familiarity with optimization and/or machine learning is useful.

It is recommended to register for this course only as a master student, but attendance is also possible for bachelor students in their last semester. There are no strict rules or regulations, but the students are required to acquire missing fundamentals on their own. Please note that this course will be offered on a regular basis (yearly).

Program

Materials will be posted under the materials tab.

Preliminary schedule (subject to change - materials will be updated under program):

April 25 (hybrid), Introduction, Convexity
May 2 (hybrid), Gradient Descent I
May 9 (hybrid), Gradient Descent II
May 16 (only online!), Stochastic Gradient Descent, Non-Convex Optimization - Please register (in teams of 3) for the mini-project.
May 23 (hybrid), Non-Convex Optimization, Accelerated Gradient Descent
May 30 (only online!), Coordinate Descent
June 6, holiday, no lecture - Project abstract deadline on June 7 (submission here on cms).
June 13 (only online!), Adaptive Methods and Delays
June 20 (hybrid), Distributed Optimization I
June 27 (hybrid), Distributed Optimization II
July 4 (only online!), Decentralized Optimization and Compression
July 11 (only online!), Opt for ML in Practice (overview)
July 18 (TBA), mini-project week

Exams (it will be announced later if the exam is written or oral)

regular exam: written exam Aug 4, 2022 (oral exam tentatively between Aug 2-5, 2022, written exam Aug 4, 2022)
re-exam: written exam Oct 4, 2022 (oral exam tentatively between Oct 3-7 2022, written exam Oct 4, 2022)

Registration

Registration opens on March 28! The slots will be handed out on a first-come-first-serve basis, but we expect the number of slots to be sufficient for everyone interested.
Students are required to register for the lecture and exam on LSF (the registration deadline is a few weeks after the first lecture, and will be visible on LSF). No assistance can be provided if you miss this deadline - no exceptions.

Organization

There will be one lecture per week, taking place on Mondays, 14:00 - 16:00. The first lecture is on April 25. Due to the high number of registered students (and still high incidence rates), the first lecture will be online.

There will be a Q&A session (office hour) Mondays 16:00 - 17:00. You can use zoom webinar link (room for offline participation: TBA).

We will use a hybrid format. All lectures can be attended online, some lectures will be held in building E9 1 (CISPA), room 0.05 (lecture hall ground floor). The offline offering will be adjusted based on current regulations and recommendations. Note that a mask is required in the CISPA lecture hall and the room capacity is limited to 96 offline participants.

Grading and Exam

The assignments will not be graded (I do not have the capacity to provide individual feedback). Nevertheless it is strongly recommended to do the exercises regularly.

A project will focus on practical implementation and encourage the students to investigate the real-world performance of one of the studied optimization algorithms or variants. The project is mandatory and done in groups of 3 students. The projects will be graded in scale of Fail, Pass, Good (top 30%, 0.3 bonus), Excellent (top 10%, 0.6 bonus). You are required to pass the project to take part in the exam. Failed projects can be resubmitted within 2 weeks.

The final exams will be (oral or written, TBA), and will cover all the material discussed in the lectures and the topics on which you did your assignments. If you pass the exam, eventual bonus points from the project will be considered to determine the final grade.