News
Exam Review 2023Written on 28.08.23 by Sebastian Stich Dear students, I have added the exam grades (incl. bonus) to LSF. I am offering an exam review session on
(I can also send you a scanned version of your graded exam if you prefer that).
|
Exam Q&A SessionsWritten on 12.07.23 by Sebastian Stich Dear students, As announced,
Most projects are now graded (except from some pending… Read more Dear students, As announced,
Most projects are now graded (except from some pending re-submissions). Students that have passed all requirements can now register for the exam.
You can directly register on LSF. The registration link on this side should redirect your there. Some students (e.g. erasmus) cannot register on LSF, for them registration in CMS suffices. If it does not work for you, please contact me (or the study office) early. |
June 19 cancelled - Lecture 8 available as a video recordingWritten on 14.06.23 (last change on 14.06.23) by Sebastian Stich Dear students, As announced last week, there will be no live lecture next week (June 19) due to conference travel. Instead, a video recording is available (as of now) under the Materials tab. Please use the tutorials to ask questions, and we will also reserve a bit of time in the following week… Read more Dear students, As announced last week, there will be no live lecture next week (June 19) due to conference travel. Instead, a video recording is available (as of now) under the Materials tab. Please use the tutorials to ask questions, and we will also reserve a bit of time in the following week (June 26) to go over questions that you might have on the lecture materials. |
Project Plan FeedbackWritten on 01.06.23 (last change on 01.06.23) by Sebastian Stich Dear students, I have uploaded a brief feedback to your project proposals (the ones that have been submitted). As a general feedback: - while most proposals had nice ideas, many were lacking rigor in describing how a 'fair comparison/evaluation' would be performed. Please carefully think… Read more Dear students, I have uploaded a brief feedback to your project proposals (the ones that have been submitted). As a general feedback: - while most proposals had nice ideas, many were lacking rigor in describing how a 'fair comparison/evaluation' would be performed. Please carefully think about this, and also include a discussion in your final report on 'why' you methodology is adequate to evaluate the properties you want to study. - many projects were submitted by teams of size 1. There might be an opportunity to merge similar projects, or for people that do not have one yet to join one of these (you can use the forum to exchange). The course materials are quite comprehensive and demanding, and therefore I think that tripling the effort for the project work is not an ideal strategy if you need the time for exercises/quizzes, etc. |
Project teams & Project planWritten on 17.05.23 by Sebastian Stich Now that we have learned about stochastic gradient descent in the lecture, it is about time to start thinking about a cool group project ;-) The registration is open on the website (note that you can register teams of size 1-3 to submit the project). Originally, the (voluntary) submission… Read more Now that we have learned about stochastic gradient descent in the lecture, it is about time to start thinking about a cool group project ;-) The registration is open on the website (note that you can register teams of size 1-3 to submit the project). Originally, the (voluntary) submission deadline for the project idea was set to be May 22. However, we can also take some time in the next lecture to address your questions. If you want to adjust your project plan based on this discussion, you can also submit it a few days later (e.g. Thursday May 25 - end of day). |
New tutorial timeslotsWritten on 11.05.23 (last change on 15.05.23) by Sebastian Stich Dear students, Based on your comments during the lecture, we would like to propose the following new timeslots for the tutorial/Q&A session: - Monday, 1-2pm (in-person, CISPA room 3.21) - Wednesday, 3-4pm (online on zoom - link will be shared under 'materials')
|
Reminder: weekly quizWritten on 20.04.23 by Sebastian Stich Dear students, As a reminder: please don't forget to submit the quiz (quiz week 1) before the next lecture. |
Zoom linkWritten on 17.04.23 (last change on 17.04.23) by Sebastian Stich Dear students, Thank you for registering to the OPT4ML (Optimization for Machine Learning) lecture. The first lecture starts today at 4.15pm. You find the zoom link under the materials tab with the other course material. |
Optimization for Machine Learning
This course teaches an overview of modern mathematical optimization methods, for applications in machine learning and data science. In particular, scalability of algorithms to large datasets will be discussed in theory and in implementation.
This advanced lecture aims to prepare students to conduct research on this topic. An interest and the ability to understand and apply mathematical proofs is essential.
The first lecture will take place on April 17. (hybrid), building E9 1 (CISPA), room 0.02 ("showroom"). Due to overbooking of the room, the first lecture will be held online. The zoom link will be send by email to registered participants.
The following lectures are scheduled to take place in the (a bit smaller, but much more comfortable) room 0.01.
Learning Prerequisites
- Previous coursework in calculus, linear algebra, and probability is required.
- Familiarity with optimization and/or machine learning is useful.
It is recommended to register for this course only as a master student, but attendance is also possible for bachelor students in their last semester. There are no strict rules or regulations, but the students are required to acquire missing fundamentals on their own. Please note that this course will be offered on a regular basis (yearly).
Program
Materials will be posted under the materials tab.
Preliminary schedule (subject to change - materials will be updated under program):
- April 17, L1, Introduction, Convexity
- April 24, L2, Gradient Descent I
- May 1, holiday, no lecture
- May 8 (only online!), L3, Gradient Descent II
- May 15, L4, Stochastic Gradient Descent, Non-Convex Optimization
- May 22 (only online!), L5, Non-Convex Optimization
- May 29, holiday, no lecture
- June 5, L6, Coordinate Descent & Accelerated Gradient Descent
- June 12 (only online!), L7, Adaptive Gradient Methods
- June 19 (only online!), L8, Distributed Optimization I
- June 26 (at CISPA), L9, Distributed Optimization II
- July 3 (at CISPA), L10, Variance Reduction
- July 10 (only online!), L11, Distributed Optimization III
- July 17 (at CISPA), L12, Communication compression & Exam Q&A Session
- August 2 (at CISPA, room 3.21, 3pm), Additional Tutorial/Exam Q&A (with Xiaowen)
Exam: written exam on August 8, 2023.
Note that only one exam will take place this year (re-exam possible after the next reading in 2024).
Registration
- Registration opens on March 28.
- Students are required to register for the lecture and exam on LSF (the registration deadline is a few weeks after the first lecture, and will be visible on LSF). No assistance can be provided if you miss this deadline - no exceptions.
Organization
There will be one lecture per week, taking place on Mondays, 16:15 - 17:45. The first lecture is on April 17.
There will be a Q&A session with the teaching assistant (Xiaowen Jiang), on Mondays 15:00 - 16:00, room 3.21. First office hour: April 24.
We might also offer office hours on zoom, if requested.
We will use a hybrid format. All lectures can be attended online, some lectures will be held in building E9 1 (CISPA), room 0.01 (presentation room, ground floor).
Grading and Exam
The assignments (exercises) will in not be graded. Nevertheless it is strongly recommended to do the exercises every week!
To encourage engagement with the lecture material, you are asked to submit a short quiz every week! Submission is mandatory, but not graded. You will get an automatic feedback on your submitted answers.
A research project (theoretical or practical) will focus on either practical implementation and the real-world performance of one of the studied optimization algorithms or variants, or a theoretical investigation with a small extension (with proof) of one of these schemes. The project is mandatory and done in groups of 2-3 students. The projects will be graded in scale of Fail, Pass, Good (top 30%, 0.3 bonus), Excellent (top 10%, 0.6 bonus). You are required to pass the project to take part in the exam. Failed projects can be resubmitted within 2 weeks.
The final exam will be (oral or written, TBA), and will cover all the material discussed in the lectures and the topics from the assignments/exercises (but not the research projects). If you pass the exam, eventual bonus points from the project will be considered to determine the final grade.