Topics in Optimization for Machine Learning Sebastian Stich

News

04.02.2023

Talks next Monday (last session)

Next Monday (meeting on zoom) we will discuss the following two papers:

Due to the late announcement of the papers, I do... Read more

Next Monday (meeting on zoom) we will discuss the following two papers:

Due to the late announcement of the papers, I do not expect you to submit a question this time. However, it would be still great if you could catch up on the topic to be prepared for the discussion.

Note that the submission deadline of report 3 is coming up this week.

12.01.2023

Hybrid Sessions in January

The next three sessions will be held in hybrid format: we are physically meeting in CISPA room 0.01 (ground floor to the right), but the sessions will also be streamed on zoom for those who cannot attend.

02.12.2022

First paper review

Dear students,

Just a small reminder that the first deadline (Dec 14) is upcoming. As this is the first submission, please plan accordingly.

The registration deadline on LSF has passed. If you have not yet registered (but still want to), please write an email... Read more

Dear students,

Just a small reminder that the first deadline (Dec 14) is upcoming. As this is the first submission, please plan accordingly.

The registration deadline on LSF has passed. If you have not yet registered (but still want to), please write an email directly to the study office.

I did a preliminary assignment for the presentation dates (see google sheet). I still need to check the room availability for the 'in-person' presentations (these could still move by a week or so). We have the first session on Dec 19 (online), both speakers have confirmed.

-Sebastian

15.11.2022

topic assigment

Dear Students,

I finalized the topic assignment (see google sheet). The column 'Talk & Report' indicates your main topic (you can choose any of these papers for final report and presentation). The columns 'Review x' refers to the review assignments where a... Read more

Dear Students,

I finalized the topic assignment (see google sheet). The column 'Talk & Report' indicates your main topic (you can choose any of these papers for final report and presentation). The columns 'Review x' refers to the review assignments where a specific paper is assigned to you.

Many got their favorite choice, but I had to break a few ties. Students that did not get assigned their top choice (ranked 4) for the talk can let me know this week if they would rather prefer any other (available) topic.

The first review assignment should be submitted no later than Dec 14 (end of day).

 

04.11.2022

First steps

Dear students,

We will meet next Monday (Nov 7) from 16:15-17:00 on zoom to discuss the seminar format and topic assignment (please add your preferences by end of next week).

I added the slide deck and link to the paper list under the 'materials' tab on CMS.... Read more

Dear students,

We will meet next Monday (Nov 7) from 16:15-17:00 on zoom to discuss the seminar format and topic assignment (please add your preferences by end of next week).

I added the slide deck and link to the paper list under the 'materials' tab on CMS. Please have a look, so that we can use the time on Monday to answer the questions you might have.

-Sebastian

 
Registration for the seminar is not possible directly. Please use the CS department assignment system to register your interest.

Seminar: Topics in Optimization for Machine Learning

Optimization lies at the heart of many machine learning algorithms. This seminar teaches an overview of modern mathematical optimization methods, for applications in machine learning and data science. In particular, we will discuss the theoretical basics of stochastic optimization, scalability of algorithms to large datasets, and challenges in distributed optimization, such as for instance in decentralized or federated machine learning. We will cover a set of foundational papers, but also a selection of recent publications. 

Previous attendance of the summer term lecture "Optimization for Machine Learning" is recommended, but not required.

Organization

In this seminar, students will learn to present, discuss, and summarize papers in different areas of optimization for machine learning. Specifically, each student will get a single topic assigned to them, consisting of two papers (a lead and follow-up paper). Each student will

  • write a short seminar paper on the topic assigned to them, for which the two papers on the topic serve as the starting point;
  • prepare a presentation on the topic assigned to them and present the paper to the other students;
  • read the papers assigned to the other students and prepare questions to ask to the presenter of this paper/topic;
  • there will be an additional writing assignment that can be chosen from the two tasks below at the beginning of the semester (please indicate your preference when applying for the seminar):
    • (A) write three short reviews on papers from a different topic. The reviews will be shared among the group (in particular with the presenter of the topic).
    • (B) write a research report on your topic. The goal is to identify an open research question in the assigned topic area and trying to work towards a (partial) solution of the identified question.

Important Dates (TBA)

  • Kick-off meeting 7.11.22, 16.15h-17.00h (to be held online, via zoom).
  • Voting for topics deadline: 11.11.22 (end of day)
  • The presentations will take place between end of November - January (hybrid session format). Tentatively on Monday 4-6pm (we need to fix 6-7 dates, depending on number of participants). Participation in the presentations is mandatory.
  • Mandatory feedback round/practice talk at least two weeks before the presentation (arrange exact time with supervisor).
  • Hand-in of report: TBA.

Deliverables

  • Questions: (10% of your final grade): submit a question for each presentation slot (except when you are presenting).

  • Option A: 3 short reviews: (each contributes 10% of your final grade): Write a short review (max 2 page) on one of the papers (not the one that you are presenting) that addresses the following questions:

    1. What is the problem addressed by the paper?
    2. What was done before, and how does the paper improve on previous work?
    3. What are the strengths and the limitations of the techniques in the paper
    4. What part of the paper was difficult to understand?
    5. What are possible improvements or extensions of the techniques in the paper?

     

    Option B: a short scientific report on a research question: (30% of your final grade). (Format and length to be discussed with supervisor).

  • Presentation: (30%). You will prepare and deliver a 30 min presentation (followed by 15 mins question/discussion) of the paper assigned to you. You will have the possibility to get feedback on your slides before the presentation.

  • Seminar Paper: (30%) You will write a seminar paper on the topic that you have presented. It must not be longer than 6 pages, not counting references and appendices. Note that appendices are not meant to provide information that is absolutely necessary to understand the paper, but rather to provide auxiliary material. Papers can be shorter, but in general the provided page limit is a good indicator of how long a paper should be.


Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators