Trustworthy Agentic Systems

 

Seminar Description

When AI systems move from pattern recognition to decision-making, they stop being just tools and start becoming agents. These agents can plan, act, and adapt, qualities that make them powerful but also raise questions about reliability, safety, and alignment with human goals.

This seminar investigates what it means to build trustworthy agentic systems. Instead of focusing on scale or efficiency, we’ll centre on issues like:

• How do we evaluate whether an AI system deserves trust?
• What design principles can reduce the risks of unintended behaviour?
• How can humans and AI collaborate without losing oversight or control?

By the end of the seminar, you’ll have a deeper grasp of the challenges that lie ahead for autonomous AI and the frameworks researchers are developing to meet them.

 

Organization

The seminar combines weekly paper discussions with individual position papers, building towards a final presentation.

• Introductory session – We will give an overview of the main themes and topics.
• Weekly paper presentations – Each student will be assigned a paper. One or two students will present each week, followed by a discussion round. All participants are expected to read the paper in advance so they can contribute meaningfully. Students will be graded not only on their own presentations but also on their engagement in discussions. 
• Position paper – Drawing on their assigned paper, weekly readings, and class discussions, students will develop their own perspective on trustworthy agentic systems. This will take the form of a position paper. Unlike a survey or summary, this paper must present and defend a clear argument.
• Mid-term abstract – By the middle of the semester, students will submit a 1-page abstract outlining their intended argument.
• Final paper and presentation – Students will submit their final position paper and present it in a conference-style session at the end of the term.

 

Topics

We will cover a set of core issues that define the emerging field of trustworthy agentic systems:

1. Defining Trust in AI
2. Agency vs. Tool Use
3. Uncertainty and Robustness
4. Interpretability and Transparency
5. Multi-Agent Safety / Cross Agent Trust Networks
6. Security by Design for Agentic Systems
7. Human-Agent Collaboration and Oversight
8. Trust in Browser and Web Agents
9. Long-term agent memory
10. Deployment and Monitoring of Agents
11. Societal Implications of Agentic AI

 

Schedule

The seminar will take place every Tuesday from 2.15–3.45 pm (room TBA). Everyone must attend!

• Oct 21: Kick-off – Introduction to the seminar and overview of topics
• Oct 28: Topic assignment and initial discussion
• Nov 4 onwards: Weekly student paper presentations and discussions
• Mid-semester: Abstract submission deadline
• Final weeks: Position paper presentations in a conference-style format

 

Assessment

• Weekly paper presentation: 20%
• Participation in weekly discussions: 15%
• Mid-term abstract submission: 15%
• Final position paper: 35%
• Final presentation: 15%