Basic Information

  • Course Code: CS 726
  • Course Name: Advanced Machine Learning
  • Course Offered In: 2023-2024
  • Semester Season: Spring
  • Instructors: Prof. Sunita Sarawagi
  • Prerequisites: Formal introductory ML courses like CS 725, CS 337, CS 419 or DS 303. Although not a formal requirement, EE782 - Advanced Topics in Machine Learning, while sounding similar, focuses more on specific model architectures and implementations, which complements this course well.
  • Difficulty (1 being easy and 5 being tough): 4

Course Content

First half: Learning and Modelling distributions using probabilistic graphical models (directed and undirected), inference algorithms like junction trees, etc. and applications to Deep Generative Models: GANs, Diffusion Models, Variational Autoencoders, and Normalizing Flows. Second Half: Markov Chain Monte Carlo sampling methods like Gibbs and Langevin, Bayesian Optimisation and Causal Inference. The second half of the syllabus is more variable and covers recent machine-learning research topics. Also, the instructor is open to new topics based on suggestions from students.

Feedback on Lectures

The lectures are very well structured but cover dense concepts, so one needs to be active in class and ask questions, which the instructor also appreciates. Some lectures are taken by guest researchers and students demonstrating applications of relevant topics. Attending all the lectures is a must; It can become challenging to keep up with the content otherwise.

Feedback on Evaluations

Weekly Quizzes(15%), Assignments(10%), Mid-semester exam(25%), End semester exam(35%), Project(15%)

Weekly quizzes cover the content from the previous week. Assignments include old-fashioned problem-solving or coding. Mid-semester and End-semester exams are open-handwritten-notes. Exams can be lengthy and challenging, which results in low averages, but relative grading helps. Questions from guest lectures are also included in exams. Final Project is very flexible, and students are encouraged to pick up recent topics; evaluation consists of a theory presentation, which the instructor also attends and a code review, which is done by TAs.

Study Material and Resources

  1. Probabilistic Graphical Models: Principles and Techniques, by Daphne Koller and Nir Friedman
  2. Brady Neal’s Causal Inference Course Lectures: https://youtu.be/CfzO4IEMVUk?si=O_HmmWK9Is54gbzr

The PGM book is relevant for the first half of the course, but the notation slightly differs from what the instructor uses. For other topics, paper references are provided. I used Brady Neal’s lectures to understand some parts of Causal Inference.

Follow-up Courses

Since the course content is very general and complements other courses like Automatic Speech Recognition and Organisation of Web Information really well.

Final Takeaway

Overall, the course content is very principled and mathematically involved. The course requires effort to keep up, but it can be very rewarding and trains you well to read and understand the latest research in ML. The instructor is approachable and will help you with projects and doubts, so I encourage you to attend office hours. Personally this course encouraged me to take up research as well.