Basic Information
- Course Code: CS 726
- Course Name: Advanced Machine Learning
- Course Offered In: 2022-23
- Semester Season: Spring
- Instructors: Prof. Sunita Sarawagi
- Prerequisites: The course is open to PhD, Masters, DD and BTech students provided they have taken an introductory course in IITB in machine learning (CS 725 or CS 337 or CS 419 or DS 303) and obtained at least a BC grade in it. (refer: https://www.cse.iitb.ac.in/~sunita/cs726/)
- Difficulty (1 being easy and 5 being tough): 4
Course Content
The initial part of the course deals with graphical models where we look at methods for representing conditional independencies between variables to arrive at a factorised representation for the joint density that can be learnt and queried efficiently. This lays the foundation for all topics to follow where we pose all learning problems as queries on this joint density and discuss generative models like GANs, diffusion models, normalizing flows, gaussian processes and variational autoencoders. This is the point where you start to appreciate the unifying power of graphical models to understand what seemed like completely different deep learning models, and I believe this was one of the best outcomes from the course for me. We then dive into sampling techniques like Gibbs sampling and Markov-Chain Monte Carlo. Once your generative model has been trained, this part lays the foundation for how to query it efficiently. Next, we discussed causal models which accounts for additional real-world information while modelling the dependencies in graphical models. This is closely tied to the professor’s research and is a very new and evolving field. Much of the course content draws from very recent topics and given the rapid pace of deep learning research you may encounter different topics from those mentioned here. There is a lot more focus on theory and software implementation is relegated to few assignments and the end-term project. After doing this course you will be well prepared to understand a wide-range of deep learning papers.
Feedback on Lectures
Very well structured lectures that have mathematically dense concepts in the initial half on graphical models. The professor often takes few lectures in between just to address doubts. In some lectures the professor was also willing to discuss extensions of certain topics based on interest from the class.
Feedback on Evaluations
There were coding assignments which required us to implement a diffusion model from a given template code. This made deep learning coding more accessible for me. The assignments are fun but need time. The midsem and endsem were challenging but most problem-solving techniques could be understood well from the homework problems. The end-term project had a lot of flexibility in what we chose to implement, and there was a focus on trying new things.
Study Material and Resources
For the first half semester you only need the book: Probabilistic Graphical Models: Principles and Techniques, by Daphne Koller and Nir Friedman For the second half semester topics are drawn from various papers so you need to be regular in class.
Follow-up Courses
This course complements all the other machine learning courses in the institute. While many other courses focus on specific models, this course address some core unifying principles that help you see the big picture.
Final Takeaway
It is a challenging but highly rewarding experience. I would highly recommend it if you enjoy probability and statistics and are curious about deep learning.