Basic Information

  • Course Code: SC646
  • Course Name: Distributed Optimization and Machine Learning
  • Course Offered In: Spring 2022
  • Instructors: Prof. Mayank Baranwal
  • Prerequisites: No hard prerequisites, having some idea about system theory is recommended (EE302 will suffice). However, professor covers them briefly.
  • Difficulty (on a scale of 5): 3

Course Content

The course starts with a decently rigorous presentation of convexity and is followed with accelerated methods for gradient descent. Several results are proven in the class and hence get a little mathematically involved and hard to follow. Then a quick introduction to Lyapunov theory is made, it could be a little hard to follow if seeing for the first time. After which, basics of graph theory is taught which is easy to follow. And the course then culminates with different algorithms for distributed optimization. An emphasis on finite time convergence algorithms were made throughout the course. From the website:

  • Review of convex optimization:

    • Properties of convex functions
    • Iterative algorithms for constrained and unconstrained optimization
    • First and second-order methods
    • Dynamical systems viewpoint
  • Review of control theory:
    • Lyapunov theory
    • Single and double-integrator systems
  • Review of graph theory:
    • Elements of graph theory
    • Network topology
    • Multi-agent systems
  • Distributed optimization problem:
    • Problem formulation
    • Distributed consensus
    • Algorithms for distributed optimization
    • Dynamical Systems viewpoint
    • Robustness

Feedback on Lectures

The lectures were decently paced and focused on the mathematical tools used to guarantee convergence in different notions. The professor and TAs were very approachable and the content to be reviewed was decided by a show of hands at times. In the beginning of each lecture a short review of the previous lecture and an overview of that day’s lecture was made which made keeping track of the content easier.

Feedback on Evaluations

There were 3 homework assignments, one paper presentation and one 30 min end semester exam. The homework assignments were not difficult to solve as ideas to use were discussed in the class and hints were provided in the lectures. A bunch of papers were given to us and were asked to make a 15min presentation in groups of not more than 3 in front of the class. The end semester paper had fairly basic and conceptual questions.

Follow-up Courses

None

Final Takeaway

Good course, a bit mathematical. Recommended for anyone who wants a formal treatment of machine learning and wants to work on multiagent control. Introduces to recent works in finite time convergence as well.