Basic Information

  • Course Code: SC607
  • Course Name: Optimisation
  • Course Offered In: 2023-2024
  • Semester Season: Spring
  • Instructors: Prof. Avishek Ghosh
  • Prerequisites: There are no formal prerequisites, but familiarity with the following topics is needed. • Probability • Linear algebra • Multivariable calculus
  • Difficulty (1 being easy and 5 being tough): 4

Course Content

Part I: Formulation / Modelling (Analysis)

• Structured Optimisation (Convexity) • Convex set • Convex functions • Convex optimisation problems examples

  1. Linear programming
  2. Quadratic programming • Necessary and sufficient conditions for optimality (both constrained and unconstrained) • Duality

Part II: Solution / Algorithms (Convergence)

• Unconstrained problem

  1. First order algorithms ex. Gradient descent and variants
  2. Second order algorithms ex. Newton method • Constrained optimisation
  3. Projected gradient and its variants
  4. Interior-point methods (by Y. Nesteron 1980) • Linear program
  5. Simplex method • Advanced constrained algorithms
  6. Augmented Lagrangian
  7. Dual ascent
  8. Cutting plane method

Feedback on Lectures

The lectures were extremely well structured. Professor would come with a detailed well-prepared notes for the lecture and relevant concepts, proofs and examples were meticulously explained on the board. Lectures were not very heavy and the instructor gave the students enough time to absorb the concepts. The instructor was very receptive to doubts. Professor would also give some HOTS questions in class to solve at home so that students get more comfortable in applying the theory taught in class. Although attendance was not mandatory, attending all lectures is highly recommended.

Feedback on Evaluations

2 HWs (30%), 1 mid term (25%), Final (35%), Scribe (10%) Lecture notes:

As a part of the course, each of us was required to volunteer for taking notes of one lecture as per the prescribed format. Before endsem, all the lecture notes were made available for reference.

Assignments:

The assignments played a very crucial part in the learning process. Each problem was an illustration of some important concept taught in the class. The difficulty was moderate and on occasions, the assignments were quite lengthy. Leaving the assignments for the last moment is discouraged.

Examination:

The midsem was relatively more tougher as compared to the endsem. Almost 70-80% of the paper was formed by slightly modifying the assignment problems and HOTS questions given in class. Remaining 20% of the paper consisted of questions which required tedious calculations and strong hold on the concepts to reach to the final answer.

Grading was moderate and sufficient marks were provided for the effort shown in the examination.

Study Material and Resources

• Convex Optimization; S Boyd and L Vandenberghe • Numerical Optimization; Nocedal and Wright • Optimization for Data Analysis; S. Wright and Ben. Recht (Rough draft available below) https://people.eecs.berkeley.edu/˜brecht/opt4ml_book/ • Lectures on Convex Optimization; Y. Nesterov

Follow-up Courses

Robotics, structural optimization (Shape and Topology Optimization), Optimal controls

Final Takeaway

It’s needless to say that the skills learnt in this course would definitely come in handy in Electrical Engineering and allied fields. For someone planning to pursue AI-ML field or robotics control, this is a must-do course. But one should keep in mind that it is a very math-extensive course filled with rigorous and non-intuitive proofs. Hence, to ace the course and grasp the concepts, regular efforts are required.