IE 601 – OPTIMIZATION TECHNIQUES

Course offered in:

Fall 2018

Prerequisites: Exposure to relevant concepts at an undergraduate level and instructor consent. Basic knowledge of linear algebra and calculus is useful.

Motivation: Applications of Optimization theory come up in various engineering disciplines. For instance, various problems in machine learning, image processing, and even finance are essentially optimization problems at their core and hence it’s important to be well versed with the basic ideas of optimization. The aim of this course is to have some basic understanding of provably convergent computational schemes for R^n constrained optimization problems. The course also covers the theory of linear programming in great detail, which is a widely applicable technique in solving real-life optimization problems.

Course Content: Linear Programming and primitives: Convex sets, Weirstrauss Theorem, Fundamental theorem of linear programming, Degenerate solutions, Duality, Complementary slackness conditions.

Non-linear programming: First and second order conditions. Iterative methods and associated issues. Line search methods: Stationarity of limit points of steepest descent, successive step-size reduction algorithms, etc.

Hessian-based algorithms: Newton, Conjugate directions and Quasi-Newton methods, Constrained optimization problems: Lagrange variables, Fritz-John and Karush-Kuhn-Tucker conditions.

Feedback on Lectures:

The lectures were quite slow-paced and easy to follow, but not enough content was covered in a lecture. The professor also skipped quite a few classes and did not make up for them later which affected the amount of material covered in this offering of the course. He encouraged doubts and discussions in the class. Weekly tutorials were also conducted, which focused primarily on solving problems and introducing new concepts through them. The content covered in the tutorial was not a major component of exams.

Feedback on Tutorials, Assignments and Exams: There were 2 quizzes of 7.5% weightage each. The assignments had a weightage of 20%, the midsem 30% and the endsem was 35%. There were three theory assignments, which were moderately difficult. Some portion of the exams (midsem and endsem) was based on the assignments, hence it was very important to solve the assignments thoroughly to do well in the course.

Difficulty:

The course covered limited content, and that too on a basic level. The proofs were a bit daunting at times, but the course was not very diverse or conceptually demanding. The exams were doable and not very difficult.

Study Material and References:

D. Bertsekas Nonlinear programming, 2nd Edition, Athena Scientific, 1999, Nashua.

Review by – Chinmay Talegaonkar.