Basic Information

  • Course Code: EE 6106
  • Course Name: Online Learning and Optimisation
  • Course Offered In: 2023-2024
  • Semester Season: Spring
  • Instructors: Prof. Jayakrishnan Nair
  • Prerequisites: Into Probability Course (EE325/EE601)
  • Difficulty (1 being easy and 5 being tough): 4

Course Content

Part 0: Online learning in adversarial settings • Learning from experts • Bandit setting

Part 1: Stochastic bandits • Regret minimization • Information theoretic bounds • Pure exploration • Bayesian setting

Part 2: Markov decision processes and RL • Basics of MDPs • Reinforcement learning algorithms

Feedback on Lectures

Lectures were very detailed and prof used to write everything on board so you have to take notes. Sometimes he used to left some part of proof as exercise so try to do that on the same day. You need to revise the previous class before coming to next class since content was very connected and lectures used to be on Tuesday and Friday so there was a big gap. Attending classes is a must even though there is no compulsory attendance you will not be able to understand properly if you miss the classes.

Feedback on Evaluations

  1. Homeworks (20%) 3 Assignments were given, exact weightage of each assignment was not shared. Assignments were slightly on the tougher side where you might need to have a look at the book for the solutions. 1 week of time were given for each assignment

  2. Midsem (30%) and Endsem (45%) Exams would be relatively easy if you have attended the classes and done the assignments because most of the questions would be a similar proof discussed in class with just a slight variation so understanding the notes are important

  3. Class participation (5%) This 5% was slightly vague as there was nothing to judge about class participation. Attendance was not taken so mostly this 5% was included in endsem only to make it 50%

For grades you can refer the asc site, but this is not the course if you are looking for an easy grade

Study Material and Resources

There were no slides in the course, Prof used to write on board and you will have to take notes. Try not to miss the lectures and if you do miss study the notes of that class before coming to class next day

Recommended Book : https://tor-lattimore.com/downloads/book/book.pdf

Follow-up Courses

There is no direct follow up course for this however if you like probability, proper mathematics and proofs you can take Stochastic Optimization (EE736) by Prof. Vivek Borkar

Final Takeaway

Being regular in class and doing the assignments should be enough to get a good score since exams mostly depend on this, if you don’t like probability or mathematical proofs I would recommend not to take this course as the whole course revolves around that