IE 611 – INTRODUCTION TO STOCHASTIC MODELS

Course offered in:

Fall 2018

Course Instructor: Prof. Manjesh K. Hanawal

Prerequisites: This course doesn’t require any hard and fast prerequisites but a basic course in probability helps although the prof did repeat the entire thing from the basics.

Motivation: This course deals with the theory of probability, renewal processes, reward and cost models, cumulative processes, poisson process and stopping times, regenerative processes, renewal theorems, discrete and continuous time Markov chains, communication classes and class properties, irreducible, and positive recurrent chains, costs and rewards for ergodic chains, transient behaviour.

Course Content: *Probability basics review *Strong Law of large numbers. *Introduction to stochastic process; sample paths and finite dimensional distributions *Kolmogorov’s consistency conditions. *Renewal processes, reward and cost models, cumulative processes, Poisson process and stopping times. *Regenerative processes, relation between a time average and mean of limiting distribution, Wald’s equation, renewal equation, renewal theorems, Markov’s Renewal Theory *Discrete time Markov chains: Hitting Times and Recurrence (First Passage Time Distribution, Number of Returns to a State), Communicating Classes and Class Properties, Positive Recurrence and the Invariant Probability Vector, Transience, The Discrete Time M/M/1 Queue, Mean Drift Criteria *Continuous time Markov Chains: Transition Probability Function, Sojourn Time in a State, Structure of a Pure Jump CTMC, Regular CTMC, Communicating Classes, Recurrence and Positivity, Birth and Death Processes, Differential Equations for transition probabilities, *Irreducible, and positive and null recurrent chains, costs and rewards for ergodic chains, transient behaviour

Feedback on Lectures:

Lectures were conducted with the prof writing down content on the white board and students copying, with it not readable sometimes as the font was too small. The pace of the lectures were slow and initial probability concepts are all done in a previous core course like EE223.

Feedback on Tutorials, Assignments and Exams: Tutorials were conducted to discuss problems weekly as posted on Moodle by the instructor, 2-3 days in advance. There wasn’t compulsory attendance even though it is an 8 credit course (3 hours lectures and 1 hour tutorial.) But they helped in understanding the course. No assignments were given.

Exams comprised of 4 quizzes (10% each) which at the end of the course prof turned to best 3 of 4 amounting to a total of 40% weightage, midsem (25%), endsem (35%). Exams were mostly based on tutorial discussions. For midsem and endsem, class note-book was allowed.

Difficulty:

Easy-Medium

Grading: Lenient although getting AA was difficult

Study Material and References:

Class notes

Discrete Event Stochastic Processes: Lecture Notes for an Engineering Curriculum: Anurag Kumar, Department of Electrical Communication Engineering, Institute of Science Review by – Dimple Kochardimplekochar99@gmail.com