ESI 6213 Stochastic Decision Models I

Objective: Get exposed to the theory of stochastic processes and build foundations for their decision support applications in engineering, healthcare, and finance.

Text:Introduction to Probability Models, S. Ross, 8-th ed. or later.

Topics

I. Review of probability theory
(elements of set theory; events & probability spaces; Bayesian concepts; statistical independence; random variables, vectors, & functions; conditional probability & expectation; distributions & transformations)

II. Discrete-Time Markov processes
(Markovian property; Chapman-Kolmogorov equations; classification of states; limiting probabilities; applications)

III. Poisson processes
(counting processes; properties of Poisson processes; non-homogeneous & compound Poisson processes; applications)

IV. Continuous-Time Markov processes
(birth & death processes; transition probability function; limiting probabilities; applications)

V. Renewal theory
(limit theorems; renewal reward processes; regenerative processes; applications)