Martin Haugh

Martin Haugh

Associate Professor of Analytics and Operations Research at Imperial College Business School

IEOR E4525: Machine Learning for OR & FE

I last taught this advanced-level MS course in spring 2017 in the IE&OR Department at Columbia University. It’s an elective course for the MS in Financial Engineering and MS in Operations Research programs at Columbia. Because the selection of topics varied over the years there is considerably more material here than could be covered in a single course. Rather than identifying what topics (or subsets of topics) were covered each year, I have simply provided a list of topics that were covered in some version of the course. A syllabus and description of the course logistics from spring 2017 (when I co-taught the course with Garud Iyengar) can be found here. I’m also grateful to the excellent textbooks of (1) James, Witten, Hastie & Tibshirani (2) David Barber and (3) Christopher Bishop. Many of the figures in the slides below were taken from these sources.

Lecture Slides (and Occasional Notes)

  1. Very Brief Introduction to Machine Learning
  2. Regression I (Linear regression, bias-variance decomposition)
  3. Classification I (k-NN, Naïve Bayes, LDA & QDA, logistic regression, optimal Bayes classifier, reduced-rank LDA)
  4. Resampling Methods (Bootstrap; cross-validation)
  5. Regression II (Subset selection, ridge regression, Lasso etc.)
  6. Classification II (Classification & Regression Trees, Bagging, Random Forests & Boosting)
  7. Clustering
  8. An Introduction to Causality
  9. Support Vector Machines
  10. Kernels & the Kernel Trick (including reproducing kernel Hilbert spaces (RKHS))
  11. The EM Algorithm (Notes and slides)
  12. Dimension Reduction Methods (PCA, kernel PCA, recommender systems and matrix factorization, PageRank)
  13. Hidden Markov Models (HMMs)
  14. Bayesian Models and MCMC (Notes and slides from my Monte-Carlo Simulation course)
    An extended tutorial on MCMC and Bayesian Modeling that grew out of these notes can be found here.
  15. Introduction to Graphical Models (Directed acyclic graphs (DAGs) and Markov random fields)
  16. Variational Inference (KL divergence, variational Bayes, expectation propagation)