HKUST

Math4432: Statistical Machine Learning (统计学习)
Spring 2018, HKUST

Course Information

Synopsis (摘要)

This course is open to graduates and senior undergraduates in applied mathematics, statistics, and engineering who are interested in learning from data. It covers hot topics in statistical learning, also known as machine learning, featured with various in-class projects in computer vision, pattern recognition, computational advertisement, bioinformatics, and social networks, etc. An emphasis this year is on deep learning with convolutional neural networks.
Prerequisite: linear algebra, basic probability and multivariate statistics, convex optimization; familiarity with R, Matlab, and/or Python, Torch for deep learning, etc.

Reference (参考教材)

An Introduction to Statistical Learning, with applications in R. By James, Witten, Hastie, and Tibshirani

ISLR-python, By Jordi Warmenhoven.

ISLR-Python: Labs and Applied, by Matt Caudill.

The Elements of Statistical Learning. 2nd Ed. By Hastie, Tibshirani, and Friedman

statlearning-notebooks, by Sujit Pal, Python implementations of the R labs for the StatLearning: Statistical Learning online course from Stanford taught by Profs Trevor Hastie and Rob Tibshirani.

Instructors:

Yuan Yao

Time and Venue:

TuTh 4:30-5:50pm
Rm4504 (Lift 25/26), Academic Bldg
Piazza discussion forum: sign-up link

Homework and Projects:

Weekly homeworks, monthly mini-projects, and a final major project. No final exam. For 3-project plan, homework and projects will be counted in grading by 20-20-20-40 in percentage.

Grading scheme: [ description ]

Teaching Assistant (助教):

Mr. ZHU, Weizhi, Email: statml.hw (add "AT gmail DOT com" afterwards)

Tutorial Material

Schedule (时间表)

Date Topic Instructor Scriber
02/01/2018, Thu Lecture 01: Introduction and Overview [ Lecture01.pdf ]
Y.Y.
02/06/2018, Thu Lecture 02: Linear Regression [ Lecture02.pdf ] : the slides may be slightly above that of ISLR
    [Homework]:
  • ISLR (Print7), Chapter 2: 1, 3, 8, 10. Deadline: Feb 20, 2018. Please submit your homework to the Email address above (statml.hw) before class, including source codes (or link) if necessary. Mark on the head of your homework: "Math4432: Name - Student ID".
 Y.Y.
02/08/2018, Thu Lecture 03: Linear Regression B [ Lecture03.pdf ]
    [Homework]:
  • ISLR (Print7), Chapter 3: 1, 5, 8, 9. Deadline: Feb 20, 2018. Please submit your homework to the Email address above (statml.hw) before class, including source codes (or link) if necessary. Mark on the head of your homework: "Math4432: Name - Student ID".
 Y.Y.
02/13/2018, Tue Lecture 04: Linear Classification A: Logistic Regression [ Lecture04 ] : logistic regression
Prof. Can YANG
02/15/2018, Thu Lecture will be rescheduled to another date, to be announced later
 Y.Y.
02/20/2018, Tue Lecture 05: Linear Classification B: LDA, QDA etc. [ Lecture05 ]
    [Homework]:
  • ISLR (Print7), Chapter 4: 1-3, 10, 11, and Bonus question 4* (You don't need to work on it; but if you work on it, bonus credit will be given to you). Deadline: Feb 27, 2018. Please submit your homework to the Email address above (statml.hw) before class, including source codes (or link) if necessary. Mark on the head of your homework: "Math4432: Name - Student ID".
 Y.Y.
02/22/2018, Thu Lecture 06: Resampling A: Cross-Validation [ slides ]
Y.Y.
02/27/2018, Thu Lecture 07: Resampling B: Bootstrap [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 5: 1, 2, 5, 6, and Bonus question 8* (You don't need to work on it; but if you work on it, bonus credit will be given to you). Deadline: Mar 6, 2018. Please submit your homework to the Email address above (statml.hw) before class, including source codes (or link) if necessary. Mark on the head of your homework: "Math4432: Name - Student ID".
 Y.Y.
03/01/2018, Thu Lecture 08: Mini-Project 1: A Warmup

Y.Y.
03/06/2018, Tue Lecture 09: Linear Model Selection: Subset/Forward/Backward selection, adjusted R-square, AIC, and BIC [ slides ]
Y.Y.
03/08/2018, Thu Lecture 10: Linear Model Selection: Ridge and Lasso [ slides ]
Y.Y.
03/13/2018, Tue Lecture 11: Linear Model Selection: Principal Component Regression and Partial Least Squares [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 6: Problems 1, 2, 3, 4, 7, 9; *Bonus question (optional): ESL (Print 10), Exercise 3.27. Deadline: Mar 20, 2018. Please submit your homework to the Email address above (statml.hw) before class, including source codes (or link) if necessary. Mark on the head of your homework: "Math4432: Name - Student ID".
Y.Y.
03/15/2018, Thu Lecture 12: Moving beyond linearity I [ slides ]
Prof. Can YANG
03/20/2018, Tue Lecture 13: Moving beyond linearity II [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 7: exercise 1, 2, 3, 5, 6, 7, 10 in Section 7.9. Mark on the head of your homework: "Math4432: Name - Student ID".
 Y.Y.
03/22/2018, Thu Lecture 14: Tree-based Methods: Classification and Regression Trees (CART) [ slides ]
Y.Y.
03/27/2018, Tue Lecture 15: Tree-based Methods: Bagging, Random Forests, and Boosting [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 8: Exercises 3, 4, 7, 8, 9, 10 of Section 8.4. Deadline: Apr 10, 2018. Mark on the head of your homework: "Math4432: Name - Student ID".
Y.Y.
03/29/2018, Thu Lecture 16: Project 2: Midterm, due: April 12 11:59pm, 2018.
    [Mini-Project 2]
  • Project description: [ pdf ].
  • Collection of student reports [ Github ]
  • Doodle voting: please input your top 3 favorite reports, excluding your own team [ Vote ]
 Y.Y.
04/10/2018, Tue Lecture 17: Support Vector Machines I. [ slides ] Y.Y.
04/12/2018, Thu Lecture 18: Support Vector Machines II. [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 9: Exercises 3, 5, 7 of Section 9.7. Deadline: Apr 19, 2018. Mark on the head of your homework: "Math4432: Name - Student ID".
Y.Y.
04/17/2018, Tue Lecture 19: Unsupervised Learning I: PCA. [ slides ] Y.Y.
04/19/2018, Thu Lecture 20: Unsupervised Learning II: K-means and Hierarchical Clustering. [ slides ]
    [Homework]:
  • ISLR (Print7), Chapter 10: Exercises 1-3 and 10 of Section 10.7. Deadline: Apr 26, 2018. Mark on the head of your homework: "Math4432: Name - Student ID".
Y.Y.
04/24/2018, Tue Lecture 21: An Introduction to Deep Learning I: Perceptrons, Neural Networks, CNNs. [ slides ]. Y.Y.
04/26/2018, Thu Lecture 22: An Introduction to Empirical Bayes. [ slides ] . Prof. Can YANG
Y.Y.
05/03/2018, Thu Lecture 23: An Introduction to Deep Learning II: Transfer Learning, Recurrent Neural Networks, LSTM, and Reinforcement Learning. [ slides ].
Y.Y.
05/08/2018, Thu Lecture 24: Final Project [ project3.pdf ].
Gijs Bruining
Y.Y.
by YAO, Yuan.