Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: M:\public_html\accessories\aau_logo.gif


Syllabus

Exercises 

Resources


Ph.D. School homepage

Ph.D. courses

 

Machine Learning (Ph.D. Course), Spring 2011

Zheng-Hua Tan, Associate Professor, Ph.D.

+45 9940-8686, zt@es.aau.dk, http://kom.aau.dk/~zt

Office: Room A6-319, Niels Jernes Vej 12, Aalborg University, Denmark

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: M:\public_html\accessories\movline.gif

Machine learning is concerned with the development of computer programs that allow computer (or machine) to learn from examples or experiences. Machine learning is of interdisciplinary nature, with roots in computer science, statistics and pattern recognition. In the past decade, this field has witnessed rapid theoretical advances and growing real-world applications. Successful applications include machine perception (speech recognition, computer vision), control (robotics), data mining, web search and text classification, time-series prediction, system modelling, bioinformatics, data compression, and many more.

 

This course will give a comprehensive introduction to machine learning both by presenting technologies proven valuable and by addressing specific problems such as pattern recognition and data mining. This course covers both theory and practices for machine learning, but with an emphasis on the practical side namely how to effectively apply machine learning to a variety of problems. Topics will include

·    Supervised learning (of classification and regression functions)

K-nearest neighbors, decision trees, naïve Bayes, support vector machines, logistic regression, evolutionary algorithms, Bayesian Networks, hidden Markov model, neural networks, boosting

·    Unsupervised learning and clustering

K-means, hierarchical clustering (agglomerative and divisive), principal component analysis, independent component analysis, Expectation Maximization algorithm

·    Reinforcement learning

 

Prerequisites: Basic probability and statistics theory, linear algebra.

ECTS: 4

Time: May 23, 24, 26, 31, June 7, 9, 2011 (each day 09:15-16:15)

Place: Niels Jernes Vej 14, Room 4-111 (the video conference auditorium), 9220 Aalborg. After 14:00 Room 4-111 and Room 3-119 will be available for exercise.

 

Information: This course consists of five-day lectures. Students are highly encouraged to do mini projects, either presented in the course or related to their own PhD research, and hand in short reports by the end of February. Coffee and bread will be served at 10:00 in the morning; coffee and cake at 14:00 in the afternoon.


Note: The schedule is indicative and subject to change, and reading is optional. 

 

DAY 1

Lecture 1: Introduction (slides)

Readings: Chapters 1 and 2 of Alpaydin’s book; or Chapter 1 of Bishop's book.


Lecture 2: Bayesian decision theory (incl. probability theory) (
slides)
Readings: Chapter 3 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 3: Parametric methods (ML, MAP & Bayesian learning) (slides)
Readings: Chapters 4 and 5 of Alpaydin’s book, or similar chapters in other books.

 

Exercises for DAY1: download dataset1_noisy and Netlab toolbox, and do Exercise1.  

 

DAY 2

Lecture 4: Dimensional reduction (slides)
Readings: Chapter 6 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 5: Clustering (slides)
Readings: Chapter 7 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 6: Nonparametric methods (Parzen windows and K-NN) (slides)
Readings: Chapter 8 of Alpaydin’s book, or similar chapters in other books.

 

Exercises for DAY2: download full dataset, which is a Matlab format of THE MNIST DATABASE of handwritten digits by Yann LeCun, and Corinna Cortes, and do Exercise2:  (1) from the 10-class database, choose three classes (5, 6 and 8) and then reduce dimension to 2; (2) perform 3-class classification based on the generated 2-dimensional data. You may want to use eigdec.m and pca.m in Netlab toolbox and the LDA code.

 

DAY 3

Lecture 7: Linear discrimination (slides)
Readings: Chapter 10 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 8: Support vector machines (slides)
Readings: Chapter 10 of Alpaydin’s book, or similar chapters in other books.

 

Exercises for DAY3: perform classification for the entire dataset based on the algorithms introduced (using LDA for dimensionality reduction). As an option, you can perform the classification by using LIBSVM -- A Library for Support Vector Machines. 

 

DAY 4

Lecture 9: Multilayer perceptrons and evolutionary computation (slides)
Readings: Chapter 11 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 10: Time series models (slides)
Readings: Chapter 13 of Alpaydin’s book, or similar chapters in other books.

 

Exercises for DAY4: develop an MLP for the MNIST database by using the dimension-reduced data from your work on DAY 2 and DAY 3. You can download the LDA projected data here. Further, you can use 10-, 20- and 30-dimensional data generated by PCA and compare their performance (at the same time, try various MLP architectures). Functions for MLP in the NETLAB toolbox include mlp.m, mlptrain.m and mlpfwd.m.

 

DAY 5

Lecture 11: Graphical models (Introduction, main slides, main slides - commented)

Readings: Chapter 8 of Bishop’s book.

 

Exercises for DAY5: choose your own images and apply Markov random field for de-noising by using Matlab code. Optionally, you can play with Bayesian Net Toolbox. 

 

DAY 6

Lecture 12: Algorithm-independent machine learning (slides)
Readings: Chapters 2, 9, 14 and15 of Alpaydin’s book, or similar chapters in other books.

 

Lecture 13: Reinforcement learning (slides)
Readings: Chapter 16 of Alpaydin’s book, or similar chapters in other books.

 

Wrap-up.

Exercises for DAY6: implement AdaBoost for the MNIST database or improve the system that you have developed by choosing algorithms you like. A tutorial on AdaBoost is available here.


Literature:

Introduction to Machine Learning, Ethem Alpaydin, The MIT Press, USA, October 2004,

Pattern Recognition and Machine Learning, Chris Bishop, Springer, UK, 2006

Pattern Classification, Second Edition, Richard O. Duda, Peter E. Hart, David G. Stork, Wiley Interscience, USA, 2001.

 

Instructor:

Zheng-Hua Tan, Department of Electronic Systems, Aalborg University, Denmark