|
Syllabus Exercises |
|
|
|
Machine Learning (Ph.D. Course), Fall 2009 http://adm.aau.dk/fak-tekn/phd/kurser/s4_1.htm Zheng-Hua Tan, Associate Professor, Ph.D. +45 9635-8686, zt@es.aau.dk, http://kom.aau.dk/~zt Office:
Room A6-319, Niels Jernes Vej 12
|
||
|
Machine learning
is concerned with the development of computer programs that allow computer
(or machine) to learn from examples or experiences. Machine learning is of
interdisciplinary nature, with roots in computer science, statistics and
pattern recognition. In the past decade, this field has witnessed rapid
theoretical advances and growing real-world applications. Successful
applications include machine perception (speech recognition, computer vision),
control (robotics), data mining, web search and text classification,
time-series prediction, system modelling, bioinformatics, data compression,
and many more. This course will
give a comprehensive introduction to machine learning both by presenting
technologies proven valuable and by addressing specific problems such as
pattern recognition and data mining. This course covers both theory and
practices for machine learning, but with an emphasis on the practical side
namely how to effectively apply machine learning to a variety of problems.
Topics will include ·
Supervised learning (of classification and regression functions) K-nearest
neighbors, decision trees, naïve Bayes, support vector machines, logistic
regression, evolutionary algorithms, Bayesian Networks, hidden Markov model,
neural networks, boosting ·
Unsupervised learning and clustering K-means,
hierarchical clustering (agglomerative and divisive), principal component analysis,
independent component analysis, Expectation Maximization algorithm ·
Reinforcement learning Prerequisites: Basic
probability and statistics theory, linear algebra. Time: November 30, December 2,4,9 and 11, 2009 (each day 09:15-16:15) Place: Niels Jernes Vej 14, Room 4-117, 9220
Aalborg Information: This course consists of five-day lectures. Students are highly encouraged to do mini projects, either presented in the course or related to their own PhD research, and hand in short reports by the end of February. Coffee and bread will be served at 10:00 in the morning; coffee and cake at 14:00 in the afternoon. Note: The schedule is indicative and subject to change, and reading is optional.
DAY 1 Lecture 1: Introduction (slides) Readings: Chapter 1, 2 of the assigned textbook; or Chapter 1 of Bishop's book.
Lecture 3: Parametric
methods (ML, MAP & Bayesian learning) (slides)
Exercises for DAY1: download dataset1_noisy and Netlab toolbox, and do Exercise1.
DAY 2 Lecture 4: Dimensional
reduction (slides)
Lecture 5: Clustering
(slides)
Lecture 6: Nonparametric
methods (Parzen windows and K-NN) (slides)
Exercises for DAY2: download full dataset, which is a Matlab format of THE MNIST DATABASE of handwritten digits by Yann LeCun, and Corinna Cortes, and do Exercise2: (1) from the 10-class database, choose three classes (5, 6 and 8) and then reduce dimension to 2; (2) perform 3-class classification based on the generated 2-dimensional data. You may want to use the LDA code.
DAY 3 Lecture 7: Linear
discrimination (slides)
Lecture 8: Support
vector machines (slides)
Exercises for DAY3: perform classification for the entire dataset based on the algorithms introduced (using LDA for dimensionality reduction).
DAY 4 Lecture 9: Multilayer
perceptrons and evolutionary computation (slides)
Lecture 10: Time
series models (slides)
Exercises for DAY4: develop an MLP for the MNIST database. It is beneficial to use the dimension-reduced data from your work on DAY 2 and DAY 3. Alternatively, you can download the LDA projected data here. You are encouraged to use the NETLAB toolbox (in particular, functions such as mlp, mlptrain, mlpfwd).
DAY 5 Lecture 11: Algorithm-independent
machine learning (slides)
Lecture 12: Reinforcement
learning (slides,
combined with Lecture 11)
Wrap-up. Exercises for DAY5: implement Adaboost for the MNIST database or imporve the system that you have developed by choosing algorithms you like.
Introduction to Machine
Learning, Ethem Alpaydin, The MIT Press, October 2004, References: Pattern Recognition and Machine Learning, Chris Bishop, Springer, 2006 Pattern Classification,
Second Edition, Richard O. Duda, Peter E. Hart, David G. Stork, Wiley
Interscience, 2001. Machine Learning, Tom Mitchell, McGraw Hill, 1997.
Instructor:
|
