Assignments
Assignment 3 - Hadoop in Machine Learning
This assignment is optional. It will not count towards your grade.
Instructions
Resources
Description
Datasets
MNIST Encoder
The encoder has 800 outputs. The architecture is Linear → SoftPlus. It was trained using predictive sparse decomposition, after the original data was filtered through a local contrast normalization with a 5×5 Gaussian window with standard deviation 0.25.
To load the encoder into Torch, please specify the format as “ascii”. For example:
encoder = torch.load("encoder.t7","ascii")
Overview
Here is a brief introduction of what it contains:
(20%) Getting hands on Hadoop streaming
(40%) One-vs-one multiclass classification using Hadoop; design output format of mapper
(40%) One-vs-all multiclass classification using Hadoop;
Assignment 2 - Deep Learning using Sparse Coding
Update April 19th 2013: the assignment deadline has been extended to April 23rd.
Update April 19th 2013: the third term in the predictive sparse decomposition energy in problem 4 should be squared. It is fixed.
Update April 19th 2013: another way of obtaining the ImageNet data from CIMS servers has been posted.
Instructions
Resources
Description
Datasets
/home/xz558/public/train256.tar
/home/xz558/public/test256.tar
/user/xz558/public/train256
/user/xz558/public/test256
/scratch/xz558/public/train256.tar
/scratch/xz558/public/test256.tar
Clément's tutorial on unsupervised learning
link (If you are stuck with implementation, these sample code may help)
Overview
Here is a brief introduction of what it contains:
(5%) Intuition behind sparsity.
(30%) Inference in sparse coding with FISTA; the shrinkage operator; line search in FISTA; trial of obtaining Lipschitz constant with spectral approximation.
(30%) Dictionary learning in sparse coding; implement the alternating direction algorithm; test on the MNIST dataset
(35%) Training predictive sparse decomposition architecture; apply to preprocessed ImageNet data; rectified linear unit and its smooth version.
Assignment 1 - Binary Linear Learning
Instructions
Deadline: March 12, 2013 before midnight.
You must answer the questions by yourself, but you may discusss the results of experiments with other students. We recommend you to submit questions to our Piazza forum where you can get help from both instructors and students.
Send an assignment report, along with all of your source code, to the TA ONLY. Please make everything in one tar ball or zip file, naming it as: LASTNAME_FIRSTNAME.tar.gz
Wirte all your answers to the questions below to the report, in pdf format. Please do not include any doc, docx, odt, html or plain texts. If you hate writing maths in computers, we can accept hand-written reports submitted before the class at the day of the deadline. You still have to send all the source code to the TA if you choose to submit a hand-written report.
Do not include any dataset in your submission.
We accept late submissions, but there will be some penalty on your score.
Resources
Description: pdf | tar.gz (LaTeX source)
Skeleton code: tar.gz
The datasets used are spambase dataset and the malicious URL dataset. You should put the file spambase.data into the src/learn directory in order to do problem 2. To prevent too many downloads of the datasets from their original distributors, we made the following back-up means of downloading the datasets:
/home/xz558/public/lsml/assign1-dat
Overview
Here is a brief introduction of what it contains:
(20%) Semantics of loss functions: square, quantile, hinge and logistic.
(45%) Demo with pre-conditioned SGD and batch GD. Implement all kinds of loss functions, and train them with SGD, bfgs, cg and lbfgs algorithms on spambase dataset.
(35%) Vowpal Wabbit with Malicious
URL dataset. Write code to compute testing error, precision and recall. Train on the dataset with different weights on different labels to see precision-recall trade-off.
The three parts are relatively independent; you can choose either one of them as a start. Please refer to the description file for more details.