Tu., Th. 9:30-10:45am
ART & ARCHITECTURE SOUTH 103
Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville. Available online
In the fast developing area of deep learning, one should not try to be a hero, but to learn from each other. Instead of just rolling your own architecture for a problem, you should look at what good methods other are using, and what architecture currently works best. Participation in classroom discussion will greatly help you to learn from each other on this new topic. Participation will be highly emphasized, and will be counted as 30% of the course grade.
During the class, we will assign short problems related to the material covered in the lectures. Many of these problems, you may answer them immediately in the class, but some require you to work out and turn in your answers later. These problems will used to help us evaluate your participation in the class.
There will be two homework assignments. In the first assignment, you will be asked to write codes (preferably in python) for a neural network with two hidden layers from scratch to implement the mathematical ideas and methods to be presented in the class that are widely used in deep learning. Doing this assignment will help you get a better understanding of the overall mechanism of the learning algorithms.
In the second assignment, you will be asked to write codes for specific components related to deep convolutional neural networks. In practice, it is very rare that you ever have to train a convolutional neural network from scratch or design one from scratch (Andrej Karpathy). However, experimenting with your own codes and comparing with the ones from Tensorflow will help you better appreciate the packages developed by Google and many contributors. On the other hand, doing this will encourage you to implement your own ideas. In an interview with Andrew Ng, Geoffrey Hinton advised that if you find something that everybody is doing wrong in deep learning, you should go ahead to figure out what should be the right way, regardless what others think.
The two assignments together count for 30% of the course grade.
The most important part of this course is to get hands-on, and perhaps the most exciting part of learning is to see that what you learned in class really works. For you to enjoy this kind of excitement, you are required to do a final project. You can either do it alone, or work with another student in the class. You should discuss the project with the instructors early; submit the proposal and give a short presentation in the class about the project. If you need thoughts about appropriate projects, you are welcome to discuss with the instructors, and we will gladly connect you to some researchers on campus. The final project weights 40% of the course grade.
The material will be presented through examples using MNIST data set, CIFAR-10 data set, Speech Commands Data Set v0.01, as well as some toy data.
In this course, we will study the mathematical foundation and implementation of Deep Learning. Through this course, students will