Contents
1 Introduction
2 From the Very Beginning to Neural Network
2.1 Connectionism and Early Stage of Artificial Neural Network
2.2 Perceptron as a Principal Component Analyzer
2.3 Neural Network, Backpropagation, Global Optimum and Universal Approximation
3 Neural Networks Extended to Deep Discriminative Models
3.1 The Necessity of Depth
3.2 Stacked Autoencoders
3.3 Convolutional Neural Network
3.4 Dropout & Maxout
4 Deep Generative Models
4.1 Self Organizing Map
4.2 Generative (Deep) Network Family
4.3 Hopfield Network
4.4 Boltzmann Machine
4.5 Restricted Boltzmann Machine
4.6 Deep Belief Networks
5 Learning Through Time: Modeling Time Series Data
5.1 Time Delay Neural Network
5.2 Recurrent Neural Network
5.3 Long Short Term Memory
5.4 Deep Recurrent Neural Networks
6 Advantages and Disadvantages of Deep Learning
6.1 Theoretically Best So Far
6.2 Some Critical Flaws