1 Introduction
1.1 Contribution
1.2 Organisation of the report
2 Activation function
2.1 Properties of activation function
2.2 Problems faced by activation functions
2.3 Widely used activation functions
2.4 Shared properties among different activation functions
3 Weight initialization
3.1 Xavier initialization
3.2 He normal initialization
4 Discussion
5 Conclusion