1 Introduction  
1.1 What neural networks can model  
1.2 How neural networks are obtained and evaluated  
1.3 Why non-linearity is important in artificial neurons  
1.4 When deep learning meets polyhedral theory  
1.5 Scope of this survey and related work  
2 The Polyhedral Perspective  
3 The Linear Regions of a Neural Network  
3.1 The combinatorial aspect of linear regions  
3.2 The algebra of linear regions  
3.3 The geometry of linear regions  
3.4 The number of linear regions  
3.5 Applications and insights  
4 Optimizing Over a Trained Neural Network  
4.1 Applications of optimization over trained networks  
4.2 Exact models using mixed-integer programming  
4.3 Scaling further: Convex relaxations and linear programming  
4.4 Generalizing the single neuron model  
5 Linear Programming and Polyhedral Theory in Training  
5.1 Training neural networks with a single hidden layer  
5.2 Convex reformulations in regularized training problems  
5.3 Frank-Wolfe in DNN training algorithms  
5.4 Polyhedral encoding of multiple training problems  
5.5 Back propagation through MILP  
5.6 Training binarized neural networks using MILP  
6 Conclusion