1. Introduction  
2. Related Work  
3. Overview  
3.1 Deep Learning System  
3.2 Risks in Deep Learning  
3.3 Commercial Off-The-Shelf  
4. Model Extraction Attack  
4.1 Introduction of Model Extraction Attack  
4.2 Approaches for Extracting Models  
4.3 Different Extracted Information  
4.4 Analysis of Model Extraction Attack  
5. Model Inversion Attack  
5.1 Introduction of Model Inversion Attack  
5.2 Membership Inference Attack  
5.3 Property Inference Attack  
5.4 Analysis of Model Inversion Attack  
6. Poisoning Attack  
6.1 Introduction of Poisoning Attack  
6.2 Poisoning Attack Approach  
6.3 Analysis of Poisoning Attack  
7. Adversarial Attack  
7.1 Introduction of Adversarial Attack  
7.2 Adversarial Attack Approach  
7.3 Analysis of Adversarial Attack  
8. Discussion  
8.1 Regulations on Privacy Protection  
8.2 Secure Implementation of Deep Learning Systems  
8.3 How Far Away from a Complete Black-Box Attack?  
8.4 Relationship Between Interpretability and Security  
8.5 Discrimination in AI  
8.6 Corresponding Defense Methods  
8.7 Future Direction of Attack and Defense  
9. Conclusion