1 INTRODUCTION  
2 PRELIMINARIES  
2.1 Pretrained Language Model Compression Problem  
2.2 Transformer Architecture  
2.3 Pretrained Language Model Compression Algorithms  
2.4 Backgrounds  
3 PRUNING  
3.1 Overview  
4 QUANTIZATION  
4.1 Overview  
5 OTHER ALGORITHMS  
5.1 Knowledge Distillation  
5.2 Low-Rank Approximation  
5.3 Parameter Sharing  
5.4 Efficient Architecture Design  
6 DISCUSSION  
6.1 Comparing Types of Compression Algorithms  
6.2 Two Desired Properties for Designing Successful Low-cost Compression  
6.3 Promising Research Topics  
7 CONCLUSION