1. Introduction

2. Preliminaries  
2.1 Transformer-Style LLMs  
2.2 Inference Process of LLMs  
2.3 Efficiency Analysis  

3. Taxonomy

4. Data-Level Optimization  
4.1 Input Compression  
4.2 Output Organization  
4.3 Knowledge, Suggestions, and Future Direction  

5. Model-Level Optimization  
5.1 Efficient Structure Design  
5.2 Model Compression  
5.3 Knowledge, Suggestions, and Future Direction  

6. System-Level Optimization  
6.1 Inference Engine  
6.2 Serving System  
6.3 Hardware Accelerator Design  
6.4 Comparison of LLM Frameworks  
6.5 Knowledge, Suggestions, and Future Direction  

7. Discussions of Key Application Scenarios

8. Conclusion