1 Introduction

2 Background  
2.1 Computation Flow for LLaMA  
2.2 Overview on Parameter Efficient Fine Tuning  
2.3 Downstream Tasks for LLM Evaluation

3 PEFT Taxonomy  
3.1 Additive PEFT  
3.2 Selective PEFT  
3.3 Re-parameterized PEFT  
3.4 Hybrid PEFT

4 Efficient PEFT Design  
4.1 KV-cache Management for PEFT Efficiency  
4.2 Pruning Strategies for PEFT  
4.3 Quantization Strategies for PEFT  
4.4 Memory-efficient PEFT Methods

5 PEFT for DNNs of Other Applications  
5.1 PEFT for LLMs – Beyond the Basics  
5.2 PEFT for ViTs  
5.3 PEFT for VLAs  
5.4 PEFT for Diffusion Models

6 System Design Challenges for PEFT  
6.1 System Design for PEFT  
6.2 Centralized PEFT Serving Frameworks  
6.3 Distributed PEFT Training Frameworks  
6.4 Parallel PEFT Training Frameworks

7 Conclusion and Future Directions  
7.1 Simplify Hyperparameter Tuning  
7.2 Establish a Unified Benchmark  
7.3 Enhance Training Efficiency  
7.4 Explore Scaling Laws  
7.5 Serve More Models and Tasks  
7.6 Enhancing Data Privacy  
7.7 PEFT with Model Compression