1 INTRODUCTION  
2 TRANSFORMER MODEL ARCHITECTURE AND PERFORMANCE BOTTLENECKS  
2.1 High-Level Overview of Transformer Architecture  
2.2 Model Analysis  
3 HARDWARE DESIGN  
3.1 Overview of Typical DNN Accelerators  
3.2 Adapting DNN Accelerators for Transformers  
3.3 Analytical Modelling  
3.4 Case Study: Building a Transformer Accelerator  
4 MODEL OPTIMIZATION  
4.1 Quantization  
4.2 Sparsity  
4.3 Transformer-specific Optimization Methods  
5 MAPPING TRANSFORMERS TO HARDWARE  
5.1 What are Mappings?  
5.2 What Are the Key Mapping Decisions?  
5.3 Finding Performant Mappings  
5.4 Performance Modeling of Mappings  
5.5 Transformer vs CNN Mapping  
6 ADAPTING TRANSFORMER ARCHITECTURE WITH NAS  
6.1 Neural Architecture Search  
6.2 Hardware-aware NAS  
6.3 Transformer-specific NAS  
6.4 Case Study: Running NAS and Co-design on the Transformer  
7 CONCLUSION