1 Introduction  
2 Preliminary  
2.1 Problem Formulation  
2.2 Taxonomy Rationale  
3 Approaches and Datasets  
3.1 Overview  
3.2 Hand-Crafted Methods  
3.3 Sequence Learning Methods  
3.4 Graph Learning Methods  
4 Model Designs  
4.1 Intra-Skeleton Modeling  
4.2 Inter-Skeleton Modeling  
4.3 Skeletal Sequential Modeling  
5 Challenges and Future Directions  
5.1 Multi-Level Skeleton Sequence Modeling  
5.2 Scarcity, Imbalance, and Noise of SRID Data  
5.3 Model Robustness, Generality, and Interpretability  
5.4 Other Directions and Discussions  
6 Conclusion