1. Introduction
1.1 Evolution of Natural Language Generation
1.2 Why a Survey on Evaluation of Natural Language Generation
1.3 Outline of The Survey
2. Human-Centric Evaluation Methods
2.1 Intrinsic Evaluation
2.2 Extrinsic Evaluation
2.3 The Evaluators
2.4 Inter-Evaluator Agreement
3. Untrained Automatic Evaluation Metrics
3.1 n-gram Overlap Metrics for Content Selection
3.2 Distance-Based Evaluation Metrics for Content Selection
3.3 n-gram-Based Diversity Metrics
3.4 Explicit Semantic Content Match Metrics
3.5 Syntactic Similarity-Based Metrics
4. Machine-Learned Evaluation Metrics
4.1 Sentence Semantic Similarity Based Evaluation
4.2 Regression-Based Evaluation
4.3 Evaluation Models with Human Judgments
4.4 BERT-Based Evaluation
4.5 Evaluating Factual Correctness
4.6 Composite Metric Scores
5. Shared Tasks for NLG Evaluation
5.1 Generating Referring Expressions
5.2 Embedded Text Generation
5.3 Regular Expression Generation (REG) in Context
5.4 Regular Expression Generation from Attribute Sets
5.5 Deep Meaning Representation to Text (SemEval)
5.6 WebNLG
5.7 E2E NLG Challenge
5.8 Data-to-Text Generation Challenge
5.9 GEM Benchmark
6. Examples of Task-Specific NLG Evaluation
6.1 Automatic Document Summarization Evaluation
6.2 Long Text Generation Evaluation
7. Conclusions and Future Directions