Contents

1. Introduction

2. The slope heuristics
2.1. Framework
2.2. Optimal penalty
2.3. Minimal penalty and the slope heuristics
2.4. Data-driven penalty algorithm
2.5. What can be proved mathematically
2.6. Bibliographical remarks

3. Generalizing the slope heuristics
3.1. General framework
3.2. Penalties known up to some constant factor
3.3. Algorithm 3 fails for linear-estimator selection
3.4. Minimal-penalty heuristics for linear estimators
3.5. General minimal-penalty algorithm
3.6. Optimal and minimal penalties
3.7. Bibliographical remarks

4. Theoretical results: a review
4.1. General approach for proving Algorithm 5
4.2. Full proofs of Algorithm 5
4.3. Partial proofs: uncertainty on the optimal penalty
4.4. Minimal penalty in terms of risk: (β′)
4.5. Partial proofs: uncertainty on the minimal penalty
4.6. Partial proofs: for some specific s★ only
4.7. Partial proofs: richer collections of models

5. Towards new theoretical results on minimal penalties
5.1. Hints for (α): how to find pen₀, pen₁, and Cₘ?
5.2. Hints for (β): how to prove that C★pen₀ is a minimal penalty?
5.3. Hints for (γ): how to prove that C★pen₁ is an optimal penalty?

6. Related procedures
6.1. Residual-variance estimation
6.2. Estimation of the residual covariance matrix
6.3. Model/estimator-selection procedures based on Cₚ/C_L
6.4. L-curve, corner, and elbow heuristics
6.5. Scree test and related methods
6.6. Thresholding under the null
6.7. Other model/estimator-selection procedures

7. Some practical remarks
7.1. Several definitions for Ĉ
7.2. Algorithmic cost
7.3. Nested minimal-penalty algorithm
7.4. Estimation of several unknown constants in the penalty
7.5. Variants for change-point detection
7.6. Other uses of minimal penalties

8. Conclusion, conjectures, and open problems
8.1. Settings and losses where minimal-penalty algorithms apply
8.2. Unavoidable assumptions
8.3. Other settings, losses, estimators
8.4. Overpenalization
8.5. Beyond Algorithms 5-6: phase transitions for estimator selection
8.6. Related challenges in probability theory