Once a month during the academic year, the statistics faculty select a paper for our students to read and discuss. Papers are selected based on their impact or historical value, or because they contain useful techniques or results.
Stein, C. (1981), Estimation of the Mean of a Multivariate Normal Distribution, The Annals of Statistics, 9(6), 1135-1151.
Notes preparer: Yuwen Gu
In this paper, Stein (1981) derived the famous Stein’s Lemma for multivariate normal distributions. Though simple, this lemma is so remarkable that it was used in the paper to derive an unbiased estimator for the risk of an arbitrary almost differentiable estimator under the squared error loss. This is known as Stein’s Unbiased Risk Estimate (SURE). SURE is an extremely useful tool for selecting tuning parameters or choosing between estimators to minimize the empirical risk, and as a theoretical tool for proving dominance results. For example, SURE can be applied to estimate the degrees of freedom for a large class of estimators, such as the various types of linear smoothers, the LARS (Efron et al. 2004), the LASSO (Zou, Hastie, and Tibshirani 2007), the reduced rank regression (Mukherjee et al. 2015), and many more. SURE can be also used to show the dominance of the James-Stein estimator over the standard least squares estimator when the dimension is above two. It is a really nice tool to have in our statistician’s toolbox.
References
- Efron, Bradley, Trevor Hastie, Iain Johnstone, and Robert Tibshirani. 2004. Least Angle Regression. The Annals of Statistics, 32 (2). Institute of Mathematical Statistics: 407-99.
- Mukherjee, Ashin, Kun Chen, Naisyin Wang, and Ji Zhu. 2015. On the Degrees of Freedom of Reduced-Rank Estimators in Multivariate Regression. Biometrika, 102 (2). Oxford University Press: 457-77.
- Zou, Hui, Trevor Hastie, and Robert Tibshirani. 2007. On the Degrees of Freedom of the Lasso. The Annals of Statistics, 35 (5). Institute of Mathematical Statistics: 2173-92.