Blyth Lecture Series

Event Information Optimal Shrinkage of Singular Values and Eigenvalues under `Spiked' Big-Data Asymptotics
16:10 on Thursday February 26, 2015
17:00 on Thursday February 26, 2015
FI230, Fields Institute, 222 College St.
David Donoho
http://statweb.stanford.edu/~donoho/
Stanford University

In the 1950's Charles Stein had the revolutionary insight that in estimating high-dimensional covariance matrices, the empirical eigenvalues ought to be dramatically outperformed by nonlinear shrinkage of the eigenvalues. This insight inspired dozens of papers in mathematical statistics over the next 6 decades.

In the last decade, mathematical analysts working in Random Matrix Theory made a great deal of progress on the so-called $\textit spiked$ $\textit covariance$ $\textit model$ introduced by Johnstone (2001).

This talk will show how this recent progress makes it now possible to elegantly derive the unique asymptotically admissible shrinkage rules in many problems of matrix de-noising and covariance matrix estimation.

The new rules are very concrete and simple, and they dramatically outperform heuristics such as scree plot truncation and bulk edge truncation which have been around for decades and are used in thousands of papers across all of science.

Joint Work with Matan Gavish and Iain Johnstone.