Syllabus

Course overview

Prerequisites: Multivariate calculus, Linear algebra, Probability (at the level of understanding the language of measure theory)

Tentative plan

Part I: Parametric statistics

Sufficient and ancillary statistics, Efficiency theory for maximum likelihood estimation, Neyman-Pearson lemma, Cramer-Rao lower bound, Regression, M-estimation, Z-estimation, LAN/CAN/RAL, Parametric Bayesian inference

Part II: Nonparametric and semiparametric statistics

Nonparametric MLE and shape constrained estimation, Resampling (bootstrap, jackknife, permutation), Edgeworth expansion, Influence functions and von Mises expansion, Semiparametric efficiency, Gaussian and Empirical processes, Function spaces and approximation theory, U-statistics and V-statistics

Part III: Minimax lower bounds

Lower bounds for testing, estimation, and inference, adaptative testing, estimation and inference

Part V: Advanced topics

CLT in high dimensions, Compressive sensing/RMT, Robust/heavy-tailed statistics and Median-of-mean tournament, Computational-statistical gap: (1) sparse PCA (2) SoS/Lasserre Hierarchy

Evaluation

Scribing (10%), Homeworks (40%), In-class exam on Part I-II (25%), Final project (short research project or literature survey) (25%)

Recommended textbooks

Note: All the books listed here have electronic version available in the library.

Basics and foundations:

1. Van der Vaart, Aad W. Asymptotic statistics. Vol. 3. Cambridge University Press, 2000.

2. Wasserman, Larry. All of statistics: A concise course in statistical inference. Springer Science & Business Media, 2013.

3. Wasserman, Larry. All of nonparametric statistics. Springer Science & Business Media, 2006.

4. Lehmann, Erich L., and Romano, Joseph P. Testing statistical hypotheses. Springer Science & Business Media, 2006.

5. Lehmann, Erich L., and Casella, George. Theory of point estimation. Springer Science & Business Media, 2006.

6. Lehmann, Erich L. Elements of large-sample theory. Springer Science & Business Media, 2004.

Advanced materials:

High-dimensional statistics

1. Vershynin, Roman. High-dimensional probability: An introduction with applications in data science. Vol. 47. Cambridge University Press, 2018.

2. Wainwright, Martin J. High-dimensional statistics: A non-asymptotic viewpoint. Vol. 48. Cambridge University Press, 2019.

Nonparametric statistics

1. Giné, Evarist, and Nickl, Richard. Mathematical foundations of infinite-dimensional statistical models. Vol. 40. Cambridge University Press, 2016.

2. Tsybakov, Alexandre B. Introduction to nonparametric estimation. Springer Science & Business Media, 2008.

3. Johnstone, Iain M. Gaussian estimation: Sequence and wavelet models.

4. Ghosal, Subhashis, and van der Vaart, Aad. Fundamentals of nonparametric Bayesian inference. Vol. 44. Cambridge University Press, 2017.

5. Nemirovski, Arkadi. Topics in non-parametric. Ecole d’Eté de Probabilités de Saint-Flour 28 (2000): 85.

6. Tsiatis, Anastasios. Semiparametric theory and missing data. Springer Science & Business Media, 2007.

7. Ibragimov, Ildar A and Has’minskii, Rafail Z. Statistical estimation: asymptotic theory. Springer Science & Business Media, 1981.

8. Ingster, Yuri N and Suslina, Irina A. Nonparametric Goodness-of-Fit Testing Under Gaussian Models. Springer Science & Business Media, 2002.

Probabilistic tools:

1. Talagrand, Michel. Upper and lower bounds for stochastic processes: Modern methods and classical problems. Vol. 60. Springer Science & Business Media, 2014.

2. Tropp, Joel A. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics 12.4 (2012): 389-434.

3. van Handel, Ramon. Probability in high dimension. 2016.

4. Dembo, Amir. Lecture Notes: Probability Theory. 2020.

Optimization tools:

1. Juditsky, Anatoli and Nemirovski, Arkadi. Statistical Inference via Convex Optimization. 2019.

Potential topics for final projects