Practicality meets Optimality: Real-Time Statistical Inference under Complex Constraints

Presented by Sen Na, University of California, Berkeley

Thursday, February 8 2024
3:30 PM-4:30 PM ET
AUST 105
Webex Meeting Link
Coffee will be served at 3:00 pm in the Noether Lounge (AUST 326)

Constrained estimation problems are prevalent in statistics, machine learning, and engineering. These problems encompass constrained generalized linear models, constrained deep neural networks, physics-inspired machine learning, algorithmic fairness, and optimal control. However, existing estimation methods under hard constraints rely on either projection or regularization, which may theoretically exhibit optimal efficiency but are impractical or unreasonably fail in reality. This talk aims to bridge the significant gap between practice and theory for constrained estimation problems.

I will begin by introducing the critical methodology used to bridge the gap, called Stochastic Sequential Quadratic Programming. We will see that SQP methods serve as the workhorse for modern scientific machine learning problems and can resolve the failure modes of prevalent regularization-based methods. I will demonstrate how to make SQP adaptive and scalable using various modern techniques, such as stochastic line search, trust region, and dimensionality reduction. Additionally, I will show how to further enhance SQP to handle inequality constraints online. Following the methodology, I will present some selective theories, emphasizing the consistency and efficiency of the SQP methods. Specifically, I will show that online SQP iterates asymptotically exhibit normal behavior with a mean of zero and optimal covariance in the Hájek and Le Cam sense. Significantly, the covariance does not deteriorate even when we apply modern techniques driven by practical concerns. The talk concludes with experiments on both synthetic and real datasets.

Speaker Bio:

Sen Na is currently a postdoctoral researcher in the Department of Statistics and the International Computer Science Institute at UC Berkeley. He received a Ph.D. degree in statistics from the University of Chicago. Sen Na’s primary research interests lie in the mathematical foundations of data science, encompassing highdimensional statistics, computational statistics, sequential decision-making, and large-scale and stochastic nonlinear optimization. Additionally, he is passionate about various applications of machine learning methods in scientific fields such as biology, neuroscience, physics, and engineering. Sen Na’s research has been recognized by the prestigious Harper Dissertation Fellowship from UChicago, and he has been selected as one of the Young Researchers in ORIE by Cornell University.