Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution

Abstract

Ridge regression is an indispensable tool in big data analysis. Yet its inherent bias poses a significant and longstanding challenge, compromising both statistical efficiency and scalability across various applications. To tackle this critical issue, we introduce an iterative strategy to correct bias effectively when the dimension p is less than the sample size n. For p>n, our method optimally mitigates the bias such that any remaining bias in the proposed de-biased estimator is unattainable through linear transformations of the response data. To address the remaining bias when p>n, we employ a Ridge-Screening (RS) method, producing a reduced model suitable for bias correction. Crucially, under certain conditions, the true model is nested within our selected one, highlighting RS as a novel variable selection approach. Through rigorous analysis, we establish the asymptotic properties and valid inferences of our de-biased ridge estimators for both p<n and p>n, where, both p and n may increase towards infinity, along with the number of iterations. We further validate these results using simulated and real-world data examples. Our method offers a transformative solution to the bias challenge in ridge regression inferences across various disciplines.

Tasks

Reproductions