Kernel based regression with robust loss function via iteratively reweighted least squares

Abstract

Least squares kernel based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottleneck. To address this issue, a generalized loss function, called _s-loss, is proposed in this paper. With the support of novel loss function, two kernel based regressors are constructed by replacing the _2-loss in LS-SVR and ELM with the proposed _s-loss for better noise robustness. Important properties of _s-loss, including robustness, asymmetry and asymptotic approximation behaviors, are verified theoretically. Moreover, iteratively reweighted least squares (IRLS) is utilized to optimize and interpret the proposed methods from a weighted viewpoint. The convergence of the proposal are proved, and detailed analyses of robustness are given. Experiments on both artificial and benchmark datasets confirm the validity of the proposed methods.

Tasks

Reproductions