Adjoint Differentiation for generic matrix functions

Abstract

We derive a formula for the adjoint A of a square-matrix operation of the form C=f(A), where f is holomorphic in the neighborhood of each eigenvalue. We then apply the formula to derive closed-form expressions in particular cases of interest such as the case when we have a spectral decomposition A=UDU^-1, the spectrum cut-off C=A_+ and the Nearest Correlation Matrix routine. Finally, we explain how to simplify the computation of adjoints for regularized linear regression coefficients.

Tasks

Reproductions