Convergence of the denoising diffusion probabilistic models for general noise schedules

Abstract

This work presents a theoretical analysis of the original formulation of denoising diffusion probabilistic models (DDPMs), introduced by Ho, Jain, and Abbeel in Advances in Neural Information Processing Systems, 33 (2020), pp. 6840-6851. An explicit upper bound is derived for the total variation distance between the distribution of the discrete-time DDPM sampling algorithm and a target data distribution, under general noise schedule parameters. The analysis assumes certain technical conditions on the data distribution and a linear growth condition on the noise estimation function. The sampling sequence emerges as an exponential integrator-type approximation of a reverse-time stochastic differential equation (SDE) over a finite time interval. Schr\"odinger's problem provides a tool for estimating the distributional error in reverse time, which connects the reverse-time error with its forward-time counterpart. The score function in DDPMs appears as an adapted solution of a forward-backward SDE, providing a foundation for analyzing the time-discretization error associated with the reverse-time SDE.

Tasks

Reproductions