SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, Robin Rombach
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/compvis/fm-boostingpytorch★ 256
- github.com/yuchen413/text2image_safetypytorch★ 197
- github.com/vision-xl/codespytorch★ 37
- github.com/bytedance/cascadevpytorch★ 35
- github.com/benearnthof/fm_boostingpytorch★ 7
- github.com/wellzline/protippytorch★ 6
- github.com/andrew-miao/RPOpytorch★ 5
- github.com/tillmannohm/fruit-SALADpytorch★ 4
Abstract
We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. We design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. We also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL shows drastically improved performance compared the previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators. In the spirit of promoting open research and fostering transparency in large model training and evaluation, we provide access to code and model weights at https://github.com/Stability-AI/generative-models
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| WISE | stable-diffusion-xl-base-0.9 | Overall | 0.43 | — | Unverified |