SOTAVerified

Audio Generation

Audio generation (synthesis) is the task of generating raw audio such as speech.

( Image credit: MelNet )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 270 papers

TitleStatusHype
Fast Text-to-Audio Generation with Adversarial Post-TrainingCode7
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio SynthesisCode7
Stable Audio OpenCode7
Long-form music generation with latent diffusionCode7
Fast Timing-Conditioned Latent Audio DiffusionCode7
AudioLM: a Language Modeling Approach to Audio GenerationCode7
SoundStorm: Efficient Parallel Audio GenerationCode6
AudioGen: Textually Guided Audio GenerationCode6
ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and EditingCode5
InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music GenerationCode5
FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio GenerationCode5
Improving Text-To-Audio Models with Synthetic CaptionsCode5
AudioLCM: Text-to-Audio Generation with Latent Consistency ModelsCode5
Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference OptimizationCode5
Latent Swap Joint Diffusion for 2D Long-Form Latent GenerationCode4
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference OptimizationCode4
SNAC: Multi-Scale Neural Audio CodecCode4
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized SoundsCode4
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised PretrainingCode4
Enhancing Suno's Bark Text-to-Speech Model: Addressing Limitations Through Meta's Encodec and Pre-Trained HubertCode4
ArchiSound: Audio Generation with DiffusionCode4
AudioLDM: Text-to-Audio Generation with Latent Diffusion ModelsCode4
OmniAudio: Generating Spatial Audio from 360-Degree VideoCode3
Movie Gen: A Cast of Media Foundation ModelsCode3
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and SpeechCode3
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language ModelCode3
Language-Codec: Bridging Discrete Codec Representations and Speech Language ModelsCode3
High-Fidelity Audio Compression with Improved RVQGANCode3
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion ModelCode3
BigVGAN: A Universal Neural Vocoder with Large-Scale TrainingCode3
PodAgent: A Comprehensive Framework for Podcast GenerationCode2
KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio GenerationCode2
TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-InstrumentCode2
Baichuan-Omni-1.5 Technical ReportCode2
RingFormer: A Neural Vocoder with Ring Attention and Convolution-Augmented TransformerCode2
ETTA: Elucidating the Design Space of Text-to-Audio ModelsCode2
Gotta Hear Them All: Sound Source Aware Vision to Audio GenerationCode2
Audio Deepfake Detection with Self-Supervised XLS-R and SLS ClassifierCode2
TSELM: Target Speaker Extraction using Discrete Tokens and Language ModelsCode2
Efficient Autoregressive Audio Modeling via Next-Scale PredictionCode2
Taming Data and Transformers for Audio GenerationCode2
SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial SoundCode2
Generative Pre-trained Speech Language Model with Efficient Hierarchical TransformerCode2
Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow MatchingCode2
SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound GenerationCode2
The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake AudioCode2
RFWave: Multi-band Rectified Flow for Audio Waveform ReconstructionCode2
EVA-GAN: Enhanced Various Audio Generation via Scalable Generative Adversarial NetworksCode2
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio GenerationCode2
WavMark: Watermarking for Audio GenerationCode2
Show:102550
← PrevPage 1 of 6Next →
All datasetsAudioCapsClassical music, 5 seconds at 12 kHzSymphony music

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AudioGenFD_openl3185.53—Unverified
2AudioLDM2-largeFD_openl3158.04—Unverified
3Stable Audio 2.0FD_openl3110.62—Unverified
4Stable AudioFD_openl3103.66—Unverified
5ETTAFD_openl380.13—Unverified
6TangoFlux-baseFD_openl379.7—Unverified
7Stable Audio OpenFD_openl378.24—Unverified
8TangoFluxFD_openl375.1—Unverified
9ETTA-FT-AC-100kFD_openl361.79—Unverified
10DiffsoundFAD7.75—Unverified
#ModelMetricClaimedVerifiedStatus
1VAB-Encodec (Ours)Bits per byte40—Unverified
2Sparse Transformer 152M (strided)Bits per byte1.97—Unverified
#ModelMetricClaimedVerifiedStatus
1SymphonyNet Human listening average results3.5—Unverified