SOTAVerified

Audio Generation

Audio generation (synthesis) is the task of generating raw audio such as speech.

( Image credit: MelNet )

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 270 papers

TitleStatusHype
Retrieval-Augmented Neural Field for HRTF Upsampling and PersonalizationCode0
CCStereo: Audio-Visual Contextual and Contrastive Learning for Binaural Audio Generation—0
Nonparametric estimation of a factorizable density using diffusion models—0
RingFormer: A Neural Vocoder with Ring Attention and Convolution-Augmented TransformerCode2
Animate and Sound an Image—0
Foley-Flow: Coordinated Video-to-Audio Generation with Masked Audio-Visual Alignment and Dynamic Conditional Flows—0
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference OptimizationCode4
Tri-Ergon: Fine-grained Video-to-Audio Generation with Multi-modal Conditions and LUFS Control—0
ETTA: Elucidating the Design Space of Text-to-Audio ModelsCode2
VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis—0
Smooth-Foley: Creating Continuous Sound for Video-to-Audio Generation Under Semantic Guidance—0
RiTTA: Modeling Event Relations in Text-to-Audio GenerationCode1
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio SynthesisCode7
FolAI: Synchronized Foley Sound Generation with Semantic and Temporal Alignment—0
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation—0
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls—0
Comprehensive Audio Query Handling System with Integrated Expert Models and Contextual Understanding—0
Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation—0
Video-Guided Foley Sound Generation with Multimodal Controls—0
Gotta Hear Them All: Sound Source Aware Vision to Audio GenerationCode2
PerceiverS: A Multi-Scale Perceiver with Effective Segmentation for Long-Term Expressive Symbolic Music Generation—0
Tell What You Hear From What You See -- Video to Audio Generation Through TextCode1
Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation—0
The NPU-HWC System for the ISCSLP 2024 Inspirational and Convincing Audio Generation Challenge—0
Audio Deepfake Detection with Self-Supervised XLS-R and SLS ClassifierCode2
Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio GenerationCode0
SNAC: Multi-Scale Neural Audio CodecCode4
Movie Gen: A Cast of Media Foundation ModelsCode3
FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio GenerationCode5
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation—0
Did You Hear That? Introducing AADG: A Framework for Generating Benchmark Data in Audio Anomaly Detection—0
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition—0
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation—0
Analyzing and Mitigating Inconsistency in Discrete Audio Tokens for Neural Codec Language Models—0
From Vision to Audio and Beyond: A Unified Model for Audio-Visual Representation and GenerationCode1
ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and SpeechCode3
Video-to-Audio Generation with Fine-grained Temporal Semantics—0
Temporally Aligned Audio for Video with AutoregressionCode1
PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models—0
NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization—0
AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions—0
EzAudio: Enhancing Text-to-Audio Generation with Efficient Diffusion Transformer—0
Learning Source Disentanglement in Neural Audio Codec—0
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation—0
TSELM: Target Speaker Extraction using Discrete Tokens and Language ModelsCode2
MetaBGM: Dynamic Soundtrack Transformation For Continuous Multi-Scene Experiences With Ambient Awareness And Personalization—0
Applications and Advances of Artificial Intelligence in Music Generation:A Review—0
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language ModelCode3
Video-Foley: Two-Stage Video-To-Sound Generation via Temporal Event Condition For Foley Sound—0
Demystifying the Communication Characteristics for Distributed Transformer Models—0
Show:102550
← PrevPage 2 of 6Next →
All datasetsAudioCapsClassical music, 5 seconds at 12 kHzSymphony music

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AudioGenFD_openl3185.53—Unverified
2AudioLDM2-largeFD_openl3158.04—Unverified
3Stable Audio 2.0FD_openl3110.62—Unverified
4Stable AudioFD_openl3103.66—Unverified
5ETTAFD_openl380.13—Unverified
6TangoFlux-baseFD_openl379.7—Unverified
7Stable Audio OpenFD_openl378.24—Unverified
8TangoFluxFD_openl375.1—Unverified
9ETTA-FT-AC-100kFD_openl361.79—Unverified
10DiffsoundFAD7.75—Unverified
#ModelMetricClaimedVerifiedStatus
1VAB-Encodec (Ours)Bits per byte40—Unverified
2Sparse Transformer 152M (strided)Bits per byte1.97—Unverified
#ModelMetricClaimedVerifiedStatus
1SymphonyNet Human listening average results3.5—Unverified