SOTAVerified

Audio captioning

Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 76–100 of 119 papers

TitleStatusHype
Audio Difference Learning for Audio Captioning—0
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities—0
Automated Audio Captioning: An Overview of Recent Progress and New Challenges—0
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization—0
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features—0
Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning—0
Automated Audio Captioning with Recurrent Neural Networks—0
Automatic Audio Captioning using Attention weighted Event based Embeddings—0
CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizer—0
Classifier-Guided Captioning Across Modalities—0
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions—0
Diverse Audio Captioning via Adversarial Training—0
Diversity and bias in audio captioning datasets—0
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning—0
Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning—0
Efficient Audio Captioning Transformer with Patchout and Text Guidance—0
EmotionCaps: Enhancing Audio Captioning Through Emotion-Augmented Data Generation—0
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models—0
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization—0
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders—0
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models—0
Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning—0
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning—0
Generating Realistic Images from In-the-wild Sounds—0
Impact of visual assistance for automated audio captioning—0
Show:102550
← PrevPage 4 of 5Next →
All datasetsAudioCapsClotho

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.78—Unverified
2VALORCIDEr0.74—Unverified
3MQ-CapSPIDEr0.52—Unverified
4SLAM-AACSPIDEr0.52—Unverified
5LAVCapSPIDEr0.52—Unverified
6EnCLAP++-largeSPIDEr0.51—Unverified
7AutoCapSPIDEr0.51—Unverified
8LOAESPIDEr0.51—Unverified
9EnCLAP++-baseSPIDEr0.5—Unverified
10EnCLAP-largeSPIDEr0.5—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.52—Unverified
2VALORCIDEr0.42—Unverified
3SLAM-AACSPIDEr0.33—Unverified
4LOAESPIDEr0.33—Unverified
5MQ-CapSPIDEr0.32—Unverified
6EnsembleSPIDEr0.32—Unverified
7Audio Flamingo (Pengi trainset)SPIDEr0.31—Unverified
8Ensemble-RLSPIDEr0.3—Unverified
9Qwen-AudioSPIDEr0.29—Unverified
10EnsembleSPIDEr0.21—Unverified