SOTAVerified

Audio captioning

Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 119 papers

TitleStatusHype
M2D2: Exploring General-purpose Audio-Language Representations Beyond CLAPCode0
DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio CaptioningCode0
Multi-task Regularization Based on Infrequent Classes for Audio CaptioningCode0
OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio SeparationCode0
AUTOMATED AUDIO CAPTIONING BY FINE-TUNING BART WITH AUDIOSET TAGSCode0
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMsCode0
Solla: Towards a Speech-Oriented LLM That Hears Acoustic ContextCode0
Temporal Sub-sampling of Audio Feature Sequences for Automated Audio CaptioningCode0
Audio Difference Captioning Utilizing Similarity-Discrepancy DisentanglementCode0
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution AlignmentCode0
Weakly-supervised Automated Audio Captioning via text only trainingCode0
Audio Caption in a Car Setting with a Sentence-Level LossCode0
Language-based Audio Retrieval Task in DCASE 2022 Challenge—0
AC/DC: LLM-based Audio Comprehension via Dialogue Continuation—0
Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning—0
An Attempt towards Interpretable Audio-Visual Video Captioning—0
An investigation on selecting audio pre-trained models for audio captioning—0
A Transformer-based Audio Captioning Model with Keyword Estimation—0
AudioCaps: Generating Captions for Audios in The Wild—0
Audio Captioning using Gated Recurrent Units—0
Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval—0
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning—0
Audio Captioning with Composition of Acoustic and Semantic Information—0
Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model—0
Audio Dialogues: Dialogues dataset for audio and music understanding—0
Audio Difference Learning for Audio Captioning—0
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities—0
Automated Audio Captioning: An Overview of Recent Progress and New Challenges—0
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization—0
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features—0
Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning—0
Automated Audio Captioning with Recurrent Neural Networks—0
Automatic Audio Captioning using Attention weighted Event based Embeddings—0
CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizer—0
Classifier-Guided Captioning Across Modalities—0
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions—0
Diverse Audio Captioning via Adversarial Training—0
Diversity and bias in audio captioning datasets—0
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning—0
Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning—0
Efficient Audio Captioning Transformer with Patchout and Text Guidance—0
EmotionCaps: Enhancing Audio Captioning Through Emotion-Augmented Data Generation—0
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models—0
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization—0
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders—0
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models—0
Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning—0
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning—0
Generating Realistic Images from In-the-wild Sounds—0
Impact of visual assistance for automated audio captioning—0
Show:102550
← PrevPage 2 of 3Next →
All datasetsAudioCapsClotho

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.78—Unverified
2VALORCIDEr0.74—Unverified
3MQ-CapSPIDEr0.52—Unverified
4SLAM-AACSPIDEr0.52—Unverified
5LAVCapSPIDEr0.52—Unverified
6EnCLAP++-largeSPIDEr0.51—Unverified
7AutoCapSPIDEr0.51—Unverified
8LOAESPIDEr0.51—Unverified
9EnCLAP++-baseSPIDEr0.5—Unverified
10EnCLAP-largeSPIDEr0.5—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.52—Unverified
2VALORCIDEr0.42—Unverified
3SLAM-AACSPIDEr0.33—Unverified
4LOAESPIDEr0.33—Unverified
5MQ-CapSPIDEr0.32—Unverified
6EnsembleSPIDEr0.32—Unverified
7Audio Flamingo (Pengi trainset)SPIDEr0.31—Unverified
8Ensemble-RLSPIDEr0.3—Unverified
9Qwen-AudioSPIDEr0.29—Unverified
10EnsembleSPIDEr0.21—Unverified