SOTAVerified

Audio captioning

Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 119 papers

TitleStatusHype
Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation—0
Language-based Audio Retrieval Task in DCASE 2022 Challenge—0
AC/DC: LLM-based Audio Comprehension via Dialogue Continuation—0
Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning—0
An Attempt towards Interpretable Audio-Visual Video Captioning—0
An investigation on selecting audio pre-trained models for audio captioning—0
A Transformer-based Audio Captioning Model with Keyword Estimation—0
AudioCaps: Generating Captions for Audios in The Wild—0
Audio Captioning using Gated Recurrent Units—0
Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval—0
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning—0
Audio Captioning with Composition of Acoustic and Semantic Information—0
Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model—0
Audio Dialogues: Dialogues dataset for audio and music understanding—0
Audio Difference Learning for Audio Captioning—0
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities—0
Automated Audio Captioning: An Overview of Recent Progress and New Challenges—0
Automated Audio Captioning using Transfer Learning and Reconstruction Latent Space Similarity Regularization—0
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features—0
Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning—0
Automated Audio Captioning with Recurrent Neural Networks—0
Automatic Audio Captioning using Attention weighted Event based Embeddings—0
CLAP-ART: Automated Audio Captioning with Semantic-rich Audio Representation Tokenizer—0
Classifier-Guided Captioning Across Modalities—0
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions—0
Diverse Audio Captioning via Adversarial Training—0
Diversity and bias in audio captioning datasets—0
DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning—0
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning—0
Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning—0
Efficient Audio Captioning Transformer with Patchout and Text Guidance—0
EmotionCaps: Enhancing Audio Captioning Through Emotion-Augmented Data Generation—0
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models—0
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization—0
Enhancing Speech Large Language Models with Prompt-Aware Mixture of Audio Encoders—0
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models—0
Evaluating Off-the-Shelf Machine Listening and Natural Language Models for Automated Audio Captioning—0
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning—0
Generating Realistic Images from In-the-wild Sounds—0
Impact of visual assistance for automated audio captioning—0
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs—0
Improving Audio Caption Fluency with Automatic Error Correction—0
Rethinking Transfer and Auxiliary Learning for Improving Audio Captioning Transformer—0
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs—0
TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining—0
Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity—0
THE DCASE 2021 CHALLENGE TASK 6 SYSTEM: AUTOMATED AUDIO CAPTIONING WITH WEAKLY SUPERVISED PRE-TRAING AND WORD SELECTION METHODS—0
The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation—0
Towards Diverse and Efficient Audio Captioning via Diffusion Models—0
Towards Generating Diverse Audio Captions via Adversarial Training—0
Show:102550
← PrevPage 2 of 3Next →
All datasetsAudioCapsClotho

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.78—Unverified
2VALORCIDEr0.74—Unverified
3MQ-CapSPIDEr0.52—Unverified
4SLAM-AACSPIDEr0.52—Unverified
5LAVCapSPIDEr0.52—Unverified
6EnCLAP++-largeSPIDEr0.51—Unverified
7AutoCapSPIDEr0.51—Unverified
8LOAESPIDEr0.51—Unverified
9EnCLAP++-baseSPIDEr0.5—Unverified
10EnCLAP-largeSPIDEr0.5—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.52—Unverified
2VALORCIDEr0.42—Unverified
3SLAM-AACSPIDEr0.33—Unverified
4LOAESPIDEr0.33—Unverified
5MQ-CapSPIDEr0.32—Unverified
6EnsembleSPIDEr0.32—Unverified
7Audio Flamingo (Pengi trainset)SPIDEr0.31—Unverified
8Ensemble-RLSPIDEr0.3—Unverified
9Qwen-AudioSPIDEr0.29—Unverified
10EnsembleSPIDEr0.21—Unverified