SOTAVerified

Audio captioning

Audio Captioning is the task of describing audio using text. The general approach is to use an audio encoder to encode the audio (example: PANN, CAV-MAE), and to use a decoder (example: transformer) to generate the text. To judge the quality of audio captions, though machine translation metrics (BLEU, METEOR, ROUGE) and image captioning metrics (SPICE, CIDER) are used, they are not very well-suited. Attempts have been made to use pretrained language model based metrics such as Sentence-BERT.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 119 papers

TitleStatusHype
CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions—0
Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model—0
Classifier-Guided Captioning Across Modalities—0
EmotionCaps: Enhancing Audio Captioning Through Emotion-Augmented Data Generation—0
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning—0
DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio CaptioningCode0
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMsCode0
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization—0
An Eye for an Ear: Zero-shot Audio Description Leveraging an Image Captioner using Audiovisual Distribution AlignmentCode0
OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio SeparationCode0
CLAIR-A: Leveraging Large Language Models to Judge Audio CaptionsCode0
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models—0
Towards Diverse and Efficient Audio Captioning via Diffusion Models—0
Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models—0
Expanding on EnCLAP with Auxiliary Retrieval Model for Automated Audio Captioning—0
Audio Dialogues: Dialogues dataset for audio and music understanding—0
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs—0
Learning Audio Concepts from Counterfactual Natural LanguageCode0
AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive LearningCode0
Weakly-supervised Automated Audio Captioning via text only trainingCode0
Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning—0
Audio Difference Learning for Audio Captioning—0
Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation—0
Generating Realistic Images from In-the-wild Sounds—0
Killing two birds with one stone: Can an audio captioning system also be used for audio-text retrieval?—0
Audio Difference Captioning Utilizing Similarity-Discrepancy DisentanglementCode0
Rethinking Transfer and Auxiliary Learning for Improving Audio Captioning Transformer—0
Crowdsourcing and Evaluating Text-Based Audio Retrieval RelevancesCode0
Improving Audio Caption Fluency with Automatic Error Correction—0
Dual Transformer Decoder based Features Fusion Network for Automated Audio Captioning—0
Efficient Audio Captioning Transformer with Patchout and Text Guidance—0
Towards Generating Diverse Audio Captions via Adversarial Training—0
Impact of visual assistance for automated audio captioning—0
Diversity and bias in audio captioning datasets—0
Investigations in Audio Captioning: Addressing Vocabulary Imbalance and Evaluating Suitability of Language-Centric Performance Metrics—0
Exploring Train and Test-Time Augmentations for Audio-Language Learning—0
Automated Audio Captioning via Fusion of Low- and High- Dimensional Features—0
Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity—0
Language-based Audio Retrieval Task in DCASE 2022 Challenge—0
An investigation on selecting audio pre-trained models for audio captioning—0
Automated Audio Captioning and Language-Based Audio RetrievalCode0
Language-based Audio Retrieval Task in DCASE 2022 ChallengeCode0
Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning—0
Automated Audio Captioning: An Overview of Recent Progress and New Challenges—0
Caption Feature Space Regularization for Audio CaptioningCode0
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning—0
Leveraging Pre-trained BERT for Audio Captioning—0
Joint Speech Recognition and Audio Captioning—0
Automatic Audio Captioning using Attention weighted Event based Embeddings—0
Local Information Assisted Attention-free Decoder for Audio CaptioningCode0
Show:102550
← PrevPage 2 of 3Next →
All datasetsAudioCapsClotho

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.78—Unverified
2VALORCIDEr0.74—Unverified
3MQ-CapSPIDEr0.52—Unverified
4SLAM-AACSPIDEr0.52—Unverified
5LAVCapSPIDEr0.52—Unverified
6EnCLAP++-largeSPIDEr0.51—Unverified
7AutoCapSPIDEr0.51—Unverified
8LOAESPIDEr0.51—Unverified
9EnCLAP++-baseSPIDEr0.5—Unverified
10EnCLAP-largeSPIDEr0.5—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTCIDEr0.52—Unverified
2VALORCIDEr0.42—Unverified
3SLAM-AACSPIDEr0.33—Unverified
4LOAESPIDEr0.33—Unverified
5MQ-CapSPIDEr0.32—Unverified
6EnsembleSPIDEr0.32—Unverified
7Audio Flamingo (Pengi trainset)SPIDEr0.31—Unverified
8Ensemble-RLSPIDEr0.3—Unverified
9Qwen-AudioSPIDEr0.29—Unverified
10EnsembleSPIDEr0.21—Unverified