| InternVideo2: Scaling Foundation Models for Multimodal Video Understanding | Mar 22, 2024 | Action ClassificationAction Recognition | CodeCode Available | 7 |
| ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities | May 18, 2023 | 1 Image, 2*2 StitchiAction Classification | CodeCode Available | 3 |
| VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset | May 29, 2023 | Audio captioningAudio-Visual Captioning | CodeCode Available | 2 |
| VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset | Apr 17, 2023 | Audio captioningAudio-Video Question Answering (AVQA) | CodeCode Available | 2 |
| The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation | Nov 16, 2023 | Music CaptioningMusic Generation | CodeCode Available | 1 |
| Cross Modal Retrieval with Querybank Normalisation | Dec 23, 2021 | Cross-Modal RetrievalMetric Learning | CodeCode Available | 1 |
| Audio Retrieval with Natural Language Queries: A Benchmark Study | Dec 17, 2021 | AudioCapsAudio captioning | CodeCode Available | 1 |
| Audio Retrieval with Natural Language Queries | May 5, 2021 | AudioCapsAudio to Text Retrieval | CodeCode Available | 1 |
| M2D2: Exploring General-purpose Audio-Language Representations Beyond CLAP | Mar 28, 2025 | Audio captioningAudio Classification | CodeCode Available | 0 |
| Do Audio-Language Models Understand Linguistic Variations? | Oct 21, 2024 | Contrastive LearningNatural Language Queries | —Unverified | 0 |
