| InternVideo2: Scaling Foundation Models for Multimodal Video Understanding | Mar 22, 2024 | Action ClassificationAction Recognition | CodeCode Available | 7 |
| ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities | May 18, 2023 | 1 Image, 2*2 StitchiAction Classification | CodeCode Available | 3 |
| VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset | Apr 17, 2023 | Audio captioningAudio-Video Question Answering (AVQA) | CodeCode Available | 2 |
| VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset | May 29, 2023 | Audio captioningAudio-Visual Captioning | CodeCode Available | 2 |
| Audio Retrieval with Natural Language Queries: A Benchmark Study | Dec 17, 2021 | AudioCapsAudio captioning | CodeCode Available | 1 |
| Audio Retrieval with Natural Language Queries | May 5, 2021 | AudioCapsAudio to Text Retrieval | CodeCode Available | 1 |
| Cross Modal Retrieval with Querybank Normalisation | Dec 23, 2021 | Cross-Modal RetrievalMetric Learning | CodeCode Available | 1 |
| The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation | Nov 16, 2023 | Music CaptioningMusic Generation | CodeCode Available | 1 |
| WikiMuTe: A web-sourced dataset of semantic descriptions for music audio | Dec 14, 2023 | ArticlesCross-Modal Retrieval | —Unverified | 0 |
| Data leakage in cross-modal retrieval training: A case study | Feb 23, 2023 | Cross-Modal RetrievalRetrieval | —Unverified | 0 |
