SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 251–300 of 522 papers

TitleStatusHype
Two-Stage Triplet Loss Training with Curriculum Augmentation for Audio-Visual Retrieval—0
Uncertainty-based Cross-Modal Retrieval with Probabilistic Representations—0
Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models—0
Uni3DL: Unified Model for 3D and Language Understanding—0
Unsupervised Multi-modal Hashing for Cross-modal retrieval—0
Unsupervised Contrastive Hashing for Cross-Modal Retrieval in Remote Sensing—0
Unsupervised Deep Cross-modality Spectral Hashing—0
Unsupervised Generative Adversarial Cross-modal Hashing—0
Using Multiple Instance Learning to Build Multimodal Representations—0
Variational Autoencoder with CCA for Audio-Visual Cross-Modal Retrieval—0
Video and Audio are Images: A Cross-Modal Mixer for Original Data on Video-Audio Retrieval—0
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval—0
VLDeformer: Vision-Language Decomposed Transformer for Fast Cross-Modal Retrieval—0
Wasserstein Coupled Graph Learning for Cross-Modal Retrieval—0
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching—0
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval—0
Webly Supervised Joint Embedding for Cross-Modal lmage-Text Retrieval—0
What If We Recaption Billions of Web Images with LLaMA-3?—0
WikiMuTe: A web-sourced dataset of semantic descriptions for music audio—0
Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding—0
X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning—0
X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation—0
Y^2Seq2Seq: Cross-Modal Representation Learning for 3D Shape and Text by Joint Reconstruction and Prediction of View and Word Sequences—0
Zero-Shot Interactive Text-to-Image Retrieval via Diffusion-Augmented Representations—0
More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching—0
Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution—0
3D Shape Knowledge Graph for Cross-domain 3D Shape Retrieval—0
ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling—0
A Comprehensive Survey on Cross-modal Retrieval—0
Adaptive Asymmetric Label-guided Hashing for Multimedia Search—0
Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation—0
Semi-Supervised Cross-Modal Retrieval with Label Prediction—0
Adversarial Attack on Deep Cross-Modal Hamming Retrieval—0
Adversarial Cross-Modal Retrieval via Learning and Transferring Single-Modal Similarities—0
Aggregating Image and Text Quantized Correlated Components—0
All in One Framework for Multimodal Re-identification in the Wild—0
Alternating Co-Quantization for Cross-Modal Hashing—0
An analysis of vision-language models for fabric retrieval—0
Anchor-aware Deep Metric Learning for Audio-visual Retrieval—0
A Novel Incremental Cross-Modal Hashing Approach—0
A Proposal-based Approach for Activity Image-to-Video Retrieval—0
A Recipe for Improving Remote Sensing VLM Zero Shot Generalization—0
Artpedia—0
A scoping review on multimodal deep learning in biomedical images and texts—0
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment—0
Asymmetric Correlation Quantization Hashing for Cross-modal Retrieval—0
A Text-Image Pair Is not Enough: Language-Vision Relation Inference with Auxiliary Modality Translation—0
Attention-aware Deep Adversarial Hashing for Cross-Modal Retrieval—0
Attribute-Guided Network for Cross-Modal Zero-Shot Hashing—0
A Unified Optimal Transport Framework for Cross-Modal Retrieval with Noisy Labels—0
Show:102550
← PrevPage 6 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified