SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 101–150 of 522 papers

TitleStatusHype
Deep Sketched Output Kernel Regression for Structured PredictionCode0
What If We Recaption Billions of Web Images with LLaMA-3?—0
Merlin: A Vision Language Foundation Model for 3D Computed TomographyCode3
Separating the "Chirp" from the "Chat": Self-supervised Visual Grounding of Sound and LanguageCode2
No Captions, No Problem: Captionless 3D-CLIP Alignment with Hard Negatives via CLIP Knowledge and LLMs—0
Multi-Modal Generative Embedding Model—0
CaLa: Complementary Association Learning for Augmenting Composed Image RetrievalCode1
RREH: Reconstruction Relations Embedded Hashing for Semi-Paired Cross-Modal Retrieval—0
Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrieval—0
Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Models—0
MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Binding—0
Global–Local Information Soft-Alignment for Cross-Modal Remote-Sensing Image–Text Retrieval—0
All in One Framework for Multimodal Re-identification in the Wild—0
COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval—0
Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models—0
Efficient Remote Sensing with Harmonized Transfer Learning and Modality AlignmentCode2
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-HighlightingCode0
Anchor-aware Deep Metric Learning for Audio-visual Retrieval—0
Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding—0
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation LearningCode0
Knowledge-enhanced Visual-Language Pretraining for Computational PathologyCode1
Bridging Vision and Language Spaces with Assignment PredictionCode0
Learning with Noisy Correspondence—0
Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and MiningCode1
VXP: Voxel-Cross-Pixel Large-scale Image-LiDAR Place RecognitionCode1
A Unified Optimal Transport Framework for Cross-Modal Retrieval with Noisy Labels—0
Improving Medical Multi-modal Contrastive Learning with Expert AnnotationsCode0
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluationCode2
Learning to Rematch Mismatched Pairs for Robust Cross-Modal RetrievalCode1
Large Language Models are In-Context Molecule LearnersCode2
Tri-Modal Motion Retrieval by Learning a Joint Embedding Space—0
Impression-CLIP: Contrastive Shape-Impression Embedding for FontsCode0
Distinctive Image Captioning: Leveraging Ground Truth Captions in CLIP Guided Reinforcement LearningCode1
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond—0
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment—0
Large Language Models for Captioning and Retrieving Remote Sensing Images—0
Zero-shot sketch-based remote sensing image retrieval based on multi-level and attention-guided tokenizationCode0
Cross-Modal Coordination Across a Diverse Set of Input Modalities—0
Enhancing medical vision-language contrastive learning via inter-matching relation modelling—0
Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answering—0
Cross-modal Retrieval for Knowledge-based Visual Question AnsweringCode1
Linguistic-Aware Patch Slimming Framework for Fine-grained Cross-Modal AlignmentCode2
Fine-grained Prototypical Voting with Heterogeneous Mixup for Semi-supervised 2D-3D Cross-modal Retrieval—0
Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation—0
LeanVec: Searching vectors faster by making them fitCode2
Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrieval—0
SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote SensingCode2
TF-CLIP: Learning Text-free CLIP for Video-based Person Re-IdentificationCode1
CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer—0
WikiMuTe: A web-sourced dataset of semantic descriptions for music audio—0
Show:102550
← PrevPage 3 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified