SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 201–250 of 522 papers

TitleStatusHype
Towards Identity-Aware Cross-Modal Retrieval: a Dataset and a BaselineCode0
Maybe you are looking for CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval—0
Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal RetrievalCode0
Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation—0
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance—0
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey—0
Fusing Physics-Driven Strategies and Cross-Modal Adversarial Learning: Toward Multi-Domain Applications—0
FLEX-CLIP: Feature-Level GEneration Network Enhanced CLIP for X-shot Cross-modal Retrieval—0
CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions—0
Improving Factuality of 3D Brain MRI Report Generation with Paired Image-domain Retrieval and Text-domain Augmentation—0
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity—0
Everything is a Video: Unifying Modalities through Next-Frame Prediction—0
Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval—0
MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs—0
Towards Cross-Modal Text-Molecule Retrieval with Better Modality AlignmentCode0
Multilingual Vision-Language Pre-training for the Remote Sensing DomainCode0
Test-time Adaptation for Cross-modal Retrieval with Query Shift—0
Deep Class-guided Hashing for Multi-label Cross-modal RetrievalCode0
GleanVec: Accelerating vector search with minimalist nonlinear dimensionality reduction—0
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models—0
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features—0
Multimodal LLM Enhanced Cross-lingual Cross-modal RetrievalCode0
Snap and Diagnose: An Advanced Multimodal Retrieval System for Identifying Plant Diseases in the Wild—0
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them—0
Leveraging Chemistry Foundation Models to Facilitate Structure Focused Retrieval Augmented Generation in Multi-Agent Workflows for Catalyst and Materials Design—0
Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach—0
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models—0
Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval—0
Disentangled Noisy Correspondence Learning—0
Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval—0
Unified Lexical Representation for Interpretable Visual-Language AlignmentCode0
DAC: 2D-3D Retrieval with Noisy Labels via Divide-and-Conquer Alignment and CorrectionCode0
Revolutionizing Text-to-Image Retrieval as Autoregressive Token-to-Voken Generation—0
ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion MapCode0
Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge—0
Semantic Compositions Enhance Vision-Language Contrastive Learning—0
Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive LearningCode0
MATE: Meet At The Embedding -- Connecting Images with Long Texts—0
ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling—0
Deep Sketched Output Kernel Regression for Structured PredictionCode0
What If We Recaption Billions of Web Images with LLaMA-3?—0
No Captions, No Problem: Captionless 3D-CLIP Alignment with Hard Negatives via CLIP Knowledge and LLMs—0
Multi-Modal Generative Embedding Model—0
RREH: Reconstruction Relations Embedded Hashing for Semi-Paired Cross-Modal Retrieval—0
Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Models—0
Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrieval—0
MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Binding—0
Global–Local Information Soft-Alignment for Cross-Modal Remote-Sensing Image–Text Retrieval—0
All in One Framework for Multimodal Re-identification in the Wild—0
COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval—0
Show:102550
← PrevPage 5 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified