SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 522 papers

TitleStatusHype
An analysis of vision-language models for fabric retrieval—0
Mask-aware Text-to-Image Retrieval: Referring Expression Segmentation Meets Cross-modal Retrieval—0
Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval—0
Multimodal Medical Image Binding via Shared Text Embeddings—0
FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models—0
ContextRefine-CLIP for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2025Code0
SA-Person: Text-Based Person Retrieval with Scene-aware Re-ranking—0
FOLIAGE: Towards Physical Intelligence World Models Via Unbounded Surface Evolution—0
EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast—0
DocMMIR: A Framework for Document Multi-modal Information RetrievalCode0
Sat2Sound: A Unified Framework for Zero-Shot Soundscape Mapping—0
SMOTExT: SMOTE meets Large Language ModelsCode0
GMM-Based Comprehensive Feature Extraction and Relative Distance Preservation For Few-Shot Cross-Modal Retrieval—0
CellCLIP -- Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning—0
Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert ReasonerCode2
Towards Cross-modal Retrieval in Chinese Cultural Heritage Documents: Dataset and Solution—0
MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological AssessmentCode1
OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval—0
Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable ModelsCode0
Disentangling and Generating Modalities for Recommendation in Missing Modality ScenariosCode1
Improving Sound Source Localization with Joint Slot Attention on Image and Audio—0
The 1st EReL@MIR Workshop on Efficient Representation Learning for Multimodal Information Retrieval—0
SemCORE: A Semantic-Enhanced Generative Cross-Modal Retrieval Framework with MLLMs—0
PATFinger: Prompt-Adapted Transferable Fingerprinting against Unauthorized Multimodal Dataset Usage—0
Learning Sparse Disentangled Representations for Multimodal Exclusion Retrieval—0
FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputs—0
LRSCLIP: A Vision-Language Foundation Model for Aligning Remote Sensing Image with Longer TextCode1
Seeing Speech and Sound: Distinguishing and Locating Audios in Visual Scenes—0
PromptHash: Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing RetrievalCode0
Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for DermatologyCode2
NeighborRetr: Balancing Hub Centrality in Cross-Modal RetrievalCode0
Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation—0
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment—0
A Recipe for Improving Remote Sensing VLM Zero Shot Generalization—0
X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learning—0
Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document RetrievalCode1
Composed Multi-modal Retrieval: A Survey of Approaches and ApplicationsCode2
Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval—0
ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence LearningCode1
Pathology Report Generation and Multimodal Representation Learning for Cutaneous Melanocytic Lesions—0
On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation—0
CLASS: Enhancing Cross-Modal Text-Molecule Retrieval Performance and Training Efficiency—0
GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image AnalysisCode1
Zero-Shot Interactive Text-to-Image Retrieval via Diffusion-Augmented Representations—0
TSVC:Tripartite Learning with Semantic Variation Consistency for Robust Image-Text Retrieval—0
Deep Reversible Consistency Learning for Cross-modal RetrievalCode0
Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy LabelsCode1
Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes—0
Incorporating Dense Knowledge Alignment into Unified Multimodal Representation Models—0
Fuzzy Multimodal Learning for Trusted Cross-modal RetrievalCode1
Show:102550
← PrevPage 1 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
6Aurora (ours, r=128)Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified