SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 451–500 of 522 papers

TitleStatusHype
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal RepresentationsCode0
Deep Supervised Cross-Modal RetrievalCode0
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and GenerationCode0
Multimodal LLM Enhanced Cross-lingual Cross-modal RetrievalCode0
Multilingual Vision-Language Pre-training for the Remote Sensing DomainCode0
Unified Visual-Semantic Embeddings: Bridging Vision and Language With Structured Meaning RepresentationsCode0
Deep Sketched Output Kernel Regression for Structured PredictionCode0
MuLan: A Joint Embedding of Music Audio and Natural LanguageCode0
MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal RetrievalCode0
Modality-specific Cross-modal Similarity Measurement with Recurrent Attention NetworkCode0
ContextRefine-CLIP for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2025Code0
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive LearningCode0
ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion MapCode0
Deep Reversible Consistency Learning for Cross-modal RetrievalCode0
Picture It In Your Mind: Generating High Level Visual Representations From Textual DescriptionsCode0
Alternative Telescopic Displacement: An Efficient Multimodal Alignment MethodCode0
Deep Joint-Semantics Reconstructing Hashing for Large-Scale Unsupervised Cross-Modal RetrievalCode0
MaMMUT: A Simple Architecture for Joint Learning for MultiModal TasksCode0
Leveraging Acoustic Images for Effective Self-Supervised Audio Representation LearningCode0
Learning Visual Actions Using Multiple Verb-Only LabelsCode0
Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval ServiceCode0
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval TaskCode0
Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature EngineeringCode0
Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable ModelsCode0
Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food ImagesCode0
Align before Search: Aligning Ads Image to Text for Accurate Cross-Modal Sponsored SearchCode0
PromptHash: Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing RetrievalCode0
Learnable PINs: Cross-Modal Embeddings for Person IdentityCode0
Language-Agnostic Visual-Semantic EmbeddingsCode0
Deep Cross-Modal Projection Learning for Image-Text MatchingCode0
Adversarial Modality Alignment Network for Cross-Modal Molecule RetrievalCode0
Deep Cross-Modal HashingCode0
InvGC: Robust Cross-Modal Retrieval by Inverse Graph ConvolutionCode0
Intra-Modal Constraint Loss For Image-Text RetrievalCode0
Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive LearningCode0
Improving Medical Multi-modal Contrastive Learning with Expert AnnotationsCode0
Impression-CLIP: Contrastive Shape-Impression Embedding for FontsCode0
Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal AnalysisCode0
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language TasksCode0
Harmonized Multimodal Learning with Gaussian Process Latent Variable ModelsCode0
Deep Class-guided Hashing for Multi-label Cross-modal RetrievalCode0
Finding beans in burgers: Deep semantic-visual embedding with localizationCode0
ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and RetrievalCode0
FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal RetrievalCode0
Deep Binary Reconstruction for Cross-modal HashingCode0
Context-Aware Embeddings for Automatic Art AnalysisCode0
Exposing and Mitigating Spurious Correlations for Cross-Modal RetrievalCode0
RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching ModelsCode0
Exploring modality-agnostic representations for music classificationCode0
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding TasksCode0
Show:102550
← PrevPage 10 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified