SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 201–250 of 522 papers

TitleStatusHype
PromptHash: Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing RetrievalCode0
COOKIE: Contrastive Cross-Modal Knowledge Sharing Pre-Training for Vision-Language RepresentationCode0
Dual-Path Convolutional Image-Text Embeddings with Instance LossCode0
Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person SearchCode0
NAPReg: Nouns As Proxies Regularization for Semantically Aware Cross-Modal EmbeddingsCode0
DocMMIR: A Framework for Document Multi-modal Information RetrievalCode0
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal RepresentationsCode0
NeighborRetr: Balancing Hub Centrality in Cross-Modal RetrievalCode0
Multimodal LLM Enhanced Cross-lingual Cross-modal RetrievalCode0
Dissecting Deep Metric Learning Losses for Image-Text RetrievalCode0
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-HighlightingCode0
ContextRefine-CLIP for EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge 2025Code0
ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and RetrievalCode0
Context-Aware Embeddings for Automatic Art AnalysisCode0
MuLan: A Joint Embedding of Music Audio and Natural LanguageCode0
Multilingual Vision-Language Pre-training for the Remote Sensing DomainCode0
Content-Based Video-Music Retrieval Using Soft Intra-Modal Structure ConstraintCode0
DIME: An Online Tool for the Visual Comparison of Cross-Modal Retrieval ModelsCode0
ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion MapCode0
Modality-specific Cross-modal Similarity Measurement with Recurrent Attention NetworkCode0
Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated ImagesCode0
MaMMUT: A Simple Architecture for Joint Learning for MultiModal TasksCode0
Deep Visual-Semantic Alignments for Generating Image DescriptionsCode0
Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-modal RetrievalCode0
Leveraging Acoustic Images for Effective Self-Supervised Audio Representation LearningCode0
Deep Supervised Cross-Modal RetrievalCode0
Learning Visual Actions Using Multiple Verb-Only LabelsCode0
Deep Sketched Output Kernel Regression for Structured PredictionCode0
Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature EngineeringCode0
Towards Cross-Modal Text-Molecule Retrieval with Better Modality AlignmentCode0
Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval ServiceCode0
Deep Reversible Consistency Learning for Cross-modal RetrievalCode0
Adversarial Modality Alignment Network for Cross-Modal Molecule RetrievalCode0
CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote SensingCode0
Deep Joint-Semantics Reconstructing Hashing for Large-Scale Unsupervised Cross-Modal RetrievalCode0
Learnable PINs: Cross-Modal Embeddings for Person IdentityCode0
Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food ImagesCode0
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language TasksCode0
Deep Cross-Modal Projection Learning for Image-Text MatchingCode0
Deep Cross-Modal HashingCode0
InvGC: Robust Cross-Modal Retrieval by Inverse Graph ConvolutionCode0
Deep Class-guided Hashing for Multi-label Cross-modal RetrievalCode0
Deep Binary Reconstruction for Cross-modal HashingCode0
DAC: 2D-3D Retrieval with Noisy Labels via Divide-and-Conquer Alignment and CorrectionCode0
Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive LearningCode0
Intra-Modal Constraint Loss For Image-Text RetrievalCode0
Language-Agnostic Visual-Semantic EmbeddingsCode0
MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal RetrievalCode0
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features—0
Cross-View Image Retrieval -- Ground to Aerial Image Retrieval through Deep Learning—0
Show:102550
← PrevPage 5 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
6Aurora (ours, r=128)Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified