SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 301–350 of 522 papers

TitleStatusHype
BagFormer: Better Cross-Modal Retrieval via bag-wise interaction—0
Calibrating Probabilistic Embeddings for Cross-Modal Retrieval—0
No Captions, No Problem: Captionless 3D-CLIP Alignment with Hard Negatives via CLIP Knowledge and LLMs—0
Category-Based Deep CCA for Fine-Grained Venue Discovery from Multimodal Data—0
CellCLIP -- Learning Perturbation Effects in Cell Painting via Text-Guided Contrastive Learning—0
CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer—0
CLASS: Enhancing Cross-Modal Text-Molecule Retrieval Performance and Training Efficiency—0
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance—0
VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval—0
CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions—0
CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning—0
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising—0
COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval—0
Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval—0
Composite Correlation Quantization for Efficient Multimodal Retrieval—0
Continual learning in cross-modal retrieval—0
Continual Vision-Language Representation Learning with Off-Diagonal Information—0
Contrastive Cross-Modal Knowledge Sharing Pre-training for Vision-Language Representation Learning and Retrieval—0
Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval—0
Correlation Hashing Network for Efficient Cross-Modal Retrieval—0
COTS: Collaborative Two-Stream Vision-Language Pre-Training Model for Cross-Modal Retrieval—0
Coupled CycleGAN: Unsupervised Hashing Network for Cross-Modal Retrieval—0
CoVLR: Coordinating Cross-Modal Consistency and Intra-Modal Structure for Vision-Language Retrieval—0
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval—0
Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model—0
Cross-Modal 3D Representation with Multi-View Images and Point Clouds—0
Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization—0
Cross-modal Center Loss—0
Cross-Modal Center Loss for 3D Cross-Modal Retrieval—0
Cross-modal Common Representation Learning by Hybrid Transfer Network—0
Cross-Modal Coordination Across a Diverse Set of Input Modalities—0
Cross-modal Deep Metric Learning with Multi-task Regularization—0
Cross-Modal Discrete Representation Learning—0
Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images and Recipes with Semantic Consistency and Attention Mechanism—0
Cross-Modal Hamming Hashing—0
Cross-modal Image Retrieval with Deep Mutual Information Maximization—0
Cross-Modality Personalization for Retrieval—0
Cross-Modal Manifold Learning for Cross-modal Retrieval—0
Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies—0
Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning—0
Cross-Modal Retrieval Augmentation for Multi-Modal Classification—0
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval—0
Cross-Modal Retrieval with Implicit Concept Association—0
Cross-modal Retrieval with Improved Graph Convolution—0
Cross-modal Search Method of Technology Video based on Adversarial Learning and Feature Fusion—0
Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval—0
Cross-View Image Retrieval -- Ground to Aerial Image Retrieval through Deep Learning—0
CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features—0
Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval—0
Data leakage in cross-modal retrieval training: A case study—0
Show:102550
← PrevPage 7 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
6Aurora (ours, r=128)Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified