SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 351–400 of 522 papers

TitleStatusHype
Data leakage in cross-modal retrieval training: A case study—0
Deep Binaries: Encoding Semantic-Rich Cues for Efficient Textual-Visual Cross Retrieval—0
Deep Cross-modal Hashing via Margin-dynamic-softmax Loss—0
Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals—0
Deep Lifelong Cross-modal Hashing—0
Deep Manifold Hashing: A Divide-and-Conquer Approach for Semi-Paired Unsupervised Cross-Modal Retrieval—0
Deep Multimodal Image-Text Embeddings for Automatic Cross-Media Retrieval—0
Deep Robust Multilevel Semantic Cross-Modal Hashing—0
Deep Semantic Correlation Learning Based Hashing for Multimedia Cross-Modal Retrieval—0
Deep Semantic Multimodal Hashing Network for Scalable Image-Text and Video-Text Retrievals—0
Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis—0
Deep Unified Multimodal Embeddings for Understanding both Content and Users in Social Media Networks—0
Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing—0
Dense Multimodal Fusion for Hierarchically Joint Representation—0
Developing ChatGPT for Biology and Medicine: A Complete Review of Biomedical Question Answering—0
Direction-Oriented Visual-semantic Embedding Model for Remote Sensing Image-text Retrieval—0
Discriminative Semantic Transitive Consistency for Cross-Modal Learning—0
Discriminative Supervised Hashing for Cross-Modal similarity Search—0
Discriminative Supervised Subspace Learning for Cross-modal Retrieval—0
Disentangled Noisy Correspondence Learning—0
Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrieval—0
Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval—0
Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA—0
Do Cross Modal Systems Leverage Semantic Relationships?—0
Dual-view Curricular Optimal Transport for Cross-lingual Cross-modal Retrieval—0
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models—0
EfficientCLIP: Efficient Cross-Modal Pre-training by Ensemble Confident Learning and Language Modeling—0
Efficient Discrete Supervised Hashing for Large-scale Cross-modal Retrieval—0
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval—0
EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast—0
Emphasizing Complementary Samples for Non-literal Cross-modal Retrieval—0
Enhancing medical vision-language contrastive learning via inter-matching relation modelling—0
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training—0
Everything is a Video: Unifying Modalities through Next-Frame Prediction—0
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey—0
Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisation—0
Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval—0
Extending Cross-Modal Retrieval with Interactive Learning to Improve Image Retrieval Performance in Forensics—0
FaD-VLP: Fashion Vision-and-Language Pre-training towards Unified Retrieval and Captioning—0
FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models—0
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity—0
Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings—0
Fine-Grained Instance-Level Sketch-Based Video Retrieval—0
Fine-grained Prototypical Voting with Heterogeneous Mixup for Semi-supervised 2D-3D Cross-modal Retrieval—0
FineLIP: Extending CLIP's Reach via Fine-Grained Alignment with Longer Text Inputs—0
FLEX-CLIP: Feature-Level GEneration Network Enhanced CLIP for X-shot Cross-modal Retrieval—0
FOLIAGE: Towards Physical Intelligence World Models Via Unbounded Surface Evolution—0
Fusing Physics-Driven Strategies and Cross-Modal Adversarial Learning: Toward Multi-Domain Applications—0
Fusion-supervised Deep Cross-modal Hashing—0
Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval—0
Show:102550
← PrevPage 8 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified