SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 351–400 of 522 papers

TitleStatusHype
Learning Program Representations for Food Images and Cooking Recipes—0
Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model—0
LILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives—0
Efficient Cross-Modal Retrieval via Deep Binary Hashing and QuantizationCode0
Discriminative Supervised Subspace Learning for Cross-modal Retrieval—0
Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing—0
A Text-Image Pair Is not Enough: Language-Vision Relation Inference with Auxiliary Modality Translation—0
EI-CLIP: Entity-Aware Interventional Contrastive Learning for E-Commerce Cross-Modal Retrieval—0
CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising—0
Multi-Modal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing—0
Variational Autoencoder with CCA for Audio-Visual Cross-Modal Retrieval—0
SwAMP: Swapped Assignment of Multi-Modal Pairs for Cross-Modal Retrieval—0
MURAL: Multimodal, Multitask Representations Across Languages—0
Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval—0
Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature EngineeringCode0
VLDeformer: Vision-Language Decomposed Transformer for Fast Cross-Modal Retrieval—0
Learning Structural Representations for Recipe Generation and Food Retrieval—0
Self-Supervised Modality-Invariant and Modality-Specific Feature Learning for 3D Objects—0
Calibrating Probabilistic Embeddings for Cross-Modal Retrieval—0
EfficientCLIP: Efficient Cross-Modal Pre-training by Ensemble Confident Learning and Language Modeling—0
MURAL: Multimodal, Multitask Retrieval Across Languages—0
Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images—0
Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval ServiceCode0
Evaluation of Audio-Visual Alignments in Visually Grounded Speech ModelsCode0
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and GenerationCode0
Graph Pattern Loss based Diversified Attention Network for Cross-Modal Retrieval—0
Cross-Modal Center Loss for 3D Cross-Modal Retrieval—0
Multi-Modal Relational Graph for Cross-Modal Video Moment Retrieval—0
Cross-Modal Discrete Representation Learning—0
Exploring modality-agnostic representations for music classificationCode0
Cross-lingual Cross-modal Pretraining for Multimodal Retrieval—0
Towards Efficient Cross-Modal Visual Textual Retrieval using Transformer-Encoder Deep Features—0
More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching—0
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching—0
FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal RetrievalCode0
Cross-Modal and Multimodal Data Analysis Based on Functional Mapping of Spectral Descriptors and Manifold Regularization—0
T-EMDE: Sketching-based global similarity for cross-modal retrieval—0
Multimodal Contrastive Training for Visual Representation Learning—0
Cross-Modal Retrieval Augmentation for Multi-Modal Classification—0
Continual learning in cross-modal retrieval—0
Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval—0
Discriminative Semantic Transitive Consistency for Cross-Modal Learning—0
Cross-modal Image Retrieval with Deep Mutual Information Maximization—0
CHEF: Cross-modal Hierarchical Embeddings for Food Domain RetrievalCode0
COOKIE: Contrastive Cross-Modal Knowledge Sharing Pre-Training for Vision-Language RepresentationCode0
Adversarial Attack on Deep Cross-Modal Hamming Retrieval—0
Wasserstein Coupled Graph Learning for Cross-Modal Retrieval—0
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive LearningCode0
Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning—0
Learning Disentangled Latent Factors from Paired Data in Cross-Modal Retrieval: An Implicit Identifiable VAE Approach—0
Show:102550
← PrevPage 8 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
6Aurora (ours, r=128)Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified