SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 522 papers

TitleStatusHype
CLIP-KD: An Empirical Study of CLIP Model DistillationCode1
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal RetrievalCode1
Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL ModelsCode1
Transformer Decoders with MultiModal Regularization for Cross-Modal Food RetrievalCode1
A Comprehensive Empirical Study of Vision-Language Pre-trained Model for Supervised Cross-Modal RetrievalCode1
Fine-grained Video-Text Retrieval with Hierarchical Graph ReasoningCode1
Learning Cross-Modal Retrieval With Noisy LabelsCode1
Single-branch Network for Multimodal TrainingCode1
StacMR: Scene-Text Aware Cross-Modal RetrievalCode1
Visual Semantic Reasoning for Image-Text MatchingCode1
RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching ModelsCode0
A Channel Mix Method for Fine-Grained Cross-Modal RetrievalCode0
CHEF: Cross-modal Hierarchical Embeddings for Food Domain RetrievalCode0
Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image EmbeddingsCode0
CAMP: Cross-Modal Adaptive Message Passing for Text-Image RetrievalCode0
Alternative Telescopic Displacement: An Efficient Multimodal Alignment MethodCode0
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility StudyCode0
PromptHash: Affinity-Prompted Collaborative Cross-Modal Learning for Adaptive Hashing RetrievalCode0
Bridging Vision and Language Spaces with Assignment PredictionCode0
Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable ModelsCode0
Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery BanksCode0
Picture It In Your Mind: Generating High Level Visual Representations From Textual DescriptionsCode0
Exposing and Mitigating Spurious Correlations for Cross-Modal RetrievalCode0
OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and GenerationCode0
Exploring modality-agnostic representations for music classificationCode0
NeighborRetr: Balancing Hub Centrality in Cross-Modal RetrievalCode0
Evaluation of Audio-Visual Alignments in Visually Grounded Speech ModelsCode0
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-trainingCode0
See, Hear, and Read: Deep Aligned RepresentationsCode0
Multimodal LLM Enhanced Cross-lingual Cross-modal RetrievalCode0
Multilingual Vision-Language Pre-training for the Remote Sensing DomainCode0
ELIP: Efficient Language-Image Pre-training with Fewer Vision TokensCode0
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval TaskCode0
MuLan: A Joint Embedding of Music Audio and Natural LanguageCode0
Modality-specific Cross-modal Similarity Measurement with Recurrent Attention NetworkCode0
Efficient Cross-Modal Retrieval via Deep Binary Hashing and QuantizationCode0
ModalChorus: Visual Probing and Alignment of Multi-modal Embeddings via Modal Fusion MapCode0
Effective and Efficient Indexing in Cross-Modal Hashing-Based DatasetsCode0
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation LearningCode0
Align before Search: Aligning Ads Image to Text for Accurate Cross-Modal Sponsored SearchCode0
Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal RetrievalCode0
MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal RetrievalCode0
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal RepresentationsCode0
COOKIE: Contrastive Cross-Modal Knowledge Sharing Pre-Training for Vision-Language RepresentationCode0
Dual-Path Convolutional Image-Text Embeddings with Instance LossCode0
Contrastive Transformer Learning with Proximity Data Generation for Text-Based Person SearchCode0
DocMMIR: A Framework for Document Multi-modal Information RetrievalCode0
Leveraging Acoustic Images for Effective Self-Supervised Audio Representation LearningCode0
Learning Visual Actions Using Multiple Verb-Only LabelsCode0
Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature EngineeringCode0
Show:102550
← PrevPage 4 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified