SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 401–450 of 522 papers

TitleStatusHype
Generalized Semantic Preserving Hashing for N-Label Cross-Modal Retrieval—0
Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond—0
GleanVec: Accelerating vector search with minimalist nonlinear dimensionality reduction—0
Global–Local Information Soft-Alignment for Cross-Modal Remote-Sensing Image–Text Retrieval—0
GMM-Based Comprehensive Feature Extraction and Relative Distance Preservation For Few-Shot Cross-Modal Retrieval—0
Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach—0
Graph Pattern Loss based Diversified Attention Network for Cross-Modal Retrieval—0
HashGAN:Attention-aware Deep Adversarial Hashing for Cross Modal Retrieval—0
Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples—0
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval—0
Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks—0
Improved Text-Image Matching by Mitigating Visual Semantic Hubs—0
Improving Factuality of 3D Brain MRI Report Generation with Paired Image-domain Retrieval and Text-domain Augmentation—0
Improving Sound Source Localization with Joint Slot Attention on Image and Audio—0
Incorporating Dense Knowledge Alignment into Unified Multimodal Representation Models—0
Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval—0
Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval—0
Ink Marker Segmentation in Histopathology Images Using Deep Learning—0
Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal—0
Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval—0
Joint Wasserstein Autoencoders for Aligning Multimodal Embeddings—0
Label Prediction Framework for Semi-Supervised Cross-Modal Retrieval—0
Large Language Models for Captioning and Retrieving Remote Sensing Images—0
Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision—0
Learning Concordant Attention via Target-aware Alignment for Visible-Infrared Person Re-identification—0
Learning Discriminative Hashing Codes for Cross-Modal Retrieval based on Multi-view Features—0
Learning Disentangled Latent Factors from Paired Data in Cross-Modal Retrieval: An Implicit Identifiable VAE Approach—0
Learning Embodied Semantics via Music and Dance Semiotic Correlations—0
Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images—0
Learning Program Representations for Food Images and Cooking Recipes—0
Learning Semantic Concepts and Order for Image and Sentence Matching—0
Learning Similarity Preserving Binary Codes for Recommender Systems—0
Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval—0
Learning Sparse Disentangled Representations for Multimodal Exclusion Retrieval—0
Learning Structural Representations for Recipe Generation and Food Retrieval—0
Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays—0
Learning with Noisy Correspondence—0
Leveraging Chemistry Foundation Models to Facilitate Structure Focused Retrieval Augmented Generation in Multi-Agent Workflows for Catalyst and Materials Design—0
Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval—0
LILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives—0
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them—0
Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models—0
Mask-aware Text-to-Image Retrieval: Referring Expression Segmentation Meets Cross-modal Retrieval—0
Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrieval—0
MATE: Meet At The Embedding -- Connecting Images with Long Texts—0
Maximal Matching Matters: Preventing Representation Collapse for Robust Cross-Modal Retrieval—0
Maximum Covariance Unfolding : Manifold Learning for Bimodal Data—0
Maybe you are looking for CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval—0
MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model—0
MEDIAPI-SKEL - A 2D-Skeleton Video Database of French Sign Language With Aligned French Subtitles—0
Show:102550
← PrevPage 9 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5Aurora (ours, r=128)Image-to-text R@197.2—Unverified
6ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified