SOTAVerified

Cross-Modal Retrieval

Cross-Modal Retrieval (CMR) is a task of retrieving items across different modalities, such as image, text, video, and audio. The core challenge of CMR is the heterogeneity gap, which arises because data from different modalities have distinct representations, making direct comparison difficult. To address this, most CMR methods focus on learning a shared latent embedding space. In this space, concepts from different modalities are projected, allowing their similarity to be measured using a distance metric.

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 201–250 of 522 papers

TitleStatusHype
Instance-Variant Loss with Gaussian RBF Kernel for 3D Cross-modal Retriveal—0
Category-Oriented Representation Learning for Image to Multi-Modal Retrieval—0
Deep Lifelong Cross-modal Hashing—0
Sample-Specific Debiasing for Better Image-Text Models—0
Rethinking Benchmarks for Cross-modal Image-text RetrievalCode1
RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching ModelsCode0
Image-text Retrieval via Preserving Main Semantics of VisionCode1
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and DatasetCode2
CoVLR: Coordinating Cross-Modal Consistency and Intra-Modal Structure for Vision-Language Retrieval—0
Noisy Correspondence Learning with Meta Similarity CorrectionCode1
Exposing and Mitigating Spurious Correlations for Cross-Modal RetrievalCode0
AToMiC: An Image/Text Retrieval Test Collection to Support Multimedia Content CreationCode3
Hindi as a Second Language: Improving Visually Grounded Speech with Semantically Similar Samples—0
MaMMUT: A Simple Architecture for Joint Learning for MultiModal TasksCode0
Plug-and-Play Regulators for Image-Text MatchingCode1
MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal RepresentationsCode0
Single-branch Network for Multimodal TrainingCode1
Adversarial Modality Alignment Network for Cross-Modal Molecule RetrievalCode0
Cross-modal Retrieval with Improved Graph Convolution—0
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion TasksCode1
Data leakage in cross-modal retrieval training: A case study—0
Cross-Modal Retrieval with Partially Mismatched PairsCode1
X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation—0
VITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval—0
Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval—0
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding TasksCode0
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility StudyCode0
Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images—0
NAPReg: Nouns As Proxies Regularization for Semantically Aware Cross-Modal EmbeddingsCode0
Learning Concordant Attention via Target-aware Alignment for Visible-Infrared Person Re-identification—0
Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks—0
Learning Semantic Relationship Among Instances for Image-Text MatchingCode1
RONO: Robust Discriminative Learning With Noisy Labels for 2D-3D Cross-Modal RetrievalCode1
BagFormer: Better Cross-Modal Retrieval via bag-wise interaction—0
Position-guided Text Prompt for Vision-Language Pre-trainingCode1
Retrieval-based Disentangled Representation Learning with Natural Language Supervision—0
Scale-Semantic Joint Decoupling Network for Image-text Retrieval in Remote Sensing—0
Using Multiple Instance Learning to Build Multimodal Representations—0
Vision and Structured-Language Pretraining for Cross-Modal Food RetrievalCode1
A Differentiable Semantic Metric Approximation in Probabilistic Embedding for Cross-Modal RetrievalCode1
Semantic-Conditional Diffusion Networks for Image CaptioningCode2
Normalized Contrastive Learning for Text-Video RetrievalCode1
Improving Cross-Modal Retrieval with Set of Diverse EmbeddingsCode1
VoP: Text-Video Co-operative Prompt Tuning for Cross-Modal RetrievalCode1
X^2-VLM: All-In-One Pre-trained Model For Vision-Language TasksCode2
TimbreCLIP: Connecting Timbre to Text and Images—0
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent AttentionCode1
AltCLIP: Altering the Language Encoder in CLIP for Extended Language CapabilitiesCode4
Complete Cross-triplet Loss in Label Space for Audio-visual Cross-modal Retrieval—0
3D Shape Knowledge Graph for Cross-domain 3D Shape Retrieval—0
Show:102550
← PrevPage 5 of 11Next →
All datasetsCOCO 2014Flickr30kRecipe1M+RSICDRSITMDChEBI-20MSCOCO-1kSoundingEarthCUHK-PEDESFlickr-8kMSCOCOMS-COCO-2014

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1MaMMUT (ours)Image-to-text R@170.7—Unverified
2VASTText-to-image R@168—Unverified
3X2-VLM (large)Text-to-image R@167.7—Unverified
4BEiT-3Text-to-image R@167.2—Unverified
5XFM (base)Text-to-image R@167—Unverified
6X2-VLM (base)Text-to-image R@166.2—Unverified
7PTP-BLIP (14M)Text-to-image R@164.9—Unverified
8OmniVL (14M)Text-to-image R@164.8—Unverified
9VSE-GradientText-to-image R@163.6—Unverified
10X-VLM (base)Text-to-image R@163.4—Unverified
#ModelMetricClaimedVerifiedStatus
1X2-VLM (large)Image-to-text R@198.8—Unverified
2X2-VLM (base)Image-to-text R@198.5—Unverified
3BEiT-3Image-to-text R@198—Unverified
4OmniVL (14M)Image-to-text R@197.3—Unverified
5ERNIE-ViL 2.0Image-to-text R@197.2—Unverified
6Aurora (ours, r=128)Image-to-text R@197.2—Unverified
7X-VLM (base)Image-to-text R@197.1—Unverified
8VSE-GradientImage-to-text R@197—Unverified
9ALIGNImage-to-text R@195.3—Unverified
10VASTText-to-image R@191—Unverified
#ModelMetricClaimedVerifiedStatus
1VLPCook (R1M+)Image-to-text R@174.9—Unverified
2VLPCookImage-to-text R@173.6—Unverified
3T-Food (CLIP)Image-to-text R@172.3—Unverified
4T-FoodImage-to-text R@168.2—Unverified
5X-MRSImage-to-text R@164—Unverified
6H-TImage-to-text R@160—Unverified
7SCANImage-to-text R@154—Unverified
8ACMEImage-to-text R@151.8—Unverified
9VLPCookImage-to-text R@145.2—Unverified
10AdaMineImage-to-text R@139.8—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Mean Recall38.95—Unverified
2GeoRSCLIP-FTMean Recall38.87—Unverified
3GLISAMean Recall37.69—Unverified
4RemoteCLIPMean Recall36.35—Unverified
5PE-RSITR (MRS-Adapter)Mean Recall31.12—Unverified
6PIRMean Recall24.46—Unverified
7DOVEMean Recall22.72—Unverified
8SWANMean Recall20.61—Unverified
9GaLRMean Recall18.96—Unverified
10AMFMNMean Recall15.53—Unverified
#ModelMetricClaimedVerifiedStatus
1HarMA (w/ GeoRSCLIP)Image-to-text R@132.74—Unverified
2GeoRSCLIP-FTImage-to-text R@132.3—Unverified
3GLISAImage-to-text R@132.08—Unverified
4RemoteCLIPImage-to-text R@128.76—Unverified
5PE-RSITR (MRS-Adapter)Image-to-text R@123.67—Unverified
6PIRImage-to-text R@118.14—Unverified
7DOVEImage-to-text R@116.81—Unverified
8GaLRImage-to-text R@114.82—Unverified
9SWANImage-to-text R@113.35—Unverified
10AMFMNImage-to-text R@110.63—Unverified
#ModelMetricClaimedVerifiedStatus
1CLASS (ORMA)Hits@167.4—Unverified
2ORMAHits@166.5—Unverified
3Song et al.Hits@156.5—Unverified
4CLASS (AMAN)Hits@151.1—Unverified
5DSOKRHits@151—Unverified
6AMANHits@149.4—Unverified
7All-EnsembleHits@134.4—Unverified
8MLP1Hits@122.4—Unverified
9GCN2Hits@122.3—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@181.9—Unverified
2Dual-path CNNImage-to-text R@141.2—Unverified
#ModelMetricClaimedVerifiedStatus
1ResNet-18Median Rank565—Unverified
2GeoCLAPMedian Rank159—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual PathText-to-image Medr2—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegImage-to-text R@156.2—Unverified
#ModelMetricClaimedVerifiedStatus
13SHNetImage-to-text R@185.8—Unverified
#ModelMetricClaimedVerifiedStatus
1NAPRegText-to-image R@143—Unverified