SOTAVerified

Visual Grounding

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

  • What is the main focus in a query?
  • How to understand an image?
  • How to locate an object?

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 451–500 of 571 papers

TitleStatusHype
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language ModelsCode1
Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue GenerationCode1
Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question AnsweringCode0
Panoptic Narrative GroundingCode1
Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models—0
INVIGORATE: Interactive Visual Grounding and Grasping in Clutter—0
A Better Loss for Visual-Textual GroundingCode0
TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Grounding—0
Attending Self-Attention: A Case Study of Visually Grounded Supervision in Vision-and-Language Transformers—0
Word2Pix: Word to Pixel Cross Attention Transformer in Visual Grounding—0
LanguageRefer: Spatial-Language Model for 3D Visual Grounding—0
VidLanKD: Improving Language Understanding via Video-Distilled Knowledge TransferCode1
Adventurer's Treasure Hunt: A Transparent System for Visually Grounded Compositional Visual Question Answering based on Scene Graphs—0
AIFit: Automatic 3D Human-Interpretable Feedback Models for Fitness Training—0
Semantic sentence similarity: size does not always matter—0
Attention-Based Keyword Localisation in Speech using Visual Grounding—0
Referring Transformer: A One-step Approach to Multi-task Visual GroundingCode1
Learning Better Visual Dialog Agents with Pretrained Visual-Linguistic RepresentationCode0
SAT: 2D Semantics Assisted Training for 3D Visual GroundingCode1
Connecting What to Say With Where to Look by Modeling Human Attention TracesCode1
MDETR -- Modulated Detection for End-to-End Multi-Modal UnderstandingCode1
TransVG: End-to-End Visual Grounding with TransformersCode1
Look Before You Leap: Learning Landmark Features for One-Stage Visual GroundingCode1
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound SeparationCode1
Visual Grounding Strategies for Text-Only Natural Language Processing—0
Relation-aware Instance Refinement for Weakly Supervised Visual GroundingCode1
Scene-Intuitive Agent for Remote Embodied Visual Grounding—0
Decoupled Spatial Temporal Graphs for Generic Visual Grounding—0
Few-Shot Visual Grounding for Natural Human-Robot Interaction—0
Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD ImagesCode1
OCID-Ref: A 3D Robotic Dataset with Embodied Language for Clutter Scene GroundingCode1
InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual ReferringCode1
Composing Pick-and-Place Tasks By Grounding LanguageCode0
Answer Questions with Right Image Regions: A Visual Attention Regularization ApproachCode0
Transformers in Vision: A Survey—0
3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds—0
Explainable Video Entailment With Grounded Visual Evidence—0
Panoptic Narrative GroundingCode1
Text-Free Image-to-Speech Synthesis Using Learned Segmental UnitsCode1
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations—0
Class-agnostic Object Detection—0
Text-to-Image Generation Grounded by Fine-Grained User AttentionCode1
Learning to ground medical text in a 3D human atlasCode0
SOrT-ing VQA Models : Contrastive Gradient Learning for Improved ConsistencyCode0
Neural Twins TalkCode0
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal TransformersCode1
Commands 4 Autonomous Vehicles (C4AV) Workshop Summary—0
Cosine meets Softmax: A tough-to-beat baseline for visual groundingCode0
AttnGrounder: Talking to Cars with AttentionCode0
Improving One-stage Visual Grounding by Recursive Sub-query ConstructionCode1
Show:102550
← PrevPage 10 of 12Next →
All datasetsRefCOCO testARefCOCO+ test BRefCoCo val

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)95.3—Unverified
2mPLUG-2Accuracy (%)92.8—Unverified
3X2-VLM (large)Accuracy (%)92.1—Unverified
4XFM (base)Accuracy (%)90.4—Unverified
5X2-VLM (base)Accuracy (%)90.3—Unverified
6X-VLM (base)Accuracy (%)89—Unverified
7HYDRAIoU61.7—Unverified
8HYDRAIoU61.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)92—Unverified
2mPLUG-2Accuracy (%)86.05—Unverified
3X2-VLM (large)Accuracy (%)81.8—Unverified
4XFM (base)Accuracy (%)79.8—Unverified
5X2-VLM (base)Accuracy (%)78.4—Unverified
6X-VLM (base)Accuracy (%)76.91—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)93.4—Unverified
2mPLUG-2Accuracy (%)90.33—Unverified
3X2-VLM (large)Accuracy (%)87.6—Unverified
4XFM (base)Accuracy (%)86.1—Unverified
5X2-VLM (base)Accuracy (%)85.2—Unverified
6X-VLM (base)Accuracy (%)84.51—Unverified