SOTAVerified

Visual Grounding

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

  • What is the main focus in a query?
  • How to understand an image?
  • How to locate an object?

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 301–350 of 571 papers

TitleStatusHype
Context Does Matter: End-to-end Panoptic Narrative Grounding with Deformable Attention Refined Matching NetworkCode0
OV-VG: A Benchmark for Open-Vocabulary Visual GroundingCode1
Visual Grounding Helps Learn Word Meanings in Low-Data RegimesCode1
InViG: Benchmarking Interactive Visual Grounding with 500K Human-Robot InteractionsCode0
NICE: Improving Panoptic Narrative Detection and Segmentation with Cascading Collaborative LearningCode0
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4VCode4
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningCode7
From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language ModelsCode2
CoT3DRef: Chain-of-Thoughts Data-Efficient 3D Visual GroundingCode1
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language ModelsCode1
Lightweight In-Context Tuning for Multimodal Unified Models—0
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an AgentCode2
Object2Scene: Putting Objects in Context for Open-Vocabulary 3D Detection—0
PROGrasp: Pragmatic Human-Robot Communication for Object GraspingCode1
Multi3DRefer: Grounding Text Description to Multiple 3D ObjectsCode1
Collecting Visually-Grounded Dialogue with A Game Of SortsCode0
Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Grounding—0
Interpretable Visual Question Answering via Reasoning Supervision—0
DetermiNet: A Large-Scale Diagnostic Dataset for Complex Visually-Grounded Referencing using DeterminersCode0
VGDiffZero: Text-to-image Diffusion Models Can Be Zero-shot Visual GroundersCode1
FACET: Fairness in Computer Vision Evaluation Benchmark—0
WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Model—0
UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and MemoryCode1
HuBo-VLM: Unified Vision-Language Model designed for HUman roBOt interaction tasksCode0
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and BeyondCode5
A Unified Framework for 3D Point Cloud Visual GroundingCode1
Target-Grounded Graph-Aware Transformer for Aerial Vision-and-Dialog NavigationCode1
Language-Guided Diffusion Model for Visual GroundingCode0
3D-VisTA: Pre-trained Transformer for 3D Vision and Text AlignmentCode2
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding—0
Iterative Robust Visual Grounding with Masked Reference based Centerpoint SupervisionCode1
Advancing Visual Grounding with Scene Knowledge: Benchmark and MethodCode1
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual GroundingCode1
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMsCode2
GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic ManipulationCode0
OG: Equip vision occupancy with instance segmentation and visual grounding—0
What Do Self-Supervised Speech Models Know About Words?Code1
Learning with Difference Attention for Visually Grounded Self-supervised Representations—0
Kosmos-2: Grounding Multimodal Large Language Models to the WorldCode1
Extending CLIP's Image-Text Alignment to Referring Image Segmentation—0
Referring to Screen Texts with Voice Assistants—0
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardsCode1
Language Adaptive Weight Generation for Multi-task Visual GroundingCode0
Leverage Points in Modality Shifts: Comparing Language-only and Multimodal Word RepresentationsCode0
Benchmarking Diverse-Modal Entity Linking with Generative Models—0
Language-Guided 3D Object Detection in Point Cloud for Autonomous Driving—0
Measuring Faithful and Plausible Visual Grounding in VQACode0
An Examination of the Robustness of Reference-Free Image Captioning Evaluation MetricsCode0
Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D ScansCode1
Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech ModelCode1
Show:102550
← PrevPage 7 of 12Next →
All datasetsRefCOCO testARefCOCO+ test BRefCoCo val

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)95.3—Unverified
2mPLUG-2Accuracy (%)92.8—Unverified
3X2-VLM (large)Accuracy (%)92.1—Unverified
4XFM (base)Accuracy (%)90.4—Unverified
5X2-VLM (base)Accuracy (%)90.3—Unverified
6X-VLM (base)Accuracy (%)89—Unverified
7HYDRAIoU61.7—Unverified
8HYDRAIoU61.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)92—Unverified
2mPLUG-2Accuracy (%)86.05—Unverified
3X2-VLM (large)Accuracy (%)81.8—Unverified
4XFM (base)Accuracy (%)79.8—Unverified
5X2-VLM (base)Accuracy (%)78.4—Unverified
6X-VLM (base)Accuracy (%)76.91—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)93.4—Unverified
2mPLUG-2Accuracy (%)90.33—Unverified
3X2-VLM (large)Accuracy (%)87.6—Unverified
4XFM (base)Accuracy (%)86.1—Unverified
5X2-VLM (base)Accuracy (%)85.2—Unverified
6X-VLM (base)Accuracy (%)84.51—Unverified