SOTAVerified

Visual Grounding

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

  • What is the main focus in a query?
  • How to understand an image?
  • How to locate an object?

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 571 papers

TitleStatusHype
VLM-R1: A Stable and Generalizable R1-style Large Vision-Language ModelCode9
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal UnderstandingCode9
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningCode7
Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric PerspectivesCode5
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMsCode5
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and BeyondCode5
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language ModelsCode4
V?: Guided Visual Search as a Core Mechanism in Multimodal LLMsCode4
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4VCode4
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and VideoCode4
MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language AnnotationsCode4
OrionBench: A Benchmark for Chart and Human-Recognizable Object Detection in InfographicsCode3
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI AgentsCode3
AgentStudio: A Toolkit for Building General Virtual AgentsCode3
DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language ModelsCode3
Champion Solution for the WSDM2023 Toloka VQA ChallengeCode3
A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future DirectionsCode3
Vision-Language Pre-training: Basics, Recent Advances, and Future TrendsCode3
Aria-UI: Visual Grounding for GUI InstructionsCode3
Towards Visual Grounding: A SurveyCode3
ShapeLLM: Universal 3D Object Understanding for Embodied InteractionCode3
BIP3D: Bridging 2D Images and 3D Perception for Embodied IntelligenceCode3
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMsCode3
Text-guided Sparse Voxel Pruning for Efficient 3D Visual GroundingCode3
Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement LearningCode3
RefMask3D: Language-Guided Transformer for 3D Referring SegmentationCode2
Aligning and Prompting Everything All at Once for Universal Visual PerceptionCode2
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator TrajectoriesCode2
Reasoning to Attend: Try to Understand How <SEG> Token WorksCode2
Referring Image MattingCode2
NExT-Chat: An LMM for Chat, Detection and SegmentationCode2
AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local AttentionCode2
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real WorldCode2
One Model to Rule them All: Towards Universal Segmentation for Medical Images with Text PromptsCode2
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator TrajectoriesCode2
LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an AgentCode2
ChatterBox: Multi-round Multimodal Referring and GroundingCode2
MedPromptX: Grounded Multimodal Prompting for Chest X-ray DiagnosisCode2
InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object RecognitionCode2
In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic SegmentationCode2
Interpreting Object-level Foundation Models via Visual Precision SearchCode2
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMsCode2
3D-VisTA: Pre-trained Transformer for 3D Vision and Text AlignmentCode2
DeepPerception: Advancing R1-like Cognitive Visual Perception in MLLMs for Knowledge-Intensive Visual GroundingCode2
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene UnderstandingCode2
List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMsCode2
VHM: Versatile and Honest Vision Language Model for Remote Sensing Image AnalysisCode2
A Simple Aerial Detection Baseline of Multimodal Language ModelsCode2
High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement LearningCode2
GTA1: GUI Test-time Scaling AgentCode2
Show:102550
← PrevPage 1 of 12Next →
All datasetsRefCOCO testARefCOCO+ test BRefCoCo val

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)95.3—Unverified
2mPLUG-2Accuracy (%)92.8—Unverified
3X2-VLM (large)Accuracy (%)92.1—Unverified
4XFM (base)Accuracy (%)90.4—Unverified
5X2-VLM (base)Accuracy (%)90.3—Unverified
6X-VLM (base)Accuracy (%)89—Unverified
7HYDRAIoU61.7—Unverified
8HYDRAIoU61.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)92—Unverified
2mPLUG-2Accuracy (%)86.05—Unverified
3X2-VLM (large)Accuracy (%)81.8—Unverified
4XFM (base)Accuracy (%)79.8—Unverified
5X2-VLM (base)Accuracy (%)78.4—Unverified
6X-VLM (base)Accuracy (%)76.91—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)93.4—Unverified
2mPLUG-2Accuracy (%)90.33—Unverified
3X2-VLM (large)Accuracy (%)87.6—Unverified
4XFM (base)Accuracy (%)86.1—Unverified
5X2-VLM (base)Accuracy (%)85.2—Unverified
6X-VLM (base)Accuracy (%)84.51—Unverified