SOTAVerified

Visual Grounding

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

  • What is the main focus in a query?
  • How to understand an image?
  • How to locate an object?

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 571 papers

TitleStatusHype
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI AgentsCode3
VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks—0
Adaptive Masking Enhances Visual GroundingCode0
World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and FilteringCode0
Individuation in Neural Models with and without Visual Grounding—0
ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue—0
SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal FusionCode2
HiFi-CS: Towards Open Vocabulary Visual Grounding For Robotic Grasping Using Vision-Language ModelsCode0
Bayesian Self-Training for Semi-Supervised 3D Segmentation—0
Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language ModelingCode0
Visual Grounding with Multi-modal Conditional AdaptationCode1
Visual Prompting in Multimodal Large Language Models: A Survey—0
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene UnderstandingCode2
NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radar—0
ResVG: Enhancing Relation and Semantic Understanding in Multiple Instances for Visual GroundingCode0
M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation—0
MMR: Evaluating Reading Ability of Large Multimodal Models—0
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal CapabilitiesCode1
Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models—0
In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic SegmentationCode2
Task-oriented Sequential Grounding in 3D Scenes—0
Visual Grounding for Object-Level Generalization in Reinforcement LearningCode1
An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual GroundingCode1
UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models—0
RefMask3D: Language-Guided Transformer for 3D Referring SegmentationCode2
Unveiling and Mitigating Bias in Audio Visual Segmentation—0
PD-APE: A Parallel Decoding Framework with Adaptive Position Encoding for 3D Visual Grounding—0
Learning Visual Grounding from Generative Vision and Language Model—0
Open-Vocabulary 3D Semantic Segmentation with Text-to-Image Diffusion Models—0
VIMI: Grounding Video Generation through Multi-modal Instruction—0
3D Vision and Language Pretraining with Large-Scale Synthetic DataCode1
Exploring Phrase-Level Grounding with Text-to-Image Diffusion ModelCode0
Multi-branch Collaborative Learning Network for 3D Visual GroundingCode1
Second Place Solution of WSDM2023 Toloka Visual Question Answering Challenge—0
Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and RepetitionCode0
Smart Vision-Language ReasonersCode0
ACTRESS: Active Retraining for Semi-supervised Visual Grounding—0
Visual Grounding with Attention-Driven Constraint Balancing—0
SegVG: Transferring Object Bounding Box to Segmentation for Visual GroundingCode2
The Solution for the ICCV 2023 Perception Test Challenge 2023 -- Task 6 -- Grounded videoQA—0
CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding EvaluationCode1
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities—0
From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models—0
FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts—0
On the Role of Visual Grounding in VQA—0
Towards Open-World Grasping with Large Vision-Language Models—0
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMsCode5
AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local AttentionCode2
VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image UnderstandingCode2
Visually Consistent Hierarchical Image Classification—0
Show:102550
← PrevPage 4 of 12Next →
All datasetsRefCOCO testARefCOCO+ test BRefCoCo val

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)95.3—Unverified
2mPLUG-2Accuracy (%)92.8—Unverified
3X2-VLM (large)Accuracy (%)92.1—Unverified
4XFM (base)Accuracy (%)90.4—Unverified
5X2-VLM (base)Accuracy (%)90.3—Unverified
6X-VLM (base)Accuracy (%)89—Unverified
7HYDRAIoU61.7—Unverified
8HYDRAIoU61.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)92—Unverified
2mPLUG-2Accuracy (%)86.05—Unverified
3X2-VLM (large)Accuracy (%)81.8—Unverified
4XFM (base)Accuracy (%)79.8—Unverified
5X2-VLM (base)Accuracy (%)78.4—Unverified
6X-VLM (base)Accuracy (%)76.91—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)93.4—Unverified
2mPLUG-2Accuracy (%)90.33—Unverified
3X2-VLM (large)Accuracy (%)87.6—Unverified
4XFM (base)Accuracy (%)86.1—Unverified
5X2-VLM (base)Accuracy (%)85.2—Unverified
6X-VLM (base)Accuracy (%)84.51—Unverified