SOTAVerified

Visual Grounding

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

  • What is the main focus in a query?
  • How to understand an image?
  • How to locate an object?

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 151–200 of 571 papers

TitleStatusHype
Collaborative Transformers for Grounded Situation RecognitionCode1
GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly DetectionCode1
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language ModelsCode1
Fine-Grained Semantically Aligned Vision-Language Pre-TrainingCode1
Kosmos-2: Grounding Multimodal Large Language Models to the WorldCode1
REX: Reasoning-aware and Grounded ExplanationCode1
PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language ModelCode1
Visual Grounding for Object-Level Generalization in Reinforcement LearningCode1
Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD ImagesCode1
Relation-aware Instance Refinement for Weakly Supervised Visual GroundingCode1
Florence-2: Advancing a Unified Representation for a Variety of Vision TasksCode1
Context-Aware Alignment and Mutual Masking for 3D-Language Pre-TrainingCode1
Improving Weakly Supervised Visual Grounding by Contrastive Knowledge DistillationCode1
CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual GroundingCode1
Context Disentangling and Prototype Inheriting for Robust Visual GroundingCode1
CLIP-Lite: Information Efficient Visual Representation Learning with Language SupervisionCode1
InfMLLM: A Unified Framework for Visual-Language TasksCode1
Advancing Grounded Multimodal Named Entity Recognition via LLM-Based Reformulation and Box-Based SegmentationCode1
Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction ProblemsCode1
Learning Cross-modal Context Graph for Visual GroundingCode1
SwimVG: Step-wise Multimodal Fusion and Adaption for Visual GroundingCode1
Evolving Symbolic 3D Visual Grounder with Weakly Supervised ReflectionCode1
Improving Visual Grounding by Encouraging Consistent Gradient-based ExplanationsCode1
Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression ComprehensionCode1
CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud DataCode1
Improving Visual Grounding with Visual-Linguistic Verification and Iterative ReasoningCode1
GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language ModelsCode1
GRAVL-BERT: Graphical Visual-Linguistic Representations for Multimodal Coreference ResolutionCode1
InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual ReferringCode1
HYDRA: A Hyper Agent for Dynamic Compositional Visual ReasoningCode1
Instruction-Following Agents with Multimodal TransformerCode1
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical AlignmentCode1
Grounded Situation Recognition with TransformersCode1
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language ModelsCode1
Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D ScansCode1
Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual GroundingCode1
GroundVLP: Harnessing Zero-shot Visual Grounding from Vision-Language Pre-training and Open-Vocabulary Object DetectionCode1
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal CapabilitiesCode1
Guessing State Tracking for Visual DialogueCode1
CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding EvaluationCode1
MDETR -- Modulated Detection for End-to-End Multi-Modal UnderstandingCode1
3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive SelectionCode1
EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual GroundingCode1
How Do Multimodal Large Language Models Handle Complex Multimodal Reasoning? Placing Them in An Extensible Escape GameCode1
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound SeparationCode1
Learning Point-Language Hierarchical Alignment for 3D Visual GroundingCode1
Improving One-stage Visual Grounding by Recursive Sub-query ConstructionCode1
Pseudo-Q: Generating Pseudo Language Queries for Visual GroundingCode1
RefChartQA: Grounding Visual Answer on Chart Images through Instruction TuningCode1
Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language ModelingCode1
Show:102550
← PrevPage 4 of 12Next →
All datasetsRefCOCO testARefCOCO+ test BRefCoCo val

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)95.3—Unverified
2mPLUG-2Accuracy (%)92.8—Unverified
3X2-VLM (large)Accuracy (%)92.1—Unverified
4XFM (base)Accuracy (%)90.4—Unverified
5X2-VLM (base)Accuracy (%)90.3—Unverified
6X-VLM (base)Accuracy (%)89—Unverified
7HYDRAIoU61.7—Unverified
8HYDRAIoU61.1—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)92—Unverified
2mPLUG-2Accuracy (%)86.05—Unverified
3X2-VLM (large)Accuracy (%)81.8—Unverified
4XFM (base)Accuracy (%)79.8—Unverified
5X2-VLM (base)Accuracy (%)78.4—Unverified
6X-VLM (base)Accuracy (%)76.91—Unverified
#ModelMetricClaimedVerifiedStatus
1Florence-2-large-ftAccuracy (%)93.4—Unverified
2mPLUG-2Accuracy (%)90.33—Unverified
3X2-VLM (large)Accuracy (%)87.6—Unverified
4XFM (base)Accuracy (%)86.1—Unverified
5X2-VLM (base)Accuracy (%)85.2—Unverified
6X-VLM (base)Accuracy (%)84.51—Unverified