Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review Mar 4, 2024 Medical Report Generation Question Answering
Code Code Available PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers Feb 13, 2024 Question Answering Retrieval
Code Code Available Common Sense Reasoning for Deepfake Detection Jan 31, 2024 Binary Classification Common Sense Reasoning
Code Code Available TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones Dec 28, 2023 Computational Efficiency Image Captioning
Code Code Available DriveLM: Driving with Graph Visual Question Answering Dec 21, 2023 Autonomous Driving Question Answering
Code Code Available Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models Nov 11, 2023 Image Captioning MMR total
Code Code Available Emu: Generative Pretraining in Multimodality Jul 11, 2023 Image Captioning Image Generation
Code Code Available CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning Jun 30, 2023 Causal Inference Medical Report Generation
Code Code Available ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities May 18, 2023 1 Image, 2*2 Stitchi Action Classification
Code Code Available Champion Solution for the WSDM2023 Toloka VQA Challenge Jan 22, 2023 Question Answering Visual Grounding
Code Code Available Unifying Vision, Text, and Layout for Universal Document Processing Dec 5, 2022 Document AI document understanding
Code Code Available Vision-Language Pre-training: Basics, Recent Advances, and Future Trends Oct 17, 2022 Few-Shot Learning Image Captioning
Code Code Available All You May Need for VQA are Image Captions May 4, 2022 All Image Captioning
Code Code Available OCR-free Document Understanding Transformer Nov 30, 2021 Document Image Classification document understanding
Code Code Available Ludwig: a type-based declarative deep learning toolbox Sep 17, 2019 Decoder Deep Learning
Code Code Available Towards VQA Models That Can Read Apr 18, 2019 TextVQA Visual Question Answering (VQA)
Code Code Available Pythia v0.1: the Winning Entry to the VQA Challenge 2018 Jul 26, 2018 Data Augmentation Visual Question Answering (VQA)
Code Code Available Bilinear Attention Networks May 21, 2018 Visual Question Answering Visual Question Answering (VQA)
Code Code Available CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models Jun 11, 2025 counterfactual Descriptive
Code Code Available Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis Mar 25, 2025 Contrastive Learning Image-text Retrieval
Code Code Available DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding Mar 13, 2025 4k Autonomous Driving
Code Code Available SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories Mar 11, 2025 Decision Making Interactive Segmentation
Code Code Available SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories Mar 11, 2025 Decision Making Interactive Segmentation
Code Code Available When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruning Mar 10, 2025 Language Modeling Language Modelling
Code Code Available Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Model Mar 8, 2025 Image Quality Assessment Language Modeling
Code Code Available OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference Feb 25, 2025 Visual Question Answering (VQA)
Code Code Available Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization Feb 18, 2025 Image Retrieval Question Answering
Code Code Available ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding Jan 9, 2025 Visual Question Answering (VQA) Visual Reasoning
Code Code Available EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation Dec 24, 2024 Image Captioning Image Generation
Code Code Available Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine Dec 12, 2024 Language Modeling Language Modelling
Code Code Available MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization Dec 9, 2024 Visual Question Answering (VQA)
Code Code Available Video Quality Assessment: A Comprehensive Survey Dec 4, 2024 Benchmarking Survey
Code Code Available Path-RAG: Knowledge-Guided Key Region Retrieval for Open-ended Pathology Visual Question Answering Nov 26, 2024 Prognosis Question Answering
Code Code Available Grounding-IQA: Multimodal Language Grounding Model for Image Quality Assessment Nov 26, 2024 Image Quality Assessment Question Answering
Code Code Available ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration Nov 25, 2024 AI Agent Visual Question Answering
Code Code Available VQA^2: Visual Question Answering for Video Quality Assessment Nov 6, 2024 Question Answering Video Quality Assessment
Code Code Available Frontiers in Intelligent Colonoscopy Oct 22, 2024 Image Captioning
Code Code Available VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment Aug 21, 2024 Video Alignment Video Editing
Code Code Available PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding Aug 18, 2024 Language Modelling Question Answering
Code Code Available GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI Aug 6, 2024 Question Answering Visual Question Answering
Code Code Available Fréchet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos Jul 23, 2024 Image Generation Point Tracking
Code Code Available SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers Jul 12, 2024 Articles Question Answering
Code Code Available WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering Jul 8, 2024 Diagnostic Generative Visual Question Answering
Code Code Available RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models Jul 6, 2024 Medical Diagnosis RAG
Code Code Available MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosis Jul 4, 2024 Diagnostic Language Modeling
Code Code Available A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding Jul 2, 2024 document understanding Key Information Extraction
Code Code Available TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy Jun 3, 2024 Language Modelling Question Answering
Code Code Available Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models Jun 3, 2024 Image Captioning Language Modelling
Code Code Available DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models May 31, 2024 cross-modal alignment Visual Localization
Code Code Available LM4LV: A Frozen Large Language Model for Low-level Vision Tasks May 24, 2024 Language Modeling Language Modelling
Code Code Available 