Linearly Mapping from Image to Text Space Sep 30, 2022 Image Captioning Image to text
Code Code Available TVLT: Textless Vision-Language Transformer Sep 28, 2022 Automatic Speech Recognition (ASR) Image Retrieval
Code Code Available RepsNet: Combining Vision with Language for Automated Medical Reports Sep 27, 2022 Contrastive Learning Decoder
— Unverified Towards Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baseline Sep 24, 2022 Question Answering Visual Question Answering
Code Code Available Exploring Modulated Detection Transformer as a Tool for Action Recognition in Videos Sep 21, 2022 Action Detection Action Recognition
Code Code Available Continual VQA for Disaster Response Systems Sep 21, 2022 Disaster Response Management
Code Code Available Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering Sep 21, 2022 Image Captioning Optical Character Recognition (OCR)
— Unverified Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Sep 20, 2022 Multimodal Deep Learning Multimodal Reasoning
Code Code Available Panoramic Vision Transformer for Saliency Detection in 360° Videos Sep 19, 2022 Saliency Detection Saliency Prediction
Code Code Available Overcoming Language Priors in Visual Question Answering via Distinguishing Superficially Similar Instances Sep 18, 2022 Attribute Question Answering
Code Code Available LAVIS: A Library for Language-Vision Intelligence Sep 15, 2022 Benchmarking Image Captioning
— Unverified OmniVL:One Foundation Model for Image-Language and Video-Language Tasks Sep 15, 2022 Action Classification Action Recognition
— Unverified PaLI: A Jointly-Scaled Multilingual Language-Image Model Sep 14, 2022 Decoder Few-Shot Image Classification
— Unverified Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Sep 14, 2022 Adversarial Robustness Question Answering
— Unverified MUST-VQA: MUltilingual Scene-text VQA Sep 14, 2022 Question Answering Visual Question Answering
— Unverified PreSTU: Pre-Training for Scene-Text Understanding Sep 12, 2022 Decoder Image Captioning
— Unverified MaXM: Towards Multilingual Visual Question Answering Sep 12, 2022 Question Answering Translation
Code Code Available Pre-training image-language transformers for open-vocabulary tasks Sep 9, 2022 Question Answering Visual Entailment
— Unverified Improving the Cross-Lingual Generalisation in Visual Question Answering Sep 7, 2022 Cross-Lingual Transfer Question Answering
Code Code Available An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Sep 4, 2022 Fill Mask Optical Flow Estimation
Code Code Available 2BiVQA: Double Bi-LSTM based Video Quality Assessment of UGC Videos Aug 31, 2022 Video Quality Assessment Visual Question Answering (VQA)
Code Code Available Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric Aug 30, 2022 Image Quality Assessment Point Cloud Quality Assessment
Code Code Available Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment Aug 29, 2022 cross-modal alignment Image-text Retrieval
Code Code Available Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task Aug 24, 2022 Continual Learning Question Answering
Code Code Available Bidirectional Contrastive Split Learning for Visual Question Answering Aug 24, 2022 Adversarial Attack Backdoor Attack
— Unverified FashionVQA: A Domain-Specific Visual Question Answering System Aug 24, 2022 Question Answering Visual Question Answering
— Unverified How good are deep models in understanding the generated images? Aug 23, 2022 Object Object Recognition
— Unverified Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Aug 22, 2022 All Cross-Modal Retrieval
Code Code Available VLMAE: Vision-Language Masked Autoencoder Aug 19, 2022 Image-text Retrieval Language Modeling
— Unverified Understanding Attention for Vision-and-Language Tasks Aug 17, 2022 Image Generation Image Retrieval
Code Code Available ILLUME: Rationalizing Vision-Language Models through Human Interactions Aug 17, 2022 Image Captioning Question Answering
Code Code Available Aesthetic Visual Question Answering of Photographs Aug 10, 2022 Question Answering Sentiment Analysis
— Unverified CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning Aug 10, 2022 Math Mathematical Reasoning
Code Code Available ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding Aug 5, 2022 Image Retrieval Question Answering
Code Code Available Prompt Tuning for Generative Multimodal Pretrained Models Aug 4, 2022 Image Captioning Visual Entailment
— Unverified TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation Aug 3, 2022 Answer Generation Question-Answer-Generation
Code Code Available NAPA: Intermediate-level Variational Native-pulse Ansatz for Variational Quantum Algorithms Aug 2, 2022 Neural Architecture Search Visual Question Answering (VQA)
— Unverified Generative Bias for Robust Visual Question Answering Aug 1, 2022 Knowledge Distillation Question Answering
Code Code Available Video Question Answering with Iterative Video-Text Co-Tokenization Aug 1, 2022 Question Answering Video Question Answering
— Unverified Parameter-Parallel Distributed Variational Quantum Algorithm Jul 31, 2022 Visual Question Answering (VQA)
— Unverified Uncertainty-based Visual Question Answering: Estimating Semantic Inconsistency between Image and Knowledge Base Jul 27, 2022 Question Answering Semantic Similarity
— Unverified LaKo: Knowledge-driven Visual Question Answering via Late Knowledge-to-Text Injection Jul 26, 2022 Decoder Knowledge Graphs
Code Code Available Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering Jul 26, 2022 Causal Inference Question Answering
Code Code Available WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models Jul 25, 2022 Common Sense Reasoning General Knowledge
Code Code Available Is GPT-3 all you need for Visual Question Answering in Cultural Heritage? Jul 25, 2022 All Question Answering
— Unverified Towards Complex Document Understanding By Discrete Reasoning Jul 25, 2022 document understanding Question Answering
— Unverified Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem Jul 24, 2022 Diagnostic Question Answering
— Unverified Semantic-aware Modular Capsule Routing for Visual Question Answering Jul 21, 2022 Question Answering Visual Question Answering
— Unverified Rethinking Data Augmentation for Robust Visual Question Answering Jul 18, 2022 Data Augmentation Knowledge Distillation
Code Code Available Clover: Towards A Unified Video-Language Alignment and Fusion Model Jul 16, 2022 Language Modeling Language Modelling
Code Code Available 