SOTAVerified

Image Captioning

Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.

( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1601–1650 of 1878 papers

TitleStatusHype
Improving Image Captioning with Conditional Generative Adversarial NetsCode0
SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure CaptioningCode0
Improving Explicit Spatial Relationships in Text-to-Image Generation through an Automatically Derived DatasetCode0
Improved Image Captioning via Policy Gradient optimization of SPIDErCode0
Disentangling Fine-Tuning from Pre-Training in Visual Captioning with Hybrid Markov LogicCode0
Discriminability objective for training descriptive captionsCode0
Improved Bengali Image Captioning via deep convolutional neural network based encoder-decoder modelCode0
MHSAN: Multi-Head Self-Attention Network for Visual Semantic EmbeddingCode0
MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report GenerationCode0
MilaNLP at SemEval-2022 Task 5: Using Perceiver IO for Detecting Misogynous Memes with Text and Image ModalitiesCode0
Mimic and Fool: A Task Agnostic Adversarial AttackCode0
Towards Diverse and Natural Image Descriptions via a Conditional GANCode0
Scratch that! An Evolution-based Adversarial Attack against Neural NetworksCode0
Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal AnalysisCode0
IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of ImagesCode0
Image Captioning with Deep Bidirectional LSTMsCode0
Towards Fairness in Visual Recognition: Effective Strategies for Bias MitigationCode0
Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial AttacksCode0
Attention on Attention for Image CaptioningCode0
Towards Interpretable Reinforcement Learning Using Attention Augmented AgentsCode0
Cognitive resilience: Unraveling the proficiency of image-captioning models to interpret masked visual contentCode0
MI-VisionShot: Few-shot adaptation of vision-language models for slide-level classification of histopathological imagesCode0
Attention-Based Models for Text-Dependent Speaker VerificationCode0
Diffusion Based Augmentation for Captioning and Retrieval in Cultural HeritageCode0
Image Captioning with Clause-Focused Metrics in a Multi-Modal Setting for MarketingCode0
Descriptive Caption Enhancement with Visual Specialists for Multimodal PerceptionCode0
MMT: Image-guided Story Ending Generation with Multimodal Memory TransformerCode0
Watch What You Just Said: Image Captioning with Text-Conditional AttentionCode0
Image Captioning with Attention for Smart Local Tourism using EfficientNetCode0
Self-adaptive Multimodal Retrieval-Augmented GenerationCode0
Model-Agnostic Gender Debiased Image CaptioningCode0
AICAttack: Adversarial Image Captioning Attack with Attention-Based OptimizationCode0
Video Summarization: Towards Entity-Aware CaptionsCode0
Cobra Effect in Reference-Free Image Captioning MetricsCode0
Describing like humans: on diversity in image captioningCode0
Image Captioning via Dynamic Path CustomizationCode0
Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of EmbeddingsCode0
Image Captioning using Deep Neural ArchitecturesCode0
Temporally-Grounded Language Generation: A Benchmark for Real-Time Vision-Language ModelsCode0
Adaptive Testing of Computer Vision ModelsCode0
Image Captioning: Transforming Objects into WordsCode0
Attend to You: Personalized Image Captioning with Context Sequence Memory NetworksCode0
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover DistanceCode0
Image Captioning for Effective Use of Language Models in Knowledge-Based Visual Question AnsweringCode0
CNN Fixations: An unraveling approach to visualize the discriminative image regionsCode0
Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 ChallengeCode0
Semantic Compositional Networks for Visual CaptioningCode0
Image CaptioningCode0
Attacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image CaptioningCode0
Image Caption Generation for News ArticlesCode0
Show:102550
← PrevPage 33 of 38Next →
All datasetsVizWiz 2020 test-devCOCO Captionsnocaps in-domainnocaps near-domainnocaps out-of-domainnocaps entireCOCO (Common Objects in Context)VizWiz 2020 testnocaps-XD entirenocaps-val-in-domainnocaps-val-overallnocaps-XD in-domain

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1IBM Research AICIDEr80.67—Unverified
2CASIA_IVACIDEr79.15—Unverified
3feixiangCIDEr77.31—Unverified
4wocaoCIDEr77.21—Unverified
5lamiwab172CIDEr75.93—Unverified
6RUC_AIM3CIDEr73.52—Unverified
7funasCIDEr73.51—Unverified
8SRC-B_VCLabCIDEr73.47—Unverified
9spartaCIDEr73.41—Unverified
10x-vizCIDEr73.26—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORCIDER152.5—Unverified
2VASTCIDER149—Unverified
3Virtex (ResNet-101)CIDER94—Unverified
4KOSMOS-1 (1.6B) (zero-shot)CIDER84.7—Unverified
5BLIP-FuseCapCLIPScore78.5—Unverified
6mPLUGBLEU-446.5—Unverified
7OFABLEU-444.9—Unverified
8GITBLEU-444.1—Unverified
9BLIP-2 ViT-G OPT 2.7B (zero-shot)BLEU-443.7—Unverified
10BLIP-2 ViT-G OPT 6.7B (zero-shot)BLEU-443.5—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr149.1—Unverified
2GIT2, Single ModelCIDEr124.18—Unverified
3GIT, Single ModelCIDEr122.4—Unverified
4PaLICIDEr121.09—Unverified
5CoCa - Google BrainCIDEr117.9—Unverified
6Microsoft Cognitive Services teamCIDEr112.82—Unverified
7Single ModelCIDEr108.98—Unverified
8GRIT (zero-shot, no VL pretraining, no CBS)CIDEr105.9—Unverified
9FudanFVLCIDEr104.9—Unverified
10FudanWYZCIDEr104.25—Unverified
#ModelMetricClaimedVerifiedStatus
1GIT2, Single ModelCIDEr125.51—Unverified
2PaLICIDEr124.35—Unverified
3GIT, Single ModelCIDEr123.92—Unverified
4CoCa - Google BrainCIDEr120.73—Unverified
5Microsoft Cognitive Services teamCIDEr115.54—Unverified
6Single ModelCIDEr110.76—Unverified
7FudanFVLCIDEr109.33—Unverified
8FudanWYZCIDEr108.04—Unverified
9IEDA-LABCIDEr100.15—Unverified
10firetheholeCIDEr99.51—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr126.67—Unverified
2GIT2, Single ModelCIDEr122.27—Unverified
3GIT, Single ModelCIDEr122.04—Unverified
4CoCa - Google BrainCIDEr121.69—Unverified
5Microsoft Cognitive Services teamCIDEr110.14—Unverified
6Single ModelCIDEr109.49—Unverified
7FudanFVLCIDEr106.55—Unverified
8FudanWYZCIDEr103.75—Unverified
9HumanCIDEr91.62—Unverified
10firetheholeCIDEr88.54—Unverified