SOTAVerified

Image Captioning

Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.

( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 901–950 of 1878 papers

TitleStatusHype
On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization—0
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionsCode1
Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam SearchCode1
It Isn't Sh!tposting, It's My CAT Posting—0
Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image CaptioningCode1
Understanding Transfer Learning for Chest Radiograph Clinical Report Generation with Modified Transformer Architectures—0
Language Models Can See: Plugging Visual Controls in Text GenerationCode2
CoCa: Contrastive Captioners are Image-Text Foundation ModelsCode1
All You May Need for VQA are Image CaptionsCode3
Diverse Image Captioning with Grounded StyleCode0
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering—0
Combine to Describe: Evaluating Compositional Generalization in Image Captioning—0
Molecular Identification from AFM images using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks—0
Controllable Image Captioning—0
CapOnImage: Context-driven Dense-Captioning on Image—0
Cross-view Brain Decoding—0
It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data CollectionCode1
Guiding Attention using Partial-Order Relationships for Image Captioning—0
Image Captioning In the Transformer AgeCode1
Robust Cross-Modal Representation Learning with Progressive Self-Distillation—0
Semantic Exploration from Language Abstractions and Pretrained Representations—0
On Distinctive Image Captioning via Comparing and Reweighting—0
Multimodal Quasi-AutoRegression: Forecasting the visual popularity of new fashion products—0
Learning Audio-Video Modalities from Image Captions—0
Socratic Models: Composing Zero-Shot Multimodal Reasoning with LanguageCode0
Quantifying Societal Bias Amplification in Image CaptioningCode1
End-to-End Transformer Based Model for Image CaptioningCode1
Linking Emergent and Natural Languages via Corpus TransferCode1
WuDaoMM: A large-scale Multi-Modal Dataset for Pre-training models—0
AlignTransformer: Hierarchical Alignment of Visual Regions and Disease Tags for Medical Report Generation—0
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training—0
On Vision Features in Multimodal Machine TranslationCode1
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer—0
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations—0
Chart-to-Text: A Large-Scale Benchmark for Chart SummarizationCode1
Taking an Emotional Look at Video Paragraph Captioning—0
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation—0
Geodesic Multi-Modal Mixup for Robust Fine-TuningCode0
Semantic Distillation Guided Salient Object Detection—0
Unpaired Image Captioning by Image-level Weakly-Supervised Visual Concept Recognition—0
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in ContextCode1
A Deep Neural Framework for Image Caption Generation Using GRU-Based Attention Mechanism—0
Interactive Machine Learning for Image Captioning—0
CaMEL: Mean Teacher Learning for Image CaptioningCode1
I-Tuning: Tuning Frozen Language Models with Image for Lightweight Image Captioning—0
ACORT: A Compact Object Relation Transformer for Parameter Efficient Image CaptioningCode1
Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm—0
Describing image focused in cognitive and visual details for visually impaired people: An approach to generating inclusive paragraphs—0
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation ModelsCode3
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning FrameworkCode0
Show:102550
← PrevPage 19 of 38Next →
All datasetsVizWiz 2020 test-devCOCO Captionsnocaps in-domainnocaps near-domainnocaps out-of-domainnocaps entireCOCO (Common Objects in Context)VizWiz 2020 testnocaps-XD entirenocaps-val-in-domainnocaps-val-overallnocaps-XD in-domain

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1IBM Research AICIDEr80.67—Unverified
2CASIA_IVACIDEr79.15—Unverified
3feixiangCIDEr77.31—Unverified
4wocaoCIDEr77.21—Unverified
5lamiwab172CIDEr75.93—Unverified
6RUC_AIM3CIDEr73.52—Unverified
7funasCIDEr73.51—Unverified
8SRC-B_VCLabCIDEr73.47—Unverified
9spartaCIDEr73.41—Unverified
10x-vizCIDEr73.26—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORCIDER152.5—Unverified
2VASTCIDER149—Unverified
3Virtex (ResNet-101)CIDER94—Unverified
4KOSMOS-1 (1.6B) (zero-shot)CIDER84.7—Unverified
5BLIP-FuseCapCLIPScore78.5—Unverified
6mPLUGBLEU-446.5—Unverified
7OFABLEU-444.9—Unverified
8GITBLEU-444.1—Unverified
9BLIP-2 ViT-G OPT 2.7B (zero-shot)BLEU-443.7—Unverified
10BLIP-2 ViT-G OPT 6.7B (zero-shot)BLEU-443.5—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr149.1—Unverified
2GIT2, Single ModelCIDEr124.18—Unverified
3GIT, Single ModelCIDEr122.4—Unverified
4PaLICIDEr121.09—Unverified
5CoCa - Google BrainCIDEr117.9—Unverified
6Microsoft Cognitive Services teamCIDEr112.82—Unverified
7Single ModelCIDEr108.98—Unverified
8GRIT (zero-shot, no VL pretraining, no CBS)CIDEr105.9—Unverified
9FudanFVLCIDEr104.9—Unverified
10FudanWYZCIDEr104.25—Unverified
#ModelMetricClaimedVerifiedStatus
1GIT2, Single ModelCIDEr125.51—Unverified
2PaLICIDEr124.35—Unverified
3GIT, Single ModelCIDEr123.92—Unverified
4CoCa - Google BrainCIDEr120.73—Unverified
5Microsoft Cognitive Services teamCIDEr115.54—Unverified
6Single ModelCIDEr110.76—Unverified
7FudanFVLCIDEr109.33—Unverified
8FudanWYZCIDEr108.04—Unverified
9IEDA-LABCIDEr100.15—Unverified
10firetheholeCIDEr99.51—Unverified
#ModelMetricClaimedVerifiedStatus
1PaLICIDEr126.67—Unverified
2GIT2, Single ModelCIDEr122.27—Unverified
3GIT, Single ModelCIDEr122.04—Unverified
4CoCa - Google BrainCIDEr121.69—Unverified
5Microsoft Cognitive Services teamCIDEr110.14—Unverified
6Single ModelCIDEr109.49—Unverified
7FudanFVLCIDEr106.55—Unverified
8FudanWYZCIDEr103.75—Unverified
9HumanCIDEr91.62—Unverified
10firetheholeCIDEr88.54—Unverified