Image Captioning

Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.

( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1851–1878 of 1878 papers

Title	Date	Tasks	Status	Hype
A Critical Review of Recurrent Neural Networks for Sequence Learning	May 29, 2015	Handwriting RecognitionImage Captioning	CodeCode Available	0
Exploring Nearest Neighbor Approaches for Image Captioning	May 17, 2015	Image Captioning	CodeCode Available	0
Language Models for Image Captioning: The Quirks and What Works	May 7, 2015	Image CaptioningLanguage Modeling	—Unverified	0
VQA: Visual Question Answering	May 3, 2015	Image CaptioningMultiple-choice	CodeCode Available	1
Déjà Image-Captions: A Corpus of Expressive Descriptions in Repetition	May 1, 2015	Image CaptioningImage Retrieval	—Unverified	0
Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images	Apr 25, 2015	Image CaptioningNovel Concepts	CodeCode Available	0
Phrase-based Image Captioning	Feb 12, 2015	DescriptiveImage Captioning	—Unverified	0
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention	Feb 10, 2015	Caption GenerationImage Captioning	CodeCode Available	1
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)	Dec 20, 2014	8kImage Captioning	CodeCode Available	0
Deep Visual-Semantic Alignments for Generating Image Descriptions	Dec 7, 2014	Cross-Modal RetrievalImage Captioning	CodeCode Available	0
CIDEr: Consensus-based Image Description Evaluation	Nov 20, 2014	Action RecognitionAttribute	CodeCode Available	1
From Captions to Visual Concepts and Back	Nov 18, 2014	Image CaptioningLanguage Modeling	CodeCode Available	0
Show and Tell: A Neural Image Caption Generator	Nov 17, 2014	Image CaptioningImage Retrieval with Multi-Modal Query	CodeCode Available	1
Recurrent Neural Network Regularization	Sep 8, 2014	Caption GenerationImage Captioning	CodeCode Available	0
Towards Succinct and Relevant Image Descriptions	Aug 1, 2014	Image CaptioningText Generation	—Unverified	0
Multi-layered Image Representation for Image Interpretation	Aug 1, 2014	Image CaptioningImage Retrieval	—Unverified	0
Domain-Specific Image Captioning	Jun 1, 2014	Image CaptioningSentence Compression	—Unverified	0
Nonparametric Method for Data-driven Image Captioning	Jun 1, 2014	Density EstimationImage Captioning	—Unverified	0
Image Annotation with ISO-Space: Distinguishing Content from Structure	May 1, 2014	Content-Based Image RetrievalImage Captioning	—Unverified	0
TreeTalk: Composition and Compression of Trees for Image Descriptions	Jan 1, 2014	Image CaptioningImage Retrieval	—Unverified	0
Generalizing Image Captions for Image-Text Parallel Corpus	Aug 1, 2013	Image CaptioningSentence Compression	—Unverified	0
Annotation of Online Shopping Images without Labeled Training Examples	Jun 1, 2013	Image CaptioningObject Recognition	—Unverified	0
Domain-Independent Captioning of Domain-Specific Images	Jun 1, 2013	Image CaptioningImage Retrieval	—Unverified	0
TakeLab: Systems for Measuring Semantic Text Similarity	Jul 1, 2012	Image CaptioningInformation Retrieval	—Unverified	0
Unsupervised Disambiguation of Image Captions	Jul 1, 2012	Image CaptioningWord Sense Disambiguation	—Unverified	0
Correlation between Similarity Measures for Inter-Language Linked Wikipedia Articles	May 1, 2012	ArticlesImage Captioning	—Unverified	0
TTT: A Tree Transduction Language for Syntactic and Semantic Processing	Apr 1, 2012	Image Captioning	—Unverified	0
Im2Text: Describing Images Using 1 Million Captioned Photographs	Dec 1, 2011	Image CaptioningImage Description	—Unverified	0

Show:10 25 50

← PrevPage 38 of 38Next →

All datasets VizWiz 2020 test-dev COCO Captions nocaps in-domain nocaps near-domain nocaps out-of-domain nocaps entire COCO (Common Objects in Context)VizWiz 2020 test nocaps-XD entire nocaps-val-in-domain nocaps-val-overall nocaps-XD in-domain

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	IBM Research AI	CIDEr	80.67	—	Unverified
2	CASIA_IVA	CIDEr	79.15	—	Unverified
3	feixiang	CIDEr	77.31	—	Unverified
4	wocao	CIDEr	77.21	—	Unverified
5	lamiwab172	CIDEr	75.93	—	Unverified
6	RUC_AIM3	CIDEr	73.52	—	Unverified
7	funas	CIDEr	73.51	—	Unverified
8	SRC-B_VCLab	CIDEr	73.47	—	Unverified
9	sparta	CIDEr	73.41	—	Unverified
10	x-viz	CIDEr	73.26	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VALOR	CIDER	152.5	—	Unverified
2	VAST	CIDER	149	—	Unverified
3	Virtex (ResNet-101)	CIDER	94	—	Unverified
4	KOSMOS-1 (1.6B) (zero-shot)	CIDER	84.7	—	Unverified
5	BLIP-FuseCap	CLIPScore	78.5	—	Unverified
6	mPLUG	BLEU-4	46.5	—	Unverified
7	OFA	BLEU-4	44.9	—	Unverified
8	GIT	BLEU-4	44.1	—	Unverified
9	BLIP-2 ViT-G OPT 2.7B (zero-shot)	BLEU-4	43.7	—	Unverified
10	BLIP-2 ViT-G OPT 6.7B (zero-shot)	BLEU-4	43.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLI	CIDEr	149.1	—	Unverified
2	GIT2, Single Model	CIDEr	124.18	—	Unverified
3	GIT, Single Model	CIDEr	122.4	—	Unverified
4	PaLI	CIDEr	121.09	—	Unverified
5	CoCa - Google Brain	CIDEr	117.9	—	Unverified
6	Microsoft Cognitive Services team	CIDEr	112.82	—	Unverified
7	Single Model	CIDEr	108.98	—	Unverified
8	GRIT (zero-shot, no VL pretraining, no CBS)	CIDEr	105.9	—	Unverified
9	FudanFVL	CIDEr	104.9	—	Unverified
10	FudanWYZ	CIDEr	104.25	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GIT2, Single Model	CIDEr	125.51	—	Unverified
2	PaLI	CIDEr	124.35	—	Unverified
3	GIT, Single Model	CIDEr	123.92	—	Unverified
4	CoCa - Google Brain	CIDEr	120.73	—	Unverified
5	Microsoft Cognitive Services team	CIDEr	115.54	—	Unverified
6	Single Model	CIDEr	110.76	—	Unverified
7	FudanFVL	CIDEr	109.33	—	Unverified
8	FudanWYZ	CIDEr	108.04	—	Unverified
9	IEDA-LAB	CIDEr	100.15	—	Unverified
10	firethehole	CIDEr	99.51	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	PaLI	CIDEr	126.67	—	Unverified
2	GIT2, Single Model	CIDEr	122.27	—	Unverified
3	GIT, Single Model	CIDEr	122.04	—	Unverified
4	CoCa - Google Brain	CIDEr	121.69	—	Unverified
5	Microsoft Cognitive Services team	CIDEr	110.14	—	Unverified
6	Single Model	CIDEr	109.49	—	Unverified
7	FudanFVL	CIDEr	106.55	—	Unverified
8	FudanWYZ	CIDEr	103.75	—	Unverified
9	Human	CIDEr	91.62	—	Unverified
10	firethehole	CIDEr	88.54	—	Unverified