Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 451–473 of 473 papers

Title	Date	Tasks	Status
Bidirectional Multirate Reconstruction for Temporal Modeling in Videos	Nov 28, 2016	Event DetectionVideo Captioning	—Unverified
Video Captioning with Transferred Semantic Attributes	Nov 23, 2016	SentenceVideo Captioning	—Unverified
Adaptive Feature Abstraction for Translating Video to Text	Nov 23, 2016	Video Captioning	—Unverified
Recurrent Memory Addressing for describing videos	Nov 20, 2016	Video Captioning	—Unverified
Multimodal Memory Modelling for Video Captioning	Nov 17, 2016	DecoderSentence	—Unverified
Spatio-Temporal Attention Models for Grounded Video Captioning	Oct 17, 2016	image-classificationImage Classification	—Unverified
End-to-end Concept Word Detection for Video Captioning, Retrieval, and Question Answering	Oct 10, 2016	Language ModelingLanguage Modelling	—Unverified
Deep Learning for Video Classification and Captioning	Sep 22, 2016	ClassificationDeep Learning	CodeCode Available
Title Generation for User Generated Videos	Aug 25, 2016	SentenceVideo Captioning	—Unverified
MUTT: Metric Unit TesTing for Language Generation Tasks	Aug 1, 2016	Image CaptioningMachine Translation	—Unverified
A Shared Task on Multimodal Machine Translation and Crosslingual Image Description	Aug 1, 2016	Image DescriptionImage Retrieval	—Unverified
Bidirectional Long-Short Term Memory for Video Description	Jun 15, 2016	Language ModelingLanguage Modelling	—Unverified
Storytelling of Photo Stream with Bidirectional Multi-thread Recurrent Neural Network	Jun 2, 2016	Video CaptioningVisual Storytelling	—Unverified
Beyond Caption To Narrative: Video Captioning With Multiple Sentences	May 18, 2016	Action LocalizationImage Captioning	—Unverified
Video Description using Bidirectional Recurrent Neural Networks	Apr 12, 2016	DecoderText Generation	CodeCode Available
A Restricted Visual Turing Test for Deep Scene and Event Understanding	Dec 6, 2015	Question AnsweringVideo Captioning	—Unverified
Delving Deeper into Convolutional Networks for Learning Video Representations	Nov 19, 2015	Action RecognitionDecoder	CodeCode Available
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data	Nov 17, 2015	Image CaptioningNovel Concepts	CodeCode Available
Oracle performance for visual captioning	Nov 14, 2015	Image CaptioningLanguage Modeling	CodeCode Available
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning	Nov 11, 2015	image-classificationImage Classification	—Unverified
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks	Oct 26, 2015	SentenceVideo Captioning	—Unverified
Translating Videos to Natural Language Using Deep Recurrent Neural Networks	Dec 15, 2014	SentenceText Generation	—Unverified
Chinese Whispers: Cooperative Paraphrase Acquisition	May 1, 2012	Machine TranslationNatural Language Inference	—Unverified

Show:10 25 50

← PrevPage 10 of 10Next →

All datasets MSR-VTT MSVD YouCook2 VATEX ActivityNet Captions MSRVTT-CTN MSVD-CTN Hindi MSR-VTT TVC ChinaOpen-1k MSVD-Indonesian Shot2Story20K

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	mPLUG-2	CIDEr	80	—	Unverified
2	VAST	CIDEr	78	—	Unverified
3	GIT2	CIDEr	75.9	—	Unverified
4	VLAB	CIDEr	74.9	—	Unverified
5	COSA	CIDEr	74.7	—	Unverified
6	VALOR	CIDEr	74	—	Unverified
7	MaMMUT (ours)	CIDEr	73.6	—	Unverified
8	VideoCoCa	CIDEr	73.2	—	Unverified
9	RTQ	CIDEr	69.3	—	Unverified
10	HowToCaption	CIDEr	65.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MaMMUT	CIDEr	195.6	—	Unverified
2	VLAB	CIDEr	179.8	—	Unverified
3	COSA	CIDEr	178.5	—	Unverified
4	VALOR	CIDEr	178.5	—	Unverified
5	mPLUG-2	CIDEr	165.8	—	Unverified
6	HowToCaption	CIDEr	154.2	—	Unverified
7	HiTeA	CIDEr	146.9	—	Unverified
8	Vid2Seq	CIDEr	146.2	—	Unverified
9	VIOLETv2	CIDEr	139.2	—	Unverified
10	RTQ	CIDEr	123.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	18.2	—	Unverified
2	UniVL + MELTR	BLEU-4	17.92	—	Unverified
3	UniVL	BLEU-4	17.35	—	Unverified
4	VideoCoCa	BLEU-4	14.2	—	Unverified
5	VLM	BLEU-4	12.27	—	Unverified
6	E2vidD6-MASSvid-BiD	BLEU-4	12.04	—	Unverified
7	TextKG	BLEU-4	11.7	—	Unverified
8	COOT	BLEU-4	11.3	—	Unverified
9	COSA	BLEU-4	10.1	—	Unverified
10	HowToCaption	BLEU-4	8.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VALOR	BLEU-4	45.6	—	Unverified
2	VAST	BLEU-4	45	—	Unverified
3	COSA	BLEU-4	43.7	—	Unverified
4	VideoCoCa	BLEU-4	39.7	—	Unverified
5	IcoCap (ViT-B/16)	BLEU-4	37.4	—	Unverified
6	IcoCap (ViT-B/32)	BLEU-4	36.9	—	Unverified
7	VASTA (Kinetics-backbone)	BLEU-4	36.25	—	Unverified
8	CoCap (ViT/L14)	BLEU-4	35.8	—	Unverified
9	ORG-TRL	BLEU-4	32.1	—	Unverified
10	NITS-VC	BLEU-4	20	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VideoCoCa	BLEU4	14.7	—	Unverified
2	VLTinT (ae-test split) C3D/Ling	BLEU4	14.5	—	Unverified
3	VLCap (ae-test split) - Appearance + Language	BLEU4	13.38	—	Unverified
4	COOT (ae-test split) - Only Appearance features	BLEU4	10.85	—	Unverified
5	MART (ae-test split) - Appearance + Flow	BLEU4	10.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	49.87	—	Unverified
2	GIT	CIDEr	32.43	—	Unverified
3	SEM-POS	CIDEr	26.01	—	Unverified
4	AKGNN	CIDEr	25.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	63.51	—	Unverified
2	GIT	CIDEr	45.63	—	Unverified
3	SEM-POS	CIDEr	37.16	—	Unverified
4	AKGNN	CIDEr	35.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SBD_Keyframe	BLEU4	41.01	—	Unverified
2	V+S-Att-based	BLEU4	36.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	19.9	—	Unverified
2	COSA	BLEU-4	18.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GVT	BLEU4	17.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VNS-GRU (Cross-Lingual)	BLEU-4	58.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Shot2Story	CIDEr	37.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Vid2Seq	CIDEr	120.5	—	Unverified