Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 401–425 of 473 papers

Title	Date	Tasks	Status
Incorporating Background Knowledge into Video Description Generation	Oct 1, 2018	DecoderText Generation	—Unverified
Temporally Grounding Natural Sentence in Video	Oct 1, 2018	SentenceVideo Captioning	—Unverified
A Dataset for Telling the Stories of Social Media Videos	Oct 1, 2018	SentenceVideo Captioning	—Unverified
Vector Learning for Cross Domain Representations	Sep 27, 2018	DecoderImage Captioning	—Unverified
MTLE: A Multitask Learning Encoder of Visual Feature Representations for Video and Movie Description	Sep 19, 2018	DecoderVideo Captioning	CodeCode Available
Bridge Video and Text with Cascade Syntactic Structure	Aug 1, 2018	AttributeObject	—Unverified
Move Forward and Tell: A Progressive Generator of Video Descriptions	Jul 26, 2018	DescriptiveSentence	—Unverified
NMT-Keras: a Very Flexible Toolkit with a Focus on Interactive NMT and Online Learning	Jul 9, 2018	General ClassificationMachine Translation	—Unverified
Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction	Jul 8, 2018	DecoderLanguage Modeling	—Unverified
Deep Reinforcement Learning for NLP	Jul 1, 2018	Atari Gamescoreference-resolution	—Unverified
Best Vision Technologies Submission to ActivityNet Challenge 2018-Task: Dense-Captioning Events in Videos	Jun 25, 2018	Dense CaptioningOptical Flow Estimation	—Unverified
RUC+CMU: System Report for Dense Captioning Events in Videos	Jun 22, 2018	Caption GenerationDense Captioning	—Unverified
Fine-Grained Video Captioning for Sports Narrative	Jun 1, 2018	2kVideo Captioning	—Unverified
M3: Multimodal Memory Modelling for Video Captioning	Jun 1, 2018	SentenceVideo Captioning	—Unverified
Interpretable Video Captioning via Trajectory Structured Localization	Jun 1, 2018	DecoderImage Captioning	—Unverified
Amortized Context Vector Inference for Sequence-to-Sequence Networks	May 23, 2018	Document SummarizationVariational Inference	—Unverified
ECO: Efficient Convolutional Network for Online Video Understanding	Apr 24, 2018	Action ClassificationAction Recognition	CodeCode Available
Jointly Localizing and Describing Events for Dense Video Captioning	Apr 23, 2018	AttributeDense Video Captioning	—Unverified
Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video Captioning	Apr 15, 2018	Video CaptioningVideo Understanding	CodeCode Available
End-to-End Dense Video Captioning with Masked Transformer	Apr 3, 2018	DecoderDense Video Captioning	CodeCode Available
Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning	Mar 31, 2018	DecoderDense Video Captioning	CodeCode Available
Reconstruction Network for Video Captioning	Mar 30, 2018	DecoderSentence	CodeCode Available
End-to-End Video Captioning with Multitask Reinforcement Learning	Mar 21, 2018	GPUreinforcement-learning	CodeCode Available
Less Is More: Picking Informative Frames for Video Captioning	Mar 5, 2018	DecoderDiversity	—Unverified
Joint Event Detection and Description in Continuous Video Streams	Feb 28, 2018	Dense CaptioningDense Video Captioning	CodeCode Available

Show:10 25 50

← PrevPage 17 of 19Next →

All datasets MSR-VTT MSVD YouCook2 VATEX ActivityNet Captions MSRVTT-CTN MSVD-CTN Hindi MSR-VTT TVC ChinaOpen-1k MSVD-Indonesian Shot2Story20K

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	mPLUG-2	CIDEr	80	—	Unverified
2	VAST	CIDEr	78	—	Unverified
3	GIT2	CIDEr	75.9	—	Unverified
4	VLAB	CIDEr	74.9	—	Unverified
5	COSA	CIDEr	74.7	—	Unverified
6	VALOR	CIDEr	74	—	Unverified
7	MaMMUT (ours)	CIDEr	73.6	—	Unverified
8	VideoCoCa	CIDEr	73.2	—	Unverified
9	RTQ	CIDEr	69.3	—	Unverified
10	HowToCaption	CIDEr	65.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MaMMUT	CIDEr	195.6	—	Unverified
2	VLAB	CIDEr	179.8	—	Unverified
3	COSA	CIDEr	178.5	—	Unverified
4	VALOR	CIDEr	178.5	—	Unverified
5	mPLUG-2	CIDEr	165.8	—	Unverified
6	HowToCaption	CIDEr	154.2	—	Unverified
7	HiTeA	CIDEr	146.9	—	Unverified
8	Vid2Seq	CIDEr	146.2	—	Unverified
9	VIOLETv2	CIDEr	139.2	—	Unverified
10	RTQ	CIDEr	123.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	18.2	—	Unverified
2	UniVL + MELTR	BLEU-4	17.92	—	Unverified
3	UniVL	BLEU-4	17.35	—	Unverified
4	VideoCoCa	BLEU-4	14.2	—	Unverified
5	VLM	BLEU-4	12.27	—	Unverified
6	E2vidD6-MASSvid-BiD	BLEU-4	12.04	—	Unverified
7	TextKG	BLEU-4	11.7	—	Unverified
8	COOT	BLEU-4	11.3	—	Unverified
9	COSA	BLEU-4	10.1	—	Unverified
10	HowToCaption	BLEU-4	8.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VALOR	BLEU-4	45.6	—	Unverified
2	VAST	BLEU-4	45	—	Unverified
3	COSA	BLEU-4	43.7	—	Unverified
4	VideoCoCa	BLEU-4	39.7	—	Unverified
5	IcoCap (ViT-B/16)	BLEU-4	37.4	—	Unverified
6	IcoCap (ViT-B/32)	BLEU-4	36.9	—	Unverified
7	VASTA (Kinetics-backbone)	BLEU-4	36.25	—	Unverified
8	CoCap (ViT/L14)	BLEU-4	35.8	—	Unverified
9	ORG-TRL	BLEU-4	32.1	—	Unverified
10	NITS-VC	BLEU-4	20	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VideoCoCa	BLEU4	14.7	—	Unverified
2	VLTinT (ae-test split) C3D/Ling	BLEU4	14.5	—	Unverified
3	VLCap (ae-test split) - Appearance + Language	BLEU4	13.38	—	Unverified
4	COOT (ae-test split) - Only Appearance features	BLEU4	10.85	—	Unverified
5	MART (ae-test split) - Appearance + Flow	BLEU4	10.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	49.87	—	Unverified
2	GIT	CIDEr	32.43	—	Unverified
3	SEM-POS	CIDEr	26.01	—	Unverified
4	AKGNN	CIDEr	25.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	63.51	—	Unverified
2	GIT	CIDEr	45.63	—	Unverified
3	SEM-POS	CIDEr	37.16	—	Unverified
4	AKGNN	CIDEr	35.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SBD_Keyframe	BLEU4	41.01	—	Unverified
2	V+S-Att-based	BLEU4	36.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	19.9	—	Unverified
2	COSA	BLEU-4	18.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GVT	BLEU4	17.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VNS-GRU (Cross-Lingual)	BLEU-4	58.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Shot2Story	CIDEr	37.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Vid2Seq	CIDEr	120.5	—	Unverified