Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 376–400 of 473 papers

Title	Date	Tasks	Status	Hype
Reconstruct and Represent Video Contents for Captioning via Reinforcement Learning	Jun 3, 2019	Decoderreinforcement-learning	—Unverified	0
Learning to Generate Grounded Visual Captions without Localization Supervision	Jun 1, 2019	Image CaptioningLanguage Modelling	CodeCode Available	1
Interactive-predictive neural multimodal systems	May 30, 2019	Machine TranslationTranslation	—Unverified	0
A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks	May 20, 2019	Machine TranslationTranslation	CodeCode Available	0
On Flow Profile Image for Video Representation	May 12, 2019	Activity RecognitionOptical Flow Estimation	—Unverified	0
Memory-Attended Recurrent Network for Video Captioning	May 10, 2019	DecoderVideo Captioning	—Unverified	0
Multimodal Semantic Attention Network for Video Captioning	May 8, 2019	AttributeDecoder	—Unverified	0
Temporal Deformable Convolutional Encoder-Decoder Networks for Video Captioning	May 3, 2019	DecoderSentence	CodeCode Available	0
Hierarchical Recurrent Neural Network for Video Summarization	Apr 28, 2019	Video CaptioningVideo Summarization	—Unverified	0
Large Scale Holistic Video Understanding	Apr 25, 2019	Action ClassificationAction Recognition	CodeCode Available	1
Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?	Apr 11, 2019	Machine TranslationTranslation	CodeCode Available	0
What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment	Apr 8, 2019	Action ClassificationAction Quality Assessment	CodeCode Available	1
Streamlined Dense Video Captioning	Apr 8, 2019	Dense Video CaptioningReinforcement Learning	CodeCode Available	0
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research	Apr 6, 2019	Machine TranslationTranslation	CodeCode Available	1
End-to-End Video Captioning	Apr 4, 2019	Action RecognitionCaption Generation	—Unverified	0
VideoBERT: A Joint Model for Video and Language Representation Learning	Apr 3, 2019	Action ClassificationGeneral Classification	CodeCode Available	0
M-VAD Names: a Dataset for Video Captioning with Naming	Mar 4, 2019	TAGVideo Captioning	CodeCode Available	0
Spatio-Temporal Dynamics and Semantic Attribute Enriched Visual Encoding for Video Captioning	Feb 27, 2019	AttributeCaption Generation	—Unverified	0
Not All Words are Equal: Video-specific Information Loss for Video Captioning	Jan 1, 2019	AllVideo Captioning	—Unverified	0
Hierarchical LSTMs with Adaptive Attention for Visual Captioning	Dec 26, 2018	Caption GenerationImage Captioning	—Unverified	0
An Attempt towards Interpretable Audio-Visual Video Captioning	Dec 7, 2018	Audio captioningAudio-Visual Video Captioning	—Unverified	0
Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning	Nov 7, 2018	Mixture-of-ExpertsVideo Captioning	—Unverified	0
Middle-Out Decoding	Oct 28, 2018	DecoderDiversity	—Unverified	0
Cross-Modal and Hierarchical Modeling of Video and Text	Oct 16, 2018	Action RecognitionRetrieval	CodeCode Available	0
Image-to-Video Person Re-Identification by Reusing Cross-modal Embeddings	Oct 4, 2018	Image CaptioningImage-To-Video Person Re-Identification	—Unverified	0

Show:10 25 50

← PrevPage 16 of 19Next →

All datasets MSR-VTT MSVD YouCook2 VATEX ActivityNet Captions MSRVTT-CTN MSVD-CTN Hindi MSR-VTT TVC ChinaOpen-1k MSVD-Indonesian Shot2Story20K

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	mPLUG-2	CIDEr	80	—	Unverified
2	VAST	CIDEr	78	—	Unverified
3	GIT2	CIDEr	75.9	—	Unverified
4	VLAB	CIDEr	74.9	—	Unverified
5	COSA	CIDEr	74.7	—	Unverified
6	VALOR	CIDEr	74	—	Unverified
7	MaMMUT (ours)	CIDEr	73.6	—	Unverified
8	VideoCoCa	CIDEr	73.2	—	Unverified
9	RTQ	CIDEr	69.3	—	Unverified
10	HowToCaption	CIDEr	65.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MaMMUT	CIDEr	195.6	—	Unverified
2	VLAB	CIDEr	179.8	—	Unverified
3	COSA	CIDEr	178.5	—	Unverified
4	VALOR	CIDEr	178.5	—	Unverified
5	mPLUG-2	CIDEr	165.8	—	Unverified
6	HowToCaption	CIDEr	154.2	—	Unverified
7	HiTeA	CIDEr	146.9	—	Unverified
8	Vid2Seq	CIDEr	146.2	—	Unverified
9	VIOLETv2	CIDEr	139.2	—	Unverified
10	RTQ	CIDEr	123.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	18.2	—	Unverified
2	UniVL + MELTR	BLEU-4	17.92	—	Unverified
3	UniVL	BLEU-4	17.35	—	Unverified
4	VideoCoCa	BLEU-4	14.2	—	Unverified
5	VLM	BLEU-4	12.27	—	Unverified
6	E2vidD6-MASSvid-BiD	BLEU-4	12.04	—	Unverified
7	TextKG	BLEU-4	11.7	—	Unverified
8	COOT	BLEU-4	11.3	—	Unverified
9	COSA	BLEU-4	10.1	—	Unverified
10	HowToCaption	BLEU-4	8.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VALOR	BLEU-4	45.6	—	Unverified
2	VAST	BLEU-4	45	—	Unverified
3	COSA	BLEU-4	43.7	—	Unverified
4	VideoCoCa	BLEU-4	39.7	—	Unverified
5	IcoCap (ViT-B/16)	BLEU-4	37.4	—	Unverified
6	IcoCap (ViT-B/32)	BLEU-4	36.9	—	Unverified
7	VASTA (Kinetics-backbone)	BLEU-4	36.25	—	Unverified
8	CoCap (ViT/L14)	BLEU-4	35.8	—	Unverified
9	ORG-TRL	BLEU-4	32.1	—	Unverified
10	NITS-VC	BLEU-4	20	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VideoCoCa	BLEU4	14.7	—	Unverified
2	VLTinT (ae-test split) C3D/Ling	BLEU4	14.5	—	Unverified
3	VLCap (ae-test split) - Appearance + Language	BLEU4	13.38	—	Unverified
4	COOT (ae-test split) - Only Appearance features	BLEU4	10.85	—	Unverified
5	MART (ae-test split) - Appearance + Flow	BLEU4	10.33	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	49.87	—	Unverified
2	GIT	CIDEr	32.43	—	Unverified
3	SEM-POS	CIDEr	26.01	—	Unverified
4	AKGNN	CIDEr	25.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	CEN	CIDEr	63.51	—	Unverified
2	GIT	CIDEr	45.63	—	Unverified
3	SEM-POS	CIDEr	37.16	—	Unverified
4	AKGNN	CIDEr	35.08	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SBD_Keyframe	BLEU4	41.01	—	Unverified
2	V+S-Att-based	BLEU4	36.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VAST	BLEU-4	19.9	—	Unverified
2	COSA	BLEU-4	18.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	GVT	BLEU4	17.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	VNS-GRU (Cross-Lingual)	BLEU-4	58.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Shot2Story	CIDEr	37.4	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Vid2Seq	CIDEr	120.5	—	Unverified