SOTAVerified

Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–25 of 473 papers

TitleStatusHype
CogVideoX: Text-to-Video Diffusion Models with An Expert TransformerCode11
ShareGPT4Video: Improving Video Understanding and Generation with Better CaptionsCode5
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMsCode5
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and VideoCode4
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality TeachersCode4
Tarsier: Recipes for Training and Evaluating Large Video Description ModelsCode4
GiT: Towards Generalist Vision Transformer through Universal Language InterfaceCode3
Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal UnderstandingCode3
CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal ReasoningCode3
Video ReCap: Recursive Captioning of Hour-Long VideosCode3
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video UnderstandingCode3
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video UnderstandingCode3
Vision-Language Pre-training: Basics, Recent Advances, and Future TrendsCode3
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and DatasetCode2
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEsCode2
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and DatasetCode2
SkyScript-100M: 1,000,000,000 Pairs of Scripts and Shooting Scripts for Short DramaCode2
TrafficVLM: A Controllable Visual Language Model for Traffic Video CaptioningCode2
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video CaptioningCode2
OmniVid: A Generative Framework for Universal Video UnderstandingCode2
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal PromptingCode2
Movie101v2: Improved Movie Narration BenchmarkCode2
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language ModelsCode2
SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts CommentariesCode2
LVD-2M: A Long-take Video Dataset with Temporally Dense CaptionsCode2
Show:102550
← PrevPage 1 of 19Next →
All datasetsMSR-VTTMSVDYouCook2VATEXActivityNet CaptionsMSRVTT-CTNMSVD-CTNHindi MSR-VTTTVCChinaOpen-1kMSVD-IndonesianShot2Story20K

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1mPLUG-2CIDEr80—Unverified
2VASTCIDEr78—Unverified
3GIT2CIDEr75.9—Unverified
4VLABCIDEr74.9—Unverified
5COSACIDEr74.7—Unverified
6VALORCIDEr74—Unverified
7MaMMUT (ours)CIDEr73.6—Unverified
8VideoCoCaCIDEr73.2—Unverified
9RTQCIDEr69.3—Unverified
10HowToCaptionCIDEr65.3—Unverified
#ModelMetricClaimedVerifiedStatus
1MaMMUTCIDEr195.6—Unverified
2VLABCIDEr179.8—Unverified
3COSACIDEr178.5—Unverified
4VALORCIDEr178.5—Unverified
5mPLUG-2CIDEr165.8—Unverified
6HowToCaptionCIDEr154.2—Unverified
7HiTeACIDEr146.9—Unverified
8Vid2SeqCIDEr146.2—Unverified
9VIOLETv2CIDEr139.2—Unverified
10RTQCIDEr123.4—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-418.2—Unverified
2UniVL + MELTRBLEU-417.92—Unverified
3UniVLBLEU-417.35—Unverified
4VideoCoCaBLEU-414.2—Unverified
5VLMBLEU-412.27—Unverified
6E2vidD6-MASSvid-BiDBLEU-412.04—Unverified
7TextKGBLEU-411.7—Unverified
8COOTBLEU-411.3—Unverified
9COSABLEU-410.1—Unverified
10HowToCaptionBLEU-48.8—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORBLEU-445.6—Unverified
2VASTBLEU-445—Unverified
3COSABLEU-443.7—Unverified
4VideoCoCaBLEU-439.7—Unverified
5IcoCap (ViT-B/16)BLEU-437.4—Unverified
6IcoCap (ViT-B/32)BLEU-436.9—Unverified
7VASTA (Kinetics-backbone)BLEU-436.25—Unverified
8CoCap (ViT/L14)BLEU-435.8—Unverified
9ORG-TRLBLEU-432.1—Unverified
10NITS-VCBLEU-420—Unverified
#ModelMetricClaimedVerifiedStatus
1VideoCoCaBLEU414.7—Unverified
2VLTinT (ae-test split) C3D/LingBLEU414.5—Unverified
3VLCap (ae-test split) - Appearance + LanguageBLEU413.38—Unverified
4COOT (ae-test split) - Only Appearance featuresBLEU410.85—Unverified
5MART (ae-test split) - Appearance + FlowBLEU410.33—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr49.87—Unverified
2GITCIDEr32.43—Unverified
3SEM-POSCIDEr26.01—Unverified
4AKGNNCIDEr25.9—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr63.51—Unverified
2GITCIDEr45.63—Unverified
3SEM-POSCIDEr37.16—Unverified
4AKGNNCIDEr35.08—Unverified
#ModelMetricClaimedVerifiedStatus
1SBD_KeyframeBLEU441.01—Unverified
2V+S-Att-basedBLEU436.2—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-419.9—Unverified
2COSABLEU-418.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GVTBLEU417.7—Unverified
#ModelMetricClaimedVerifiedStatus
1VNS-GRU (Cross-Lingual)BLEU-458.68—Unverified
#ModelMetricClaimedVerifiedStatus
1Shot2StoryCIDEr37.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Vid2SeqCIDEr120.5—Unverified