SOTAVerified

Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 473 papers

TitleStatusHype
UGC-VideoCaptioner: An Omni UGC Video Detail Caption Model and New BenchmarksCode1
Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization—0
Dense Video Captioning using Graph-based Sentence Summarization—0
video-SALMONN 2: Captioning-Enhanced Audio-Visual Large Language ModelsCode2
VersaVid-R1: A Versatile Video Understanding and Reasoning Model from Question Answering to Captioning Tasks—0
ARGUS: Hallucination and Omission Evaluation in Video-LLMs—0
Temporal Object Captioning for Street Scene Videos from LiDAR Tracks—0
FLASH: Latent-Aware Semi-Autoregressive Speculative Decoding for Multimodal TasksCode0
TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation—0
Describe Anything: Detailed Localized Image and Video Captioning—0
FocusedAD: Character-centric Movie Audio DescriptionCode0
Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal PromptingCode2
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning—0
Watch and Learn: Leveraging Expert Knowledge and Language for Surgical Video Understanding—0
Get In Video: Add Anything You Want to the Video—0
Fine-Grained Video Captioning through Scene Graph Consolidation—0
LongCaptioning: Unlocking the Power of Long Caption Generation in Large Multimodal Models—0
Capturing Rich Behavior Representations: A Dynamic Action Semantic-Aware Graph Transformer for Video Captioning—0
Pretrained Image-Text Models are Secretly Video CaptionersCode0
VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video GenerationCode1
Temporal Working Memory: Query-Guided Segment Refinement for Enhanced Multimodal UnderstandingCode3
MAMS: Model-Agnostic Module Selection Framework for Video Captioning—0
VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video CaptioningCode1
Classifier-Guided Captioning Across Modalities—0
AdaCM^2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction—0
Event-Equalized Dense Video Captioning—0
CaReBench: A Fine-Grained Benchmark for Video Captioning and Retrieval—0
Hierarchical Banzhaf Interaction for General Video-Language Representation LearningCode0
PolySmart @ TRECVid 2024 Video Captioning (VTT)—0
HiCM^2: Hierarchical Compact Memory Modeling for Dense Video CaptioningCode1
G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4oCode1
Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video CaptioningCode0
Exploring Temporal Event Cues for Dense Video Captioning in Cyclic Co-learning—0
VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video PromptingCode0
Bridging Vision and Language: Modeling Causality and Temporality in Video Narratives—0
ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation—0
Agent-based Video Trimming—0
Video LLMs for Temporal Reasoning in Long Videos—0
Progress-Aware Video Frame Captioning—0
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation—0
VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction FormatCode1
Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Grounding—0
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity—0
Whats in a Video: Factorized Autoregressive Decoding for Online Dense Video Captioning—0
AdaCM^2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction—0
Multi-Modal interpretable automatic video captioning—0
Pseudo-labeling with Keyword Refining for Few-Supervised Video CaptioningCode0
SPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities—0
Technical Report for Soccernet 2023 -- Dense Video Captioning—0
EVC-MF: End-to-end Video Captioning Network with Multi-scale Features—0
Show:102550
← PrevPage 1 of 10Next →
All datasetsMSR-VTTMSVDYouCook2VATEXActivityNet CaptionsMSRVTT-CTNMSVD-CTNHindi MSR-VTTTVCChinaOpen-1kMSVD-IndonesianShot2Story20K

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1mPLUG-2CIDEr80—Unverified
2VASTCIDEr78—Unverified
3GIT2CIDEr75.9—Unverified
4VLABCIDEr74.9—Unverified
5COSACIDEr74.7—Unverified
6VALORCIDEr74—Unverified
7MaMMUT (ours)CIDEr73.6—Unverified
8VideoCoCaCIDEr73.2—Unverified
9RTQCIDEr69.3—Unverified
10HowToCaptionCIDEr65.3—Unverified
#ModelMetricClaimedVerifiedStatus
1MaMMUTCIDEr195.6—Unverified
2VLABCIDEr179.8—Unverified
3COSACIDEr178.5—Unverified
4VALORCIDEr178.5—Unverified
5mPLUG-2CIDEr165.8—Unverified
6HowToCaptionCIDEr154.2—Unverified
7HiTeACIDEr146.9—Unverified
8Vid2SeqCIDEr146.2—Unverified
9VIOLETv2CIDEr139.2—Unverified
10RTQCIDEr123.4—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-418.2—Unverified
2UniVL + MELTRBLEU-417.92—Unverified
3UniVLBLEU-417.35—Unverified
4VideoCoCaBLEU-414.2—Unverified
5VLMBLEU-412.27—Unverified
6E2vidD6-MASSvid-BiDBLEU-412.04—Unverified
7TextKGBLEU-411.7—Unverified
8COOTBLEU-411.3—Unverified
9COSABLEU-410.1—Unverified
10HowToCaptionBLEU-48.8—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORBLEU-445.6—Unverified
2VASTBLEU-445—Unverified
3COSABLEU-443.7—Unverified
4VideoCoCaBLEU-439.7—Unverified
5IcoCap (ViT-B/16)BLEU-437.4—Unverified
6IcoCap (ViT-B/32)BLEU-436.9—Unverified
7VASTA (Kinetics-backbone)BLEU-436.25—Unverified
8CoCap (ViT/L14)BLEU-435.8—Unverified
9ORG-TRLBLEU-432.1—Unverified
10NITS-VCBLEU-420—Unverified
#ModelMetricClaimedVerifiedStatus
1VideoCoCaBLEU414.7—Unverified
2VLTinT (ae-test split) C3D/LingBLEU414.5—Unverified
3VLCap (ae-test split) - Appearance + LanguageBLEU413.38—Unverified
4COOT (ae-test split) - Only Appearance featuresBLEU410.85—Unverified
5MART (ae-test split) - Appearance + FlowBLEU410.33—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr49.87—Unverified
2GITCIDEr32.43—Unverified
3SEM-POSCIDEr26.01—Unverified
4AKGNNCIDEr25.9—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr63.51—Unverified
2GITCIDEr45.63—Unverified
3SEM-POSCIDEr37.16—Unverified
4AKGNNCIDEr35.08—Unverified
#ModelMetricClaimedVerifiedStatus
1SBD_KeyframeBLEU441.01—Unverified
2V+S-Att-basedBLEU436.2—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-419.9—Unverified
2COSABLEU-418.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GVTBLEU417.7—Unverified
#ModelMetricClaimedVerifiedStatus
1VNS-GRU (Cross-Lingual)BLEU-458.68—Unverified
#ModelMetricClaimedVerifiedStatus
1Shot2StoryCIDEr37.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Vid2SeqCIDEr120.5—Unverified