SOTAVerified

Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 101–150 of 473 papers

TitleStatusHype
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality TeachersCode4
MCF-VC: Mitigate Catastrophic Forgetting in Class-Incremental Learning for Multimodal Video Captioning—0
Video ReCap: Recursive Captioning of Hour-Long VideosCode3
LVCHAT: Facilitating Long Video ComprehensionCode1
Knowledge Guided Entity-aware Video Captioning and A Basketball Benchmark—0
Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal DataCode1
SnapCap: Efficient Snapshot Compressive Video Captioning—0
On Scaling Up a Multilingual Vision and Language Model—0
Retrieval-Augmented Egocentric Video Captioning—0
A Recipe for Scaling up Text-to-Video Generation with Text-free VideosCode0
Set Prediction Guided by Semantic Concepts for Diverse Video Captioning—0
SOVC: Subject-Oriented Video Captioning—0
Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot VideosCode1
Attention Based Encoder Decoder Model for Video Captioning in Nepali (2023)—0
Video Summarization: Towards Entity-Aware CaptionsCode0
RTQ: Rethinking Video-language Understanding Based on Image-text ModelCode1
VTimeLLM: Empower LLM to Grasp Video MomentsCode2
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos—0
Incorporating granularity bias as the margin into contrastive loss for video captioning—0
Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols—0
Nepali Video Captioning using CNN-RNN Architecture—0
Learning Interactive Real-World Simulators—0
HowToCaption: Prompting LLMs to Transform Video Annotations at ScaleCode1
Analyzing Zero-Shot Abilities of Vision-Language Models on Video Understanding TasksCode0
IcoCap: Improving Video Captioning by Compounding Images—0
Human-centric Behavior Description in Videos: New Benchmark and Model—0
Encoder-Decoder Based Long Short-Term Memory (LSTM) Model for Video Captioning—0
VidChapters-7M: Video Chapters at ScaleCode2
Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges—0
Accurate and Fast Compressed Video CaptioningCode1
Collaborative Three-Stream Transformers for Video Captioning—0
SoccerNet 2023 Challenges ResultsCode1
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual CaptioningCode1
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity ControlCode1
Prompt Switch: Efficient CLIP Adaptation for Text-Video RetrievalCode1
Video Captioning with Aggregated Features Based on Dual Graphs and Gated Fusion—0
OmniDataComposer: A Unified Data Structure for Multimodal Data Fusion and Infinite Data GenerationCode1
Learning Multi-modal Representations by Watching Hundreds of Surgical Video LecturesCode1
Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment—0
CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal ReasoningCode3
Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos—0
Exploring the Role of Audio in Video Captioning—0
MSVD-Indonesian: A Benchmark for Multimodal Video-Text Tasks in IndonesianCode0
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary CaptioningCode1
Knowledge Distillation for Efficient Audio-Visual Video Captioning—0
COSA: Concatenated Sample Pretrained Vision-Language Foundation ModelCode1
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and BenchmarksCode2
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and DatasetCode2
PaLI-X: On Scaling up a Multilingual Vision and Language ModelCode1
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending—0
Show:102550
← PrevPage 3 of 10Next →
All datasetsMSR-VTTMSVDYouCook2VATEXActivityNet CaptionsMSRVTT-CTNMSVD-CTNHindi MSR-VTTTVCChinaOpen-1kMSVD-IndonesianShot2Story20K

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1mPLUG-2CIDEr80—Unverified
2VASTCIDEr78—Unverified
3GIT2CIDEr75.9—Unverified
4VLABCIDEr74.9—Unverified
5COSACIDEr74.7—Unverified
6VALORCIDEr74—Unverified
7MaMMUT (ours)CIDEr73.6—Unverified
8VideoCoCaCIDEr73.2—Unverified
9RTQCIDEr69.3—Unverified
10HowToCaptionCIDEr65.3—Unverified
#ModelMetricClaimedVerifiedStatus
1MaMMUTCIDEr195.6—Unverified
2VLABCIDEr179.8—Unverified
3COSACIDEr178.5—Unverified
4VALORCIDEr178.5—Unverified
5mPLUG-2CIDEr165.8—Unverified
6HowToCaptionCIDEr154.2—Unverified
7HiTeACIDEr146.9—Unverified
8Vid2SeqCIDEr146.2—Unverified
9VIOLETv2CIDEr139.2—Unverified
10RTQCIDEr123.4—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-418.2—Unverified
2UniVL + MELTRBLEU-417.92—Unverified
3UniVLBLEU-417.35—Unverified
4VideoCoCaBLEU-414.2—Unverified
5VLMBLEU-412.27—Unverified
6E2vidD6-MASSvid-BiDBLEU-412.04—Unverified
7TextKGBLEU-411.7—Unverified
8COOTBLEU-411.3—Unverified
9COSABLEU-410.1—Unverified
10HowToCaptionBLEU-48.8—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORBLEU-445.6—Unverified
2VASTBLEU-445—Unverified
3COSABLEU-443.7—Unverified
4VideoCoCaBLEU-439.7—Unverified
5IcoCap (ViT-B/16)BLEU-437.4—Unverified
6IcoCap (ViT-B/32)BLEU-436.9—Unverified
7VASTA (Kinetics-backbone)BLEU-436.25—Unverified
8CoCap (ViT/L14)BLEU-435.8—Unverified
9ORG-TRLBLEU-432.1—Unverified
10NITS-VCBLEU-420—Unverified
#ModelMetricClaimedVerifiedStatus
1VideoCoCaBLEU414.7—Unverified
2VLTinT (ae-test split) C3D/LingBLEU414.5—Unverified
3VLCap (ae-test split) - Appearance + LanguageBLEU413.38—Unverified
4COOT (ae-test split) - Only Appearance featuresBLEU410.85—Unverified
5MART (ae-test split) - Appearance + FlowBLEU410.33—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr49.87—Unverified
2GITCIDEr32.43—Unverified
3SEM-POSCIDEr26.01—Unverified
4AKGNNCIDEr25.9—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr63.51—Unverified
2GITCIDEr45.63—Unverified
3SEM-POSCIDEr37.16—Unverified
4AKGNNCIDEr35.08—Unverified
#ModelMetricClaimedVerifiedStatus
1SBD_KeyframeBLEU441.01—Unverified
2V+S-Att-basedBLEU436.2—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-419.9—Unverified
2COSABLEU-418.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GVTBLEU417.7—Unverified
#ModelMetricClaimedVerifiedStatus
1VNS-GRU (Cross-Lingual)BLEU-458.68—Unverified
#ModelMetricClaimedVerifiedStatus
1Shot2StoryCIDEr37.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Vid2SeqCIDEr120.5—Unverified