SOTAVerified

Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 401–450 of 473 papers

TitleStatusHype
Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment—0
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training DataCode0
Live Video CaptioningCode0
Video captioning with stacked attention and semantic hard pullCode0
Event and Entity Extraction from Generated Video CaptionsCode0
Joint Event Detection and Description in Continuous Video StreamsCode0
Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video CaptioningCode0
Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal AttentionCode0
https://arxiv.org/abs/2407.00634Code0
FocusedAD: Character-centric Movie Audio DescriptionCode0
Screencast Tutorial Video UnderstandingCode0
Refined Semantic Enhancement towards Frequency Diffusion for Video CaptioningCode0
Reconstruction Network for Video CaptioningCode0
Video Summarization: Towards Entity-Aware CaptionsCode0
Sketch, Ground, and Refine: Top-Down Dense Video CaptioningCode0
Pseudo-labeling with Keyword Refining for Few-Supervised Video CaptioningCode0
FLASH: Latent-Aware Semi-Autoregressive Speculative Decoding for Multimodal TasksCode0
FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation FrameworkCode0
Pretrained Image-Text Models are Secretly Video CaptionersCode0
SoccerNet 2024 Challenges ResultsCode0
OSVidCap: A Framework for the Simultaneous Recognition and Description of Concurrent Actions in Videos in an Open-Set ScenarioCode0
Oracle performance for visual captioningCode0
Watch, Listen, and Describe: Globally and Locally Aligned Cross-Modal Attentions for Video CaptioningCode0
Cross-Modal Graph with Meta Concepts for Video CaptioningCode0
A Neural, Interactive-predictive System for Multimodal Sequence to Sequence TasksCode0
VG-TVP: Multimodal Procedural Planning via Visually Grounded Text-Video PromptingCode0
OmniNet: A unified architecture for multi-modal multi-task learningCode0
Cross-Modal and Hierarchical Modeling of Video and TextCode0
Excitation Backprop for RNNsCode0
Enriching Video Captions With Contextual TextCode0
StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story ContinuationCode0
End-to-End Video Captioning with Multitask Reinforcement LearningCode0
End-to-End Dense Video Captioning with Masked TransformerCode0
Effectively Leveraging CLIP for Generating Situational Summaries of Images and VideosCode0
Streamlined Dense Video CaptioningCode0
VideoBERT: A Joint Model for Video and Language Representation LearningCode0
ActBERT: Learning Global-Local Video-Text RepresentationsCode0
Controllable Video Captioning with POS Sequence Guidance Based on Gated Fusion NetworkCode0
Support-set based Multi-modal Representation Enhancement for Video CaptioningCode0
Non-Autoregressive Coarse-to-Fine Video CaptioningCode0
M-VAD Names: a Dataset for Video Captioning with NamingCode0
Syntax Customized Video Captioning by Imitating Exemplar SentencesCode0
Multi-attention Networks for Temporal Localization of Video-level LabelsCode0
Visual Transformation TellingCode0
A Survey of Video Datasets for Grounded Event UnderstandingCode0
Continual and Multi-Task Architecture SearchCode0
Accommodating Audio Modality in CLIP for Multimodal ProcessingCode0
MTLE: A Multitask Learning Encoder of Visual Feature Representations for Video and Movie DescriptionCode0
Temporal Deformable Convolutional Encoder-Decoder Networks for Video CaptioningCode0
Contextual Explainable Video Representation: Human Perception-based UnderstandingCode0
Show:102550
← PrevPage 9 of 10Next →
All datasetsMSR-VTTMSVDYouCook2VATEXActivityNet CaptionsMSRVTT-CTNMSVD-CTNHindi MSR-VTTTVCChinaOpen-1kMSVD-IndonesianShot2Story20K

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1mPLUG-2CIDEr80—Unverified
2VASTCIDEr78—Unverified
3GIT2CIDEr75.9—Unverified
4VLABCIDEr74.9—Unverified
5COSACIDEr74.7—Unverified
6VALORCIDEr74—Unverified
7MaMMUT (ours)CIDEr73.6—Unverified
8VideoCoCaCIDEr73.2—Unverified
9RTQCIDEr69.3—Unverified
10HowToCaptionCIDEr65.3—Unverified
#ModelMetricClaimedVerifiedStatus
1MaMMUTCIDEr195.6—Unverified
2VLABCIDEr179.8—Unverified
3COSACIDEr178.5—Unverified
4VALORCIDEr178.5—Unverified
5mPLUG-2CIDEr165.8—Unverified
6HowToCaptionCIDEr154.2—Unverified
7HiTeACIDEr146.9—Unverified
8Vid2SeqCIDEr146.2—Unverified
9VIOLETv2CIDEr139.2—Unverified
10RTQCIDEr123.4—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-418.2—Unverified
2UniVL + MELTRBLEU-417.92—Unverified
3UniVLBLEU-417.35—Unverified
4VideoCoCaBLEU-414.2—Unverified
5VLMBLEU-412.27—Unverified
6E2vidD6-MASSvid-BiDBLEU-412.04—Unverified
7TextKGBLEU-411.7—Unverified
8COOTBLEU-411.3—Unverified
9COSABLEU-410.1—Unverified
10HowToCaptionBLEU-48.8—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORBLEU-445.6—Unverified
2VASTBLEU-445—Unverified
3COSABLEU-443.7—Unverified
4VideoCoCaBLEU-439.7—Unverified
5IcoCap (ViT-B/16)BLEU-437.4—Unverified
6IcoCap (ViT-B/32)BLEU-436.9—Unverified
7VASTA (Kinetics-backbone)BLEU-436.25—Unverified
8CoCap (ViT/L14)BLEU-435.8—Unverified
9ORG-TRLBLEU-432.1—Unverified
10NITS-VCBLEU-420—Unverified
#ModelMetricClaimedVerifiedStatus
1VideoCoCaBLEU414.7—Unverified
2VLTinT (ae-test split) C3D/LingBLEU414.5—Unverified
3VLCap (ae-test split) - Appearance + LanguageBLEU413.38—Unverified
4COOT (ae-test split) - Only Appearance featuresBLEU410.85—Unverified
5MART (ae-test split) - Appearance + FlowBLEU410.33—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr49.87—Unverified
2GITCIDEr32.43—Unverified
3SEM-POSCIDEr26.01—Unverified
4AKGNNCIDEr25.9—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr63.51—Unverified
2GITCIDEr45.63—Unverified
3SEM-POSCIDEr37.16—Unverified
4AKGNNCIDEr35.08—Unverified
#ModelMetricClaimedVerifiedStatus
1SBD_KeyframeBLEU441.01—Unverified
2V+S-Att-basedBLEU436.2—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-419.9—Unverified
2COSABLEU-418.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GVTBLEU417.7—Unverified
#ModelMetricClaimedVerifiedStatus
1VNS-GRU (Cross-Lingual)BLEU-458.68—Unverified
#ModelMetricClaimedVerifiedStatus
1Shot2StoryCIDEr37.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Vid2SeqCIDEr120.5—Unverified