SOTAVerified

Video Captioning

Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.

Source: NITS-VC System for VATEX Video Captioning Challenge 2020

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 473 papers

TitleStatusHype
Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot VideosCode1
RTQ: Rethinking Video-language Understanding Based on Image-text ModelCode1
HowToCaption: Prompting LLMs to Transform Video Annotations at ScaleCode1
Accurate and Fast Compressed Video CaptioningCode1
SoccerNet 2023 Challenges ResultsCode1
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual CaptioningCode1
VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity ControlCode1
Prompt Switch: Efficient CLIP Adaptation for Text-Video RetrievalCode1
OmniDataComposer: A Unified Data Structure for Multimodal Data Fusion and Infinite Data GenerationCode1
Learning Multi-modal Representations by Watching Hundreds of Surgical Video LecturesCode1
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary CaptioningCode1
COSA: Concatenated Sample Pretrained Vision-Language Foundation ModelCode1
PaLI-X: On Scaling up a Multilingual Vision and Language ModelCode1
Movie101: A New Movie Understanding BenchmarkCode1
From Association to Generation: Text-only Captioning by Unsupervised Cross-modal MappingCode1
Hierarchical Video-Moment Retrieval and Step-CaptioningCode1
Fine-grained Audible Video DescriptionCode1
GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary GenerationCode1
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation ModelsCode1
Positive-Augmented Contrastive Learning for Image and Video Captioning EvaluationCode1
Action knowledge for video captioning with graph neural networksCode1
ZeroNLG: Aligning and Autoencoding Domains for Zero-Shot Multimodal and Multilingual Natural Language GenerationCode1
Learning Grounded Vision-Language Representation for Versatile Understanding in Untrimmed VideosCode1
VLTinT: Visual-Linguistic Transformer-in-Transformer for Coherent Video Paragraph CaptioningCode1
Expectation-Maximization Contrastive Learning for Compact Video-and-Language RepresentationsCode1
Visual Commonsense-aware Representation Network for Video CaptioningCode1
Why is Winoground Hard? Investigating Failures in Visuolinguistic CompositionalityCode1
Thinking Hallucination for Video CaptioningCode1
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual ModelingCode1
Partially Relevant Video RetrievalCode1
Zero-Shot Video Captioning with Evolving Pseudo-TokensCode1
Unifying Event Detection and Captioning as Sequence Generation via Pre-TrainingCode1
Rethinking Surgical Captioning: End-to-End Window-Based MLP Transformer Using PatchesCode1
VLCap: Vision-Language with Contrastive Learning for Coherent Video Paragraph CaptioningCode1
LAVENDER: Unifying Video-Language Understanding as Masked Language ModelingCode1
GL-RG: Global-Local Representation Granularity for Video CaptioningCode1
Language Models with Image Descriptors are Strong Few-Shot Video-Language LearnersCode1
MUGEN: A Playground for Video-Audio-Text Multimodal Understanding and GENerationCode1
Tell me what you see: A zero-shot action recognition method based on natural language descriptionsCode1
Controllable Video Captioning with an Exemplar SentenceCode1
SwinBERT: End-to-End Transformers with Sparse Attention for Video CaptioningCode1
Hierarchical Modular Network for Video CaptioningCode1
EMScore: Evaluating Video Captioning via Coarse-Grained and Fine-Grained Embedding MatchingCode1
Co-segmentation Inspired Attention Module for Video-based Computer Vision TasksCode1
X-modaler: A Versatile and High-performance Codebase for Cross-modal AnalyticsCode1
End-to-End Dense Video Captioning with Parallel DecodingCode1
Discriminative Latent Semantic Graph for Video CaptioningCode1
VALUE: A Multi-Task Benchmark for Video-and-Language Understanding EvaluationCode1
DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy MinimizationCode1
Improving Generation and Evaluation of Visual Stories via Semantic ConsistencyCode1
Show:102550
← PrevPage 2 of 10Next →
All datasetsMSR-VTTMSVDYouCook2VATEXActivityNet CaptionsMSRVTT-CTNMSVD-CTNHindi MSR-VTTTVCChinaOpen-1kMSVD-IndonesianShot2Story20K

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1mPLUG-2CIDEr80—Unverified
2VASTCIDEr78—Unverified
3GIT2CIDEr75.9—Unverified
4VLABCIDEr74.9—Unverified
5COSACIDEr74.7—Unverified
6VALORCIDEr74—Unverified
7MaMMUT (ours)CIDEr73.6—Unverified
8VideoCoCaCIDEr73.2—Unverified
9RTQCIDEr69.3—Unverified
10HowToCaptionCIDEr65.3—Unverified
#ModelMetricClaimedVerifiedStatus
1MaMMUTCIDEr195.6—Unverified
2VLABCIDEr179.8—Unverified
3COSACIDEr178.5—Unverified
4VALORCIDEr178.5—Unverified
5mPLUG-2CIDEr165.8—Unverified
6HowToCaptionCIDEr154.2—Unverified
7HiTeACIDEr146.9—Unverified
8Vid2SeqCIDEr146.2—Unverified
9VIOLETv2CIDEr139.2—Unverified
10RTQCIDEr123.4—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-418.2—Unverified
2UniVL + MELTRBLEU-417.92—Unverified
3UniVLBLEU-417.35—Unverified
4VideoCoCaBLEU-414.2—Unverified
5VLMBLEU-412.27—Unverified
6E2vidD6-MASSvid-BiDBLEU-412.04—Unverified
7TextKGBLEU-411.7—Unverified
8COOTBLEU-411.3—Unverified
9COSABLEU-410.1—Unverified
10HowToCaptionBLEU-48.8—Unverified
#ModelMetricClaimedVerifiedStatus
1VALORBLEU-445.6—Unverified
2VASTBLEU-445—Unverified
3COSABLEU-443.7—Unverified
4VideoCoCaBLEU-439.7—Unverified
5IcoCap (ViT-B/16)BLEU-437.4—Unverified
6IcoCap (ViT-B/32)BLEU-436.9—Unverified
7VASTA (Kinetics-backbone)BLEU-436.25—Unverified
8CoCap (ViT/L14)BLEU-435.8—Unverified
9ORG-TRLBLEU-432.1—Unverified
10NITS-VCBLEU-420—Unverified
#ModelMetricClaimedVerifiedStatus
1VideoCoCaBLEU414.7—Unverified
2VLTinT (ae-test split) C3D/LingBLEU414.5—Unverified
3VLCap (ae-test split) - Appearance + LanguageBLEU413.38—Unverified
4COOT (ae-test split) - Only Appearance featuresBLEU410.85—Unverified
5MART (ae-test split) - Appearance + FlowBLEU410.33—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr49.87—Unverified
2GITCIDEr32.43—Unverified
3SEM-POSCIDEr26.01—Unverified
4AKGNNCIDEr25.9—Unverified
#ModelMetricClaimedVerifiedStatus
1CENCIDEr63.51—Unverified
2GITCIDEr45.63—Unverified
3SEM-POSCIDEr37.16—Unverified
4AKGNNCIDEr35.08—Unverified
#ModelMetricClaimedVerifiedStatus
1SBD_KeyframeBLEU441.01—Unverified
2V+S-Att-basedBLEU436.2—Unverified
#ModelMetricClaimedVerifiedStatus
1VASTBLEU-419.9—Unverified
2COSABLEU-418.8—Unverified
#ModelMetricClaimedVerifiedStatus
1GVTBLEU417.7—Unverified
#ModelMetricClaimedVerifiedStatus
1VNS-GRU (Cross-Lingual)BLEU-458.68—Unverified
#ModelMetricClaimedVerifiedStatus
1Shot2StoryCIDEr37.4—Unverified
#ModelMetricClaimedVerifiedStatus
1Vid2SeqCIDEr120.5—Unverified