SOTAVerified

Video Description

The goal of automatic Video Description is to tell a story about events happening in a video. While early Video Description methods produced captions for short clips that were manually segmented to contain a single event of interest, more recently dense video captioning has been proposed to both segment distinct events in time and describe them in a series of coherent sentences. This problem is a generalization of dense image region captioning and has many practical applications, such as generating textual summaries for the visually impaired, or detecting and describing important events in surveillance footage.

Source: Joint Event Detection and Description in Continuous Video Streams

Papers

Showing 125 of 104 papers

TitleStatusHype
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality TeachersCode4
Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video UnderstandingCode4
Tarsier: Recipes for Training and Evaluating Large Video Description ModelsCode4
Hawk: Learning to Understand Open-World Video AnomaliesCode3
TrafficVLM: A Controllable Visual Language Model for Traffic Video CaptioningCode2
StoryTeller: Improving Long Video Description through Global Audio-Visual Character IdentificationCode2
FunQA: Towards Surprising Video ComprehensionCode1
Grounded Video DescriptionCode1
Fine-grained Audible Video DescriptionCode1
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language ResearchCode1
Identity-Aware Multi-Sentence Video DescriptionCode1
What's in a Caption? Dataset-Specific Linguistic Diversity and Its Effect on Visual Description Models and MetricsCode1
Thinking Hallucination for Video CaptioningCode1
Audio Visual Scene-Aware Dialog (AVSD) Challenge at DSTC7Code1
Delving Deeper into the Decoder for Video CaptioningCode1
Using Descriptive Video Services to Create a Large Data Source for Video Annotation ResearchCode1
A Mid-level Video Representation based on Binary Descriptors: A Case Study for Pornography DetectionCode0
Predicting Visual Features from Text for Image and Video Caption RetrievalCode0
Memory-augmented Attention Modelling for VideosCode0
Improving LSTM-based Video Description with Linguistic Knowledge Mined from TextCode0
Learn to Understand Negation in Video RetrievalCode0
MSVD-Indonesian: A Benchmark for Multimodal Video-Text Tasks in IndonesianCode0
https://arxiv.org/abs/2407.00634Code0
JMI at SemEval 2024 Task 3: Two-step approach for multimodal ECAC using in-context learning with GPT and instruction-tuned Llama modelsCode0
Adversarial Inference for Multi-Sentence Video DescriptionCode0
Show:102550
← PrevPage 1 of 5Next →

No leaderboard results yet.