SOTAVerified

Video Instance Segmentation

The goal of video instance segmentation is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain.

To facilitate research on this new task, a large-scale benchmark called YouTube-VIS, which consists of 2,883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks is built.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 101–148 of 148 papers

TitleStatusHype
Less than Few: Self-Shot Video Instance Segmentation—0
A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects—0
Self-supervised Video Instance Segmentation Can Boost Geographic Entity Alignment in Historical Maps—0
LSVOS Challenge 3rd Place Report: SAM2 and Cutie based VOS—0
Decoupled Motion Expression Video Segmentation—0
Towards Robust Video Instance Segmentation with Temporal-Aware Transformer—0
1st Place Solution for YouTubeVOS Challenge 2021:Video Instance Segmentation—0
Efficient Video Instance Segmentation via Tracklet Query and Proposal—0
Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation—0
Efficient Video Segmentation Models with Per-frame Inference—0
Video Instance Segmentation by Instance Flow Assembly—0
A Temporal Modeling Framework for Video Pre-Training on Video Instance Segmentation—0
Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation—0
2nd Place Solution for MeViS Track in CVPR 2024 PVUW Workshop: Motion Expression guided Video Segmentation—0
FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matching—0
Minimizing Labeled, Maximizing Unlabeled: An Image-Driven Approach for Video Instance Segmentation—0
Contextual Guided Segmentation Framework for Semi-supervised Video Instance Segmentation—0
A Reinforcement-Learning-Based Energy-Efficient Framework for Multi-Task Video Analytics Pipeline—0
Human Instance Segmentation and Tracking via Data Association and Single-stage Detector—0
Hybrid Instance-aware Temporal Fusion for Online Video Instance Segmentation—0
A2VIS: Amodal-Aware Approach to Video Instance Segmentation—0
MobileInst: Video Instance Segmentation on the Mobile—0
STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation—0
Consistent Video Instance Segmentation with Inter-Frame Recurrent Attention—0
InsPro: Propagating Instance Query and Proposal for Online Video Instance Segmentation—0
Video Instance Segmentation Tracking With a Modified VAE Architecture—0
3rd Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation—0
Tag-Based Attention Guided Bottom-Up Approach for Video Instance Segmentation—0
1st Place Solution for CVPR2023 BURST Long Tail and Open World Challenges—0
CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation—0
Object Segmentation with Audio Context—0
The Runner-up Solution for YouTube-VIS Long Video Challenge 2022—0
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge—0
Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?—0
Offline-to-Online Knowledge Distillation for Video Instance Segmentation—0
Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation—0
Online Video Instance Segmentation via Robust Context Fusion—0
MoSAM: Motion-Guided Segment Anything Model with Spatial-Temporal Memory Selection—0
OW-VISCapTor: Abstractors for Open-World Video Instance Segmentation and Captioning—0
PM-VIS: High-Performance Box-Supervised Video Instance Segmentation—0
NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation—0
UVIS: Unsupervised Video Instance Segmentation—0
Quantifying and Learning Static vs. Dynamic Information in Deep Spatiotemporal Networks—0
1st Place Winner of the 2024 Pixel-level Video Understanding in the Wild (CVPR'24 PVUW) Challenge in Video Panoptic Segmentation and Best Long Video Consistency of Video Semantic Segmentation—0
What is Point Supervision Worth in Video Instance Segmentation?—0
RefineVIS: Video Instance Segmentation with Temporal Attention Refinement—0
A Graph Matching Perspective With Transformers on Video Instance Segmentation—0
RipVIS: Rip Currents Video Instance Segmentation Benchmark for Beach Monitoring and Safety—0
Show:102550
← PrevPage 3 of 3Next →
All datasetsOVIS validationYouTube-VIS validationYouTube-VIS 2021Youtube-VIS 2022 ValidationBDD100K valHQ-YTVISYouTube-VISYoutube-VIS (trained with no video masks)

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1CAVIS(VIT-L, Offline)mask AP57.1—Unverified
2DVIS-DAQ(VIT-L, Offline)mask AP57.1—Unverified
3DVIS++(VIT-L,Offline)mask AP53.4—Unverified
4GLEE-Promask AP50.4—Unverified
5DVIS(Swin-L, Offline)mask AP49.9—Unverified
6DVIS++(VIT-L, Online)mask AP49.6—Unverified
7UNINEXT (ViT-H, Online)mask AP49—Unverified
8DVIS(Swin-L, Online)mask AP47.1—Unverified
9CTVIS (Swin-L)mask AP46.9—Unverified
10RefineVIS (Swin-L, offline)mask AP46—Unverified
#ModelMetricClaimedVerifiedStatus
1CAVIS(ViT-L, Online)mask AP68.9—Unverified
2DVIS++(ViT-L, Online)mask AP67.7—Unverified
3DVISmask AP64.9—Unverified
4Tube-Linkmask AP64.6—Unverified
5MinVIS (Swin-L)mask AP61.6—Unverified
6Mask2Former (Swin-L)mask AP60.4—Unverified
7UniVS(Swin-L)mask AP60—Unverified
8MDQE(Swin-L)mask AP59.9—Unverified
9SeqFormer (Swin-L)mask AP59.3—Unverified
10DeVIS (Swin-L)mask AP57.1—Unverified
#ModelMetricClaimedVerifiedStatus
1CAVIS(VIT-L, Offline)mask AP65.3—Unverified
2DVIS-DAQ(VIT-L, Offline)mask AP64.5—Unverified
3DVIS++(VIT-L, Offline)mask AP63.9—Unverified
4DVIS++(VIT-L, Online)mask AP62.3—Unverified
5RefineVIS (Swin-L, online)mask AP61.4—Unverified
6GRAtt-VIS (Swin-L)mask AP60.3—Unverified
7TarViS (Swin-L)mask AP60.2—Unverified
8DVIS(Swin-L)mask AP60.1—Unverified
9GenVIS (Swin-L)mask AP60.1—Unverified
10NOVIS (Swin-L)mask AP59.8—Unverified
#ModelMetricClaimedVerifiedStatus
1DVIS++(VIT-L)mAP_L50.9—Unverified
2CAVIS (VIT-L)mAP_L48.6—Unverified
3CTVIS (Swin-L)mAP_L46.4—Unverified
4DVIS(Swin-L)mAP_L45.9—Unverified
5CTVIS (ResNet-50)mAP_L39.4—Unverified
6InstanceFormer (Swin)mAP_L26.3—Unverified
7InstanceFormer (Resnet-50)mAP_L24.8—Unverified
#ModelMetricClaimedVerifiedStatus
1PCANmMOTSA27.4—Unverified
2QDTrack-mots-fixmMOTSA23.5—Unverified
3QDTrack-motsmMOTSA22.5—Unverified
4MaskTrackRCNNmMOTSA12.3—Unverified
5STEm-SegmMOTSA12.2—Unverified
6SortIoUmMOTSA10.3—Unverified
#ModelMetricClaimedVerifiedStatus
1VMT (Swin-L)Tube-Boundary AP44.8—Unverified
2SeqFormer (Swin-L)Tube-Boundary AP43.3—Unverified
3VMT (R101)Tube-Boundary AP32.5—Unverified
4VMT (R50)Tube-Boundary AP30.7—Unverified
#ModelMetricClaimedVerifiedStatus
1Temporal ROI Alignmask AP38—Unverified
#ModelMetricClaimedVerifiedStatus
1MaskFreeVISAP55.3—Unverified