Video Instance Segmentation
The goal of video instance segmentation is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first time that the image instance segmentation problem is extended to the video domain.
To facilitate research on this new task, a large-scale benchmark called YouTube-VIS, which consists of 2,883 high-resolution YouTube videos, a 40-category label set and 131k high-quality instance masks is built.
Papers
Showing 1–10 of 148 papers
All datasetsOVIS validationYouTube-VIS validationYouTube-VIS 2021Youtube-VIS 2022 ValidationBDD100K valHQ-YTVISYouTube-VISYoutube-VIS (trained with no video masks)
Benchmark Results
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CAVIS(VIT-L, Offline) | mask AP | 57.1 | — | Unverified |
| 2 | DVIS-DAQ(VIT-L, Offline) | mask AP | 57.1 | — | Unverified |
| 3 | DVIS++(VIT-L,Offline) | mask AP | 53.4 | — | Unverified |
| 4 | GLEE-Pro | mask AP | 50.4 | — | Unverified |
| 5 | DVIS(Swin-L, Offline) | mask AP | 49.9 | — | Unverified |
| 6 | DVIS++(VIT-L, Online) | mask AP | 49.6 | — | Unverified |
| 7 | UNINEXT (ViT-H, Online) | mask AP | 49 | — | Unverified |
| 8 | DVIS(Swin-L, Online) | mask AP | 47.1 | — | Unverified |
| 9 | CTVIS (Swin-L) | mask AP | 46.9 | — | Unverified |
| 10 | RefineVIS (Swin-L, offline) | mask AP | 46 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CAVIS(ViT-L, Online) | mask AP | 68.9 | — | Unverified |
| 2 | DVIS++(ViT-L, Online) | mask AP | 67.7 | — | Unverified |
| 3 | DVIS | mask AP | 64.9 | — | Unverified |
| 4 | Tube-Link | mask AP | 64.6 | — | Unverified |
| 5 | MinVIS (Swin-L) | mask AP | 61.6 | — | Unverified |
| 6 | Mask2Former (Swin-L) | mask AP | 60.4 | — | Unverified |
| 7 | UniVS(Swin-L) | mask AP | 60 | — | Unverified |
| 8 | MDQE(Swin-L) | mask AP | 59.9 | — | Unverified |
| 9 | SeqFormer (Swin-L) | mask AP | 59.3 | — | Unverified |
| 10 | DeVIS (Swin-L) | mask AP | 57.1 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | CAVIS(VIT-L, Offline) | mask AP | 65.3 | — | Unverified |
| 2 | DVIS-DAQ(VIT-L, Offline) | mask AP | 64.5 | — | Unverified |
| 3 | DVIS++(VIT-L, Offline) | mask AP | 63.9 | — | Unverified |
| 4 | DVIS++(VIT-L, Online) | mask AP | 62.3 | — | Unverified |
| 5 | RefineVIS (Swin-L, online) | mask AP | 61.4 | — | Unverified |
| 6 | GRAtt-VIS (Swin-L) | mask AP | 60.3 | — | Unverified |
| 7 | TarViS (Swin-L) | mask AP | 60.2 | — | Unverified |
| 8 | DVIS(Swin-L) | mask AP | 60.1 | — | Unverified |
| 9 | GenVIS (Swin-L) | mask AP | 60.1 | — | Unverified |
| 10 | NOVIS (Swin-L) | mask AP | 59.8 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | DVIS++(VIT-L) | mAP_L | 50.9 | — | Unverified |
| 2 | CAVIS (VIT-L) | mAP_L | 48.6 | — | Unverified |
| 3 | CTVIS (Swin-L) | mAP_L | 46.4 | — | Unverified |
| 4 | DVIS(Swin-L) | mAP_L | 45.9 | — | Unverified |
| 5 | CTVIS (ResNet-50) | mAP_L | 39.4 | — | Unverified |
| 6 | InstanceFormer (Swin) | mAP_L | 26.3 | — | Unverified |
| 7 | InstanceFormer (Resnet-50) | mAP_L | 24.8 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | PCAN | mMOTSA | 27.4 | — | Unverified |
| 2 | QDTrack-mots-fix | mMOTSA | 23.5 | — | Unverified |
| 3 | QDTrack-mots | mMOTSA | 22.5 | — | Unverified |
| 4 | MaskTrackRCNN | mMOTSA | 12.3 | — | Unverified |
| 5 | STEm-Seg | mMOTSA | 12.2 | — | Unverified |
| 6 | SortIoU | mMOTSA | 10.3 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | VMT (Swin-L) | Tube-Boundary AP | 44.8 | — | Unverified |
| 2 | SeqFormer (Swin-L) | Tube-Boundary AP | 43.3 | — | Unverified |
| 3 | VMT (R101) | Tube-Boundary AP | 32.5 | — | Unverified |
| 4 | VMT (R50) | Tube-Boundary AP | 30.7 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | Temporal ROI Align | mask AP | 38 | — | Unverified |
| # | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| 1 | MaskFreeVIS | AP | 55.3 | — | Unverified |