SOTAVerified

Align and Distill: Unifying and Improving Domain Adaptive Object Detection

2024-03-18Code Available1· sign in to hype

Justin Kay, Timm Haucke, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, Grant van Horn

Code Available — Be the first to reproduce this paper.

Reproduce

Code

Abstract

Object detectors often perform poorly on data that differs from their training set. Domain adaptive object detection (DAOD) methods have recently demonstrated strong results on addressing this challenge. Unfortunately, we identify systemic benchmarking pitfalls that call past results into question and hamper further progress: (a) Overestimation of performance due to underpowered baselines, (b) Inconsistent implementation practices preventing transparent comparisons of methods, and (c) Lack of generality due to outdated backbones and lack of diversity in benchmarks. We address these problems by introducing: (1) A unified benchmarking and implementation framework, Align and Distill (ALDI), enabling comparison of DAOD methods and supporting future development, (2) A fair and modern training and evaluation protocol for DAOD that addresses benchmarking pitfalls, (3) A new DAOD benchmark dataset, CFC-DAOD, enabling evaluation on diverse real-world data, and (4) A new method, ALDI++, that achieves state-of-the-art results by a large margin. ALDI++ outperforms the previous state-of-the-art by +3.5 AP50 on Cityscapes to Foggy Cityscapes, +5.7 AP50 on Sim10k to Cityscapes (where ours is the only method to outperform a fair baseline), and +0.6 AP50 on CFC Kenai to Channel. Our framework, dataset, and state-of-the-art method offer a critical reset for DAOD and provide a strong foundation for future research. Code and data are available: https://github.com/justinkay/aldi and https://github.com/visipedia/caltech-fish-counting.

Tasks

Benchmark Results

DatasetModelMetricClaimedVerifiedStatus
CFC-DAODALDI++ (ResNet50-FPN)AP@0.576.1Unverified
CFC-DAODSADA (ResNet50-FPN)AP@0.558.9Unverified
CFC-DAODUMT (ResNet50-FPN)AP@0.561.2Unverified
CFC-DAODPT (ResNet50-FPN)AP@0.569Unverified
CFC-DAODAT (ResNet50-FPN)AP@0.569.1Unverified
CFC-DAODMIC (ResNet50-FPN)AP@0.574.1Unverified
Cityscapes-to-Foggy CityscapesALDI-DETR (ResNet-50, 800px)mAP@0.544.8Unverified
SIM10K to CityscapesPT(ALDI frame)mAP@0.570.6Unverified
SIM10K to CityscapesSADA(ALDI frame)mAP@0.571.8Unverified
SIM10K to CityscapesAT(ALDI frame)mAP@0.572Unverified
SIM10K to CityscapesMIC(ALDI frame)mAP@0.573.1Unverified
SIM10K to CityscapesALDI-YOLOmAP@0.575Unverified
SIM10K to CityscapesALDI++mAP@0.577.8Unverified

Reproductions