Test-Time Adaptation and Adversarial Robustness

Abstract

This paper studies test-time adaptation in the context of adversarial robustness. We first formulate an adversarial threat model for test-time adaptation, where the defender may have a unique advantage as the adversarial game becomes a maximin game, instead of a minimax game as in the classic adversarial robustness threat model. We then study whether the maximin threat model admits more ``good solutions'' than the minimax threat model, and is thus strictly weaker. On the positive side, we show that, if one is allowed to access the training data, then Domain Adversarial Neural Networks ( DANN), an algorithm designed for unsupervised domain adaptation, can provide nontrivial robustness in the test-time maximin threat model against strong transfer attacks and adaptive fixed point attacks. This is somewhat surprising since DANN is not designed specifically for adversarial robustness (e.g. against norm-based attacks), and provides no robustness in the minimax model. On the negative side, we show that recent data-oblivious test-time adaptations, in contrast to DANN, can be easily attacked. We take a step to discuss moving towards adversarially robust test-time adaptation and examine its various implications.

Tasks

Reproductions