PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search

Code

• github.com/yuhuixu1993/PC-DARTS
  OfficialIn paperpytorch★ 0
• github.com/ddghost/new_darts
  pytorch★ 0
• github.com/Mind23-2/MindCode-5/tree/main/PDarts
  mindspore★ 0
• github.com/MS-Mind/MS-Code-06/tree/main/PDarts
  mindspore★ 0
• github.com/chenxin061/pdarts
  pytorch★ 0
• github.com/xkp793003821/PC-DARTS-COOPER
  pytorch★ 0
• github.com/peteryuX/pcdarts-tf2
  tf★ 0
• github.com/aragakiyuiii/gumbel-pdarts-master
  pytorch★ 0

Abstract

Differentiable architecture search (DARTS) provided a fast solution in finding effective network architectures, but suffered from large memory and computing overheads in jointly training a super-network and searching for an optimal architecture. In this paper, we present a novel approach, namely, Partially-Connected DARTS, by sampling a small part of super-network to reduce the redundancy in exploring the network space, thereby performing a more efficient search without comprising the performance. In particular, we perform operation search in a subset of channels while bypassing the held out part in a shortcut. This strategy may suffer from an undesired inconsistency on selecting the edges of super-net caused by sampling different channels. We alleviate it using edge normalization, which adds a new set of edge-level parameters to reduce uncertainty in search. Thanks to the reduced memory cost, PC-DARTS can be trained with a larger batch size and, consequently, enjoys both faster speed and higher training stability. Experimental results demonstrate the effectiveness of the proposed method. Specifically, we achieve an error rate of 2.57% on CIFAR10 with merely 0.1 GPU-days for architecture search, and a state-of-the-art top-1 error rate of 24.2% on ImageNet (under the mobile setting) using 3.8 GPU-days for search. Our code has been made available at: https://github.com/yuhuixu1993/PC-DARTS.

Tasks

Benchmark Results

Reproductions