Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

Code

• github.com/dstamoulis/single-path-nas
  OfficialIn papertf★ 0
• github.com/mindspore-ai/models/tree/master/research/cv/single_path_nas
  mindspore★ 0
• github.com/Mind23-2/MindCode-88/tree/main/single_path_nas
  mindspore★ 0
• github.com/2023-MindSpore-4/Code7/tree/main/single_path_nas
  mindspore★ 0
• github.com/code-implementation1/Code8/tree/main/single_path_nas
  mindspore★ 0
• github.com/Mind23-2/MindCode-64
  mindspore★ 0
• github.com/2023-MindSpore-4/Code11/tree/main/single_path_nas
  mindspore★ 0

Abstract

Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one single-path over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar constraints (<80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPU-hours), which is up to 5,000x faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.

Tasks

Benchmark Results

Reproductions