When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search
Guocheng Qian, Xuanyang Zhang, Guohao Li, Chen Zhao, Yukang Chen, Xiangyu Zhang, Bernard Ghanem, Jian Sun
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/guochengqian/tnasOfficialIn paperpytorch★ 7
Abstract
The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37\% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35\%, which outperforms the state-of-the-art. Code is available at: https://github.com/guochengqian/TNAS.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| NAS-Bench-201, CIFAR-10 | TNAS | Accuracy (Test) | 94.35 | — | Unverified |
| NAS-Bench-201, CIFAR-100 | TNAS | Accuracy (Test) | 73.02 | — | Unverified |
| NAS-Bench-201, ImageNet-16-120 | TNAS | Accuracy (Test) | 46.31 | — | Unverified |