Robust Differentiable Architecture Search (DARTS-) for AutoML

High‑quality model design and iterative updates are major pain points in AI production. In this context, Automated Machine Learning (AutoML) has emerged, with Neural Architecture Search (NAS) becoming its core component after Google introduced it in 2017.

Meituan’s visual intelligence team, in collaboration with Shanghai Jiao Tong University, contributed a paper (DARTS‑) accepted at ICLR 2021, which analyzes the robustness issues of the original DARTS method and proposes improvements.

Background : Meituan’s AI services span over 200 scenarios, requiring scalable data and advanced deep‑learning models. Designing and updating high‑quality models is a bottleneck, motivating the use of AutoML and NAS.

NAS Overview : NAS searches for optimal architectures within a defined search space, using algorithms based on reinforcement learning, evolutionary strategies, or gradient‑based optimization. Gradient‑based methods, especially DARTS, have become mainstream due to their efficiency.

Limitations of DARTS : The differentiable approach suffers from performance collapse, where the super‑network performs well but the derived subnet exhibits many skip connections, degrading final accuracy.

Proposed DARTS‑ Method : To mitigate collapse, an auxiliary skip connection with a linearly decaying weight (β) is added, separating the residual benefit of skip connections from their role as selectable operators. The optimization alternates between updating network weights (w) and architecture weights (α) while applying the β decay schedule.

Analysis and Validation : Experiments show that DARTS‑ reduces Hessian eigenvalue spikes and yields smoother loss landscapes compared to vanilla DARTS and R‑DARTS. On CIFAR‑10 and ImageNet, DARTS‑ achieves state‑of‑the‑art accuracy, outperforming other NAS baselines on NAS‑Bench‑201 and demonstrating strong transferability to COCO object detection (mAP 32.5%).

Conclusion : DARTS‑ retains DARTS’s efficiency while improving robustness and generalization, offering a practical solution for large‑scale AutoML deployments across vision, speech, NLP, and recommendation tasks. The code is open‑sourced on GitHub.