{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T17:02:27Z","timestamp":1770742947192,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,11,25]],"date-time":"2023-11-25T00:00:00Z","timestamp":1700870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Neural architecture search (NAS) has shown great potential in discovering powerful and flexible network models, becoming an important branch of automatic machine learning (AutoML). Although search methods based on reinforcement learning and evolutionary algorithms can find high-performance architectures, these search methods typically require hundreds of GPU days. Unlike searching in a discrete search space based on reinforcement learning and evolutionary algorithms, the differentiable neural architecture search (DARTS) continuously relaxes the search space, allowing for optimization using gradient-based methods. Based on DARTS, we propose NDARTS in this article. The new algorithm uses the Implicit Function Theorem and the Neumann series to approximate the hyper-gradient, which obtains better results than DARTS. In the simulation experiment, an ablation experiment was carried out to study the influence of the different parameters on the NDARTS algorithm and to determine the optimal weight, then the best performance of the NDARTS algorithm was searched for in the DARTS search space and the NAS-BENCH-201 search space. Compared with other NAS algorithms, the results showed that NDARTS achieved excellent results on the CIFAR-10, CIFAR-100, and ImageNet datasets, and was an effective neural architecture search algorithm.<\/jats:p>","DOI":"10.3390\/a16120536","type":"journal-article","created":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T03:35:06Z","timestamp":1701056106000},"page":"536","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["NDARTS: A Differentiable Architecture Search Based on the Neumann Series"],"prefix":"10.3390","volume":"16","author":[{"given":"Xiaoyu","family":"Han","sequence":"first","affiliation":[{"name":"School of Mathematics, Southeast University, Nanjing 211189, China"}]},{"given":"Chenyu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mathematics, Southeast University, Nanjing 211189, China"}]},{"given":"Zifan","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mathematics, Southeast University, Nanjing 211189, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0445-2795","authenticated-orcid":false,"given":"Guohua","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Mathematics, Southeast University, Nanjing 211189, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,25]]},"reference":[{"key":"ref_1","unstructured":"Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., and Macherey, K. (2016). Google\u2019s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv."},{"key":"ref_2","unstructured":"Chen, M.X., Firat, O., Bapna, A., Johnson, M., Macherey, W., Foster, G., Jones, L., Parmar, N., Schuster, M., and Chen, Z. (2016). The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation. arXiv."},{"key":"ref_3","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations.Computational and Biological Learning Society, San Diego, CA, USA."},{"key":"ref_4","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Tech Report (v5). arXiv.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Dong, X., Huang, J., Yang, Y., and Yan, S. (2017, January 21\u201326). More is Less: A More Complicated Network with Less Inference Complexity. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.205"},{"key":"ref_9","first-page":"1997","article-title":"Neural architecture search: A survey","volume":"20","author":"Elsken","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref_10","unstructured":"Baker, B., Gupta, O., Naik, N., and Raskar, R. (2016). Designing Neural Network Architectures using Reinforcement Learning. arXiv."},{"key":"ref_11","first-page":"1","article-title":"A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions","volume":"54","author":"Ren","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2017). Learning Transferable Architectures for Scalable Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_13","unstructured":"Enzo, L.A., Eduardo, L., Vasty, Z., Claudia, R., and John, M. (2016). Neural Architecture Search with Reinforcement Learning. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cai, H., Chen, T., Zhang, W., Yu, Y., and Wang, J. (2017). Efficient Architecture Search by Network Transformation. arXiv.","DOI":"10.1609\/aaai.v32i1.11709"},{"key":"ref_15","first-page":"4780","article-title":"Regularized Evolution for Image Classifier Architecture Search","volume":"33","author":"Real","year":"2018","journal-title":"Proc. Aaai Conf. Artif. Intell."},{"key":"ref_16","unstructured":"Liu, H., Simonyan, K., Vinyals, O., Fernando, C., and Kavukcuoglu, K. (2018). Hierarchical Representations for Efficient Architecture Search.International Conference on Learning Representations. arXiv."},{"key":"ref_17","unstructured":"Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Tan, J., Le, Q.V., and Kurakin, A. (2017). Large-Scale Evolution of Image Classifiers. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1162\/106365602320169811","article-title":"Evolving neural networks through augmenting topologies","volume":"10","author":"Stanley","year":"2002","journal-title":"Evol. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xie, L., and Yuille, A. (2017, January 22\u201329). Genetic CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.154"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Li, F.-F., Yuille, A., Huang, J., and Murphy, K. (2017). Progressive Neural Architecture Search. arXiv.","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"ref_21","unstructured":"Kandasamy, K., Neiswanger, W., Schneider, J., P\u00f3czos, B., and Xing, E.P. (2018). Neural Architecture Search with Bayesian Optimisation and Optimal Transport. arXiv."},{"key":"ref_22","unstructured":"Negrinho, R., and Gordon, G. (2017). DeepArchitect: Automatically Designing and Training Deep Architectures. arXiv."},{"key":"ref_23","unstructured":"Liu, H., Simonyan, K., and Yang, Y. (2018). Darts: Differentiable architecture search. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, X., Xie, L., Wu, J., and Tian, Q. (2019). Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation. arXiv.","DOI":"10.1109\/ICCV.2019.00138"},{"key":"ref_25","unstructured":"Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.J., Tian, Q., and Xiong, H. (2019). PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. arXiv."},{"key":"ref_26","unstructured":"Cai, H., Zhu, L., and Han, S. (2018). ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. arXiv."},{"key":"ref_27","unstructured":"Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., and Hutter, F. (2020, January 26\u201330). Understanding and Robustifying Differentiable Architecture Search. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_28","unstructured":"Chen, X., and Hsieh, C.J. (2020). Stabilizing Differentiable Architecture Search via Perturbation-based Regularization. arXiv."},{"key":"ref_29","unstructured":"Xie, S., Zheng, H., Liu, C., and Lin, L. (2018). SNAS: Stochastic Neural Architecture Search. arXiv."},{"key":"ref_30","unstructured":"Liang, H., Zhang, S., Sun, J., He, X., Huang, W., Zhuang, K., and Li, Z. (2019). DARTS+: Improved Differentiable Architecture Search with Early Stopping. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hou, P., Jin, Y., and Chen, Y. (2021, January 11\u201317). Single-DARTS: Towards stable architecture search. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00046"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhu, X., Li, J., Liu, Y., and Wang, W. (2023). Improving Differentiable Architecture Search via Self-Distillation. arXiv.","DOI":"10.2139\/ssrn.4361706"},{"key":"ref_33","unstructured":"Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv."},{"key":"ref_34","unstructured":"Dong, X., and Yang, Y. (2020). NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search. arXiv."},{"key":"ref_35","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Handbook of Systemic Autoimmune Diseases, Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009, January 20\u201325). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_37","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Chu, X., Zhou, T., Zhang, B., and Li, J. (2019). Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search. arXiv.","DOI":"10.1007\/978-3-030-58555-6_28"},{"key":"ref_39","unstructured":"Liam, L.I., University, C.M., and Determined, A.I. (2019). Random Search and Reproducibility for Neural Architecture Search. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Dong, X., and Yang, Y. (2019, January 15\u201320). Searching for A Robust Neural Architecture in Four GPU Hours. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2019, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00186"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Dong, X., and Yang, Y. (November, January 27). One-Shot Neural Architecture Search via Self-Evaluated Template Network. Proceedings of the IEEE\/CVF International Conference on Computer Vision 2019 ICCV, Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00378"},{"key":"ref_42","unstructured":"Hundt, A., Jain, V., and Hager, G.D. (2019). SharpDARTS: Faster and More Accurate Differentiable Architecture Search. arXiv."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/12\/536\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:30:04Z","timestamp":1760131804000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/12\/536"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,25]]},"references-count":42,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["a16120536"],"URL":"https:\/\/doi.org\/10.3390\/a16120536","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,25]]}}}