{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T15:12:57Z","timestamp":1780413177088,"version":"3.54.1"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T00:00:00Z","timestamp":1780358400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Neural architecture search (NAS) is a core technology in AutoML, but it faces challenges such as high computational costs and inefficient evaluation. Traditional NAS methods require fully training each candidate architecture, leading to substantial resource consumption and long evaluation times. This paper introduces a four-stage progressive multi-task learning framework that shifts from brute-force search to performance prediction. By progressively training from simple synthetic data to complex real data, the framework enables efficient architecture performance prediction. The main contributions are a unified progressive predictor paradigm, a deployment-aware multi-task prediction mechanism with dual lightweighting, and a benchmark-aware data-and-transfer framework based on NATS-Bench reconstruction and progressive knowledge distillation. Experiments on 15,000 NATS-Bench architectures with a fixed train\u2013validation\u2013test split (8:1:1) and consistent hyperparameters show a 95.56% correlation-based prediction score, computed as Pearson correlation expressed as a percentage (Pearson correlation 0.9556, R-squared 0.9134), 32-fold training efficiency improvement (37.87 s \u00b1 2.1 s vs. 1247.6 s), and 95.7% convergence stability across five random seeds. Ablation studies, including a Direct Stage-3-Only comparison, quantify component contributions, and benchmarks compare against random forest, XGBoost, and MLP under identical data splits and feature spaces.<\/jats:p>","DOI":"10.3390\/a19060451","type":"journal-article","created":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T14:38:42Z","timestamp":1780411122000},"page":"451","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Dual Lightweighting in Neural Architecture Search: Progressive Multi-Task Learning with Adaptive Model Complexity"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-4594-7970","authenticated-orcid":false,"given":"Tao","family":"Wang","sequence":"first","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanqiang","family":"Di","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaochong","family":"Feng","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haohao","family":"Cui","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-6345-2997","authenticated-orcid":false,"given":"Qing","family":"Liu","sequence":"additional","affiliation":[{"name":"Shijiazhuang Campus, Army Engineering University of PLA, Shijiazhuang 050003, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,2]]},"reference":[{"key":"ref_1","unstructured":"Zoph, B., and Le, Q.V. (2017). Neural Architecture Search with Reinforcement Learning. arXiv."},{"key":"ref_2","unstructured":"Baker, B., Gupta, O., Naik, N., and Raskar, R. (2017). Designing Neural Network Architectures using Reinforcement Learning. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18\u201323). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_4","unstructured":"Pham, H., Guan, M., Zoph, B., Le, Q.V., and Dean, J. (2018, January 10\u201315). Efficient Neural Architecture Search via Parameter Sharing. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_5","unstructured":"Liu, H., Simonyan, K., and Yang, Y. (2019, January 6\u20139). DARTS: Differentiable Architecture Search. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, X., Xie, L., Wu, J., and Tian, Q. (2019, January 27\u201328). Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00138"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.J., Fei-Fei, L., Yuille, A., Huang, J., and Murphy, K. (2018, January 8\u201314). Progressive Neural Architecture Search. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"ref_8","unstructured":"Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18\u201323). Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., and Sun, J. (2020, January 23\u201328). Single Path One-Shot Neural Architecture Search with Uniform Sampling. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58517-4_32"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"White, C., Neiswanger, W., and Savani, Y. (2021, January 2\u20139). BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.","DOI":"10.1609\/aaai.v35i12.17233"},{"key":"ref_11","unstructured":"Ying, C., Klein, A., Christiansen, E., Real, E., Murphy, K., and Hutter, F. (2019, January 9\u201315). NAS-Bench-101: Towards Reproducible Neural Architecture Search. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_12","unstructured":"Mellor, J., Turner, J., Storkey, A., and Crowley, E.J. (2021, January 18\u201324). Neural Architecture Search without Training. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_13","unstructured":"Deng, B., Yan, J., and Lin, D. (2017). Peephole: Predicting Network Performance Before Training. arXiv."},{"key":"ref_14","unstructured":"Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.J., Tian, Q., and Xiong, H. (2020, January 26\u201330). PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ning, X., Zheng, Y., Zhao, T., Wang, Y., and Yang, H. (2020, January 23\u201328). A Generic Graph-Based Neural Architecture Encoding Scheme for Predictor-Based NAS. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58601-0_12"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"8441","DOI":"10.1109\/TNNLS.2022.3151160","article-title":"NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search","volume":"34","author":"Wei","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_17","unstructured":"Istrate, R., Scheidegger, F., Mariani, G., Nikolopoulos, D.S., Bekas, C., and Malossi, A.C.I. (February, January 27). TAPAS: Train-Less Accuracy Predictor for Architecture Search. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1109\/TNNLS.2019.2919608","article-title":"Completely Automated CNN Architecture Design Based on Blocks","volume":"31","author":"Sun","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_19","unstructured":"Luo, R., Tian, F., Qin, T., Chen, E., and Liu, T.Y. (2018, January 3\u20138). Neural Architecture Optimization. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_20","unstructured":"Bender, G., Kindermans, P.J., Zoph, B., Vasudevan, V., and Le, Q. (2018, January 10\u201315). Understanding and Simplifying One-Shot Architecture Search. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_21","unstructured":"Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., and Xing, E.P. (2018, January 3\u20138). Neural Architecture Search with Bayesian Optimisation and Optimal Transport. Proceedings of the Advances in Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask Learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."},{"key":"ref_23","unstructured":"Ruder, S. (2017). An Overview of Multi-Task Learning in Deep Neural Networks. arXiv."},{"key":"ref_24","first-page":"3614","article-title":"Multi-Task Learning for Dense Prediction Tasks: A Survey","volume":"44","author":"Vandenhende","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","unstructured":"Standley, T., Zamir, A.R., Chen, D., Guibas, L., Malik, J., and Savarese, S. (2020, January 13\u201318). Which Tasks Should Be Learned Together in Multi-task Learning?. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Guo, M., Haque, A., Huang, D.A., Yeung, S., and Fei-Fei, L. (2018, January 8\u201314). Dynamic Task Prioritization for Multitask Learning. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01270-0_17"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yu, J., Jin, P., Liu, H., Bender, G., Kindermans, P.J., Tan, M., Huang, T., Song, X., Pang, R., and Le, Q. (2020, January 23\u201328). BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58571-6_41"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14\u201318). Curriculum Learning. Proceedings of the International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553380"},{"key":"ref_29","unstructured":"Weinshall, D., Cohen, G., and Amir, D. (2018, January 10\u201315). Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_30","unstructured":"Hacohen, G., and Weinshall, D. (2019, January 9\u201315). On The Power of Curriculum Learning in Training Deep Networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1526","DOI":"10.1007\/s11263-022-01611-x","article-title":"Curriculum Learning: A Survey","volume":"130","author":"Soviany","year":"2022","journal-title":"Int. J. Comput. Vis."},{"key":"ref_32","unstructured":"Wang, C., Min, S., Chen, X., Sun, X., and Li, H. (2021, January 6\u201314). Dual Progressive Prototype Network for Generalized Zero-Shot Learning. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_33","first-page":"3634","article-title":"NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size","volume":"44","author":"Dong","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","first-page":"1","article-title":"The Benefit of Multitask Representation Learning","volume":"17","author":"Maurer","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_35","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Li, C., Peng, J., Yuan, L., Wang, G., Liang, X., Lin, L., and Chang, X. (2020, January 13\u201319). Block-Wisely Supervised Neural Architecture Search with Knowledge Distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00206"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/19\/6\/451\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T14:40:15Z","timestamp":1780411215000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/19\/6\/451"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,2]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["a19060451"],"URL":"https:\/\/doi.org\/10.3390\/a19060451","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,2]]}}}