{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T21:54:58Z","timestamp":1766267698271,"version":"3.37.3"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,12,4]],"date-time":"2021-12-04T00:00:00Z","timestamp":1638576000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,4]],"date-time":"2021-12-04T00:00:00Z","timestamp":1638576000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"natural science foundation of china","doi-asserted-by":"crossref","award":["61903178","61906081"],"award-info":[{"award-number":["61903178","61906081"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"natural science foundation of china","doi-asserted-by":"crossref","award":["U20A20306"],"award-info":[{"award-number":["U20A20306"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>For the goal of automated design of high-performance deep convolutional neural networks (CNNs), neural architecture search (NAS) methodology is becoming increasingly important for both academia and industries. Due to the costly stochastic gradient descent training of CNNs for performance evaluation, most existing NAS methods are computationally expensive for real-world deployments. To address this issue, we first introduce a new performance estimation metric, named random-weight evaluation (RWE) to quantify the quality of CNNs in a cost-efficient manner. Instead of fully training the entire CNN, the RWE only trains its last layer and leaves the remainders with randomly initialized weights, which results in a single network evaluation in seconds. Second, a complexity metric is adopted for multi-objective NAS to balance the model size and performance. Overall, our proposed method obtains a set of efficient models with state-of-the-art performance in two real-world search spaces. Then the results obtained on the CIFAR-10 dataset are transferred to the ImageNet dataset to validate the practicality of the proposed algorithm. Moreover, ablation studies on NAS-Bench-301 datasets reveal the effectiveness of the proposed RWE in estimating the performance compared to existing methods.<\/jats:p>","DOI":"10.1007\/s40747-021-00594-5","type":"journal-article","created":{"date-parts":[[2021,12,4]],"date-time":"2021-12-04T10:02:29Z","timestamp":1638612149000},"page":"1183-1192","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Accelerating multi-objective neural architecture search by random-weight evaluation"],"prefix":"10.1007","volume":"9","author":[{"given":"Shengran","family":"Hu","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9410-8263","authenticated-orcid":false,"given":"Ran","family":"Cheng","sequence":"additional","affiliation":[]},{"given":"Cheng","family":"He","sequence":"additional","affiliation":[]},{"given":"Zhichao","family":"Lu","sequence":"additional","affiliation":[]},{"given":"Jing","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Miao","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,12,4]]},"reference":[{"key":"594_CR1","unstructured":"Abdelfattah MS, Mehrotra A, Dudziak \u0141, Lane ND (2021) Zero-cost proxies for lightweight NAS. In: International conference on learning representations (ICLR 2021). Virtual only"},{"key":"594_CR2","unstructured":"Adebayo J, Gilmer J, Goodfellow IJ, Kim B (2018) Local explanation methods for deep neural networks lack sensitivity to parameter values. In: International conference on learning representations (ICLR 2018). Vancouver, Canada"},{"key":"594_CR3","unstructured":"Cai H, Gan C, Wang T, Zhang Z, Han S (2020) Once for all: train one network and specialize it for efficient deployment. In: International conference on learning representations (ICLR 2020). Virtual Only"},{"key":"594_CR4","unstructured":"Cai H, Zhu L, Han S (2019) ProxylessNAS: direct neural architecture search on target task and hardware. In: International conference on learning representations (ICLR 2019). New Orleans, United States"},{"issue":"2","key":"594_CR5","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182\u2013197","journal-title":"IEEE Trans Evol Comput"},{"key":"594_CR6","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L, Li K, Li F (2009) Imagenet: a large-scale hierarchical image database. In: IEEE computer society conference on computer vision and pattern recognition. (CVPR 2009). Miami Beach, United States, pp 248\u2013255","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"594_CR7","unstructured":"DeVries T, Taylor GW (2017) Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552"},{"key":"594_CR8","unstructured":"Dong X, Yang Y (2020) NAS-Bench-201: extending the scope of reproducible neural architecture search. In: International conference on learning representations (ICLR 2020). Virtual Only"},{"key":"594_CR9","unstructured":"Gaier A, Ha D (2019) Weight agnostic neural networks. In: Advances in neural information processing systems, (NeurIPS 2019) vol\u00a032. Vancouver, Canada, pp 5365\u20135379"},{"key":"594_CR10","unstructured":"Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: International conference on artificial intelligence and statistics. (AISTATS 2010) Sardinia, Italy, pp 249\u2013256"},{"issue":"10","key":"594_CR11","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1109\/34.58871","volume":"12","author":"LK Hansen","year":"1990","unstructured":"Hansen LK, Salamon P (1990) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12(10):993\u20131001","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"594_CR12","doi-asserted-by":"publisher","first-page":"100894","DOI":"10.1016\/j.swevo.2021.100894","volume":"64","author":"C He","year":"2021","unstructured":"He C, Tan H, Huang S, Cheng R (2021) Efficient evolutionary neural architecture search by modular inheritable crossover. Swarm Evol Comput 64:100894","journal-title":"Swarm Evol Comput"},{"key":"594_CR13","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: International conference on computer vision. (ICCV 2015) Santiago, Chile, pp 1026\u20131034","DOI":"10.1109\/ICCV.2015.123"},{"key":"594_CR14","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2016). Las Vegas, United States, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"594_CR15","unstructured":"Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861"},{"key":"594_CR16","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, Van Der\u00a0Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2017). Honolulu, Hawaii, pp 4700\u20134708","DOI":"10.1109\/CVPR.2017.243"},{"key":"594_CR17","doi-asserted-by":"crossref","unstructured":"Jarrett K, Kavukcuoglu K, Ranzato M, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: International conference on computer vision. IEEE, pp 2146\u20132153","DOI":"10.1109\/ICCV.2009.5459469"},{"key":"594_CR18","doi-asserted-by":"crossref","unstructured":"Juefei-Xu F, Naresh\u00a0Boddeti V, Savvides M (2017) Local binary convolutional neural networks. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2017). Honolulu, Hawaii pp 19\u201328","DOI":"10.1109\/CVPR.2017.456"},{"key":"594_CR19","unstructured":"Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images. University of Toronto, Tech. rep"},{"key":"594_CR20","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, (NeurIPS 2012) vol\u00a025. Lake Tahoe, United States, pp 1097\u20131105"},{"key":"594_CR21","doi-asserted-by":"crossref","unstructured":"Liu C, Zoph B, Neumann M, Shlens J, Hua W, Li L.J., Fei-Fei L, Yuille A, Huang J, Murphy K (2018) Progressive neural architecture search. In: European conference on computer vision (ECCV 2018). Munich, Germany, pp 19\u201334","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"594_CR22","unstructured":"Liu H, Simonyan K, Vinyals O, Fernando C, Kavukcuoglu K (2018) Hierarchical representations for efficient architecture search. In: International conference on learning representations (ICLR 2018). Vancouver, Canada"},{"key":"594_CR23","unstructured":"Liu H, Simonyan K, Yang Y (2019) DARTS: differentiable architecture search. In: International conference on learning representations (ICLR 2019). New Orleans, United States"},{"key":"594_CR24","doi-asserted-by":"crossref","unstructured":"Liu Y, Tang Y, Sun Y (2021) Homogeneous architecture augmentation for neural predictor. In: International conference on computer vision (ICCV 2021). Virtual only","DOI":"10.1109\/ICCV48922.2021.01203"},{"key":"594_CR25","unstructured":"Loshchilov I, Hutter F (2016) SGDR: stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983"},{"key":"594_CR26","doi-asserted-by":"crossref","unstructured":"Lu Z, Deb K, Boddeti VN (2020) MUXConv: information multiplexing in convolutional neural networks. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2020). Virtual only, pp 12044\u201312053","DOI":"10.1109\/CVPR42600.2020.01206"},{"key":"594_CR27","doi-asserted-by":"crossref","unstructured":"Lu Z, Deb K, Goodman E, Banzhaf W, Boddeti VN (2020) NSGANetV2: evolutionary multi-objective surrogate-assisted neural architecture search. In: European conference on computer vision (ECCV 2020). Virtual only, pp 35\u201351","DOI":"10.1007\/978-3-030-58452-8_3"},{"issue":"09","key":"594_CR28","doi-asserted-by":"publisher","first-page":"2971","DOI":"10.1109\/TPAMI.2021.3052758","volume":"43","author":"Z Lu","year":"2021","unstructured":"Lu Z, Sreekumar G, Goodman E, Banzhaf W, Deb K, Boddeti VN (2021) Neural architecture transfer. IEEE Trans Pattern Anal Mach Intell 43(09):2971\u20132989","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"594_CR29","doi-asserted-by":"crossref","unstructured":"Lu Z, Whalen I, Boddeti V, Dhebar Y, Deb K, Goodman E, Banzhaf W (2019) NSGA-net: neural architecture search using multi-objective genetic algorithm. In: Genetic and evolutionary computation conference (GECCO 2019). Prague, Czech Republic, pp 419\u2013427","DOI":"10.1145\/3321707.3321729"},{"issue":"2","key":"594_CR30","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1109\/TEVC.2020.3024708","volume":"25","author":"Z Lu","year":"2020","unstructured":"Lu Z, Whalen I, Dhebar Y, Deb K, Goodman E, Banzhaf W, Boddeti VN (2020) Multi-objective evolutionary design of deep convolutional neural networks for image classification. IEEE Trans Evol Comput 25(2):277\u2013291","journal-title":"IEEE Trans Evol Comput"},{"key":"594_CR31","doi-asserted-by":"crossref","unstructured":"Ma N, Zhang X, Zheng HT, Sun J (2018) Shufflenet v2: practical guidelines for efficient CNN architecture design. In: European conference on computer vision (ECCV 2018). Munich, Germany, pp 116\u2013131","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"594_CR32","unstructured":"Mellor J, Turner J, Storkey A, Crowley EJ (2021) Neural architecture search without training. In: International conference on machine learning. PMLR, pp 7588\u20137598"},{"key":"594_CR33","doi-asserted-by":"crossref","unstructured":"Ning X, Zheng Y, Zhao T, Wang Y, Yang H (2020) A generic graph-based neural architecture encoding scheme for predictor-based NAS. In: European conference on computer vision. Springer, pp 189\u2013204","DOI":"10.1007\/978-3-030-58601-0_12"},{"issue":"01","key":"594_CR34","first-page":"4780","volume":"33","author":"E Real","year":"2019","unstructured":"Real E, Aggarwal A, Huang Y, Le QV (2019) Regularized evolution for image classifier architecture search. Proceed AAAI Conf Artif Intell 33(01):4780\u20134789","journal-title":"Proceed AAAI Conf Artif Intell"},{"key":"594_CR35","doi-asserted-by":"crossref","unstructured":"Rosenfeld A, Tsotsos JK (2019) Intriguing properties of randomly weighted networks: generalizing while learning next to nothing. In: International conference on robotics and vision (ICRV 2019). Singapore, pp 9\u201316","DOI":"10.1109\/CRV.2019.00010"},{"key":"594_CR36","doi-asserted-by":"crossref","unstructured":"Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2018). Salt Lake City, United States, pp 4510\u20134520","DOI":"10.1109\/CVPR.2018.00474"},{"key":"594_CR37","unstructured":"Saxe AM, Koh PW, Chen Z, Bhand M, Suresh B, Ng AY (2011) On random weights and unsupervised feature learning. In: International conference on machine learning (ICML 2011). Bellevue, United States"},{"key":"594_CR38","unstructured":"Siems J, Zimmer L, Zela A, Lukasik J, Keuper M, Hutter F (2020) NAS-Bench-301 and the case for surrogate benchmarks for neural architecture search. arXiv preprint arXiv:2008.09777"},{"key":"594_CR39","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR 2015). San Diego, United States"},{"issue":"2","key":"594_CR40","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1109\/TEVC.2019.2924461","volume":"24","author":"Y Sun","year":"2020","unstructured":"Sun Y, Wang H, Xue B, Jin Y, Yen GG, Zhang M (2020) Surrogate-assisted evolutionary deep learning using an end-to-end random forest-based performance predictor. IEEE Trans Evol Comput 24(2):350\u2013364","journal-title":"IEEE Trans Evol Comput"},{"key":"594_CR41","doi-asserted-by":"crossref","unstructured":"Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2015). Boston, United States, pp 1\u20139","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"594_CR42","unstructured":"Tan H, Cheng R, Huang S, He C, Qiu C, Yang F, Luo P (2021) RelativeNAS: relative neural architecture search via slow-fast learning. IEEE IEEE transactions on neural networks and learning systems. pp 1\u20131"},{"key":"594_CR43","doi-asserted-by":"crossref","unstructured":"Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard A, Le Q.V. (2019) MnasNet: platform-aware neural architecture search for mobile. In: IEEE conference on computer vision and pattern recognition. pp 2820\u20132828","DOI":"10.1109\/CVPR.2019.00293"},{"key":"594_CR44","unstructured":"Ulyanov D, Vedaldi A, Lempitsky V (2018) Deep image prior. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2018). Salt Lake City, United States, pp 9446\u20139454"},{"key":"594_CR45","doi-asserted-by":"publisher","unstructured":"Wang B, Xue B, Zhang M (2021) Surrogate-assisted particle swarm optimization for evolving variable-length transferable blocks for image classification. IEEE transactions on neural networks and learning systems. pp 1\u201314. https:\/\/doi.org\/10.1109\/TNNLS.2021.3054400","DOI":"10.1109\/TNNLS.2021.3054400"},{"key":"594_CR46","doi-asserted-by":"crossref","unstructured":"Wen W, Liu H, Chen Y, Li H, Bender G, Kindermans PJ (2020) Neural predictor for neural architecture search. In: European conference on computer vision. Springer, pp 660\u2013676","DOI":"10.1007\/978-3-030-58526-6_39"},{"key":"594_CR47","doi-asserted-by":"crossref","unstructured":"Xie L, Yuille A (2017) Genetic CNN. In: International conference on computer vision (ICCV 2017). Venice, Italy","DOI":"10.1109\/ICCV.2017.154"},{"key":"594_CR48","unstructured":"Xie S, Zheng H, Liu C, Lin L (2019) SNAS: stochastic neural architecture search. In: International conference on learning representations (ICLR 2019). New Orleans, United States"},{"key":"594_CR49","unstructured":"Yan S, Zheng Y, Ao W, Zeng X, Zhang M (2020) Does unsupervised architecture representation learning help neural architecture search? Adv Neural Inf Process Syst 33: 12486\u201312498"},{"key":"594_CR50","doi-asserted-by":"crossref","unstructured":"Zagoruyko S, Komodakis N (2016) Wide residual networks. In: British machine vision conference British machine vision conference (BMVC 2016). York, United Kingdom","DOI":"10.5244\/C.30.87"},{"key":"594_CR51","unstructured":"Zela A, Klein A, Falkner S, Hutter F (2018) Towards automated deep learning: efficient joint neural architecture and hyperparameter search. arXiv preprint arXiv:1807.06906"},{"key":"594_CR52","doi-asserted-by":"crossref","unstructured":"Zhang X, Zhou X, Lin M, Sun J (2018) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2018). Salt Lake City, United States, pp 6848\u20136856","DOI":"10.1109\/CVPR.2018.00716"},{"issue":"7","key":"594_CR53","doi-asserted-by":"publisher","first-page":"2314","DOI":"10.1109\/TPAMI.2020.2969193","volume":"43","author":"Z Zhong","year":"2021","unstructured":"Zhong Z, Yang Z, Deng B, Yan J, Wu W, Shao J, Liu C (2021) BlockQNN: efficient block-wise neural network architecture generation. IEEE Trans Pattern Anal Mach Intell 43(7):2314\u20132328","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"594_CR54","doi-asserted-by":"crossref","unstructured":"Zhou D, Zhou X, Zhang W, Loy CC, Yi S, Zhang X, Ouyang W (2020) Econas: finding proxies for economical neural architecture search. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2020). Virtal only, pp 11396\u201311404","DOI":"10.1109\/CVPR42600.2020.01141"},{"key":"594_CR55","doi-asserted-by":"crossref","unstructured":"Zoph B, Vasudevan V, Shlens J, Le QV (2018) Learning transferable architectures for scalable image recognition. In: IEEE computer society conference on computer vision and pattern recognition (CVPR 2018), Salt Lake City, United States, pp 8697\u20138710","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00594-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-021-00594-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00594-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T09:20:53Z","timestamp":1681809653000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-021-00594-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,4]]},"references-count":55,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["594"],"URL":"https:\/\/doi.org\/10.1007\/s40747-021-00594-5","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2021,12,4]]},"assertion":[{"value":"21 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}