{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T04:29:39Z","timestamp":1779337779224,"version":"3.51.4"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"crossref","award":["1SC8821N"],"award-info":[{"award-number":["1SC8821N"]}],"id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002913","name":"Vlaamse Overheid","doi-asserted-by":"publisher","award":["Flanders AI Program"],"award-info":[{"award-number":["Flanders AI Program"]}],"id":[{"id":"10.13039\/501100002913","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>We assess the feasibility of a reusable neural architecture search agent aimed at amortizing the initial time-investment in building a good search strategy. We do this through the use of Reinforcement Learning, where an agent learns to iteratively select the best way to modify a given neural network architecture. This is achieved using a transformer-based agent design trained using the Ape-X algorithm. We consider both the NAS-Bench-101 and NAS-Bench-301 settings, and compare against various known strong baselines, such as local search and random search. While achieving competitive performance on both benchmarks, the amount of training required for the much larger NAS-Bench-301 is only marginally greater than NAS-Bench-101, illustrating the strong scaling properties of our agent. Our agent is able to achieve strong performance, but the choice of values for certain parameters are crucial to ensuring the succesful training of the agent. We provide some guidance for the selection of appropriate values for hyperparameters through a detailed description of our experimental setup and several ablation studies.<\/jats:p>","DOI":"10.1007\/s00521-024-10445-2","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T06:46:44Z","timestamp":1731912404000},"page":"231-261","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Scalable reinforcement learning-based neural architecture search"],"prefix":"10.1007","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7471-2508","authenticated-orcid":false,"given":"Amber","family":"Cassimon","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siegfried","family":"Mercelis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin","family":"Mets","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"10445_CR1","unstructured":"Asthana R, Conrad J, Dawoud Y, et\u00a0al (2024) Multi-conditioned graph diffusion for neural architecture search. Tran Mach Learn Res. https:\/\/openreview.net\/forum?id=5VotySkajV"},{"key":"10445_CR2","unstructured":"Ba JL, Kiros JR, Hinton GE (2016) Layer normalization. arXiv:1607.06450"},{"key":"10445_CR3","unstructured":"Baker B, Gupta O, Naik N, et\u00a0al (2017) Designing neural network architectures using reinforcement learning. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=S1c2cvqee"},{"key":"10445_CR4","doi-asserted-by":"publisher","unstructured":"Cai Z, Chen L, Liu HL (2023) Bhe-darts: bilevel optimization based on hypergradient estimation for differentiable architecture search. In: ICASSP 2023\u20142023 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1\u20135. https:\/\/doi.org\/10.1109\/ICASSP49357.2023.10095940","DOI":"10.1109\/ICASSP49357.2023.10095940"},{"key":"10445_CR5","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1007\/978-3-030-58555-6_28","volume-title":"Computer vision\u2014ECCV 2020","author":"X Chu","year":"2020","unstructured":"Chu X, Zhou T, Zhang B et al (2020) Fair darts: eliminating unfair advantages in differentiable architecture search. In: Vedaldi A, Bischof H, Brox T et al (eds) Computer vision\u2014ECCV 2020. Springer, Cham, pp 465\u2013480"},{"key":"10445_CR6","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1007\/978-3-030-72062-9_37","volume-title":"Evolutionary multi-criterion optimization","author":"T Den Ottelander","year":"2021","unstructured":"Den Ottelander T, Dushatskiy A, Virgolin M et al (2021) Local search is a remarkably strong baseline for neural architecture search. In: Ishibuchi H, Zhang Q, Cheng R et al (eds) Evolutionary multi-criterion optimization. Springer, Cham, pp 465\u2013479"},{"key":"10445_CR7","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, et\u00a0al (2021) An image is worth 16$$\\times$$16 words: transformers for image recognition at scale. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"10445_CR8","doi-asserted-by":"crossref","unstructured":"Elsken T, Metzen JH, Hutter F (2019a) Efficient multi-objective neural architecture search via Lamarckian evolution. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=ByME42AqK7","DOI":"10.1007\/978-3-030-05318-5_3"},{"issue":"55","key":"10445_CR9","first-page":"1","volume":"20","author":"T Elsken","year":"2019","unstructured":"Elsken T, Metzen JH, Hutter F (2019) Neural architecture search: a survey. J Mach Learn Res 20(55):1\u201321","journal-title":"J Mach Learn Res"},{"key":"10445_CR10","doi-asserted-by":"crossref","unstructured":"Han FX, Mills KG, Chudak F, et\u00a0al (2023) A general-purpose transferable predictor for neural architecture search. In: Proceedings of the 2023 SIAM international conference on data mining (SDM), SIAM, pp 721\u2013729","DOI":"10.1137\/1.9781611977653.ch81"},{"key":"10445_CR11","unstructured":"Hasselt H (2010) Double q-learning. In: Lafferty J, Williams C, Shawe-Taylor J, et\u00a0al (eds) Advances in neural information processing systems, vol\u00a023. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2010\/file\/091d584fced301b442654dd8c23b3fc9-Paper.pdf"},{"key":"10445_CR12","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, et\u00a0al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2016.90"},{"key":"10445_CR13","doi-asserted-by":"crossref","unstructured":"Hendrickx L, Van\u00a0Ranst W, Goedem\u00e9 T (2022) Hot-started NAS for task-specific embedded applications. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR) workshops, pp 1971\u20131978","DOI":"10.1109\/CVPRW56347.2022.00214"},{"key":"10445_CR14","unstructured":"Horgan D, Quan J, Budden D, et\u00a0al (2018) Distributed prioritized experience replay. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=H1Dy---0Z"},{"key":"10445_CR15","unstructured":"Kadlecov\u00e1 G, Lukasik J, Pil\u00e1t M, et\u00a0al (2024) Surprisingly strong performance prediction with neural graph features. In: Forty-first international conference on machine learning. https:\/\/openreview.net\/forum?id=EhPpZV6KLk"},{"key":"10445_CR16","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges C, Bottou L, et\u00a0al (eds) Advances in neural information processing systems, vol\u00a025. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2012\/file\/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf"},{"key":"10445_CR17","unstructured":"Li M, Liu JY, Sigal L, et\u00a0al (2022) GraphPNAS: learning distribution of good neural architectures via deep graph generative models. arXiv:2211.15155"},{"key":"10445_CR18","doi-asserted-by":"publisher","first-page":"106300","DOI":"10.1016\/j.engappai.2023.106300","volume":"123","author":"Y Li","year":"2023","unstructured":"Li Y, Wu J, Deng T (2023) Meta-GNAS: meta-reinforcement learning for graph neural architecture search. Eng Appl Artif Intell 123:106300. https:\/\/doi.org\/10.1016\/j.engappai.2023.106300","journal-title":"Eng Appl Artif Intell"},{"key":"10445_CR19","unstructured":"Liang E, Liaw R, Nishihara R, et\u00a0al (2018) RLlib: abstractions for distributed reinforcement learning. In: Dy J, Krause A (eds) Proceedings of the 35th international conference on machine learning, proceedings of machine learning research, vol\u00a080, pp 3053\u20133062. PMLR. https:\/\/proceedings.mlr.press\/v80\/liang18b.html"},{"key":"10445_CR20","unstructured":"Liu H, Simonyan K, Yang Y (2019) DARTS: differentiable architecture search. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=S1eYHoC5FX"},{"key":"10445_CR21","doi-asserted-by":"publisher","unstructured":"Lu Z, Whalen I, Boddeti V, et\u00a0al (2019) NSGA-Net: neural architecture search using multi-objective genetic algorithm. In: Proceedings of the genetic and evolutionary computation conference. Association for Computing Machinery, New York, GECCO \u201919, pp 419\u2013427. https:\/\/doi.org\/10.1145\/3321707.3321729","DOI":"10.1145\/3321707.3321729"},{"key":"10445_CR22","first-page":"10547","volume":"33","author":"R Luo","year":"2020","unstructured":"Luo R, Tan X, Wang R et al (2020) Semi-supervised neural architecture search. Adv Neural Inf Process Syst 33:10547\u201310557","journal-title":"Adv Neural Inf Process Syst"},{"key":"10445_CR23","unstructured":"Mellor J, Turner J, Storkey A, et\u00a0al (2021) Neural architecture search without training. In: Meila M, Zhang T (eds) Proceedings of the 38th international conference on machine learning, proceedings of machine learning research, vol 139, pp 7588\u20137598. PMLR. https:\/\/proceedings.mlr.press\/v139\/mellor21a.html"},{"key":"10445_CR24","unstructured":"Mnih V, Kavukcuoglu K, Silver D, et\u00a0al (2013) Playing atari with deep reinforcement learning. CoRR abs\/1312.5602. arXiv:1312.5602"},{"key":"10445_CR25","doi-asserted-by":"crossref","unstructured":"Mok J, Na B, Kim JH, et\u00a0al (2022) Demystifying the neural tangent kernel from a practical perspective: Can it be trusted for neural architecture search without training? In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 11861\u201311870","DOI":"10.1109\/CVPR52688.2022.01156"},{"key":"10445_CR26","unstructured":"Pardo F, Tavakoli A, Levdik V, et\u00a0al (2018) Time limits in reinforcement learning. https:\/\/openreview.net\/forum?id=HyDAQl-AW"},{"key":"10445_CR27","unstructured":"Pham H, Guan M, Zoph B, et\u00a0al (2018) Efficient neural architecture search via parameters sharing. In: Proceedings of the 35th international conference on machine learning, pp 4095\u20134104. https:\/\/proceedings.mlr.press\/v80\/pham18a.html"},{"key":"10445_CR28","unstructured":"Rao X, Zhao B, Yi X, et\u00a0al (2022) CR-LSO: convex neural architecture optimization in the latent space of graph variational autoencoder with input convex neural networks. arXiv:2211.05950"},{"key":"10445_CR29","unstructured":"Real E, Moore S, Selle A, et\u00a0al (2017) Large-scale evolution of image classifiers. In: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning, proceedings of machine learning research, vol\u00a070, pp 2902\u20132911. PMLR. https:\/\/proceedings.mlr.press\/v70\/real17a.html"},{"key":"10445_CR30","unstructured":"Schaul T, Quan J, Antonoglou I, et\u00a0al (2016) Prioritized experience replay [c\/ol]. In: Proceedings of the 4th international conference on learning representations, ICLR"},{"key":"10445_CR31","unstructured":"Siems JN, Zimmer L, Zela A, et\u00a0al (2021) {NAS}-bench-301 and the case for surrogate benchmarks for neural architecture search. https:\/\/openreview.net\/forum?id=1flmvXGGJaa"},{"key":"10445_CR32","doi-asserted-by":"crossref","unstructured":"Szegedy C, Liu W, Jia Y, et\u00a0al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"10445_CR33","unstructured":"Vaswani A, Shazeer N, Parmar N, et\u00a0al (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, et\u00a0al (eds) Advances in neural information processing systems, vol\u00a030. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"10445_CR34","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","volume":"17","author":"P Virtanen","year":"2020","unstructured":"Virtanen P, Gommers R, Oliphant TE et al (2020) SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods 17:261\u2013272. https:\/\/doi.org\/10.1038\/s41592-019-0686-2","journal-title":"Nat Methods"},{"issue":"06","key":"10445_CR35","doi-asserted-by":"publisher","first-page":"9983","DOI":"10.1609\/aaai.v34i06.6554","volume":"34","author":"L Wang","year":"2020","unstructured":"Wang L, Zhao Y, Jinnai Y et al (2020) Neural architecture search using deep neural networks and Monte Carlo tree search. Proc AAAI Conf Artif Intell 34(06):9983\u20139991. https:\/\/doi.org\/10.1609\/aaai.v34i06.6554","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"10445_CR36","unstructured":"Wang Z, Schaul T, Hessel M, et\u00a0al (2016) Dueling network architectures for deep reinforcement learning. In: Balcan MF, Weinberger KQ (eds) Proceedings of the 33rd international conference on machine learning, proceedings of machine learning research, vol\u00a048, pp 1995\u20132003. PMLR, New York. https:\/\/proceedings.mlr.press\/v48\/wangf16.html"},{"key":"10445_CR37","unstructured":"White C, Nolen S, Savani Y (2020) Local search is state of the art for NAS benchmarks. CoRR. arXiv:2005.02960"},{"key":"10445_CR38","doi-asserted-by":"crossref","unstructured":"White C, Neiswanger W, Savani Y (2021a) Bananas: Bayesian optimization with neural architectures for neural architecture search. In: Proceedings of the AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v35i12.17233"},{"key":"10445_CR39","unstructured":"White C, Nolen S, Savani Y (2021b) Exploring the loss landscape in neural architecture search. In: de\u00a0Campos C, Maathuis MH (eds) Proceedings of the thirty-seventh conference on uncertainty in artificial intelligence, proceedings of machine learning research, vol 161, pp 654\u2013664. PMLR. https:\/\/proceedings.mlr.press\/v161\/white21a.html"},{"key":"10445_CR40","unstructured":"White C, Safari M, Sukthanker R, et\u00a0al (2023) Neural architecture search: insights from 1000 papers. arXiv preprint arXiv:2301.08727"},{"key":"10445_CR41","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229\u2013256","journal-title":"Mach Learn"},{"issue":"12","key":"10445_CR42","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.8051","volume":"36","author":"S Xiao","year":"2024","unstructured":"Xiao S, Wang W (2024) Ranking-based architecture generation for surrogate-assisted neural architecture search. Concurr Comput Pract Exp 36(12):e8051","journal-title":"Concurr Comput Pract Exp"},{"key":"10445_CR43","unstructured":"Ying C, Klein A, Christiansen E, et\u00a0al (2019) NAS-bench-101: towards reproducible neural architecture search. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of the 36th international conference on machine learning, proceedings of machine learning research, vol\u00a097, pp 7105\u20137114. PMLR, Long Beach, California. http:\/\/proceedings.mlr.press\/v97\/ying19a.html"},{"key":"10445_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2023.3284995","volume":"61","author":"Q Zhang","year":"2023","unstructured":"Zhang Q, Peng Y, Zhang Z et al (2023) Semantic segmentation of spectral lidar point clouds based on neural architecture search. IEEE Trans Geosci Remote Sens 61:1\u201311. https:\/\/doi.org\/10.1109\/TGRS.2023.3284995","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"10445_CR45","unstructured":"Zhang S, Sutton RS (2018) A deeper look at experience replay. arXiv:1712.01275"},{"key":"10445_CR46","doi-asserted-by":"publisher","first-page":"1029307","DOI":"10.3389\/fdata.2022.1029307","volume":"5","author":"K Zhou","year":"2022","unstructured":"Zhou K, Huang X, Song Q et al (2022) Auto-GNN: neural architecture search of graph neural networks. Front Big Data 5:1029307","journal-title":"Front Big Data"},{"key":"10445_CR47","unstructured":"Zoph B, Le Q (2017) Neural architecture search with reinforcement learning. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=r1Ue8Hcxg"},{"key":"10445_CR48","doi-asserted-by":"crossref","unstructured":"Zoph B, Vasudevan V, Shlens J, et\u00a0al (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8697\u20138710","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10445-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-10445-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10445-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T16:37:11Z","timestamp":1737563831000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-10445-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10445"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-10445-2","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,18]]},"assertion":[{"value":"29 May 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}