{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T21:40:55Z","timestamp":1777671655419,"version":"3.51.4"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Algorithm\u00a0selection and hyperparameter tuning are critical steps in both academic and applied machine learning (ML). These steps are becoming increasingly delicate due to the extensive rise in the number, diversity, and distributed nature of ML resources. Multi-agent systems, when applied to the design of ML platforms, bring about several distinctive characteristics, such as scalability, flexibility, and robustness, just to name a few. This article proposes a fully automatic and collaborative agent-based mechanism for selecting distributed ML algorithms and simultaneously tuning their hyperparameters. Our method builds upon an existing agent-based hierarchical ML platform and augments its query structure to support the aforementioned functionalities without being limited to specific learning, selection, and tuning mechanisms. We have conducted theoretical assessments, formal verification, and analytical study to demonstrate the correctness, resource utilization, and computational efficiency of our technique. According to the results, our solution is algorithmically correct and exhibits linear time and space complexity in relation to the size of available resources. To further verify its correctness and demonstrate its effectiveness and flexibility across a range of algorithmic options and datasets, the article also presents a series of empirical results on a system composed of 24 algorithms and 9 datasets. The findings not only highlight the efficiency and scalability of the proposed approach, but also show its flexibility and openness to responding to the dynamic and distributed ML ecosystem.<\/jats:p>","DOI":"10.1145\/3697834","type":"journal-article","created":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T11:14:53Z","timestamp":1727349293000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Hybrid Algorithm Selection and Hyperparameter Tuning on Distribute Machine Learning Resources: Hierarchical Agent-based Approach"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0612-2351","authenticated-orcid":false,"given":"Ahmad","family":"Esmaeili","sequence":"first","affiliation":[{"name":"School of Computing, Wichita State University, Wichita, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3786-2416","authenticated-orcid":false,"given":"Julia","family":"Rayz","sequence":"additional","affiliation":[{"name":"Computer and Information Technology, Purdue University, West Lafayette, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9200-4903","authenticated-orcid":false,"given":"Eric","family":"Matson","sequence":"additional","affiliation":[{"name":"Computer and Information Technology, Purdue University, West Lafayette, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"[n. d.]. Scikit-learn API Reference. Retrieved December 12 2023 from https:\/\/scikit-learn.org\/stable\/modules\/classes.html"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2008.10.023"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.13922"},{"key":"e_1_3_1_5_2","first-page":"263","volume-title":"International Conference on Innovative Techniques and Applications of Artificial Intelligence","author":"Albashiri Kamal Ali","year":"2008","unstructured":"Kamal Ali Albashiri, Frans Coenen, and Paul Leng. 2008. EMADS: An extendible multi-agent data miner. In International Conference on Innovative Techniques and Applications of Artificial Intelligence. Springer, 263\u2013275."},{"issue":"4","key":"e_1_3_1_6_2","first-page":"369","article-title":"Cascading classifiers","volume":"34","author":"Alpaydin Ethem","year":"1998","unstructured":"Ethem Alpaydin and Cenk Kaynak. 1998. Cascading classifiers. Kybernetika 34, 4 (1998), 369\u2013374.","journal-title":"Kybernetika"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/1390681.1390696"},{"key":"e_1_3_1_8_2","first-page":"199","volume-title":"International Conference on Machine Learning","author":"Bardenet R\u00e9mi","year":"2013","unstructured":"R\u00e9mi Bardenet, M\u00e1ty\u00e1s Brendel, Bal\u00e1zs K\u00e9gl, and Michele Sebag. 2013. Collaborative hyperparameter tuning. In International Conference on Machine Learning. PMLR, 199\u2013207."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1142\/9789812799623_0002"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015187"},{"key":"e_1_3_1_11_2","article-title":"Algorithms for hyper-parameter optimization","volume":"24","author":"Bergstra James","year":"2011","unstructured":"James Bergstra, R\u00e9mi Bardenet, Yoshua Bengio, and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems 24 (2011).","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"2","key":"e_1_3_1_12_2","article-title":"Random search for hyper-parameter optimization.","volume":"13","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, 2 (2012).","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1088\/1749-4699\/8\/1\/014008"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1021713901879"},{"key":"e_1_3_1_15_2","volume-title":"International Conference on Learning Representations","author":"Cai Han","year":"2018","unstructured":"Han Cai, Ligeng Zhu, and Song Han. 2018. ProxylessNAS: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013943418833"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/W-FiCloud.2018.00009"},{"key":"e_1_3_1_19_2","article-title":"HPN: Personalized Federated Hyperparameter Optimization","author":"Cheng Anda","year":"2023","unstructured":"Anda Cheng, Zhen Wang, Yaliang Li, and Jian Cheng. 2023. HPN: Personalized Federated Hyperparameter Optimization. arXiv preprint arXiv:2304.05195 (2023).","journal-title":"arXiv preprint arXiv:2304.05195"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.115225"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1214\/009053604000000067"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3530191"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-18192-4_11"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.3390\/systems11050228"},{"key":"e_1_3_1_25_2","first-page":"226","volume-title":"KDD","author":"Ester Martin","year":"1996","unstructured":"Martin Ester, Hans-Peter Kriegel, J\u00f6rg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, Vol. 96. 226\u2013231."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.5555\/3586589.3586850"},{"key":"e_1_3_1_27_2","article-title":"Efficient and robust automated machine learning","volume":"28","author":"Feurer Matthias","year":"2015","unstructured":"Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and robust automated machine learning. Advances in Neural Information Processing Systems 28 (2015).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-45185-3_7"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-1809.1936.tb02137.x"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1983.10478008"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/IAT.2003.1241116"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1080\/0952813X.2020.1864784"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/SSCI44817.2019.9003174"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/1656274.1656278"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","unstructured":"David Harrison Jr and Daniel L. Rubinfeld. 1978. Hedonic housing prices and the demand for clean air. Journal of Environmental Economics and Management 5 1 (1978) 81\u2013102.","DOI":"10.1016\/0095-0696(78)90006-2"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1970.10488634"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"e_1_3_1_39_2","article-title":"Multi-agent reinforcement learning for hyperparameter optimization of convolutional neural networks","author":"Iranfar Arman","year":"2021","unstructured":"Arman Iranfar, Marina Zapater, and David Atienza. 2021. Multi-agent reinforcement learning for hyperparameter optimization of convolutional neural networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2021).","journal-title":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-022-03621-3"},{"key":"e_1_3_1_41_2","first-page":"211","volume-title":"KDD","author":"Kargupta Hillol","year":"1997","unstructured":"Hillol Kargupta, Ilker Hamzaoglu, and Brian Stafford. 1997. Scalable, Distributed Data Mining\u2014An Agent Architecture. In KDD. 211\u2013214."},{"key":"e_1_3_1_42_2","first-page":"131","article-title":"Collective data mining: A new perspective toward distributed data mining","volume":"2","author":"Kargupta Hillol","year":"1999","unstructured":"Hillol Kargupta, B. Park, Daryl Hershberger, and Erik Johnson. 1999. Collective data mining: A new perspective toward distributed data mining. Advances in Distributed and Parallel Knowledge Discovery 2 (1999), 131\u2013174.","journal-title":"Advances in Distributed and Parallel Knowledge Discovery"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-7152(96)00140-X"},{"key":"e_1_3_1_44_2","first-page":"19184","article-title":"Federated hyperparameter tuning: Challenges, baselines, and connections to weight-sharing","volume":"34","author":"Khodak Mikhail","year":"2021","unstructured":"Mikhail Khodak, Renbo Tu, Tian Li, Liam Li, Maria-Florina F. Balcan, Virginia Smith, and Ameet Talwalkar. 2021. Federated hyperparameter tuning: Challenges, baselines, and connections to weight-sharing. Advances in Neural Information Processing Systems 34 (2021), 19184\u201319197.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219837"},{"key":"e_1_3_1_46_2","first-page":"1137","volume-title":"IJCAI","author":"Kohavi Ron","year":"1995","unstructured":"Ron Kohavi. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI, Vol. 14. Montreal, Canada, 1137\u20131145."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","unstructured":"S. Aeberhard and M. Forina. 1992. Wine. UCI Machine Learning Repository. DOI:10.24432\/C5PC7J","DOI":"10.24432\/C5PC7J"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.5555\/3586589.3586643"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.5555\/2831071.2831088"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1982.1056489"},{"key":"e_1_3_1_51_2","first-page":"2113","volume-title":"International Conference on Machine Learning","author":"Maclaurin Dougal","year":"2015","unstructured":"Dougal Maclaurin, David Duvenaud, and Ryan Adams. 2015. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning. PMLR, 2113\u20132122."},{"key":"e_1_3_1_52_2","volume-title":"Machine Learning: A Probabilistic Perspective","author":"Murphy Kevin P.","year":"2012","unstructured":"Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. MIT Press."},{"key":"e_1_3_1_53_2","unstructured":"Tom O\u2019Malley Elie Bursztein James Long Fran\u00e7ois Chollet Haifeng Jin and Luca Invernizzi. 2019. Keras Tuner. Retrieved from https:\/\/github.com\/keras-team\/keras-tuner"},{"key":"e_1_3_1_54_2","first-page":"17200","article-title":"Provably efficient online hyperparameter optimization with population-based bandits","volume":"33","author":"Parker-Holder Jack","year":"2020","unstructured":"Jack Parker-Holder, Vu Nguyen, and Stephen J. Roberts. 2020. Provably efficient online hyperparameter optimization with population-based bandits. Advances in Neural Information Processing Systems 33 (2020), 17200\u201317211.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555776.3577847"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3074125"},{"key":"e_1_3_1_58_2","article-title":"Model evaluation, model selection, and algorithm selection in machine learning","author":"Raschka Sebastian","year":"2018","unstructured":"Sebastian Raschka. 2018. Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808 (2018).","journal-title":"arXiv preprint arXiv:1811.12808"},{"key":"e_1_3_1_59_2","first-page":"616","volume-title":"Proceedings of the 20th International Conference on Machine Learning (ICML-03)","author":"Rennie Jason D.","year":"2003","unstructured":"Jason D. Rennie, Lawrence Shih, Jaime Teevan, and David R. Karger. 2003. Tackling the poor assumptions of naive Bayes text classifiers. In Proceedings of the 20th International Conference on Machine Learning (ICML-03). 616\u2013623."},{"key":"e_1_3_1_60_2","first-page":"65","volume-title":"Advances in Computers","author":"Rice John R.","year":"1976","unstructured":"John R. Rice. 1976. The algorithm selection problem. In Advances in Computers. Vol. 15. Elsevier, 65\u2013118."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-25465-X_15"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781119597926"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772862"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1980.1675516"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487629"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1996.tb02080.x"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.082099299"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1198\/106186005X59243"},{"key":"e_1_3_1_69_2","article-title":"Automatic configuration of deep neural networks with EGO","author":"Stein Bas van","year":"2018","unstructured":"Bas van Stein, Hao Wang, and Thomas B\u00e4ck. 2018. Automatic configuration of deep neural networks with EGO. arXiv preprint arXiv:1810.05526 (2018).","journal-title":"arXiv preprint arXiv:1810.05526"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1016\/0304-3835(94)90099-X"},{"key":"e_1_3_1_71_2","first-page":"20147","article-title":"Multi-agent dynamic algorithm configuration","volume":"35","author":"Xue Ke","year":"2022","unstructured":"Ke Xue, Jiacheng Xu, Lei Yuan, Miqing Li, Chao Qian, Zongzhang Zhang, and Yang Yu. 2022. Multi-agent dynamic algorithm configuration. Advances in Neural Information Processing Systems 35 (2022), 20147\u201320161.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2019.8852200"},{"key":"e_1_3_1_73_2","first-page":"7124","volume-title":"International Conference on Machine Learning","author":"You Kaichao","year":"2019","unstructured":"Kaichao You, Ximei Wang, Mingsheng Long, and Michael Jordan. 2019. Towards accurate model selection in deep unsupervised domain adaptation. In International Conference on Machine Learning. PMLR, 7124\u20137133."},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13755-017-0023-z"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/235968.233324"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2005.00503.x"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3697834","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3697834","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:30Z","timestamp":1750295850000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3697834"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":75,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3697834"],"URL":"https:\/\/doi.org\/10.1145\/3697834","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"value":"1533-5399","type":"print"},{"value":"1557-6051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,18]]},"assertion":[{"value":"2024-01-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}