{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T16:54:22Z","timestamp":1758041662380,"version":"3.44.0"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"5","funder":[{"name":"NI, an Emerson company"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Des. Autom. Electron. Syst."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>We present a solution to the complex design-automation problem of scheduling test operations in a validation laboratory or production facility. Our goal is to maximize the utilization of a fleet of test stations and minimize the overall test time for a set of products. We consider the realistic scenario where tests can have dependency graphs, implying that some tests must be completed and passed before others can proceed. We also consider a mix of product types that require different kinds of tests and a mix of testers, which implies that each product can only be tested only on a specific set of testers. To ensure scalability and flexibility, we have formulated this scheduling problem as a \u201cpartially observable stochastic game\u201d, a multi-agent extension of a partially observable Markov decision process. We have implemented multi-agent reinforcement learning agents to maximize parallelization in a manner that speeds up both training and inferencing. We present scheduling results for synthetic test cases as well as real-life data from a production facility.<\/jats:p>","DOI":"10.1145\/3749985","type":"journal-article","created":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T09:26:55Z","timestamp":1753435615000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Test-Fleet Scheduling in Complex Validation and Production Environments"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1619-4586","authenticated-orcid":false,"given":"Aniruddha","family":"Datta","sequence":"first","affiliation":[{"name":"School of Electrical, Computer and Energy Engineering, Arizona State University","place":["Tempe, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0723-4614","authenticated-orcid":false,"given":"Bhanu","family":"Yaganti","sequence":"additional","affiliation":[{"name":"Amazon Web Services Inc","place":["Seattle, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-3350-8614","authenticated-orcid":false,"given":"Mate","family":"Palocska","sequence":"additional","affiliation":[{"name":"National Instruments Hungary","place":["Debrecen, Hungary"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2788-893X","authenticated-orcid":false,"given":"Andrew","family":"Dove","sequence":"additional","affiliation":[{"name":"National Instruments Corporation","place":["Austin, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8834-111X","authenticated-orcid":false,"given":"Arik","family":"Peltz","sequence":"additional","affiliation":[{"name":"Advantest","place":["Petach Tikva, Israel"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4475-6435","authenticated-orcid":false,"given":"Krishnendu","family":"Chakrabarty","sequence":"additional","affiliation":[{"name":"School of Electrical, Computer and Energy Engineering, Arizona State University","place":["Tempe, United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,9,11]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTTAS.2001.929862"},{"key":"e_1_3_1_3_2","first-page":"1639","volume-title":"Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics.","volume":"108","author":"Abel David","year":"2020","unstructured":"David Abel, Nate Umbanhowar, Khimya Khetarpal, Dilip Arumugam, Doina Precup, and Michael Littman. 2020. Value preserving state-action abstractions. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics.Silvia Chiappa and Roberto Calandra (Eds.), Proceedings of Machine Learning Research, Vol. 108, PMLR, 1639\u20131650. Retrieved from https:\/\/proceedings.mlr.press\/v108\/abel20a.html"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-08696-5_13"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9904-1954-09848-8"},{"key":"e_1_3_1_6_2","volume-title":"The SCIP Optimization Suite 8.0","author":"Bestuzheva Ksenia","year":"2021","unstructured":"Ksenia Bestuzheva et\u00a0al. 2021. The SCIP Optimization Suite 8.0. ZIB-Report 21-41. Zuse Institute Berlin. Retrieved from http:\/\/nbn-resolving.de\/urn:nbn:de:0297-zib-85309"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2010.2066630"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/43.875306"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1146909.1147025"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3546871"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ETS61313.2024.10567053"},{"key":"e_1_3_1_12_1","unstructured":"Andrew Dove. [n. d.]. A National Instruments Testing floor. Accessed: 2024-3-14."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","unstructured":"V\u00edctor Elvira and Luca Martino. 2021. Advances in Importance Sampling. 14 pages. DOI:10.1002\/9781118445112.stat08284","DOI":"10.1002\/9781118445112.stat08284"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/1018409.1018741"},{"key":"e_1_3_1_15_2","unstructured":"Lasse. Espeholt et\u00a0al. 2018. IMPALA: Scalable distributed deep-RL importance weighted actor-learner architectures. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research Vol. 80) Jennifer Dy and Andreas Krause (Eds.). PMLR 1407\u20131416. https:\/\/proceedings.mlr.press\/v80\/espeholt18a.html"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2023.109255"},{"key":"e_1_3_1_17_2","series-title":"Proceedings of Machine Learning Research","first-page":"1861","volume-title":"Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja et\u00a0al. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning. Jennifer Dy and Andreas Krause (Eds.), Proceedings of Machine Learning Research, Vol. 80, PMLR, 1861\u20131870. Retrieved from https:\/\/proceedings.mlr.press\/v80\/haarnoja18b.html"},{"key":"e_1_3_1_18_2","unstructured":"Tuomas. Haarnojah et\u00a0al. 2019. Soft actor-critic algorithms and applications. arXiv:1812.05905 [cs.LG] https:\/\/arxiv.org\/abs\/1812.05905"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cja.2020.12.027"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2023.3275532"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITC51656.2023.00038"},{"key":"e_1_3_1_22_2","unstructured":"B. Ravi Kiran Ibrahim Sobh Victor Talpaert Patrick Mannion Ahmad A. Al Sallab Senthil Yogamani and Patrick P\u00e9rez. 2020. Deep reinforcement learning for autonomous driving: A survey. CoRR abs\/2002.00444 (2020). arXiv:2002.00444 https:\/\/arxiv.org\/abs\/2002.00444"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/785411.785416"},{"key":"e_1_3_1_24_2","unstructured":"Arsenii. Kuznetsov et\u00a0al. 2020. Controlling overestimation bias with truncated mixture of continuous distributional quantile critics. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research Vol. 119) Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR 5556\u20135566. https:\/\/proceedings.mlr.press\/v119\/kuznetsov20a.html"},{"key":"e_1_3_1_25_2","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Liang Eric","year":"2018","unstructured":"Eric Liang, Richard Liaw, Philipp Moritz, Robert Nishihara, Roy Fox, Ken Goldberg, Joseph E. Gonzalez, Michael I. Jordan, and Ion Stoica. 2018. RLlib: Abstractions for distributed reinforcement learning. In Proceedings of the International Conference on Machine Learning (ICML)."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2021.3065919"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2020.3031865"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-42432-3_37"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","unstructured":"David Silver et\u00a0al. 2018. A general reinforcement learning algo-rithm that masters chess shogi and Go through self-play. Science 362 6419 (2018) 1140\u20131144. 10.1126\/science.aar6404 arXiv:https:\/\/www.science.org\/doi\/pdf\/10.1126\/science.aar6404","DOI":"10.1126\/science.aar6404"},{"key":"e_1_3_1_30_2","unstructured":"Volodymyr Mnih Adria Puigdomenech Badia Mehdi Mirza Alex Graves Timothy Lillicrap Tim Harley David Silver and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In Proceedings of The 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research Vol. 48) Maria Florina Balcan and Kilian Q. Weinberger (Eds.). PMLR New York New York USA 1928\u20131937. https:\/\/proceedings.mlr.press\/v48\/mniha16.html"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","unstructured":"Volodymyr Mnih Koray Kavukcuoglu David Silver Andrei A. Rusu Jo[40el Veness Marc G. Bellemare Alex Graves Martin Riedmiller Andreas K. Fidjeland Georg Ostrovski Stig Petersen Charles Beattie Amir Sadik Ioannis Antonoglou Helen King Dharshan Kumaran Daan Wierstra Shane Legg and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518 7540 (2015) 529\u2013533. 10.1038\/nature14236","DOI":"10.1038\/nature14236"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITC44170.2019.9000171"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/RTSS.2017.00009"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-0255(03)00072-0"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2019.2956762"},{"key":"e_1_3_1_36_2","volume-title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke et\u00a0al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Curran Associates Inc., Red Hook, NY, USA."},{"key":"e_1_3_1_37_2","unstructured":"Rafael Rafailov Archit Sharma Eric Mitchell Christopher D. Manning Stefano Ermon and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems A. Oh T. Naumann A. Globerson K. Saenko M. Hardt and S. Levine (Eds.). Vol. 36. Curran Associates Inc. 53728\u201353741. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/a85b405ed65c6477a4fe8302b5e06ce7-Paper-Conference.pdf"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.epsr.2019.01.040"},{"key":"e_1_3_1_39_2","series-title":"Proceedings of Machine Learning Research","first-page":"1889","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","volume":"37","author":"Schulman John","year":"2015","unstructured":"John Schulman et\u00a0al. 2015. Trust region policy optimization. In Proceedings of the 32nd International Conference on Machine Learning. Francis Bach and David Blei (Eds.), Proceedings of Machine Learning Research, Vol. 37, PMLR, Lille, France, 1889\u20131897. Retrieved from https:\/\/proceedings.mlr.press\/v37\/schulman15.html"},{"key":"e_1_3_1_40_2","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal policy optimization algorithms. ArXiv abs\/1707.06347 (2017). https:\/\/api.semanticscholar.org\/CorpusID:28695052"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","unstructured":"Pierre Tassel Martin Gebser and Konstantin Schekotihin. 2023. An end-to-end reinforcement learning approach for job-shop scheduling problemsbased on constraint programming. In Proceedings of the Thirty-Third International Conference on Automated Planning and Scheduling (Prague Czech Republic) (ICAPS\u201923). AAAI Press Article 72 9 pages. 10.1609\/icaps.v33i1.27243","DOI":"10.1609\/icaps.v33i1.27243"},{"key":"e_1_3_1_42_2","unstructured":"Benjamin Sch\u00f6nfu\u00df. 2024. How AI is transforming the factory floor. Retrieved July 4 2025 from https:\/\/www.weforum.org\/stories\/2024\/10\/ai-transforming-factory-floor-artificial-intelligence\/"},{"key":"e_1_3_1_43_2","unstructured":"Lenovo. 2023. Lenovo keeps manufacturing lines rolling around the clock with AI-powered production scheduling. Retrieved July 4 2025 from https:\/\/news.lenovo.com\/manufacturing-lines-ai-powered-production-scheduling\/"},{"key":"e_1_3_1_44_2","unstructured":"Tetrapak. 2025. Making vision a reality for Mengniu in Ningxia. Retrieved July 4 2025 from https:\/\/www.tetrapak.com\/insights\/cases-articles\/future-of-dairy-production"},{"key":"e_1_3_1_45_2","unstructured":"Haoyu Wang Jialin Liu Xiaohan Chen Xinshang Wang Pan Li and Wotao Yin. 2024. DIG-MILP: A Deep Instance Generator for Mixed-Integer Linear Programming with Feasibility Guarantee. Retrieved August 19 2025 from https:\/\/openreview.net\/forum?id=psDvcWtFdE"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-60990-0_12"}],"container-title":["ACM Transactions on Design Automation of Electronic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749985","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T12:32:18Z","timestamp":1757593938000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749985"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,11]]},"references-count":45,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3749985"],"URL":"https:\/\/doi.org\/10.1145\/3749985","relation":{},"ISSN":["1084-4309","1557-7309"],"issn-type":[{"type":"print","value":"1084-4309"},{"type":"electronic","value":"1557-7309"}],"subject":[],"published":{"date-parts":[[2025,9,11]]},"assertion":[{"value":"2024-12-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}