{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,5]],"date-time":"2025-08-05T12:54:32Z","timestamp":1754398472260,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,9,12]],"date-time":"2022-09-12T00:00:00Z","timestamp":1662940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Bundesministerium f\u00fcr Bildung und Forschung","award":["13FH197PX8"],"award-info":[{"award-number":["13FH197PX8"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,9,12]]},"DOI":"10.1145\/3551901.3556474","type":"proceedings-article","created":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T22:11:12Z","timestamp":1662502272000},"page":"21-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Deep Reinforcement Learning for Analog Circuit Sizing with an Electrical Design Space and Sparse Rewards"],"prefix":"10.1145","author":[{"given":"Yannick","family":"Uhlmann","sequence":"first","affiliation":[{"name":"Reutlingen University, Reutlingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Essich","sequence":"additional","affiliation":[{"name":"Reutlingen University, Reutlingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lennart","family":"Bramlage","sequence":"additional","affiliation":[{"name":"Reutlingen University, Reutlingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J\u00fcrgen","family":"Scheible","sequence":"additional","affiliation":[{"name":"Reutlingen University, Reutlingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Crist\u00f3bal","family":"Curio","sequence":"additional","affiliation":[{"name":"Reutlingen University, Reutlingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,9,12]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"An application of reinforcement learning to aerobatic helicopter flight. Advances in neural information processing systems","author":"Abbeel Pieter","year":"2006","unstructured":"Pieter Abbeel , Adam Coates , Morgan Quigley , and Andrew Ng. 2006. An application of reinforcement learning to aerobatic helicopter flight. Advances in neural information processing systems , Vol. 19 ( 2006 ). Pieter Abbeel, Adam Coates, Morgan Quigley, and Andrew Ng. 2006. An application of reinforcement learning to aerobatic helicopter flight. Advances in neural information processing systems, Vol. 19 (2006)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2018.e00938"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2966414"},{"key":"e_1_3_2_1_4_1","volume-title":"Advances in Neural Information Processing Systems","volume":"30","author":"Andrychowicz Marcin","year":"2017","unstructured":"Marcin Andrychowicz , Filip Wolski , Alex Ray , Jonas Schneider , Rachel Fong , Peter Welinder , Bob McGrew , Josh Tobin , Open AI Pieter Abbeel , and Wojciech Zaremba . 2017 . Hindsight Experience Replay . In Advances in Neural Information Processing Systems , Vol. 30 . Curran Associates, Inc. Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. 2017. Hindsight Experience Replay. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc."},{"key":"e_1_3_2_1_5_1","volume-title":"Openai gym. arXiv preprint arXiv:1606.01540","author":"Brockman Greg","year":"2016","unstructured":"Greg Brockman , Vicki Cheung , Ludwig Pettersson , Jonas Schneider , John Schulman , Jie Tang , and Wojciech Zaremba . 2016. Openai gym. arXiv preprint arXiv:1606.01540 ( 2016 ). Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. Openai gym. arXiv preprint arXiv:1606.01540 (2016)."},{"key":"e_1_3_2_1_6_1","volume-title":"Carmona","author":"Castejon Federico","year":"2020","unstructured":"Federico Castejon and Enrique J . Carmona . 2020 . Introducing Modularity and Homology in Grammatical Evolution to Address the Analog Electronic Circuit Design Problem ., Vol. 8 (2020). https:\/\/doi.org\/10.1109\/access.2020.3011641 10.1109\/access.2020.3011641 Federico Castejon and Enrique J. Carmona. 2020. Introducing Modularity and Homology in Grammatical Evolution to Address the Analog Electronic Circuit Design Problem., Vol. 8 (2020). https:\/\/doi.org\/10.1109\/access.2020.3011641"},{"key":"e_1_3_2_1_9_1","volume-title":"IEEE\/ACM International Conference on Computer Aided Design. ICCAD","author":"Graeb H.","year":"2001","unstructured":"H. Graeb , S. Zizala , J. Eckmueller , and K. Antreich . 2001. The sizing rules method for analog integrated circuit design . In IEEE\/ACM International Conference on Computer Aided Design. ICCAD 2001 . 343--349. H. Graeb, S. Zizala, J. Eckmueller, and K. Antreich. 2001. The sizing rules method for analog integrated circuit design. In IEEE\/ACM International Conference on Computer Aided Design. ICCAD 2001. 343--349."},{"volume-title":"Compact Models for Initial MOSFET Sizing Based on Higher-order Artificial Neural Networks. In 2020 ACM\/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD). 111--116","author":"Habal H.","key":"e_1_3_2_1_10_1","unstructured":"H. Habal , D. Tsonev , and M. Schweikardt . 2020 . Compact Models for Initial MOSFET Sizing Based on Higher-order Artificial Neural Networks. In 2020 ACM\/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD). 111--116 . H. Habal, D. Tsonev, and M. Schweikardt. 2020. Compact Models for Initial MOSFET Sizing Based on Higher-order Artificial Neural Networks. In 2020 ACM\/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD). 111--116."},{"key":"e_1_3_2_1_11_1","unstructured":"Austin Huang Junji Hashimoto Sam Stites and Torsten Scholak. 2017. HaskTorch. https:\/\/github.com\/hasktorch\/hasktorch  Austin Huang Junji Hashimoto Sam Stites and Torsten Scholak. 2017. HaskTorch. https:\/\/github.com\/hasktorch\/hasktorch"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2946645.2946684"},{"key":"e_1_3_2_1_13_1","unstructured":"Yaguang Li Yishuang Lin Meghna Madhusudan Arvind Sharma Sachin Sapatnekar Ramesh Harjani and Jiang Hu. 2021. A Circuit Attention Network-Based Actor-Critic Learning Approach to Robust Analog Transistor Sizing. In 2021 ACM\/IEEE 3textsuperscriptrd Workshop on Machine Learning for CAD (MLCAD). 1--6.  Yaguang Li Yishuang Lin Meghna Madhusudan Arvind Sharma Sachin Sapatnekar Ramesh Harjani and Jiang Hu. 2021. A Circuit Attention Network-Based Actor-Critic Learning Approach to Robust Analog Transistor Sizing. In 2021 ACM\/IEEE 3textsuperscriptrd Workshop on Machine Learning for CAD (MLCAD). 1--6."},{"key":"e_1_3_2_1_14_1","volume-title":"ICLR 2016, San Juan, Puerto Rico, May 2--4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.).arxiv: 1509","author":"Lillicrap Timothy P.","year":"2016","unstructured":"Timothy P. Lillicrap , Jonathan J. Hunt , Alexander Pritzel , Nicolas Heess , Tom Erez , Yuval Tassa , David Silver , and Daan Wierstra . 2016 . Continuous control with deep reinforcement learning. In 4textsuperscriptth International Conference on Learning Representations , ICLR 2016, San Juan, Puerto Rico, May 2--4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.).arxiv: 1509 .02971 Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016. Continuous control with deep reinforcement learning. In 4textsuperscriptth International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2--4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.).arxiv: 1509.02971"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSI.2017.2768826"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11030435"},{"key":"e_1_3_2_1_17_1","volume-title":"Playing Atari with Deep Reinforcement Learning. (Dec","author":"Mnih Volodymyr","year":"2013","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Alex Graves , Ioannis Antonoglou , Daan Wierstra , and Martin Riedmiller . 2013. Playing Atari with Deep Reinforcement Learning. (Dec . 2013 ). arxiv: 1312.5602 [cs.LG] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. (Dec. 2013). arxiv: 1312.5602 [cs.LG]"},{"key":"e_1_3_2_1_18_1","first-page":"278","article-title":"Policy invariance under reward transformations: Theory and application to reward shaping","volume":"99","author":"Ng Andrew Y","year":"1999","unstructured":"Andrew Y Ng , Daishi Harada , and Stuart Russell . 1999 . Policy invariance under reward transformations: Theory and application to reward shaping . In Icml , Vol. 99. 278 -- 287 . Andrew Y Ng, Daishi Harada, and Stuart Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In Icml, Vol. 99. 278--287.","journal-title":"Icml"},{"key":"e_1_3_2_1_19_1","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin Antonin","year":"2021","unstructured":"Antonin Raffin , Ashley Hill , Adam Gleave , Anssi Kanervisto , Maximilian Ernestus , and Noah Dormann . 2021 . Stable-Baselines3: Reliable Reinforcement Learning Implementations . Journal of Machine Learning Research , Vol. 22 , 268 (2021), 1 -- 8 . Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research, Vol. 22, 268 (2021), 1--8.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_20_1","volume-title":"Advances in Neural Information Processing Systems","volume":"28","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren , Kaiming He , Ross Girshick , and Jian Sun . 2015 . Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks . In Advances in Neural Information Processing Systems , Vol. 28 . Curran Associates, Inc. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems, Vol. 28. Curran Associates, Inc."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"1320","author":"Schaul Tom","year":"2015","unstructured":"Tom Schaul , Daniel Horgan , Karol Gregor , and David Silver . 2015 . Universal Value Function Approximators . In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research , Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 1312-- 1320 . Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. 2015. Universal Value Function Approximators. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 1312--1320."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3505170.3511042"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2717764.2717781"},{"key":"e_1_3_2_1_24_1","volume-title":"Proximal Policy Optimization Algorithms. (July","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . 2017. Proximal Policy Optimization Algorithms. (July 2017 ). arxiv: 1707.06347 [cs.LG] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. (July 2017). arxiv: 1707.06347 [cs.LG]"},{"key":"e_1_3_2_1_25_1","unstructured":"M. Schweikardt and J. Scheible. 2021. Improvement of Simulation-Based Analog Circuit Sizing using Design-Space Transformation. In SMACD \/ PRIME 2021; International Conference on SMACD and 16th Conference on PRIME. 1--4.  M. Schweikardt and J. Scheible. 2021. Improvement of Simulation-Based Analog Circuit Sizing using Design-Space Transformation. In SMACD \/ PRIME 2021; International Conference on SMACD and 16th Conference on PRIME. 1--4."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","unstructured":"M. Schweikardt and J. Scheible. 2022. Expert Design Plan: A Toolbox for Procedural Analog Integrated Circuit Design. In SMACD \/ PRIME 2022; International Conference on SMACD and 17th Conference on PRIME. 1--4.  M. Schweikardt and J. Scheible. 2022. Expert Design Plan: A Toolbox for Procedural Analog Integrated Circuit Design. In SMACD \/ PRIME 2022; International Conference on SMACD and 17th Conference on PRIME. 1--4.","DOI":"10.1109\/SMACD55068.2022.9816336"},{"key":"#cr-split#-e_1_3_2_1_27_1.1","doi-asserted-by":"crossref","unstructured":"Keertana Settaluri Ameer Haj-Ali Qijing Huang Kourosh Hakhamaneshi and Borivoje Nikolic. 2020. AutoCkt: Deep Reinforcement Learning of Analog Circuit Designs. https:\/\/doi.org\/10.23919\/date48585.2020.9116200 10.23919\/date48585.2020.9116200","DOI":"10.23919\/DATE48585.2020.9116200"},{"key":"#cr-split#-e_1_3_2_1_27_1.2","doi-asserted-by":"crossref","unstructured":"Keertana Settaluri Ameer Haj-Ali Qijing Huang Kourosh Hakhamaneshi and Borivoje Nikolic. 2020. AutoCkt: Deep Reinforcement Learning of Analog Circuit Designs. https:\/\/doi.org\/10.23919\/date48585.2020.9116200","DOI":"10.23919\/DATE48585.2020.9116200"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"volume-title":"SMACD \/ PRIME 2021","author":"Uhlmann Yannick","key":"e_1_3_2_1_30_1","unstructured":"Yannick Uhlmann , Michael Essich , Matthias Schweikardt , Juergen Scheible , and Cristobal Curio . 2021. Machine Learning Based Procedural Circuit Sizing and DC Operating Point Prediction . In SMACD \/ PRIME 2021 ; International Conference on SMACD and 16textsuperscriptth Conference on PRIME. 1--4. Yannick Uhlmann, Michael Essich, Matthias Schweikardt, Juergen Scheible, and Cristobal Curio. 2021. Machine Learning Based Procedural Circuit Sizing and DC Operating Point Prediction. In SMACD \/ PRIME 2021; International Conference on SMACD and 16textsuperscriptth Conference on PRIME. 1--4."},{"key":"#cr-split#-e_1_3_2_1_31_1.1","doi-asserted-by":"crossref","unstructured":"Hanrui Wang Kuan Wang Jiacheng Yang Linxiao Shen Nan Sun Hae-Seung Lee and Song Han. 2020. GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning. https:\/\/doi.org\/10.1109\/dac18072.2020.9218757 10.1109\/dac18072.2020.9218757","DOI":"10.1109\/DAC18072.2020.9218757"},{"key":"#cr-split#-e_1_3_2_1_31_1.2","doi-asserted-by":"crossref","unstructured":"Hanrui Wang Kuan Wang Jiacheng Yang Linxiao Shen Nan Sun Hae-Seung Lee and Song Han. 2020. GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning. https:\/\/doi.org\/10.1109\/dac18072.2020.9218757","DOI":"10.1109\/DAC18072.2020.9218757"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3010875"},{"key":"e_1_3_2_1_34_1","volume-title":"Deep Reinforcement Learning for Analog Circuit Sizing. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS). 1--5. https:\/\/doi.org\/10","author":"Zhao Zhenxin","year":"2020","unstructured":"Zhenxin Zhao and Lihong Zhang . 2020 . Deep Reinforcement Learning for Analog Circuit Sizing. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS). 1--5. https:\/\/doi.org\/10 .1109\/ISCAS45731.2020.9181149 10.1109\/ISCAS45731.2020.9181149 Zhenxin Zhao and Lihong Zhang. 2020. Deep Reinforcement Learning for Analog Circuit Sizing. In 2020 IEEE International Symposium on Circuits and Systems (ISCAS). 1--5. https:\/\/doi.org\/10.1109\/ISCAS45731.2020.9181149"}],"event":{"name":"MLCAD '22: 2022 ACM\/IEEE Workshop on Machine Learning for CAD","sponsor":["SIGDA ACM Special Interest Group on Design Automation","IEEE CEDA"],"location":"Virtual Event China","acronym":"MLCAD '22"},"container-title":["Proceedings of the 2022 ACM\/IEEE Workshop on Machine Learning for CAD"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551901.3556474","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3551901.3556474","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:17Z","timestamp":1750186817000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551901.3556474"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,12]]},"references-count":32,"alternative-id":["10.1145\/3551901.3556474","10.1145\/3551901"],"URL":"https:\/\/doi.org\/10.1145\/3551901.3556474","relation":{},"subject":[],"published":{"date-parts":[[2022,9,12]]},"assertion":[{"value":"2022-09-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}