{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T20:07:42Z","timestamp":1783714062154,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,9,7]],"date-time":"2020-09-07T00:00:00Z","timestamp":1599436800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"German Federal Ministry of Education and Research (BMBF)","award":["01IW19001"],"award-info":[{"award-number":["01IW19001"]}]},{"name":"German Federal Ministry of Education and Research (BMBF)","award":["16KIS0821K"],"award-info":[{"award-number":["16KIS0821K"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,9,7]]},"DOI":"10.1145\/3386263.3406901","type":"proceedings-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T21:34:20Z","timestamp":1599255260000},"page":"297-302","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Early Verification of ISA Extension Specifications using Deep Reinforcement Learning"],"prefix":"10.1145","author":[{"given":"Niklas","family":"Bruns","sequence":"first","affiliation":[{"name":"Cyber-Physical Systems, DFKI GmbH, Bremen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Gro\u00dfe","sequence":"additional","affiliation":[{"name":"Johannes Kepler University & Cyber-Physical Systems, DFKI GmbH, Linz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rolf","family":"Drechsler","sequence":"additional","affiliation":[{"name":"University of Bremen & Cyber-Physical Systems, DFKI GmbH, Bremen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,9,7]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Genesys-Pro: Innovations in Test Program Generation for Functional Processor Verification. D$&$T","author":"Adir Allon","year":"2004","unstructured":"Allon Adir , E. Almog , L. Fournier , E. Marcus , Michal Rimon , Michael Vinov , and Avi Ziv . 2004. Genesys-Pro: Innovations in Test Program Generation for Functional Processor Verification. D$&$T , Vol. 21 (04 2004 ), 84 -- 93. Allon Adir, E. Almog, L. Fournier, E. Marcus, Michal Rimon, Michael Vinov, and Avi Ziv. 2004. Genesys-Pro: Innovations in Test Program Generation for Functional Processor Verification. D$&$T, Vol. 21 (04 2004), 84 -- 93."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"crossref","unstructured":"Takumi Akazaki Shuang Liu Yoriyuki Yamagata Yihai Duan and Jianye Hao. 2018. Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning. In Formal Methods. 456--465.  Takumi Akazaki Shuang Liu Yoriyuki Yamagata Yihai Duan and Jianye Hao. 2018. Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning. In Formal Methods. 456--465.","DOI":"10.1007\/978-3-319-95582-7_27"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2743240"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3102980.3103002"},{"key":"e_1_3_2_2_5_1","unstructured":"Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540.  Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. arXiv:1606.01540."},{"key":"e_1_3_2_2_6_1","volume-title":"The design of Rijndael: AES-the advanced encryption standard","author":"Daemen Joan","unstructured":"Joan Daemen and Vincent Rijmen . 2013. The design of Rijndael: AES-the advanced encryption standard . Springer Science & Business Media . Joan Daemen and Vincent Rijmen. 2013. The design of Rijndael: AES-the advanced encryption standard .Springer Science & Business Media."},{"key":"e_1_3_2_2_7_1","volume-title":"Best Practices in Virtual Prototyping","author":"Schutter Tom De","unstructured":"Tom De Schutter . 2014. Better Software . Faster! : Best Practices in Virtual Prototyping . Synopsys Press . Tom De Schutter. 2014. Better Software. Faster!: Best Practices in Virtual Prototyping .Synopsys Press."},{"key":"e_1_3_2_2_8_1","volume-title":"FuzzerGym: A Competitive Framework for Fuzzing and Learning. arXiv preprint arXiv:1807.07490","author":"Drozd William","year":"2018","unstructured":"William Drozd and Michael D Wagner . 2018. FuzzerGym: A Competitive Framework for Fuzzing and Learning. arXiv preprint arXiv:1807.07490 ( 2018 ). William Drozd and Michael D Wagner. 2018. FuzzerGym: A Competitive Framework for Fuzzing and Learning. arXiv preprint arXiv:1807.07490 (2018)."},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"crossref","unstructured":"Laurent Fournier and Avi Ziv. 2008. Using Virtual Coverage to Hit Hard-To-Reach Events. In HVC. 104--119.  Laurent Fournier and Avi Ziv. 2008. Using Virtual Coverage to Hit Hard-To-Reach Events. In HVC. 104--119.","DOI":"10.1007\/978-3-540-77966-7_11"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1968502.1968509"},{"key":"e_1_3_2_2_12_1","volume-title":"Quality-Driven SystemC Design","author":"Gro\u00dfe Daniel","unstructured":"Daniel Gro\u00dfe and Rolf Drechsler . 2010. Quality-Driven SystemC Design . Springer . Daniel Gro\u00dfe and Rolf Drechsler. 2010. Quality-Driven SystemC Design .Springer."},{"key":"e_1_3_2_2_13_1","unstructured":"Shay Gueron. 2008. Advanced encryption standard (AES) instructions set. Intel http:\/\/softwarecommunity.intel.com\/articles\/eng\/3788.htm Vol. 25 (2008).  Shay Gueron. 2008. Advanced encryption standard (AES) instructions set. Intel http:\/\/softwarecommunity.intel.com\/articles\/eng\/3788.htm Vol. 25 (2008)."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Vladimir Herdt Daniel Gro\u00dfe Hoang M. Le and Rolf Drechsler. 2018. Extensible and Configurable RISC-V based Virtual Prototype. In FDL. 5--16.  Vladimir Herdt Daniel Gro\u00dfe Hoang M. Le and Rolf Drechsler. 2018. Extensible and Configurable RISC-V based Virtual Prototype. In FDL. 5--16.","DOI":"10.1109\/FDL.2018.8524047"},{"key":"e_1_3_2_2_15_1","volume-title":"RISC-V based Virtual Prototype: An Extensible and Configurable Platform for the System-level. JSA","author":"Herdt Vladimir","year":"2020","unstructured":"Vladimir Herdt , Daniel Gro\u00dfe , Pascal Pieper , and Rolf Drechsler . 2020. RISC-V based Virtual Prototype: An Extensible and Configurable Platform for the System-level. JSA ( 2020 ). Vladimir Herdt, Daniel Gro\u00dfe, Pascal Pieper, and Rolf Drechsler. 2020. RISC-V based Virtual Prototype: An Extensible and Configurable Platform for the System-level. JSA (2020)."},{"key":"e_1_3_2_2_16_1","unstructured":"Ashley Hill Antonin Raffin Maximilian Ernestus Adam Gleave Rene Traore Prafulla Dhariwal Christopher Hesse Oleg Klimov Alex Nichol Matthias Plappert Alec Radford John Schulman Szymon Sidor and Yuhuai Wu. 2018. Stable Baselines. https:\/\/github.com\/hill-a\/stable-baselines.  Ashley Hill Antonin Raffin Maximilian Ernestus Adam Gleave Rene Traore Prafulla Dhariwal Christopher Hesse Oleg Klimov Alex Nichol Matthias Plappert Alec Radford John Schulman Szymon Sidor and Yuhuai Wu. 2018. Stable Baselines. https:\/\/github.com\/hill-a\/stable-baselines."},{"key":"e_1_3_2_2_17_1","volume-title":"IEEE Standard SystemC Language Reference Manual","author":"Std IEEE","unstructured":"IEEE Std . 1666 2011. IEEE Standard SystemC Language Reference Manual . IEEE Std . 1666. IEEE Std. 1666 2011. IEEE Standard SystemC Language Reference Manual. IEEE Std. 1666."},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Kingshuk Karuri Rainer Leupers Gerd Ascheid and Heinrich Meyr. 2009. A Generic Design Flow for Application Specific Processor Customization through Instruction-Set Extensions (ISEs). In SAMOS. 204--214.  Kingshuk Karuri Rainer Leupers Gerd Ascheid and Heinrich Meyr. 2009. A Generic Design Flow for Application Specific Processor Customization through Instruction-Set Extensions (ISEs). In SAMOS. 204--214.","DOI":"10.1007\/978-3-642-03138-0_22"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"crossref","unstructured":"Guillaume Lample and Devendra Singh Chaplot. 2017. Playing FPS games with deep reinforcement learning. In AAAI. 2140--2146.  Guillaume Lample and Devendra Singh Chaplot. 2017. Playing FPS games with deep reinforcement learning. In AAAI. 2140--2146.","DOI":"10.1609\/aaai.v31i1.10827"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Prabhat Mishra and Nikil Dutt. 2008. Processor Description Languages .Morgan Kaufmann.  Prabhat Mishra and Nikil Dutt. 2008. Processor Description Languages .Morgan Kaufmann.","DOI":"10.1016\/B978-012369526-0\/50005-X"},{"key":"e_1_3_2_2_21_1","volume-title":"Cycle Approximate Simulation of RISC-V Processors. In Embedded World Conference.","author":"Moore Lee","year":"2018","unstructured":"Lee Moore , Duncan Graham , Simon Davidmann , and Felipe Rosa . 2018 . Cycle Approximate Simulation of RISC-V Processors. In Embedded World Conference. Lee Moore, Duncan Graham, Simon Davidmann, and Felipe Rosa. 2018. Cycle Approximate Simulation of RISC-V Processors. In Embedded World Conference."},{"key":"e_1_3_2_2_22_1","first-page":"278","article-title":"Policy invariance under reward transformations: Theory and application to reward shaping","volume":"99","author":"Ng Andrew Y","year":"1999","unstructured":"Andrew Y Ng , Daishi Harada , and Stuart Russell . 1999 . Policy invariance under reward transformations: Theory and application to reward shaping . In ICML , Vol. 99. 278 -- 287 . Andrew Y Ng, Daishi Harada, and Stuart Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, Vol. 99. 278--287.","journal-title":"ICML"},{"key":"e_1_3_2_2_23_1","volume-title":"Proximal policy optimization algorithms. arXiv:1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . 2017. Proximal policy optimization algorithms. arXiv:1707.06347 ( 2017 ). John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347 (2017)."},{"key":"e_1_3_2_2_24_1","volume-title":"International Embedded Systems Symposium. 128--138","author":"Marcela","year":"2013","unstructured":"Marcela vS imkov\u00e1, Zdenve k Pvr ikryl, Zdenve k Kot\u00e1sek , and Tom\u00e1vs Hruvs ka. 2013 . Automated functional verification of application specific instruction-set processors . In International Embedded Systems Symposium. 128--138 . Marcela vS imkov\u00e1, Zdenve k Pvr ikryl, Zdenve k Kot\u00e1sek, and Tom\u00e1vs Hruvs ka. 2013. Automated functional verification of application specific instruction-set processors. In International Embedded Systems Symposium. 128--138."},{"key":"e_1_3_2_2_25_1","volume-title":"Barto","author":"Sutton Richard S.","year":"1998","unstructured":"Richard S. Sutton and Andrew G . Barto . 1998 . Introduction to Reinforcement Learning 1st ed.). MIT Press , Cambridge, MA, USA. Richard S. Sutton and Andrew G. Barto. 1998. Introduction to Reinforcement Learning 1st ed.). MIT Press, Cambridge, MA, USA."},{"key":"e_1_3_2_2_26_1","volume-title":"Reinforcement Learning and Markov Decision Processes","author":"van Otterlo Martijn","unstructured":"Martijn van Otterlo and Marco Wiering . 2012. Reinforcement Learning and Markov Decision Processes . Springer Berlin Heidelberg , Berlin, Heidelberg , 3--42. https:\/\/doi.org\/10.1007\/978--3--642--27645--3_1 10.1007\/978--3--642--27645--3_1 Martijn van Otterlo and Marco Wiering. 2012. Reinforcement Learning and Markov Decision Processes .Springer Berlin Heidelberg, Berlin, Heidelberg, 3--42. https:\/\/doi.org\/10.1007\/978--3--642--27645--3_1"},{"key":"e_1_3_2_2_27_1","volume-title":"The RISC-V Instruction Set Manual","author":"Waterman Andrew","unstructured":"Andrew Waterman and Krste Asanovi\u0107 . 2017a. The RISC-V Instruction Set Manual ; Volume I: User- Level ISA. SiFive Inc. and CS Division, EECS Department, University of California , Berkeley. Andrew Waterman and Krste Asanovi\u0107. 2017a. The RISC-V Instruction Set Manual; Volume I: User-Level ISA. SiFive Inc. and CS Division, EECS Department, University of California, Berkeley."},{"key":"e_1_3_2_2_28_1","volume-title":"The RISC-V Instruction Set Manual","author":"Waterman Andrew","unstructured":"Andrew Waterman and Krste Asanovi\u0107 . 2017b. The RISC-V Instruction Set Manual ; Volume II : Privileged Architecture. SiFive Inc. and CS Division, EECS Department, University of California , Berkeley. Andrew Waterman and Krste Asanovi\u0107. 2017b. The RISC-V Instruction Set Manual; Volume II: Privileged Architecture. SiFive Inc. and CS Division, EECS Department, University of California, Berkeley."},{"key":"e_1_3_2_2_29_1","volume-title":"Sponge-Based Control-Flow Protection for IoT Devices. In 2018 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 214--226","author":"Werner Mario","year":"2018","unstructured":"Mario Werner , Thomas Unterluggauer , David Schaffenrath , and Stefan Mangard . 2018 . Sponge-Based Control-Flow Protection for IoT Devices. In 2018 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 214--226 . Mario Werner, Thomas Unterluggauer, David Schaffenrath, and Stefan Mangard. 2018. Sponge-Based Control-Flow Protection for IoT Devices. In 2018 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 214--226."},{"key":"e_1_3_2_2_30_1","volume-title":"Constraint-based verification","author":"Yuan Jun","unstructured":"Jun Yuan , Carl Pixley , and Adnan Aziz . 2006. Constraint-based verification . Springer Science & Business Media . Jun Yuan, Carl Pixley, and Adnan Aziz. 2006. Constraint-based verification .Springer Science & Business Media."}],"event":{"name":"GLSVLSI '20: Great Lakes Symposium on VLSI 2020","location":"Virtual Event China","acronym":"GLSVLSI '20"},"container-title":["Proceedings of the 2020 on Great Lakes Symposium on VLSI"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386263.3406901","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3386263.3406901","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:13Z","timestamp":1750201993000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386263.3406901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,7]]},"references-count":29,"alternative-id":["10.1145\/3386263.3406901","10.1145\/3386263"],"URL":"https:\/\/doi.org\/10.1145\/3386263.3406901","relation":{},"subject":[],"published":{"date-parts":[[2020,9,7]]},"assertion":[{"value":"2020-09-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}