{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T16:01:42Z","timestamp":1781020902599,"version":"3.54.1"},"reference-count":29,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2017,2,1]],"date-time":"2017-02-01T00:00:00Z","timestamp":1485907200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Spanish Ministry of Economy and Competitivity","award":["TIN2013-40968-P"],"award-info":[{"award-number":["TIN2013-40968-P"]}]},{"name":"Spanish Ministry of Economy and Competitivity","award":["TIN2014-54806-R"],"award-info":[{"award-number":["TIN2014-54806-R"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["SIMULATION"],"published-print":{"date-parts":[[2017,6]]},"abstract":"<jats:p>The discrete event system specification formalism, which supports hierarchical and modular model composition, has been widely used to understand, analyze and develop a variety of systems. Discrete event system specification has been implemented in various languages and platforms over the years. The DEVStone benchmark was conceived to generate a set of models with varied structure and behavior, and to automate the evaluation of the performance of discrete event system specification-based simulators. However, DEVStone is still in a preliminary phase and more model analysis is required. In this paper, we revisit DEVStone introducing new equations to compute the number of events triggered. We also introduce a new benchmark with a similar central processing unit and memory requirements to the most complex benchmark in DEVStone, but with an easier implementation and with it being more manageable analytically. Finally, we compare both the performance and memory footprint of five different discrete event system specification simulators in two different hardware platforms.<\/jats:p>","DOI":"10.1177\/0037549717690447","type":"journal-article","created":{"date-parts":[[2017,2,23]],"date-time":"2017-02-23T04:32:12Z","timestamp":1487824332000},"page":"459-476","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":27,"title":["Reconsidering the performance of DEVS modeling and simulation environments using the DEVStone benchmark"],"prefix":"10.1177","volume":"93","author":[{"given":"Jos\u00e9 L.","family":"Risco-Mart\u00edn","sequence":"first","affiliation":[{"name":"Department of Computer Architecture and Automation, Complutense University of Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saurabh","family":"Mittal","sequence":"additional","affiliation":[{"name":"Complex Systems Research Lab, Dunip Technologies, LLC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan Carlos","family":"Fabero Jim\u00e9nez","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture and Automation, Complutense University of Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marina","family":"Zapater","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture and Automation, Complutense University of Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rom\u00e1n","family":"Hermida Correa","sequence":"additional","affiliation":[{"name":"Department of Computer Architecture and Automation, Complutense University of Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2017,2]]},"reference":[{"key":"e_1_3_4_2_2","volume-title":"Theory of modelling and simulation","author":"Zeigler BP","year":"1976","unstructured":"Zeigler BP. Theory of modelling and simulation. New York: John Wiley, 1976."},{"key":"e_1_3_4_3_2","volume-title":"Theory of modeling and simulation. Integrating discrete event and continuous complex dynamic systems","author":"Zeigler BP","year":"2000","unstructured":"Zeigler BP, Praehofer H, Kim TG. Theory of modeling and simulation. Integrating discrete event and continuous complex dynamic systems. 2nd ed. San Diego, California, USA: Academic Press, 2000.","edition":"2"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.809"},{"key":"e_1_3_4_5_2","first-page":"1093","article-title":"Simulation of wind farm operations and maintenance using discrete event system specification","volume":"87","author":"Byon E","year":"2011","unstructured":"Byon E, P\u00e9rez E, Ding Y, . Simulation of wind farm operations and maintenance using discrete event system specification. Simul T Soc Mod Sim 2011; 87: 1093\u20131117.","journal-title":"Simul T Soc Mod Sim"},{"key":"e_1_3_4_6_2","first-page":"179","article-title":"Experiences in modeling and simulation of computer architectures using DEVS","volume":"18","author":"Wainer GA","year":"2001","unstructured":"Wainer GA, Daicz S, Troccoli A. Experiences in modeling and simulation of computer architectures using DEVS. Simul T Soc Mod Sim 2001; 18: 179\u2013202.","journal-title":"Simul T Soc Mod Sim"},{"key":"e_1_3_4_7_2","first-page":"103","volume-title":"Formal languages for computer simulation: Transdisciplinary models and applications","author":"Moreno A","year":"2013","unstructured":"Moreno A, Risco-Mart\u00edn JL, Besada-Portas E, . Thermal analysis of the MIPS processor formulated within DEVS conventions. In: Formal languages for computer simulation: Transdisciplinary models and applications. IGI Global, Hershey, Pennsylvania, USA, 2013, pp.103\u2013144."},{"key":"e_1_3_4_8_2","first-page":"95","volume-title":"TMS-DEVS \u201811 proceedings of the 2011 symposium on theory of modeling & simulation: DEVS integrative M&S symposium","author":"Moallemi M","unstructured":"Moallemi M, Wainer G. I-DEVS: Imprecise real-time and embedded DEVS modeling. In: TMS-DEVS \u201811 proceedings of the 2011 symposium on theory of modeling & simulation: DEVS integrative M&S symposium, Boston, MA, USA, 03\u201307 April, pp.95\u2013102. San Diego, CA, USA: Society for Computer Simulation International."},{"key":"e_1_3_4_9_2","volume-title":"13th IEEE\/ACM international symposium on distributed simulation and real time applications","author":"Moreno A","unstructured":"Moreno A, Risco-Mart\u00edn JL, Besada E, . DEVS\/SOA: Towards DEVS interoperability in distributed M&S. In 13th IEEE\/ACM international symposium on distributed simulation and real time applications, 25\u2013October 2009. Washington, D.C: IEEE Computer Society."},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549709358294"},{"key":"e_1_3_4_11_2","unstructured":"DEVSJAVA http:\/\/acims.asu.edu\/software\/devsjava (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_12_2","unstructured":"DEVS-Suite http:\/\/devs-suitesim.sourceforge.net (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_13_2","unstructured":"CoSMoS. http:\/\/acims.asu.edu\/software\/cosmos (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_14_2","unstructured":"CD++ http:\/\/cell-devs.sce.carleton.ca (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_15_2","first-page":"1","volume-title":"Symposium on theory of modeling and simulation - DEVS integrative M&S symposium","author":"Tendeloo YV","unstructured":"Tendeloo YV, Vangheluwe H. The modular architecture of the python(P)DEVS simulation kernel. In Symposium on theory of modeling and simulation - DEVS integrative M&S symposium, pp.1\u20136, Tampa, Florida, April 13\u201316, 2014. Society for Computer Simulation International San Diego, CA, USA."},{"key":"e_1_3_4_16_2","unstructured":"aDEVS http:\/\/web.ornl.gov\/~1qn\/adevs\/ (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_17_2","unstructured":"JAMES II http:\/\/wwwmosi.informatik.uni-rostock.de (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.1177\/1548512910389203"},{"key":"e_1_3_4_19_2","unstructured":"DUNIP Technologies http:\/\/www.duniptechnologies.com (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_20_2","first-page":"265","volume-title":"Ninth IEEE international symposium on distributed simulation and real-time applications","author":"Glinsky E","unstructured":"Glinsky E, Wainer G. DEVStone: A benchmarking technique for studying performance of DEVS modeling simulation environments. In: Ninth IEEE international symposium on distributed simulation and real-time applications, Montreal, Canada, 10\u201311 October 2005, pp.265\u2013272. Washington, DC, USA: IEEE Computer Society."},{"key":"e_1_3_4_21_2","first-page":"447","volume-title":"Proceedings of the 2008 spring simulation multiconference","author":"Gutierrez-Alcaraz M","unstructured":"Gutierrez-Alcaraz M, Wainer GA. Experiences with the DEVStone benchmark. In: Proceedings of the 2008 spring simulation multiconference, Ottawa, Canada, 14\u201317 April 2008, pp.447\u2013455. San Diego, CA, USA: Society for Computer Simulation International."},{"issue":"7","key":"e_1_3_4_22_2","first-page":"555","article-title":"Studying performance of DEVS modeling and simulation environments using the DEVStone benchmark","volume":"87","author":"Wainer G","year":"2011","unstructured":"Wainer G, Glinsky E, Gutierrez-Alcaraz M. Studying performance of DEVS modeling and simulation environments using the DEVStone benchmark. Simul T Soc Mod Sim 2011; 87(7): 555\u2013580.","journal-title":"Simul T Soc Mod Sim"},{"key":"e_1_3_4_23_2","first-page":"52","volume-title":"International conference on modeling, simulation and neural networks (MSNN 2000)","author":"Davila J","unstructured":"Davila J, Uzcategui MY. GALATEA: A multi-agent, simulation platform. In: International conference on modeling, simulation and neural networks (MSNN 2000), M\u00e9rida, Venezuela, 1\u20135 April 2000, pp.52\u201367 Berlin: AMSE."},{"key":"e_1_3_4_24_2","volume-title":"International ICST conference on simulation tools and techniques for communications, networks and systems","author":"Traor\u00e9 MK","unstructured":"Traor\u00e9 MK. SimStudio: A next generation modeling simulation framework. In: International ICST conference on simulation tools and techniques for communications, networks and systems, Marseille, France, 03\u201307 March 2008, Brussels:ICST."},{"key":"e_1_3_4_25_2","unstructured":"PowerDEVS http:\/\/powerdevs.sourceforge.net (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_26_2","unstructured":"MS4Me http:\/\/www.ms4systems.com\/pages\/main.php (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_27_2","unstructured":"VLE: The virtual laboratory environment http:\/\/www.vle-project.org\/wiki (2015 accessed 1 July 2016)."},{"key":"e_1_3_4_28_2","first-page":"1","volume-title":"Proceedings of the symposium on theory of modeling & simulation - DEVS integrative","author":"Van Tendeloo Y","unstructured":"Van Tendeloo Y, Vangheluwe H. The modular architecture of the Python(P)DEVS simulation kernel: Work in progress paper. In: Proceedings of the symposium on theory of modeling & simulation - DEVS integrative, Tampa, Florida, 13\u201316 April 2014, pp.1\u20136. San Diego, CA, USA: Society for Computer Simulation International."},{"key":"e_1_3_4_29_2","volume-title":"Symposium on theory of modeling and simulation (TMS\u201915)","author":"Vicino D","unstructured":"Vicino D, Niyonkuru D, Wainer G, . Sequential PDEVS architecture. In: Symposium on theory of modeling and simulation (TMS\u201915), Alexandria, Virginia, 12\u201315 April 2015, San Diego, CA, USA: Society for Computer Simulation International."},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/358274.358283"}],"container-title":["SIMULATION"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0037549717690447","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0037549717690447","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0037549717690447","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T11:27:56Z","timestamp":1777634876000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0037549717690447"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2]]},"references-count":29,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2017,6]]}},"alternative-id":["10.1177\/0037549717690447"],"URL":"https:\/\/doi.org\/10.1177\/0037549717690447","relation":{},"ISSN":["0037-5497","1741-3133"],"issn-type":[{"value":"0037-5497","type":"print"},{"value":"1741-3133","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2]]}}}