{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T06:48:52Z","timestamp":1762325332865,"version":"3.41.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T00:00:00Z","timestamp":1658707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Spanish Ministry of Science and Innovation","doi-asserted-by":"crossref","award":["PID2019-110866RB-I00"],"award-info":[{"award-number":["PID2019-110866RB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Comput. Simul."],"published-print":{"date-parts":[[2022,7,31]]},"abstract":"<jats:p>The DEVStone benchmark allows us to evaluate the performance of discrete-event simulators based on the Discrete Event System (DEVS) formalism. It provides model sets with different characteristics, enabling the analysis of specific issues of simulation engines. However, this heterogeneity hinders the comparison of the results among studies, as the results obtained on each research work depend on the chosen subset of DEVStone models. We define the DEVStone metric based on the DEVStone synthetic benchmark and provide a mechanism for specifying objective ratings for DEVS-based simulators. This metric corresponds to the average number of times that a simulator can execute a selection of 12 DEVStone models in 1 minute. The variety of the chosen models ensures that we measure different particularities provided by DEVStone. The proposed metric allows us to compare various simulators and to assess the impact of new features on their performance. We use the DEVStone metric to compare some popular DEVS-based simulators.<\/jats:p>","DOI":"10.1145\/3543849","type":"journal-article","created":{"date-parts":[[2022,6,11]],"date-time":"2022-06-11T22:41:53Z","timestamp":1654987313000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["The DEVStone Metric: Performance Analysis of DEVS Simulation Engines"],"prefix":"10.1145","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0762-4425","authenticated-orcid":false,"given":"Rom\u00e1n","family":"C\u00e1rdenas","sequence":"first","affiliation":[{"name":"Universidad Polit\u00e9cnica de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2637-5316","authenticated-orcid":false,"given":"Kevin","family":"Henares","sequence":"additional","affiliation":[{"name":"Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0587-997X","authenticated-orcid":false,"given":"Patricia","family":"Arroba","sequence":"additional","affiliation":[{"name":"Universidad Polit\u00e9cnica de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3127-6507","authenticated-orcid":false,"given":"Jos\u00e9 L.","family":"Risco-Mart\u00edn","sequence":"additional","affiliation":[{"name":"Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3366-9184","authenticated-orcid":false,"given":"Gabriel A.","family":"Wainer","sequence":"additional","affiliation":[{"name":"Carleton University, Ottawa, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,7,25]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Theory of Modeling and Simulation: Discrete Event and Iterative System Computational Foundations","author":"Zeigler Bernard P.","year":"2018","unstructured":"Bernard P. Zeigler, Alexandre Muzy, and Ernesto Kofman. 2018. Theory of Modeling and Simulation: Discrete Event and Iterative System Computational Foundations (3rd ed.). Academic Press, San Diego, CA.","edition":"3"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/DS-RT.2013.8"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.2009.5429701"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/DS-RT.2009.18"},{"key":"e_1_3_2_6_2","volume-title":"Introduction to Discrete Event Systems","author":"Cassandras Christos G.","year":"2009","unstructured":"Christos G. Cassandras and Stephane Lafortune. 2009. Introduction to Discrete Event Systems (2nd ed.). Springer Science & Business Media, Luxembourg.","edition":"2"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CACSD.2000.900199"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549709358294"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.5555\/3140065.3140089"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549710376841"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.04.144"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/SIMSYM.2000.844902"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/19.1.43"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/DISTRA.2005.18"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/WSC.1994.717419"},{"key":"e_1_3_2_16_2","unstructured":"James J. Nutaro. 1999. adevs: A Discrete EVent System Simulator. Retrieved May 20 2021 from https:\/\/web.ornl.gov\/\u223cnutarojj\/adevs\/."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/WSC40007.2019.9004917"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/2665008.2665022"},{"key":"e_1_3_2_19_2","unstructured":"Jos\u00e9 L. Risco-Mart\u00edn. 2014. xDEVS: M&S Framework. Retrieved May 20 2021 from https:\/\/github.com\/iscar-ucm\/xdevs."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/99.615432"},{"key":"e_1_3_2_21_2","first-page":"3","volume-title":"Proceedings of the High Performance Computing Symposium (HPC\u201904)","author":"Hu Xiaolin","year":"2004","unstructured":"Xiaolin Hu and Bernard P. Zeigler. 2004. A high performance simulation engine for large-scale cellular DEVS models. 2004. In Proceedings of the High Performance Computing Symposium (HPC\u201904). 3\u20138."},{"key":"e_1_3_2_22_2","first-page":"273","volume-title":"Proceedings of the 1st Open International Conference on Modeling and Simulation (OICMS\u201905)","author":"Muzy Alexander","year":"2005","unstructured":"Alexander Muzy and James J. Nutaro. 2005. Algorithms for efficient implementations of the DEVS & DSDEVS abstract simulators. In Proceedings of the 1st Open International Conference on Modeling and Simulation (OICMS\u201905). 273\u2013279."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.5555\/2872965.2872988"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44733-8_42"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/3466184.3466433"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.4230\/OASIcs.ICCSW.2014.40"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549716678330"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549717690447"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/358274.358283"},{"key":"e_1_3_2_30_2","volume-title":"Probability and Statistical Inference","author":"Hogg Robert V.","year":"2018","unstructured":"Robert V. Hogg and Elliot A. Tanis. 2018. Probability and Statistical Inference (10th ed.). Pearson\/Prentice Hall, Upper Saddle River, NJ.","edition":"10"},{"key":"e_1_3_2_31_2","unstructured":"Kevin Henares and Rom\u00e1n C\u00e1rdenas. 2020. DEVStone comparative. Retrieved May 20 2021 from https:\/\/github.com\/SimulationEverywhere\/devstone-comparative."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.2172\/1343541"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/3408207.3408242"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/3427510.3427527"}],"container-title":["ACM Transactions on Modeling and Computer Simulation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3543849","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3543849","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:39Z","timestamp":1750268979000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3543849"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,25]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,7,31]]}},"alternative-id":["10.1145\/3543849"],"URL":"https:\/\/doi.org\/10.1145\/3543849","relation":{},"ISSN":["1049-3301","1558-1195"],"issn-type":[{"type":"print","value":"1049-3301"},{"type":"electronic","value":"1558-1195"}],"subject":[],"published":{"date-parts":[[2022,7,25]]},"assertion":[{"value":"2021-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}