{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:16:15Z","timestamp":1740122175683,"version":"3.37.3"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2020,6,19]],"date-time":"2020-06-19T00:00:00Z","timestamp":1592524800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,6,19]],"date-time":"2020-06-19T00:00:00Z","timestamp":1592524800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Agent Multi-Agent Syst"],"published-print":{"date-parts":[[2020,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The importance of Agent-Based Simulation (ABS) as scientific method to generate data for scientific models in general and for informed policy decisions in particular has been widely recognised. However, the important technique of code testing of implementations like unit testing has not generated much research interested so far. As a possible solution, in previous work we have explored the conceptual use of <jats:italic>property-based testing<\/jats:italic>. In this code testing method, model specifications and invariants are expressed directly in code and tested through <jats:italic>automated<\/jats:italic> and <jats:italic>randomised<\/jats:italic> test data generation. This paper expands on our previous work and explores how to use property-based testing on a technical level to encode and test specifications of ABS. As use case the simple agent-based SIR model is used, where it is shown how to test agent behaviour, transition probabilities and model invariants. The outcome are specifications expressed directly in code, which relate whole classes of random input to expected classes of output. During test execution, random test data is generated automatically, potentially covering the equivalent of thousands of unit tests, run within seconds on modern hardware. This makes property-based testing in the context of ABS strictly more powerful than unit testing, as it is a much more natural fit due to its stochastic nature.<\/jats:p>","DOI":"10.1007\/s10458-020-09473-8","type":"journal-article","created":{"date-parts":[[2020,6,19]],"date-time":"2020-06-19T14:02:52Z","timestamp":1592575372000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Specification testing of agent-based simulation using property-based testing"],"prefix":"10.1007","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8736-0479","authenticated-orcid":false,"given":"Jonathan","family":"Thaler","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peer-Olaf","family":"Siebers","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,6,19]]},"reference":[{"key":"9473_CR1","unstructured":"Asta, S., \u00d6zcan, E., & Siebers, P. O. (2014). An investigation on test driven discrete event simulation. In Operational research society simulation workshop 2014 (SW14). http:\/\/eprints.nottingham.ac.uk\/28211\/."},{"key":"9473_CR2","unstructured":"Axelrod, R. (1995). The convergence and stability of cultures: Local convergence and global polarization. Working paper, Santa Fe Institute. http:\/\/econpapers.repec.org\/paper\/wopsafiwp\/95-03-028.htm."},{"key":"9473_CR3","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/978-3-662-03366-1_2","volume-title":"Simulating social phenomena","author":"R Axelrod","year":"1997","unstructured":"Axelrod, R. (1997). Advancing the art of simulation in the social sciences. In R. Conte, R. Hegselmann, & P. Terna (Eds.), Simulating social phenomena (pp. 21\u201340). Berlin: Springer. https:\/\/doi.org\/10.1007\/978-3-662-03366-1_2."},{"key":"9473_CR4","doi-asserted-by":"publisher","first-page":"1565","DOI":"10.1016\/S1574-0021(05)02033-2","volume-title":"Handbook of computational economics","author":"R Axelrod","year":"2006","unstructured":"Axelrod, R. (2006). Chapter 33 agent-based modeling as a bridge between disciplines. In L. T. A. K. L. Judd (Ed.), Handbook of computational economics (Vol. 2, pp. 1565\u20131584). Amsterdam: Elsevier. https:\/\/doi.org\/10.1016\/S1574-0021(05)02033-2."},{"key":"9473_CR5","doi-asserted-by":"crossref","unstructured":"Axelrod, R., & Tesfatsion, L. (2006). A guide for newcomers to agent-based modeling in the social sciences. Staff general research papers archive, Iowa State University, Department of Economics. http:\/\/econpapers.repec.org\/paper\/isugenres\/12515.htm.","DOI":"10.1016\/S1574-0021(05)02044-7"},{"key":"9473_CR6","volume-title":"Test Driven Development: By Example","author":"K Beck","year":"2002","unstructured":"Beck, K. (2002). Test Driven Development: By Example (01st ed.). Boston: Addison-Wesley Professional.","edition":"01"},{"key":"9473_CR7","unstructured":"Botta, N., Mandel, A., Hofmann, M., Schupp, S., & Ionescu, C. (2013). Mathematical specification of an agent-based model of exchange. In Proceedings of the AISB convention."},{"issue":"8","key":"9473_CR8","doi-asserted-by":"publisher","first-page":"4025","DOI":"10.1016\/j.amc.2011.08.051","volume":"218","author":"N Botta","year":"2011","unstructured":"Botta, N., Mandel, A., Ionescu, C., Hofmann, M., Lincke, D., Schupp, S., et al. (2011). A functional framework for agent-based models of exchange. Applied Mathematics and Computation, 218(8), 4025\u20134040. https:\/\/doi.org\/10.1016\/j.amc.2011.08.051.","journal-title":"Applied Mathematics and Computation"},{"issue":"05","key":"9473_CR9","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1017\/S095679681300018X","volume":"23","author":"E Brady","year":"2013","unstructured":"Brady, E. (2013). Idris, a general-purpose dependently typed programming language: Design and implementation. Journal of Functional Programming, 23(05), 552\u2013593. https:\/\/doi.org\/10.1017\/S095679681300018X.","journal-title":"Journal of Functional Programming"},{"key":"9473_CR10","volume-title":"Type-driven development with Idris","author":"E Brady","year":"2017","unstructured":"Brady, E. (2017). Type-driven development with Idris. New York: Manning Publications Company. (Google-Books-ID: eWzEjwEACAAJ) ."},{"key":"9473_CR11","volume-title":"Practical software testing: A process-oriented approach","author":"I Burnstein","year":"2010","unstructured":"Burnstein, I. (2010). Practical software testing: A process-oriented approach (1st ed.). Berlin: Springer Publishing Company, Incorporated.","edition":"1"},{"key":"9473_CR12","doi-asserted-by":"publisher","unstructured":"Claessen, K., & Hughes, J. (2000). QuickCheck\u2014A lightweight tool for random testing of Haskell programs. In Proceedings of the fifth ACM SIGPLAN international conference on functional programming, ICFP \u201900 (pp. 268\u2013279). New York, NY, USA: ACM. https:\/\/doi.org\/10.1145\/351240.351266.","DOI":"10.1145\/351240.351266"},{"issue":"12","key":"9473_CR13","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1145\/636517.636527","volume":"37","author":"K Claessen","year":"2002","unstructured":"Claessen, K., & Hughes, J. (2002). Testing monadic code with QuickCheck. SIGPLAN Notices, 37(12), 47\u201359. https:\/\/doi.org\/10.1145\/636517.636527.","journal-title":"SIGPLAN Notices"},{"key":"9473_CR14","volume-title":"Microeconomic theory","author":"AM Colell","year":"1995","unstructured":"Colell, A. M. (1995). Microeconomic theory. Oxford: Oxford University Press. (Google-Books-ID: dFS2AQAACAAJ) ."},{"key":"9473_CR15","doi-asserted-by":"publisher","unstructured":"Collier, N., & Ozik, J. (2013). Test-driven agent-based simulation development. In 2013 winter simulations conference (WSC) (pp. 1551\u20131559). https:\/\/doi.org\/10.1109\/WSC.2013.6721538.","DOI":"10.1109\/WSC.2013.6721538"},{"key":"9473_CR16","unstructured":"ComSES: Computational Model Library. (2019). Retrieved June 18, 2020 from https:\/\/www.comses.net\/codebases\/."},{"key":"9473_CR17","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3374.001.0001","volume-title":"Growing artificial societies: Social science from the bottom up","author":"JM Epstein","year":"1996","unstructured":"Epstein, J. M., & Axtell, R. (1996). Growing artificial societies: Social science from the bottom up. Washington, DC: The Brookings Institution."},{"key":"9473_CR18","unstructured":"Evensen, P., & M\u00e4rdin, M. (2010). An extensible and scalable agent-based simulation of Barter economics. Master\u2019s thesis. Chalmers University of Technology, G\u00f6teborg. https:\/\/gupea.ub.gu.se\/handle\/2077\/22063."},{"issue":"1","key":"9473_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2202\/1534-5971.1302","volume":"6","author":"H Gintis","year":"2006","unstructured":"Gintis, H. (2006). The emergence of a price system from decentralized bilateral exchange. Contributions in Theoretical Economics, 6(1), 1\u201315. https:\/\/doi.org\/10.2202\/1534-5971.1302.","journal-title":"Contributions in Theoretical Economics"},{"key":"9473_CR20","unstructured":"Gintis, H. (2019). Herbert Gintis Website. Retrieved January 14, 2020 from https:\/\/people.umass.edu\/gintis\/. https:\/\/people.umass.edu\/gintis\/."},{"issue":"3","key":"9473_CR21","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1057\/jos.2012.26","volume":"7","author":"O Gurcan","year":"2013","unstructured":"Gurcan, O., Dikenelli, O., & Bernon, C. (2013). A generic testing framework for agent-based simulation models. Journal of Simulation, 7(3), 183\u2013201. https:\/\/doi.org\/10.1057\/jos.2012.26.","journal-title":"Journal of Simulation"},{"key":"9473_CR22","doi-asserted-by":"publisher","unstructured":"Hughes, J. (2007). QuickCheck testing for fun and profit. In Proceedings of the 9th international conference on practical aspects of declarative languages, PADL\u201907 (pp. 1\u201332). Berlin, Heidelberg: Springer-Verlag. https:\/\/doi.org\/10.1007\/978-3-540-69611-7_1. http:\/\/dx.doi.org\/10.1007\/978-3-540-69611-7_1.","DOI":"10.1007\/978-3-540-69611-7_1"},{"issue":"772","key":"9473_CR23","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1098\/rspa.1927.0118","volume":"115","author":"WO Kermack","year":"1927","unstructured":"Kermack, W. O., & McKendrick, A. G. (1927). A contribution to the mathematical theory of epidemics. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 115(772), 700\u2013721. https:\/\/doi.org\/10.1098\/rspa.1927.0118.","journal-title":"Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences"},{"key":"9473_CR24","unstructured":"Macal, C. M. (2010). To agent-based simulation from system dynamics. In Proceedings of the winter simulation conference, WSC \u201910 (pp. 371\u2013382). Baltimore, Maryland: Winter Simulation Conference. http:\/\/dl.acm.org\/citation.cfm?id=2433508.2433551."},{"issue":"2","key":"9473_CR25","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1057\/jos.2016.7","volume":"10","author":"CM Macal","year":"2016","unstructured":"Macal, C. M. (2016). Everything you need to know about agent-based modelling and simulation. Journal of Simulation, 10(2), 144\u2013156. https:\/\/doi.org\/10.1057\/jos.2016.7.","journal-title":"Journal of Simulation"},{"key":"9473_CR26","unstructured":"McMillan, K. L. (1992). Symbolic model checking: An approach to the state explosion problem. Ph.D. thesis, USA . UMI Order No. GAX92-24209."},{"key":"9473_CR27","doi-asserted-by":"publisher","unstructured":"Meyer, R. (2014). Event-driven multi-agent simulation. In Multi-agent-based simulation XV, lecture notes in computer science (pp. 3\u201316). Cham: Springer. https:\/\/doi.org\/10.1007\/978-3-319-14627-0_1. https:\/\/link.springer.com\/chapter\/10.1007\/978-3-319-14627-0_1.","DOI":"10.1007\/978-3-319-14627-0_1"},{"key":"9473_CR28","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-319-94580-4_1","volume-title":"Advances in practical applications of agents, multi-agent systems, and complexity: The PAAMS collection","author":"MJ North","year":"2018","unstructured":"North, M. J. (2018). Hammer or tongs: How best to build agent-based models? In Y. Demazeau, B. An, J. Bajo, & A. Fern\u00e1ndez-Caballero (Eds.), Advances in practical applications of agents, multi-agent systems, and complexity: The PAAMS collection (pp. 3\u201311). Cham: Springer."},{"key":"9473_CR29","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1016\/j.ejor.2016.03.050","volume":"254","author":"BSS Onggo","year":"2016","unstructured":"Onggo, B. S. S., & Karatas, M. (2016). Test-driven simulation modelling: A case study using agent-based maritime search-operation simulation. European Journal of Operational Research, 254, 517\u2013531. https:\/\/doi.org\/10.1016\/j.ejor.2016.03.050.","journal-title":"European Journal of Operational Research"},{"key":"9473_CR30","doi-asserted-by":"publisher","unstructured":"Runciman, C., Naylor, M., Lindblad, F. (2008). Smallcheck and lazy Smallcheck: Automatic exhaustive testing for small values. In Proceedings of the first ACM SIGPLAN symposium on Haskell, Haskell \u201908 (pp. 37\u201348). New York, NY, USA: ACM. https:\/\/doi.org\/10.1145\/1411286.1411292.","DOI":"10.1145\/1411286.1411292"},{"key":"9473_CR31","doi-asserted-by":"crossref","unstructured":"Siebers, P.O., & Aickelin, U. (2008). Introduction to multi-agent simulation. arXiv:0803.3905 [cs].","DOI":"10.2139\/ssrn.2827970"},{"key":"9473_CR32","unstructured":"Thaler, J. (2019). Repository of agent-based SIR implementation in Haskell. Retrieved June 18, 2020 fromhttps:\/\/github.com\/thalerjonathan\/haskell-sir."},{"key":"9473_CR33","doi-asserted-by":"publisher","unstructured":"Thaler, J., Altenkirch, T., & Siebers, P. O. (2018). Pure functional epidemics: An agent-based approach. In Proceedings of the 30th symposium on implementation and application of functional languages, IFL 2018 (pp. 1\u201312). New York, NY, USA: ACM. https:\/\/doi.org\/10.1145\/3310232.3310372. http:\/\/doi.acm.org\/10.1145\/3310232.3310372. Event-place: Lowell, MA, USA.","DOI":"10.1145\/3310232.3310372"},{"key":"9473_CR34","unstructured":"Thaler, J., & Siebers, P. O. (2019). Show me your properties: The potential of property-based testing in agent-based simulation. In Proceedings of the 2019 summer simulation conference, SummerSim \u201919 (pp. 1:1\u20131:12). San Diego, CA, USA: Society for Computer Simulation International. http:\/\/dl.acm.org\/citation.cfm?id=3374138.3374139."},{"key":"9473_CR35","series-title":"Springer series in statistics","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1007\/978-1-4612-0919-5_18","volume-title":"Breakthroughs in statistics: Foundations and basic theory","author":"A Wald","year":"1992","unstructured":"Wald, A. (1992). Sequential tests of statistical hypotheses. In S. Kotz & N. L. Johnson (Eds.), Breakthroughs in statistics: Foundations and basic theory (pp. 256\u2013298)., Springer series in statistics New York, NY: Springer. https:\/\/doi.org\/10.1007\/978-1-4612-0919-5_18."},{"key":"9473_CR36","volume-title":"An introduction to multiagent systems","author":"M Wooldridge","year":"2009","unstructured":"Wooldridge, M. (2009). An introduction to multiagent systems (2nd ed.). New York: Wiley.","edition":"2"}],"container-title":["Autonomous Agents and Multi-Agent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-020-09473-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10458-020-09473-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-020-09473-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T23:04:59Z","timestamp":1624057499000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10458-020-09473-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,19]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,10]]}},"alternative-id":["9473"],"URL":"https:\/\/doi.org\/10.1007\/s10458-020-09473-8","relation":{},"ISSN":["1387-2532","1573-7454"],"issn-type":[{"type":"print","value":"1387-2532"},{"type":"electronic","value":"1573-7454"}],"subject":[],"published":{"date-parts":[[2020,6,19]]},"assertion":[{"value":"19 June 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"47"}}