{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:50:44Z","timestamp":1740124244077,"version":"3.37.3"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T00:00:00Z","timestamp":1713744000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T00:00:00Z","timestamp":1713744000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"ABB"},{"DOI":"10.13039\/501100003170","name":"Stiftelsen f\u00f6r Kunskaps- och Kompetensutveckling","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003170","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011914","name":"M\u00e4lardalen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011914","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Software Qual J"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Understanding the behaviour of a system\u2019s API can be hard. Giving users access to <jats:italic>relevant<\/jats:italic> examples of how an API behaves has been shown to make this easier for them. In addition, such examples can be used to verify expected behaviour or identify unwanted behaviours. Methods for automatically generating examples have existed for a long time. However, state-of-the-art methods rely on either white-box information, such as source code, or on formal specifications of the system behaviour. But what if you do not have access to either? This may be the case, for example, when interacting with a third-party API. In this paper, we present an approach to automatically generate relevant examples of behaviours of an API, without requiring either source code or a formal specification of behaviour. Evaluation on an industry-grade REST API shows that our method can produce small and relevant examples that can help engineers to understand the system under exploration.<\/jats:p>","DOI":"10.1007\/s11219-024-09668-2","type":"journal-article","created":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T05:01:48Z","timestamp":1713762108000},"page":"729-763","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Exploring API behaviours through generated examples"],"prefix":"10.1007","volume":"32","author":[{"given":"Stefan","family":"Karlsson","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Hughes","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robbert","family":"Jongeling","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adnan","family":"\u010cau\u0161evi\u0107","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Sundmark","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,4,22]]},"reference":[{"key":"9668_CR1","doi-asserted-by":"publisher","unstructured":"Arcuri, A., & Briand, L. (2014). A Hitchhiker\u2019s guide to statistical tests for assessing randomized algorithms in software engineering. 24, 219\u2013250.\u00a0https:\/\/doi.org\/10.1002\/stvr.1486","DOI":"10.1002\/stvr.1486"},{"key":"9668_CR2","doi-asserted-by":"publisher","unstructured":"Atlidakis, V., Godefroid, P., & Polishchuk, M. (2019). RESTler: Stateful REST API Fuzzing. In 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE) (pp. 748\u2013758).\u00a0https:\/\/doi.org\/10.1109\/ICSE.2019.00083","DOI":"10.1109\/ICSE.2019.00083"},{"key":"9668_CR3","doi-asserted-by":"publisher","unstructured":"Barnaby, C., Sen, K., Zhang, T., Glassman, E., & Chandra, S. (2020). Exempla Gratis (E.G.): Code examples for free. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering ESEC\/FSE 2020 (pp. 1353\u20131364).\u00a0https:\/\/doi.org\/10.1145\/3368089.3417052","DOI":"10.1145\/3368089.3417052"},{"key":"9668_CR4","doi-asserted-by":"crossref","unstructured":"Burke, D.\u00a0A., & Johannisson, K. (2005). Translating formal software specifications to natural language. In Logical Aspects of Computational Linguistics (pp. 51\u201366).","DOI":"10.1007\/11422532_4"},{"key":"9668_CR5","doi-asserted-by":"publisher","unstructured":"Buse, R. P.\u00a0L., & Weimer, W. (2012). Synthesizing API usage examples. In 2012 34th International Conference on Software Engineering (ICSE) (pp. 782\u2013792).\u00a0https:\/\/doi.org\/10.1109\/ICSE.2012.6227140","DOI":"10.1109\/ICSE.2012.6227140"},{"key":"9668_CR6","doi-asserted-by":"publisher","unstructured":"Claessen, K., & Hughes, J. (2000). QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming ICFP \u201900 (pp. 268\u2013279).\u00a0https:\/\/doi.org\/10.1145\/351240.351266","DOI":"10.1145\/351240.351266"},{"key":"9668_CR7","volume-title":"Architectural styles and the design of network-based software architectures","author":"R Fielding","year":"2000","unstructured":"Fielding, R. (2000). Architectural styles and the design of network-based software architectures. Irvine, US: University of California."},{"key":"9668_CR8","volume-title":"Production-ready microservices","author":"SJ Fowler","year":"2016","unstructured":"Fowler, S. J. (2016). Production-ready microservices. O\u2019Reilly."},{"key":"9668_CR9","doi-asserted-by":"publisher","unstructured":"Gerdes, A., Hughes, J., Smallbone, N., Hanenberg, S., Ivarsson, S., & Wang, M. (2018). Understanding formal specifications through good examples. In Proceedings of the 17th ACM SIGPLAN International Workshop on Erlang Erlang 2018 (pp. 13\u201324).\u00a0https:\/\/doi.org\/10.1145\/3239332.3242763","DOI":"10.1145\/3239332.3242763"},{"key":"9668_CR10","doi-asserted-by":"publisher","unstructured":"Gu, X., Zhang, H., & Kim, S. (2019). CodeKernel: A graph kernel based approach to the selection of API usage examples. In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE) (pp. 590\u2013601).\u00a0https:\/\/doi.org\/10.1109\/ASE.2019.00061","DOI":"10.1109\/ASE.2019.00061"},{"key":"9668_CR11","doi-asserted-by":"crossref","unstructured":"Head, A., Glassman, E.\u00a0L., Hartmann, B., & Hearst, M.\u00a0A. (2018). Interactive extraction of examples from existing code. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1\u201312).","DOI":"10.1145\/3173574.3173659"},{"key":"9668_CR12","doi-asserted-by":"publisher","unstructured":"Hickey, R. (2020). A history of Clojure (vol.\u00a04). New York, NY, USA: Association for Computing Machinery.\u00a0https:\/\/doi.org\/10.1145\/3386321","DOI":"10.1145\/3386321"},{"key":"9668_CR13","doi-asserted-by":"publisher","unstructured":"Holmes, R., Walker, R.\u00a0J., & Murphy, G.\u00a0C. (2006). Approximate structural context matching: An approach to recommend relevant examples. 32, 952\u2013970.\u00a0https:\/\/doi.org\/10.1109\/TSE.2006.117","DOI":"10.1109\/TSE.2006.117"},{"key":"9668_CR14","first-page":"1","volume-title":"Practical aspects of declarative languages","author":"J Hughes","year":"2007","unstructured":"Hughes, J. (2007). QuickCheck testing for fun and profit. In M. Hanus (Ed.), Practical aspects of declarative languages (pp. 1\u201332). Berlin, Heidelberg: Springer, Berlin Heidelberg."},{"key":"9668_CR15","doi-asserted-by":"publisher","unstructured":"Karlsson, S., \u010cau\u0161evi\u015b, A., & Sundmark, D. (2020). QuickREST: Property-based test generation of OpenAPI-described RESTful APIs. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST) (pp. 131\u2013141).\u00a0https:\/\/doi.org\/10.1109\/ICST46399.2020.00023","DOI":"10.1109\/ICST46399.2020.00023"},{"key":"9668_CR16","doi-asserted-by":"publisher","unstructured":"Kim, J., Lee, S., Hwang, S. -W., & Kim, S. (2009). Adding examples into Java documents. In 2009 IEEE\/ACM International Conference on Automated Software Engineering (pp. 540\u2013544).\u00a0https:\/\/doi.org\/10.1109\/ASE.2009.39","DOI":"10.1109\/ASE.2009.39"},{"key":"9668_CR17","doi-asserted-by":"publisher","unstructured":"Kim, M., Xin, Q., Sinha, S., & Orso, A. (2022). Automated test generation for REST APIs: No time to rest yet. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis ISSTA 2022 (pp. 289\u2013301).\u00a0https:\/\/doi.org\/10.1145\/3533767.3534401","DOI":"10.1145\/3533767.3534401"},{"key":"9668_CR18","unstructured":"Lavoie, B., Rambow, O., & Reiter, E. (1996). The ModelExplainer. In Proceedings of the 8th International Workshop on Natural Language Generation\u00a0(pp. 9\u201312)."},{"key":"9668_CR19","doi-asserted-by":"publisher","unstructured":"MacIver, D.\u00a0R., & Donaldson, A.\u00a0F. (2020). Test-case reduction via test-case generation: Insights from the hypothesis reducer. In R.\u00a0Hirschfeld, & T.\u00a0Pape (Eds.), 34th European Conference on Object-Oriented Programming (ECOOP 2020) (pp. 13:1\u201313:27). Dagstuhl, Germany: Schloss Dagstuhl \u2013 Leibniz-Zentrum f\u00fcr Informatik volume 166 of Leibniz International Proceedings in Informatics (LIPIcs).\u00a0Retrieved from: https:\/\/drops.dagstuhl.de\/entities\/document\/10.4230\/LIPIcs.ECOOP.2020.13, https:\/\/doi.org\/10.4230\/LIPIcs.ECOOP.2020.13","DOI":"10.4230\/LIPIcs.ECOOP.2020.13"},{"key":"9668_CR20","doi-asserted-by":"publisher","unstructured":"Mar, L.\u00a0W., Wu, Y. -C., & Jiau, H.\u00a0C. (2011). Recommending proper API code examples for documentation purpose. In 2011 18th Asia-Pacific Software Engineering Conference (pp. 331\u2013338).\u00a0https:\/\/doi.org\/10.1109\/APSEC.2011.18","DOI":"10.1109\/APSEC.2011.18"},{"key":"9668_CR21","doi-asserted-by":"publisher","unstructured":"McLellan, S., Roesler, A., Tempest, J., & Spinuzzi, C. (1998). Building more usable APIs.\u00a015, 78\u201386.\u00a0https:\/\/doi.org\/10.1109\/52.676963","DOI":"10.1109\/52.676963"},{"key":"9668_CR22","doi-asserted-by":"crossref","unstructured":"Mittal, V.\u00a0O., & Paris, C. (1994). Generating examples for use in tutorial explanations: Using a subsumption based classifier. In In Proceedings of the 11th European Conference on Artificial Intelligence.","DOI":"10.21236\/ADA286028"},{"key":"9668_CR23","doi-asserted-by":"publisher","unstructured":"Montandon, J.\u00a0E., Borges, H., Felix, D., & Valente, M.\u00a0T. (2013). Documenting APIs with examples: Lessons learned with the APIMiner platform. In 2013 20th Working Conference on Reverse Engineering (WCRE) (pp. 401\u2013408).\u00a0https:\/\/doi.org\/10.1109\/WCRE.2013.6671315","DOI":"10.1109\/WCRE.2013.6671315"},{"key":"9668_CR24","doi-asserted-by":"publisher","unstructured":"Moreno, L., Bavota, G., Di\u00a0Penta, M., Oliveto, R., & Marcus, A. (2015). How can I use this method? In 2015 IEEE\/ACM 37th IEEE International Conference on Software Engineering (vol. 1, pp. 880\u2013890).\u00a0https:\/\/doi.org\/10.1109\/ICSE.2015.98","DOI":"10.1109\/ICSE.2015.98"},{"key":"9668_CR25","doi-asserted-by":"publisher","unstructured":"Novick, D.\u00a0G., & Ward, K. (2006). What users say they want in documentation. In Proceedings of the 24th Annual ACM International Conference on Design of Communication SIGDOC \u201906 (pp. 84\u201391).\u00a0https:\/\/doi.org\/10.1145\/1166324.1166346","DOI":"10.1145\/1166324.1166346"},{"key":"9668_CR26","doi-asserted-by":"publisher","unstructured":"Nykaza, J., Messinger, R., Boehme, F., Norman, C.\u00a0L., Mace, M., & Gordon, M. (2002). What programmers really want: Results of a needs assessment for SDK documentation. In Proceedings of the 20th Annual International Conference on Computer Documentation SIGDOC \u201902 (pp. 133\u2013141).\u00a0https:\/\/doi.org\/10.1145\/584955.584976","DOI":"10.1145\/584955.584976"},{"key":"9668_CR27","doi-asserted-by":"publisher","unstructured":"Robillard, M. P. (2009). What makes APIs hard to learn? Answers from developers.\u00a026, 27\u201334. https:\/\/doi.org\/10.1109\/MS.2009.193","DOI":"10.1109\/MS.2009.193"},{"key":"9668_CR28","doi-asserted-by":"publisher","unstructured":"Robillard, M.\u00a0P., & DeLine, R. (2011). A field study of API learning obstacles. 16, 703\u2013732.\u00a0https:\/\/doi.org\/10.1007\/s10664-010-9150-8","DOI":"10.1007\/s10664-010-9150-8"},{"key":"9668_CR29","doi-asserted-by":"publisher","unstructured":"Shull, F., Lanubile, F., & Basili, V. (2000). Investigating reading techniques for object-oriented framework learning.\u00a026, 1101\u20131118.\u00a0https:\/\/doi.org\/10.1109\/32.881720","DOI":"10.1109\/32.881720"},{"key":"9668_CR30","unstructured":"Swartout, W.\u00a0R. (1982). GIST English Generator. In In AAAI, pp. 404\u2013409."},{"key":"9668_CR31","doi-asserted-by":"publisher","unstructured":"Wu, H., Xu, L., Niu, X., & Nie, C. (2022). Combinatorial testing of RESTful APIs. In 44th International Conference on Software Engineering (ICSE \u201922).\u00a0https:\/\/doi.org\/10.1145\/3510003.3510151","DOI":"10.1145\/3510003.3510151"},{"key":"9668_CR32","doi-asserted-by":"publisher","unstructured":"Zhang, M., Arcuri, A., Li, Y., Xue, K., Wang, Z., Huo, J., & Huang, W. (2022). Fuzzing microservices in industry: Experience of applying EvoMaster at Meituan.\u00a0https:\/\/doi.org\/10.48550\/ARXIV.2208.03988","DOI":"10.48550\/ARXIV.2208.03988"}],"container-title":["Software Quality Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-024-09668-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11219-024-09668-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11219-024-09668-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T09:17:13Z","timestamp":1718875033000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11219-024-09668-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,22]]},"references-count":32,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["9668"],"URL":"https:\/\/doi.org\/10.1007\/s11219-024-09668-2","relation":{},"ISSN":["0963-9314","1573-1367"],"issn-type":[{"type":"print","value":"0963-9314"},{"type":"electronic","value":"1573-1367"}],"subject":[],"published":{"date-parts":[[2024,4,22]]},"assertion":[{"value":"17 March 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 April 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interest"}}]}}