{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T19:58:54Z","timestamp":1757620734785,"version":"3.44.0"},"reference-count":46,"publisher":"Cambridge University Press (CUP)","license":[{"start":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T00:00:00Z","timestamp":1757289600000},"content-version":"unspecified","delay-in-days":250,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["J. Funct. Prog."],"published-print":{"date-parts":[[2025]]},"abstract":"<jats:title>Abstract<\/jats:title>\n\t  <jats:p>Good test-suites are an important tool to check the correctness of programs. They are also essential in unsupervised educational settings, like automatic grading or for students to check their solution to some programming task by themselves. For most Haskell programming tasks, one can easily provide high-quality test-suites using standard tools like QuickCheck. Unfortunately, this is no longer the case once we leave the purely functional world and enter the lands of console I\/O. Nonetheless, understanding console I\/O is an important part of learning Haskell, and we would like to provide students the same support as with other subject matters. The difficulty in testing console I\/O programs arises from the standard tools\u2019 lack of support for specifying intended console interactions as simple declarative properties. These interactions are however essential in order to determine whether a program behaves as desired. We describe the console interactions of a program by tracing its text input and output actions. In order to describe which traces match the intended behavior of the program under test, we present a formal specification language. The language is designed to capture interactive behavior found in commonly used textbook exercises and examples, or as much of it as possible, as well as in our own teaching, while at the same time retaining simplicity and clarity of specifications. We intentionally restrict the language, ensuring that expressed behavior is truly interactive and not simply a pure string-builder function in disguise. Based on this specification language, we build a testing framework that allows testing against specifications in an automated way. A central feature of the testing procedure is the use of a constraint solver in order to find meaningful input sequences for the program under test.<\/jats:p>","DOI":"10.1017\/s0956796825100075","type":"journal-article","created":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T10:22:41Z","timestamp":1757326961000},"update-policy":"https:\/\/doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":0,"title":["Automatically testing console I\/O behavior of student submissions in Haskell"],"prefix":"10.1017","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8947-0348","authenticated-orcid":false,"given":"OLIVER","family":"WESTPHAL","sequence":"first","affiliation":[{"name":"Universit\u00e4t Duisburg-Essen"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2411-9909","authenticated-orcid":false,"given":"JANIS","family":"VOIGTL\u00c4NDER","sequence":"additional","affiliation":[{"name":"Universit\u00e4t Duisburg-Essen"}]}],"member":"56","published-online":{"date-parts":[[2025,9,8]]},"reference":[{"key":"S0956796825100075_ref28","first-page":"2639","volume-title":"Proc. ACM Program. Lang","author":"Moy","year":"2024"},{"key":"S0956796825100075_ref29","first-page":"133","volume-title":"7th International Working Conference on Source Code Analysis and Manipulation (SCAM)","author":"Naylor","year":"2007"},{"key":"S0956796825100075_ref7","first-page":"268","volume-title":"5th International Conference on Functional Programming (ICFP)","author":"Claessen","year":"2000"},{"key":"S0956796825100075_ref2","first-page":"194","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems (TACAS)","author":"Bj\u00f8rner","year":"2015"},{"key":"S0956796825100075_ref17","doi-asserted-by":"publisher","DOI":"10.1145\/1993498.1993529"},{"key":"S0956796825100075_ref13","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-39110-6_2"},{"key":"S0956796825100075_ref14","first-page":"36","volume-title":"9th Interaction and Concurrency Experience Workshop (ICE)","author":"Fowler","year":"2016"},{"key":"S0956796825100075_ref24","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781316784099"},{"key":"S0956796825100075_ref26","doi-asserted-by":"publisher","DOI":"10.1145\/360248.360252"},{"key":"S0956796825100075_ref4","doi-asserted-by":"publisher","DOI":"10.1145\/2408776.2408795"},{"key":"S0956796825100075_ref6","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-07151-0_2"},{"key":"S0956796825100075_ref12","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180248"},{"key":"S0956796825100075_ref18","first-page":"213","volume-title":"26th Conference on Programming Language Design and Implementation (PLDI)","author":"Godefroid","year":"2005"},{"key":"S0956796825100075_ref20","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0053567"},{"key":"S0956796825100075_ref1","first-page":"367","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems (TACAS)","author":"Anand","year":"2008"},{"key":"S0956796825100075_ref11","doi-asserted-by":"publisher","DOI":"10.1145\/2364506.2364515"},{"key":"S0956796825100075_ref8","first-page":"337","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems (TACAS)","author":"de Moura","year":"2008"},{"key":"S0956796825100075_ref37","first-page":"19","volume-title":"4th Workshop \u201cAutomatische Bewertung von Programmieraufgaben\u201d (ABP). Gesellschaft f\u00fcr Informatik","author":"Siegburg","year":"2019"},{"key":"S0956796825100075_ref23","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-47147-7_4"},{"key":"S0956796825100075_ref38","doi-asserted-by":"publisher","DOI":"10.3991\/ijep.v12i5.31283"},{"key":"S0956796825100075_ref3","doi-asserted-by":"publisher","DOI":"10.1007\/11537328_2"},{"key":"S0956796825100075_ref15","first-page":"137","volume-title":"17th International Symposium on Principles and Practice of Declarative Programming (PPDP)","author":"Giantsios","year":"2015"},{"key":"S0956796825100075_ref22","first-page":"169","volume-title":"A List of Successes That Can Change the World \u2013 Essays Dedicated to Philip Wadler on the Occasion of His 60th Birthday","author":"Hughes","year":"2016"},{"volume-title":"Packt Publishing","year":"2023","author":"Schrijvers","key":"S0956796825100075_ref34"},{"key":"S0956796825100075_ref21","first-page":"1","volume-title":"Practical Aspects of Declarative Languages (PADL)","author":"Hughes","year":"2007"},{"key":"S0956796825100075_ref9","first-page":"97","volume-title":"2023 Haskell Symposium","author":"de Vries","year":"2023"},{"key":"S0956796825100075_ref35","first-page":"1","volume-title":"Proc. ACM Program. Lang","author":"Seidel","year":"2017"},{"volume-title":"3rd Workshop \u201cAutomatische Bewertung von Programmieraufgaben\u201d (ABP). CEUR-WS.org","year":"2017","author":"Waldmann","key":"S0956796825100075_ref42"},{"key":"S0956796825100075_ref5","doi-asserted-by":"publisher","DOI":"10.1145\/2364506.2364516"},{"key":"S0956796825100075_ref27","first-page":"54","volume-title":"35th International Conference on Automated Software Engineering (ASE)","author":"Liu","year":"2021"},{"key":"S0956796825100075_ref43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-75333-7_6"},{"key":"S0956796825100075_ref33","first-page":"27","volume-title":"2022 Conference on International Computing Education Research (ICER)","author":"Sarsa","year":"2022"},{"key":"S0956796825100075_ref44","doi-asserted-by":"publisher","DOI":"10.4204\/EPTCS.424.6"},{"key":"S0956796825100075_ref45","first-page":"19","volume-title":"8th and 9th International Workshop on Trends in Functional Programming in Education (TFPIE)","author":"Westphal","year":"2020"},{"key":"S0956796825100075_ref46","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59025-3_11"},{"key":"S0956796825100075_ref19","doi-asserted-by":"publisher","DOI":"10.1007\/11737414_15"},{"volume-title":"Pearson Education","year":"2011","author":"Thompson","key":"S0956796825100075_ref40"},{"key":"S0956796825100075_ref25","doi-asserted-by":"publisher","DOI":"10.1145\/3231711"},{"key":"S0956796825100075_ref31","doi-asserted-by":"publisher","DOI":"10.1145\/2633357.2633365"},{"key":"S0956796825100075_ref32","first-page":"37","volume-title":"2008 Haskell Symposium","author":"Runciman","year":"2008"},{"key":"S0956796825100075_ref16","doi-asserted-by":"publisher","DOI":"10.1017\/S0956796821000113"},{"key":"S0956796825100075_ref30","first-page":"95","volume-title":"5th Workshop on Programming Language Approaches to Concurrency and Communication-centric Software (PLACES)","author":"Neykova","year":"2013"},{"key":"S0956796825100075_ref36","first-page":"571","volume-title":"22nd International Conference on Automated Software Engineering (ASE)","author":"Sen","year":"2007"},{"key":"S0956796825100075_ref10","first-page":"117","volume-title":"21st International Conference on Functional Programming (ICFP)","author":"Dimoulas","year":"2016"},{"key":"S0956796825100075_ref39","first-page":"25","volume-title":"2007 Haskell Workshop","author":"Swierstra","year":"2007"},{"key":"S0956796825100075_ref41","doi-asserted-by":"publisher","DOI":"10.1016\/j.ic.2012.05.002"}],"container-title":["Journal of Functional Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0956796825100075","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T10:22:45Z","timestamp":1757326965000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0956796825100075\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":46,"alternative-id":["S0956796825100075"],"URL":"https:\/\/doi.org\/10.1017\/s0956796825100075","relation":{},"ISSN":["0956-7968","1469-7653"],"issn-type":[{"type":"print","value":"0956-7968"},{"type":"electronic","value":"1469-7653"}],"subject":[],"published":{"date-parts":[[2025]]},"assertion":[{"value":"\u00a9 The Author(s), 2025. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https:\/\/creativecommons.org\/licenses\/by\/4.0\/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.","name":"license","label":"License","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}},{"value":"This content has been made available to all.","name":"free","label":"Free to read"}],"article-number":"e18"}}