{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T14:47:12Z","timestamp":1770994032242,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T00:00:00Z","timestamp":1700697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>Software developers use a significant amount of time reading and interpreting error messages. However, error messages have often been based on either anecdotal evidence or expert opinion, disregarding novices, who arguably are the ones who benefit the most from effective error messages. Furthermore, the usability aspects of Structured Query Language (SQL) error messages have not received much scientific attention. In this mixed-methods study, we coded a total of 128 error messages from eight database management systems (DBMS), and using data from 311 participants, analysed 4,796 queries using regression analysis to find out if and how acknowledged error message qualities explain SQL syntax error fixing success rates. Additionally, we performed a conventional content analysis on 1,505 suggestions on how to improve SQL error messages, and based on the analysis, formulated a framework consisting of nine guidelines for SQL error message design. The results indicate that general error message qualities do not necessarily explain query fixing success in the context of SQL syntax errors and that even some novel NewSQL systems fail to account for basic error message design guidelines. The error message design framework and examples of its practical applications shown in this study are applicable in educational contexts as well as by DBMS vendors in understanding novice perspectives in error message design.<\/jats:p>","DOI":"10.1145\/3607180","type":"journal-article","created":{"date-parts":[[2023,7,5]],"date-time":"2023-07-05T12:21:33Z","timestamp":1688559693000},"page":"1-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Framework for SQL Error Message Design: A Data-Driven Approach"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4060-3431","authenticated-orcid":false,"given":"Toni","family":"Taipalus","sequence":"first","affiliation":[{"name":"University of Jyv\u00e4skyl\u00e4, Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7567-7807","authenticated-orcid":false,"given":"Hilkka","family":"Grahn","sequence":"additional","affiliation":[{"name":"University of Jyv\u00e4skyl\u00e4, Finland"}]}],"member":"320","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844640"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2899415.2899464"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ase.2019.00039"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.5555\/313458.313470"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/icse.2017.59"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844584"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3344429.3372508"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2016.1225464"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159450.3159453"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/S1467-0895(01)00023-9"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2005.06.028"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1656255.1656257"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1080\/00098650903267784"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2723163"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2591748"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445696"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/52.903161"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9523-3"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9655-0"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367539"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/152610.152611"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1561\/1900000002"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1080\/03075070120099368"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1177\/1049732305276687"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.4380160703"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2048237.2048241"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3335814"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3446871.3469759"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551392"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3529158"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/77481.77486"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/1352135.1352193"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.110"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/259963.260333"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/97243.97281"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/1352322.1352192"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428757.3429131"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/hcc.2002.1046372"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3003665.3003674"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3017680.3017768"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3105726.3106169"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/947121.947174"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/tse.1977.231131"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/356835.356837"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/1499949.1500036"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2997934"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/358628.358639"},{"key":"e_1_3_3_49_2","volume-title":"Designing the User Interface: Strategies for Effective Human-Computer Interaction","author":"Shneiderman Ben","year":"2016","unstructured":"Ben Shneiderman, Catherine Plaisant, Maxine S. Cohen, Steven Jacobs, Niklas Elmqvist, and Nicholas Diakopoulos. 2016. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson."},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.1995.1017"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110576"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/FIE44824.2020.9274114"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3585794"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2022.2108648"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111034"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287359"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3398377"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3231712"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554854"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2593310"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1155\/2010\/602570"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452822"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/s0020-7373(85)80051-1"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/319628.319656"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133850.3133862"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607180","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3607180","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:34Z","timestamp":1750178254000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607180"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,23]]},"references-count":64,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3607180"],"URL":"https:\/\/doi.org\/10.1145\/3607180","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,23]]},"assertion":[{"value":"2022-06-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}