{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T17:27:15Z","timestamp":1767374835554,"version":"3.41.0"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,7,20]],"date-time":"2019-07-20T00:00:00Z","timestamp":1563580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2019,12,31]]},"abstract":"<jats:p>Given its societal impacts and applications to numerous fields, machine learning (ML) is an important topic to understand for many students outside of computer science and statistics. However, machine-learning education research is nascent, and research on this subject for non-majors thus far has only focused on curricula and courseware. We interviewed 10 instructors of ML courses for non-majors, inquiring as to what their students find both easy and difficult about machine learning. While ML has a reputation for having algorithms that are difficult to understand, in practice our participating instructors reported that it was not the algorithms that were difficult to teach, but the higher-level design decisions. We found that the learning goals that participants described as hard to teach were consistent with higher levels of the Structure of Observed Learning Outcomes (SOLO) taxonomy, such as making design decisions and comparing\/contrasting models. We also found that the learning goals that were described as easy to teach, such as following the steps of particular algorithms, were consistent with the lower levels of the SOLO taxonomy. Realizing that higher-SOLO learning goals are more difficult to teach is useful for informing course design, public outreach, and the design of educational tools for teaching ML.<\/jats:p>","DOI":"10.1145\/3336124","type":"journal-article","created":{"date-parts":[[2019,7,22]],"date-time":"2019-07-22T12:15:03Z","timestamp":1563797703000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":53,"title":["What Is Hard about Teaching Machine Learning to Non-Majors? Insights from Classifying Instructors\u2019 Learning Goals"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3475-8031","authenticated-orcid":false,"given":"Elisabeth","family":"Sulmont","sequence":"first","affiliation":[{"name":"McGill University, Canada"}]},{"given":"Elizabeth","family":"Patitsas","sequence":"additional","affiliation":[{"name":"McGill University, Canada"}]},{"given":"Jeremy R.","family":"Cooperstock","sequence":"additional","affiliation":[{"name":"McGill University, Canada"}]}],"member":"320","published-online":{"date-parts":[[2019,7,20]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"1","article-title":"A comparison of primary mathematics curriculum in England and Qatar: The SOLO taxonomy","volume":"21","author":"Alsaadi Aziza","year":"2001","journal-title":"Res. Learn. Math."},{"volume-title":"Teaching for Quality Learning at University","author":"Biggs John","key":"e_1_2_1_2_1"},{"volume-title":"Teaching for Quality Learning at University: What the Student Does","author":"Biggs John B.","key":"e_1_2_1_3_1"},{"volume-title":"Collis","year":"1982","author":"Biggs John B.","key":"e_1_2_1_4_1"},{"volume-title":"Bloom et al","year":"1956","author":"Benjamin","key":"e_1_2_1_5_1"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1080\/0729436950140201"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 7th Baltic Sea Conference on Computing Education Research","volume":"88","author":"Brabrand Claus","year":"2007"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10734-009-9210-4"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3141880.3141891"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF03217340"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1345375.1345438"},{"volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI\u201916)","year":"2016","author":"Gil Yolanda","key":"e_1_2_1_12_1"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2676723.2677311"},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the 14th Australasian Computing Education Conference","volume":"123","author":"Gluga Richard","year":"2012"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2361276.2361284"},{"key":"e_1_2_1_16_1","unstructured":"Michael Guerzhoy and Renjie Liao. 2018. Understanding How Recurrent Neural Networks Model Text. Retrieved from http:\/\/modelai.gettysburg.edu\/2018\/rnntext\/index.html.  Michael Guerzhoy and Renjie Liao. 2018. Understanding How Recurrent Neural Networks Model Text. Retrieved from http:\/\/modelai.gettysburg.edu\/2018\/rnntext\/index.html."},{"key":"e_1_2_1_17_1","unstructured":"Mark Guzdial. 2018. Teaching Two Programming Languages in the First CS Course. Retrieved from https:\/\/cacm.acm.org\/blogs\/blog-cacm\/228006-teaching-two-programming-languages-in-the-first-cs-course\/fulltext.  Mark Guzdial. 2018. Teaching Two Programming Languages in the First CS Course. Retrieved from https:\/\/cacm.acm.org\/blogs\/blog-cacm\/228006-teaching-two-programming-languages-in-the-first-cs-course\/fulltext."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog2801_7"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2960310.2960324"},{"volume-title":"Rasch analysis of math SOLO taxonomy levels using hierarchical items in testlets. ERIC-ED398271","year":"1996","author":"Lam Peter","key":"e_1_2_1_20_1"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TE.2009.2038992"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1140123.1140157"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.26858\/wtetev15i1y2017p6873"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 6th Australasian Conference on Computing Education","volume":"30","author":"Oliver Dave","year":"2004"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1597849.1384328"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 11th Australasian Conference on Computing Education","volume":"95","author":"Shuhidan Shuhaida","year":"2009"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X015002004"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1352322.1352227"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287392"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538910"},{"key":"e_1_2_1_31_1","unstructured":"Douglas Turnbull. 2012. Music Genre Classification. Retrieved from http:\/\/modelai.gettysburg.edu\/2012\/music\/.  Douglas Turnbull. 2012. Music Genre Classification. Retrieved from http:\/\/modelai.gettysburg.edu\/2012\/music\/."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF03217125"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2899415.2925485"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3072979"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 8th Australasian Conference on Computing Education","volume":"52","author":"Whalley Jacqueline L.","year":"2006"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3336124","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3336124","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:26:09Z","timestamp":1750206369000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3336124"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,20]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,12,31]]}},"alternative-id":["10.1145\/3336124"],"URL":"https:\/\/doi.org\/10.1145\/3336124","relation":{},"ISSN":["1946-6226"],"issn-type":[{"type":"electronic","value":"1946-6226"}],"subject":[],"published":{"date-parts":[[2019,7,20]]},"assertion":[{"value":"2018-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}