{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T18:45:11Z","timestamp":1767206711121,"version":"build-2238731810"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,4,20]],"date-time":"2023-04-20T00:00:00Z","timestamp":1681948800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100011661","name":"Pacific Northwest National Laboratory","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100011661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>The field of machine learning and its subfield of deep learning have grown rapidly in recent years. With the speed of advancement, it is nearly impossible for data scientists to maintain expert knowledge of cutting-edge techniques. This study applies human factors methods to the field of machine learning to address these difficulties.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Using semi-structured interviews with data scientists at a National Laboratory, we sought to understand the process used when working with machine learning models, the challenges encountered, and the ways that human factors might contribute to addressing those challenges.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Results of the interviews were analyzed to create a generalization of the process of working with machine learning models. Issues encountered during each process step are described.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Recommendations and areas for collaboration between data scientists and human factors experts are provided, with the goal of creating better tools, knowledge, and guidance for machine learning scientists.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2023.1130190","type":"journal-article","created":{"date-parts":[[2023,4,20]],"date-time":"2023-04-20T15:49:12Z","timestamp":1682005752000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Opportunities for human factors in machine learning"],"prefix":"10.3389","volume":"6","author":[{"given":"Jessica A.","family":"Baweja","sequence":"first","affiliation":[]},{"given":"Corey K.","family":"Fallon","sequence":"additional","affiliation":[]},{"given":"Brett A.","family":"Jefferson","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,4,20]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1201\/9781410607775.ch2","article-title":"Hierarchical task analysis","volume":"2","author":"Annett","year":"2003","journal-title":"Handbook Cog. Task Design"},{"key":"B2","volume-title":"Memory Practices in the Sciences","author":"Bowker","year":"2008"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1037.\/13620-004","article-title":"\u201cThematic analysis,\u201d","author":"Braun","year":"2012","journal-title":"APA Handbook of Research Methods in Psychology, Vol. 2. Research designs: Quantitative, Qualitative, Neuropsychological, and Biological"},{"key":"B4","first-page":"3081","article-title":"Hyperparameter optimization is deceiving us, and how to stop it","volume":"34","author":"Cooper","year":"2021","journal-title":"Adva. Neural Inform. Process. Sys."},{"key":"B5","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.1080.\/00140139.2021.1905882","article-title":"Why human factors science is demonstrably necessary: historical and evolutionary foundations","volume":"64","author":"De Winter","year":"2021","journal-title":"Ergonomics"},{"key":"B6","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1145.\/2500499","article-title":"Data science and prediction","volume":"56","author":"Dhar","year":"2013","journal-title":"Commun. ACM"},{"key":"B7","first-page":"5","article-title":"Understanding task analysis for human-computer interaction","author":"Diaper","year":"2004","journal-title":"The handbook of task analysis for human-computer interaction"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.1037\/h0061470","article-title":"The critical incident technique","author":"Flanagan","year":"1954","journal-title":"Psychol. Bull"},{"key":"B9","first-page":"1","author":"Gitelman","year":"2013"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-99740-7_21","article-title":"Explainable AI: the new 42?","author":"Goebel","year":"2018","journal-title":"International cross-domain conference for machine learning and knowledge extraction"},{"key":"B11","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1075\/is.8.3.10gro","article-title":"Can robots be teammates? Benchmarks in human\u2013robot teams","volume":"8","author":"Groom","year":"2007","journal-title":"Interaction Studies"},{"key":"B12","article-title":"Metrics for explainable AI: Challenges and prospects","author":"Hoffman","year":"2018","journal-title":"arXiv preprint arXiv:"},{"key":"B13","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1126\/science.359.6377.725","article-title":"Artificial intelligence faces reproducibility crisis","volume":"359","author":"Hutson","year":"2018","journal-title":"Science"},{"key":"B14","article-title":"Introduction to data science","author":"Igual","year":"2017","journal-title":"Introduction to data science"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1126.\/science.aaa8415","article-title":"Machine learning: Trends, perspectives, and prospects","author":"Jordan","year":"2015","journal-title":"Science"},{"key":"B16","article-title":"Leakage and the Reproducibility Crisis in ML-based Science","author":"Kapoor","year":"2022","journal-title":"arXiv preprint arXiv:"},{"key":"B17","article-title":"Interpreting interpretability: understanding data scientists' use of interpretability tools for machine learning","author":"Kaur","year":"2020","journal-title":"In Proceedings of the"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1038.\/nature14539","article-title":"Deep learning. N","author":"LeCun","year":"2015","journal-title":"ature"},{"key":"B19","author":"Li","year":"2018","journal-title":"Hypertuner: Visual analytics for hyperparameter tuning by professionals"},{"key":"B20","doi-asserted-by":"crossref","first-page":"1618","DOI":"10.1080\/001401398186108","article-title":"Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands","volume":"41","author":"Militello","year":"1998","journal-title":"Ergonomics"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1145.\/3290605.3300356","article-title":"\u201cHow data science workers work with data: discovery, capture, curation, design, creation,\u201d","author":"Muller","year":"2019","journal-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems"},{"key":"B22","author":"Phillips","year":"2020","journal-title":"Four Principles of Explainable Artificial Intelligence"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1145.\/3411764.3445518","article-title":"\u201cEveryone wants to do the model work, not the data work\u201d: Data Cascades in High-Stakes AI,\u201d","author":"Sambasivan","year":"2021","journal-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems"},{"key":"B24","volume-title":"Human Factors Methods: A Practical Guide for Engineering and Design","author":"Stanton","year":"2017"},{"key":"B25","volume-title":"An Introduction to Human Factors Engineering","author":"Wickens","year":"2004"},{"key":"B26","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1016\/j.neucom.07061","article-title":"On hyperparameter optimization of machine learning algorithms: theory and practice","volume":"415","author":"Yang","year":"2020","journal-title":"Neurocomputing"}],"updated-by":[{"DOI":"10.3389\/frai.2023.1327954","type":"corrigendum","label":"Corrigendum","source":"publisher","updated":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T00:00:00Z","timestamp":1700179200000}}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1130190\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T03:32:16Z","timestamp":1700191936000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1130190\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,20]]},"references-count":26,"alternative-id":["10.3389\/frai.2023.1130190"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1130190","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,20]]},"article-number":"1130190"}}