{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T10:32:34Z","timestamp":1790245954137,"version":"4.1.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"GROUP","license":[{"start":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T00:00:00Z","timestamp":1625702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,7,8]]},"abstract":"<jats:p>Machine learning (ML) has become a crucial component in software products, either as part of the user experience or used internally by software teams. Prior studies have explored how ML is affecting development team roles beyond data scientists, including user experience designers, program managers, developers and operations engineers. However, there has been little investigation of how team members in different roles on the team communicate about ML, in particular about the quality of models. We use the general term quality to look beyond technical issues of model evaluation, such as accuracy and overfitting, to any issue affecting whether a model is suitable for use, including ethical, engineering, operations, and legal considerations. What challenges do teams face in discussing the quality of ML models? What work practices mitigate those challenges? To address these questions, we conducted a mixed-methods study at a large software company, first interviewing15 employees in a variety of roles, then surveying 168 employees to broaden our understanding. We found several challenges, including a mismatch between user-focused and model-focused notions of performance, misunderstandings about the capabilities and limitations of evolving ML technology, and difficulties in understanding concerns beyond one's own role. We found several mitigation strategies, including the use of demos during discussions to keep the team customer-focused.<\/jats:p>","DOI":"10.1145\/3463934","type":"journal-article","created":{"date-parts":[[2021,7,13]],"date-time":"2021-07-13T16:52:15Z","timestamp":1626195135000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["How Teams Communicate about the Quality of ML Models: A Case Study at an International Technology Company"],"prefix":"10.1145","volume":"5","author":[{"given":"Jumana","family":"Almahmoud","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge , MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"DeLine","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven M.","family":"Drucker","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,7,13]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"[n.d.]. 2017 Kaggle ML & DS Survey. https:\/\/kaggle.com\/kaggle\/kaggle-survey-2017 Library Catalog: www.kaggle.com.  [n.d.]. 2017 Kaggle ML & DS Survey. https:\/\/kaggle.com\/kaggle\/kaggle-survey-2017 Library Catalog: www.kaggle.com."},{"key":"e_1_2_1_2_1","unstructured":"[n.d.]. How to Prevent Discriminatory Outcomes in Machine Learning. https:\/\/www.weforum.org\/whitepapers\/how-to-prevent-discriminatory-outcomes-in-machine-learning\/ Library Catalog: www.weforum.org.  [n.d.]. How to Prevent Discriminatory Outcomes in Machine Learning. https:\/\/www.weforum.org\/whitepapers\/how-to-prevent-discriminatory-outcomes-in-machine-learning\/ Library Catalog: www.weforum.org."},{"key":"e_1_2_1_3_1","unstructured":"2014. 3 Data Careers Decoded and What It Means for You. https:\/\/blog.udacity.com\/2014\/12\/data-analyst-vs-data-scientist-vs-data-engineer.html Library Catalog: blog.udacity.com Section: Career Guidance.  2014. 3 Data Careers Decoded and What It Means for You. https:\/\/blog.udacity.com\/2014\/12\/data-analyst-vs-data-scientist-vs-data-engineer.html Library Catalog: blog.udacity.com Section: Career Guidance."},{"key":"e_1_2_1_4_1","volume-title":"Amazon scraps secret AI recruiting tool that showed bias against women","year":"2018","unstructured":"2018. Amazon scraps secret AI recruiting tool that showed bias against women . Reuters(Oct . 2018 ). https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G 2018. Amazon scraps secret AI recruiting tool that showed bias against women. Reuters(Oct. 2018). https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G"},{"key":"e_1_2_1_5_1","unstructured":"2018. Tutorial: 21 fairness definitions and their politics. https:\/\/www.youtube.com\/watch?v=jIXIuYdnyyk&ab_channel=ArvindNarayanan  2018. Tutorial: 21 fairness definitions and their politics. https:\/\/www.youtube.com\/watch?v=jIXIuYdnyyk&ab_channel=ArvindNarayanan"},{"key":"e_1_2_1_6_1","volume-title":"Keuper","author":"Aerts Jeroen C. J. H.","year":"2003","unstructured":"Jeroen C. J. H. Aerts , Keith C. Clarke , and Alex D . Keuper . 2003 . Testing Popular Visualization Techniques for Representing Model Uncertainty.Cartography and Geographic Information Science 30, 3 (Jan. 2003), 249--261. https:\/\/doi.org\/10.1559\/152304003100011180 10.1559\/152304003100011180 Jeroen C. J. H. Aerts, Keith C. Clarke, and Alex D. Keuper. 2003. Testing Popular Visualization Techniques for Representing Model Uncertainty.Cartography and Geographic Information Science 30, 3 (Jan. 2003), 249--261. https:\/\/doi.org\/10.1559\/152304003100011180"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702509"},{"key":"e_1_2_1_9_1","unstructured":"Gennady Andrienko Natalia Andrienko Steven Drucker Jean-Daniel Fekete Danyel Fisher Stavros Idreos Tim Kraska Guoliang Li Kwan-Liu Ma Jock Mackinlay Antti Oulasvirta Tobias Schreck Heidrun Schmann Michael Stonebraker David Auber Nikos Bikakis Panos Chrysanthis George Papastefanatos and Mohamed Sharaf. 2020. Big Data Visualization and Analytics: Future Research Challenges and Emerging Applications. https:\/\/hal.inria.fr\/hal-02568845  Gennady Andrienko Natalia Andrienko Steven Drucker Jean-Daniel Fekete Danyel Fisher Stavros Idreos Tim Kraska Guoliang Li Kwan-Liu Ma Jock Mackinlay Antti Oulasvirta Tobias Schreck Heidrun Schmann Michael Stonebraker David Auber Nikos Bikakis Panos Chrysanthis George Papastefanatos and Mohamed Sharaf. 2020. Big Data Visualization and Analytics: Future Research Challenges and Emerging Applications. https:\/\/hal.inria.fr\/hal-02568845"},{"key":"e_1_2_1_10_1","volume-title":"Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney.","author":"Arnold Matthew","year":"2018","unstructured":"Matthew Arnold , Rachel K. E. Bellamy , Michael Hind , Stephanie Houde , Sameep Mehta , Aleksandra Mojsilovic , Ravi Nair , Karthikeyan Natesan Ramamurthy , Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney. 2018 . FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity . arXiv:1808.07261 [cs] (Aug. 2018). http:\/\/arxiv.org\/abs\/1808.07261 arXiv: 1808.07261. Matthew Arnold, Rachel K. E. Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney. 2018. FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity. arXiv:1808.07261 [cs] (Aug. 2018). http:\/\/arxiv.org\/abs\/1808.07261 arXiv: 1808.07261."},{"key":"e_1_2_1_11_1","unstructured":"author. [n.d.]. ONNX: Open Neural Network Exchange Format. https:\/\/onnx.ai\/  author. [n.d.]. ONNX: Open Neural Network Exchange Format. https:\/\/onnx.ai\/"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908131.2908135"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025738"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1976.10480949"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258038"},{"key":"e_1_2_1_16_1","unstructured":"Carrie J Cai and Philip J Guo. [n.d.]. Software Developers Learning Machine Learning: Motivations Hurdles and Desires. ([n. d.]) 10.  Carrie J Cai and Philip J Guo. [n.d.]. Software Developers Learning Machine Learning: Motivations Hurdles and Desires. ([n. d.]) 10."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172950"},{"key":"e_1_2_1_18_1","unstructured":"Comet.ml. [n.d.]. Comet.ml - Supercharging Machine Learning. https:\/\/www.comet.ml\/  Comet.ml. [n.d.]. Comet.ml - Supercharging Machine Learning. https:\/\/www.comet.ml\/"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025739"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312905"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1080\/14639220110110324"},{"key":"e_1_2_1_22_1","volume-title":"Hanna Wallach, Hal Daume\u00e9 III,and Kate Crawford.","author":"Gebru Timnit","year":"2018","unstructured":"Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daume\u00e9 III,and Kate Crawford. 2018 . Datasheets for Datasets . arXiv:1803.09010 [cs](March 2018). http:\/\/arxiv.org\/abs\/1803.09010arXiv: 1803.09010. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume\u00e9 III,and Kate Crawford. 2018. Datasheets for Datasets. arXiv:1803.09010 [cs](March 2018). http:\/\/arxiv.org\/abs\/1803.09010arXiv: 1803.09010."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10606-018-9333-1"},{"key":"e_1_2_1_24_1","unstructured":"Philip Guo. [n.d.]. Data Science Workflow: Overview and Challenges. https:\/\/cacm.acm.org\/blogs\/blog-cacm\/169199-data-science-workflow-overview-and-challenges\/fulltext Library Catalog: cacm.acm.org.  Philip Guo. [n.d.]. Data Science Workflow: Overview and Challenges. https:\/\/cacm.acm.org\/blogs\/blog-cacm\/169199-data-science-workflow-overview-and-challenges\/fulltext Library Catalog: cacm.acm.org."},{"key":"e_1_2_1_25_1","volume-title":"The structure and function of communication in society. InThe Communication of Ideas","author":"Lasswell Harold D.","unstructured":"Lasswell Harold D. 1948. The structure and function of communication in society. InThe Communication of Ideas . Harper's , New York, N.Y. Lasswell Harold D. 1948. The structure and function of communication in society. InThe Communication of Ideas.Harper's, New York, N.Y."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372831"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300809"},{"key":"e_1_2_1_28_1","volume-title":"The Dataset Nutrition Label:A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 [cs](May","author":"Holland Sarah","year":"2018","unstructured":"Sarah Holland , Ahmed Hosny , Sarah Newman , Joshua Joseph , and Kasia Chmielinski . 2018. The Dataset Nutrition Label:A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 [cs](May 2018 ). http:\/\/arxiv.org\/abs\/1805.03677arXiv: 1805.03677. Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018. The Dataset Nutrition Label:A Framework To Drive Higher Data Quality Standards. arXiv:1805.03677 [cs](May 2018). http:\/\/arxiv.org\/abs\/1805.03677arXiv: 1805.03677."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392878"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934287"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300432"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858558"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702603"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3299059"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884783"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3182515"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300493"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14236\/ewic\/HCI2018.32"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2908698.2908717"},{"key":"e_1_2_1_41_1","volume-title":"Manifold: A Model-Agnostic Visual Debugging Tool for Machine Learning at Uber.","author":"Li Lezhi","year":"2019","unstructured":"Lezhi Li , Yunfeng Bai , and Yang Wang . 2019 . Manifold: A Model-Agnostic Visual Debugging Tool for Machine Learning at Uber. (2019). https:\/\/www.usenix.org\/conference\/opml19\/presentation\/li-lezhi Lezhi Li, Yunfeng Bai, and Yang Wang. 2019. Manifold: A Model-Agnostic Visual Debugging Tool for Machine Learning at Uber. (2019). https:\/\/www.usenix.org\/conference\/opml19\/presentation\/li-lezhi"},{"key":"e_1_2_1_42_1","unstructured":"John Lofland and John Lofland (Eds.). 2006.Analyzing social settings: a guide to qualitative observation and analysis(4th ed ed.). Wadsworth\/Thomson Learning Belmont CA.  John Lofland and John Lofland (Eds.). 2006.Analyzing social settings: a guide to qualitative observation and analysis(4th ed ed.). Wadsworth\/Thomson Learning Belmont CA."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3361118"},{"key":"e_1_2_1_44_1","unstructured":"Lauren Kirchner Surya Julia Angwin Mattu Jeff Larson. 2016. Machine Bias. ProPublica(2016). https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing?token=pB6i06IyoO0LwmE2vf YUQBGseZmS8U0E  Lauren Kirchner Surya Julia Angwin Mattu Jeff Larson. 2016. Machine Bias. ProPublica(2016). https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing?token=pB6i06IyoO0LwmE2vf YUQBGseZmS8U0E"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3299018"},{"key":"e_1_2_1_47_1","unstructured":"Syed Sadat Nazrul. 2018. DevOps for Data Scientists: Taming the Unicorn. https:\/\/towardsdatascience.com\/devops-for-data-scientists-taming-the-unicorn-6410843990de Library Catalog: towards datascience.com.  Syed Sadat Nazrul. 2018. DevOps for Data Scientists: Taming the Unicorn. https:\/\/towardsdatascience.com\/devops-for-data-scientists-taming-the-unicorn-6410843990de Library Catalog: towards datascience.com."},{"key":"e_1_2_1_48_1","volume-title":"Filippo Menczer, and Alessandro Flammini.","author":"Nematzadeh Azadeh","year":"2018","unstructured":"Azadeh Nematzadeh , Giovanni Luca Ciampaglia , Filippo Menczer, and Alessandro Flammini. 2018 . How algorithmic popularity bias hinders or promotes quality.Scientific Reports 8, 1 (Dec. 2018), 15951. https:\/\/doi.org\/10.1038\/s41598-018--34203--2 arXiv: 1707.00574. 10.1038\/s41598-018--34203--2 Azadeh Nematzadeh, Giovanni Luca Ciampaglia, Filippo Menczer, and Alessandro Flammini. 2018. How algorithmic popularity bias hinders or promotes quality.Scientific Reports8, 1 (Dec. 2018), 15951. https:\/\/doi.org\/10.1038\/s41598-018--34203--2 arXiv: 1707.00574."},{"key":"e_1_2_1_49_1","unstructured":"Gagan Bansal Besmira Nushi and Ece Kamar. [n.d.]. Updates in Human-AI Teams: Understanding and Addressing the Performance\/Compatibility Tradeoff. ([n. d.]) 9.  Gagan Bansal Besmira Nushi and Ece Kamar. [n.d.]. Updates in Human-AI Teams: Understanding and Addressing the Performance\/Compatibility Tradeoff. ([n. d.]) 9."},{"key":"#cr-split#-e_1_2_1_50_1.1","doi-asserted-by":"crossref","unstructured":"Lace Padilla Matthew Kay and Jessica Hullman. 2020.Uncertainty Visualization. preprint. PsyArXiv. https:\/\/doi.org\/10.31234\/osf.io\/ebd6r 10.31234\/osf.io","DOI":"10.31234\/osf.io\/ebd6r"},{"key":"#cr-split#-e_1_2_1_50_1.2","doi-asserted-by":"crossref","unstructured":"Lace Padilla Matthew Kay and Jessica Hullman. 2020.Uncertainty Visualization. preprint. PsyArXiv. https:\/\/doi.org\/10.31234\/osf.io\/ebd6r","DOI":"10.31234\/osf.io\/ebd6r"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287567"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274405"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357160"},{"key":"e_1_2_1_54_1","volume-title":"Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes.","author":"Raji Inioluwa Deborah","year":"2020","unstructured":"Inioluwa Deborah Raji , Andrew Smart , Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020 . Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing .arXiv:2001.00973 [cs] (Jan. 2020). http:\/\/arxiv.org\/abs\/2001.00973 arXiv:2001.00973. Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing.arXiv:2001.00973 [cs] (Jan. 2020). http:\/\/arxiv.org\/abs\/2001.00973 arXiv:2001.00973."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598828"},{"key":"e_1_2_1_56_1","volume-title":"Tackling Climate Change with Machine Learning.arXiv:1906.05433[cs, stat](June","author":"Rolnick David","year":"2019","unstructured":"David Rolnick , Priya L. Donti , Lynn H. Kaack , Kelly Kochanski , Alexandre Lacoste , Kris Sankaran , Andrew SlavinRoss , Nikola Milojevic-Dupont , Natasha Jaques , Anna Waldman-Brown , Alexandra Luccioni , Tegan Maharaj , Evan D. Sherwin , S. Karthik Mukkavilli , Konrad P. Kording , Carla Gomes , Andrew Y. Ng , Demis Hassabis , John C. Platt , FelixCreutzig, Jennifer Chayes , and Yoshua Bengio . 2019. Tackling Climate Change with Machine Learning.arXiv:1906.05433[cs, stat](June 2019 ). http:\/\/arxiv.org\/abs\/1906.05433 arXiv: 1906.05433. David Rolnick, Priya L. Donti, Lynn H. Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew SlavinRoss, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, Alexandra Luccioni, Tegan Maharaj, Evan D.Sherwin, S. Karthik Mukkavilli, Konrad P. Kording, Carla Gomes, Andrew Y. Ng, Demis Hassabis, John C. Platt, FelixCreutzig, Jennifer Chayes, and Yoshua Bengio. 2019. Tackling Climate Change with Machine Learning.arXiv:1906.05433[cs, stat](June 2019). http:\/\/arxiv.org\/abs\/1906.05433 arXiv: 1906.05433."},{"key":"e_1_2_1_57_1","unstructured":"Marck Harlan Harris Vaisman Sean Murphy. [n.d.]. Analyzing the Analyzers - O'Reilly Media. https:\/\/www.oreilly.com\/data\/free\/analyzing-the-analyzers.csp Library Catalog: www.oreilly.com.  Marck Harlan Harris Vaisman Sean Murphy. [n.d.]. Analyzing the Analyzers - O'Reilly Media. https:\/\/www.oreilly.com\/data\/free\/analyzing-the-analyzers.csp Library Catalog: www.oreilly.com."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359141"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300911"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372833"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196730"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196729"},{"key":"e_1_2_1_63_1","volume-title":"Workflows,and Tools. arXiv:2001.06684 [cs, stat](Jan.","author":"Zhang Amy X.","year":"2020","unstructured":"Amy X. Zhang , Michael Muller , and Dakuo Wang . 2020. How do Data Science Workers Collaborate? Roles , Workflows,and Tools. arXiv:2001.06684 [cs, stat](Jan. 2020 ). http:\/\/arxiv.org\/abs\/2001.06684 arXiv: 2001.06684. Amy X. Zhang, Michael Muller, and Dakuo Wang. 2020. How do Data Science Workers Collaborate? Roles, Workflows,and Tools. arXiv:2001.06684 [cs, stat](Jan. 2020). http:\/\/arxiv.org\/abs\/2001.06684 arXiv: 2001.06684."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3463934","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3463934","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:15Z","timestamp":1750191135000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3463934"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,8]]},"references-count":64,"journal-issue":{"issue":"GROUP","published-print":{"date-parts":[[2021,7,8]]}},"alternative-id":["10.1145\/3463934"],"URL":"https:\/\/doi.org\/10.1145\/3463934","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,8]]},"assertion":[{"value":"2021-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}