{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T05:36:12Z","timestamp":1777008972446,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,10,6]],"date-time":"2022-10-06T00:00:00Z","timestamp":1665014400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,6]]},"DOI":"10.1145\/3551624.3555301","type":"proceedings-article","created":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T11:16:15Z","timestamp":1666005375000},"page":"1-8","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Understanding Implementation Challenges in Machine Learning Documentation"],"prefix":"10.1145","author":[{"given":"Jiyoo","family":"Chang","sequence":"first","affiliation":[{"name":"Partnership on AI, USA"}]},{"given":"Christine","family":"Custis","sequence":"additional","affiliation":[{"name":"Partnership on AI, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney.","author":"Arnold Matthew","year":"2019","unstructured":"Matthew Arnold , Rachel K. E. Bellamy , Michael Hind , Stephanie Houde , Sameep Mehta , Aleksandra Mojsilovic , Ravi Nair , Karthikeyan Natesan Ramamurthy , Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney. 2019 . FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity . arXiv:1808.07261 [cs] (Feb. 2019). http:\/\/arxiv.org\/abs\/1808.07261 arXiv: 1808.07261. Matthew Arnold, Rachel K. E. Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, and Kush R. Varshney. 2019. FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity. arXiv:1808.07261 [cs] (Feb. 2019). http:\/\/arxiv.org\/abs\/1808.07261 arXiv: 1808.07261."},{"key":"e_1_3_2_1_2_1","volume-title":"A Retrospective Datasheet for BookCorpus. arXiv:2105.05241 [cs] (May","author":"Bandy Jack","year":"2021","unstructured":"Jack Bandy and Nicholas Vincent . 2021. Addressing \" Documentation Debt\" in Machine Learning Research : A Retrospective Datasheet for BookCorpus. arXiv:2105.05241 [cs] (May 2021 ). http:\/\/arxiv.org\/abs\/2105.05241 arXiv: 2105.05241. Jack Bandy and Nicholas Vincent. 2021. Addressing \"Documentation Debt\" in Machine Learning Research: A Retrospective Datasheet for BookCorpus. arXiv:2105.05241 [cs] (May 2021). http:\/\/arxiv.org\/abs\/2105.05241 arXiv: 2105.05241."},{"key":"e_1_3_2_1_3_1","volume-title":"Bender and Batya Friedman","author":"Emily","year":"2018","unstructured":"Emily M. Bender and Batya Friedman . 2018 . Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics 6 (Dec. 2018), 587\u2013604. https:\/\/doi.org\/10.1162\/tacl_a_00041 10.1162\/tacl_a_00041 Emily M. Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics 6 (Dec. 2018), 587\u2013604. https:\/\/doi.org\/10.1162\/tacl_a_00041"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479582"},{"key":"e_1_3_2_1_5_1","unstructured":"Kasia S Chmielinski Sarah Newman Matt Taylor Josh Joseph Kemi Thomas Jessica Yurkofsky and Yue Chelsea Qiu. 2020. The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence. (2020) 7.  Kasia S Chmielinski Sarah Newman Matt Taylor Josh Joseph Kemi Thomas Jessica Yurkofsky and Yue Chelsea Qiu. 2020. The Dataset Nutrition Label (2nd Gen): Leveraging Context to Mitigate Harms in Artificial Intelligence. (2020) 7."},{"key":"e_1_3_2_1_6_1","volume-title":"Question Answering in Context. arXiv:1808.07036 [cs] (Aug","author":"Choi Eunsol","year":"2018","unstructured":"Eunsol Choi , He He , Mohit Iyyer , Mark Yatskar , Wen-tau Yih, Yejin Choi , Percy Liang , and Luke Zettlemoyer . 2018. QuAC : Question Answering in Context. arXiv:1808.07036 [cs] (Aug . 2018 ). http:\/\/arxiv.org\/abs\/1808.07036 arXiv: 1808.07036. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC : Question Answering in Context. arXiv:1808.07036 [cs] (Aug. 2018). http:\/\/arxiv.org\/abs\/1808.07036 arXiv: 1808.07036."},{"key":"e_1_3_2_1_7_1","unstructured":"European Commission. 2020. Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment. (2020).  European Commission. 2020. Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment. (2020)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3299057"},{"key":"e_1_3_2_1_9_1","volume-title":"Responsible Autonomy. arXiv:1706.02513 [cs] (June","author":"Dignum Virginia","year":"2017","unstructured":"Virginia Dignum . 2017. Responsible Autonomy. arXiv:1706.02513 [cs] (June 2017 ). http:\/\/arxiv.org\/abs\/1706.02513 arXiv: 1706.02513. Virginia Dignum. 2017. Responsible Autonomy. arXiv:1706.02513 [cs] (June 2017). http:\/\/arxiv.org\/abs\/1706.02513 arXiv: 1706.02513."},{"key":"e_1_3_2_1_10_1","volume-title":"Smith","author":"Dodge Jesse","year":"2019","unstructured":"Jesse Dodge , Suchin Gururangan , Dallas Card , Roy Schwartz , and Noah A . Smith . 2019 . Show Your Work: Improved Reporting of Experimental Results . arXiv:1909.03004 [cs, stat] (Sept. 2019). http:\/\/arxiv.org\/abs\/1909.03004 arXiv: 1909.03004. Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, and Noah A. Smith. 2019. Show Your Work: Improved Reporting of Experimental Results. arXiv:1909.03004 [cs, stat] (Sept. 2019). http:\/\/arxiv.org\/abs\/1909.03004 arXiv: 1909.03004."},{"key":"e_1_3_2_1_11_1","volume-title":"Structured dataset documentation: a datasheet for CheXpert. arXiv:2105.03020 [cs, eess] (May","author":"Garbin Christian","year":"2021","unstructured":"Christian Garbin , Pranav Rajpurkar , Jeremy Irvin , Matthew P. Lungren , and Oge Marques . 2021. Structured dataset documentation: a datasheet for CheXpert. arXiv:2105.03020 [cs, eess] (May 2021 ). http:\/\/arxiv.org\/abs\/2105.03020 arXiv: 2105.03020. Christian Garbin, Pranav Rajpurkar, Jeremy Irvin, Matthew P. Lungren, and Oge Marques. 2021. Structured dataset documentation: a datasheet for CheXpert. arXiv:2105.03020 [cs, eess] (May 2021). http:\/\/arxiv.org\/abs\/2105.03020 arXiv: 2105.03020."},{"key":"e_1_3_2_1_12_1","volume-title":"Hanna Wallach, Hal Daum\u00e9 III, and Kate Crawford.","author":"Gebru Timnit","year":"2020","unstructured":"Timnit Gebru , Jamie Morgenstern , Briana Vecchione , Jennifer Wortman Vaughan , Hanna Wallach, Hal Daum\u00e9 III, and Kate Crawford. 2020 . Datasheets for Datasets . arXiv:1803.09010 [cs] (March 2020). http:\/\/arxiv.org\/abs\/1803.09010 arXiv: 1803.09010. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daum\u00e9 III, and Kate Crawford. 2020. Datasheets for Datasets. arXiv:1803.09010 [cs] (March 2020). http:\/\/arxiv.org\/abs\/1803.09010 arXiv: 1803.09010."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372862"},{"key":"e_1_3_2_1_14_1","volume-title":"Varshney","author":"Hind Michael","year":"2019","unstructured":"Michael Hind , Stephanie Houde , Jacquelyn Martino , Aleksandra Mojsilovic , David Piorkowski , John Richards , and Kush R . Varshney . 2019 . Experiences with Improving the Transparency of AI Models and Services . arXiv:1911.08293 [cs] (Nov. 2019). http:\/\/arxiv.org\/abs\/1911.08293 arXiv: 1911.08293. Michael Hind, Stephanie Houde, Jacquelyn Martino, Aleksandra Mojsilovic, David Piorkowski, John Richards, and Kush R. Varshney. 2019. Experiences with Improving the Transparency of AI Models and Services. arXiv:1911.08293 [cs] (Nov. 2019). http:\/\/arxiv.org\/abs\/1911.08293 arXiv: 1911.08293."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2021.100241"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462527"},{"key":"e_1_3_2_1_17_1","volume-title":"Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell.","author":"Hutchinson Ben","year":"2021","unstructured":"Ben Hutchinson , Andrew Smart , Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021 . Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure . arXiv:2010.13561 [cs] (Jan. 2021). http:\/\/arxiv.org\/abs\/2010.13561 arXiv: 2010.13561. Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021. Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure. arXiv:2010.13561 [cs] (Jan. 2021). http:\/\/arxiv.org\/abs\/2010.13561 arXiv: 2010.13561."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0088-2"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376445"},{"key":"e_1_3_2_1_20_1","volume-title":"Ranjit Singh, and Madeleine Clare Elish.","author":"Metcalf Jacob","year":"2021","unstructured":"Jacob Metcalf , Emanuel Moss , Elizabeth Anne Watkins , Ranjit Singh, and Madeleine Clare Elish. 2021 . Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts . (2021), 19. Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. 2021. Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts. (2021), 19."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445880"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0114-4"},{"key":"e_1_3_2_1_24_1","volume-title":"Towards evaluating and eliciting high-quality documentation for intelligent systems. arXiv:2011.08774 [cs] (Nov","author":"Piorkowski David","year":"2020","unstructured":"David Piorkowski , Daniel Gonz\u00e1lez , John Richards , and Stephanie Houde . 2020. Towards evaluating and eliciting high-quality documentation for intelligent systems. arXiv:2011.08774 [cs] (Nov . 2020 ). http:\/\/arxiv.org\/abs\/2011.08774 arXiv: 2011.08774. David Piorkowski, Daniel Gonz\u00e1lez, John Richards, and Stephanie Houde. 2020. Towards evaluating and eliciting high-quality documentation for intelligent systems. arXiv:2011.08774 [cs] (Nov. 2020). http:\/\/arxiv.org\/abs\/2011.08774 arXiv: 2011.08774."},{"key":"e_1_3_2_1_25_1","volume-title":"Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes.","author":"Raji Inioluwa Deborah","year":"2020","unstructured":"Inioluwa Deborah Raji , Andrew Smart , Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020 . Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing . arXiv:2001.00973 [cs] (Jan. 2020). http:\/\/arxiv.org\/abs\/2001.00973 arXiv: 2001.00973. Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. arXiv:2001.00973 [cs] (Jan. 2020). http:\/\/arxiv.org\/abs\/2001.00973 arXiv: 2001.00973."},{"key":"e_1_3_2_1_26_1","volume-title":"ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles. arXiv:1912.06166 [cs, stat] (Jan.","author":"Raji Inioluwa Deborah","year":"2020","unstructured":"Inioluwa Deborah Raji and Jingying Yang . 2020 . ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles. arXiv:1912.06166 [cs, stat] (Jan. 2020). http:\/\/arxiv.org\/abs\/1912.06166 arXiv: 1912.06166. Inioluwa Deborah Raji and Jingying Yang. 2020. ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles. arXiv:1912.06166 [cs, stat] (Jan. 2020). http:\/\/arxiv.org\/abs\/1912.06166 arXiv: 1912.06166."},{"key":"e_1_3_2_1_27_1","volume-title":"Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices. arXiv:2006.12358 [cs] (March","author":"Rakova Bogdana","year":"2021","unstructured":"Bogdana Rakova , Jingying Yang , Henriette Cramer , and Rumman Chowdhury . 2021. Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices. arXiv:2006.12358 [cs] (March 2021 ). https:\/\/doi.org\/10.1145\/3449081 arXiv: 2006.12358. 10.1145\/3449081 Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices. arXiv:2006.12358 [cs] (March 2021). https:\/\/doi.org\/10.1145\/3449081 arXiv: 2006.12358."},{"key":"e_1_3_2_1_28_1","volume-title":"Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1\u201312","author":"Rule Adam","unstructured":"Adam Rule , Aur\u00e9lien Tabard , and James D. Hollan . 2018. Exploration and Explanation in Computational Notebooks . In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1\u201312 . https:\/\/doi.org\/10.1145\/3173574.3173606 10.1145\/3173574.3173606 Adam Rule, Aur\u00e9lien Tabard, and James D. Hollan. 2018. Exploration and Explanation in Computational Notebooks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1\u201312. https:\/\/doi.org\/10.1145\/3173574.3173606"},{"key":"e_1_3_2_1_29_1","volume-title":"4","author":"Sculley D","year":"2018","unstructured":"D Sculley , Jasper Snoek , Ali Rahimi , and Alex Wiltschko . 2018. ON PACE, PROGRESS, AND EMPIRICAL RIGOR. ( 2018 ), 4 . D Sculley, Jasper Snoek, Ali Rahimi, and Alex Wiltschko. 2018. ON PACE, PROGRESS, AND EMPIRICAL RIGOR. (2018), 4."},{"key":"e_1_3_2_1_30_1","volume-title":"Baselines and a datasheet for the Cerema AWP dataset. arXiv:1806.04016 [cs, stat] (June","author":"Seck Isma\u00efla","year":"2018","unstructured":"Isma\u00efla Seck , Khouloud Dahmane , Pierre Duthon , and Ga\u00eblle Loosli . 2018. Baselines and a datasheet for the Cerema AWP dataset. arXiv:1806.04016 [cs, stat] (June 2018 ). http:\/\/arxiv.org\/abs\/1806.04016 arXiv: 1806.04016. Isma\u00efla Seck, Khouloud Dahmane, Pierre Duthon, and Ga\u00eblle Loosli. 2018. Baselines and a datasheet for the Cerema AWP dataset. arXiv:1806.04016 [cs, stat] (June 2018). http:\/\/arxiv.org\/abs\/1806.04016 arXiv: 1806.04016."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1049023X21000649"},{"key":"e_1_3_2_1_32_1","volume-title":"How do Data Science Workers Collaborate? Roles, Workflows, and Tools. arXiv:2001.06684 [cs, stat] (April","author":"Zhang Amy X.","year":"2020","unstructured":"Amy X. Zhang , Michael Muller , and Dakuo Wang . 2020. How do Data Science Workers Collaborate? Roles, Workflows, and Tools. arXiv:2001.06684 [cs, stat] (April 2020 ). http:\/\/arxiv.org\/abs\/2001.06684 arXiv: 2001.06684. Amy X. Zhang, Michael Muller, and Dakuo Wang. 2020. How do Data Science Workers Collaborate? Roles, Workflows, and Tools. arXiv:2001.06684 [cs, stat] (April 2020). http:\/\/arxiv.org\/abs\/2001.06684 arXiv: 2001.06684."}],"event":{"name":"EAAMO '22: Equity and Access in Algorithms, Mechanisms, and Optimization","location":"Arlington VA USA","acronym":"EAAMO '22","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","SIGecom Special Interest Group on Economics and Computation"]},"container-title":["Equity and Access in Algorithms, Mechanisms, and Optimization"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551624.3555301","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3551624.3555301","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:25Z","timestamp":1750186825000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3551624.3555301"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,6]]},"references-count":32,"alternative-id":["10.1145\/3551624.3555301","10.1145\/3551624"],"URL":"https:\/\/doi.org\/10.1145\/3551624.3555301","relation":{},"subject":[],"published":{"date-parts":[[2022,10,6]]},"assertion":[{"value":"2022-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}