{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T12:48:32Z","timestamp":1786193312223,"version":"3.56.0"},"reference-count":100,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T00:00:00Z","timestamp":1646870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Manage. Inf. Syst."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>Data provenance, a record that describes the origins and processing of data, offers new promises in the increasingly important role of artificial intelligence (AI)-based systems in guiding human decision making. To avoid disastrous outcomes that can result from bias-laden AI systems, responsible AI builds on four important characteristics: fairness, accountability, transparency, and explainability. To stimulate further research on data provenance that enables responsible AI, this study outlines existing biases and discusses possible implementations of data provenance to mitigate them. We first review biases stemming from the data's origins and pre-processing. We then discuss the current state of practice, the challenges it presents, and corresponding recommendations to address them. We present a summary highlighting how our recommendations can help establish data provenance and thereby mitigate biases stemming from the data's origins and pre-processing to realize responsible AI-based systems. We conclude with a research agenda suggesting further research avenues.<\/jats:p>","DOI":"10.1145\/3503488","type":"journal-article","created":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T14:05:52Z","timestamp":1646921152000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":88,"title":["Establishing Data Provenance for Responsible Artificial Intelligence Systems"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8481-1596","authenticated-orcid":false,"given":"Karl","family":"Werder","sequence":"first","affiliation":[{"name":"Cologne Institute for Information Systems, University of Cologne, Albertus-Magnus-Platz, K\u00f6ln, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9629-328X","authenticated-orcid":false,"given":"Balasubramaniam","family":"Ramesh","sequence":"additional","affiliation":[{"name":"Computer Information Systems, Georgia State University, Atlanta, GA, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2883-1546","authenticated-orcid":false,"given":"Rongen (Sophia)","family":"Zhang","sequence":"additional","affiliation":[{"name":"Computer Information Systems, Georgia State University, Atlanta, GA, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,3,10]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"ACM U.S. Technology Policy Committee. 2020. Statement on principles and prerequisites for the development evaluation and use of unbiased facial recognition technologies. Retrieved August 24 2021 from https:\/\/www.acm.org\/binaries\/content\/assets\/public-policy\/ustpc-facial-recognition-tech-statement.pdf."},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"issue":"4","key":"e_1_3_1_4_2","first-page":"18","article-title":"Reducing recommender systems biases: An investigation of rating display designs","volume":"43","author":"Adomavicius Gediminas","year":"2019","unstructured":"Gediminas Adomavicius, Jesse Bockstedt, Shawn Curley, and Jingjng Zhang. 2019. Reducing recommender systems biases: An investigation of rating display designs. MIS Quarterly 43, 4 (February 2019), 18\u201319.","journal-title":"MIS Quarterly"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3446944"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacr.2018.09.050"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/11890850_14"},{"key":"e_1_3_1_8_2","article-title":"How India's data labellers are powering the global AI race","author":"Murali Anand","year":"2019","unstructured":"Anand Murali. 2019. How India's data labellers are powering the global AI race. FactorDaily. Retrieved August 24, 2021 from https:\/\/archive.factordaily.com\/indian-data-labellers-powering-the-global-ai-race\/.","journal-title":"FactorDaily"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0105-1"},{"key":"e_1_3_1_10_2","unstructured":"Khalid Belhajjame Reza B'Far James Cheney Sam Coppens Stephen Cresswell Yolanda Gil Paul Groth Graham Klyne Timothy Lebo Jim McCusker Simon Miles James Myers Satya Sahoo and Curt Tilmes. 2013. PROV-DM: The PROV Data Model. (2013)."},{"key":"e_1_3_1_11_2","unstructured":"Tellen Bennett Seth Russell James King Lisa Schilling Chan Voong Nancy Rogers Bonnie Adrian Nicholas Bruce and Debashis Ghosh. 2019. Accuracy of the epic sepsis prediction model in a regional health system. arXiv:1902.07276. Retrieved September 5 2021 from http:\/\/arxiv.org\/abs\/1902.07276."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3188721"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2669368"},{"key":"e_1_3_1_14_2","unstructured":"Peter Buneman and Susan B. Davidson. Data provenance\u2014the foundation of data quality. 8."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44503-X_20"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3430368"},{"key":"e_1_3_1_17_2","unstructured":"FDA 2020 Classify your medical device"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1561\/1900000006"},{"key":"e_1_3_1_19_2","unstructured":"Medium 2020 Abolish the #TechToPrisonPipeline"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.2196\/16323"},{"key":"e_1_3_1_21_2","unstructured":"Alexander D'Amour Katherine Heller Dan Moldovan Ben Adlam Babak Alipanahi Alex Beutel Christina Chen Jonathan Deaton Jacob Eisenstein Matthew D. Hoffman Farhad Hormozdiari Neil Houlsby Shaobo Hou Ghassen Jerfel Alan Karthikesalingam Mario Lucic Yian Ma Cory McLean Diana Mincu Akinori Mitani Andrea Montanari Zachary Nado Vivek Natarajan Christopher Nielson Thomas F. Osborne Rajiv Raman Kim Ramasamy Rory Sayres Jessica Schrouff Martin Seneviratne Shannon Sequeira Harini Suresh Victor Veitch Max Vladymyrov Xuezhi Wang Kellie Webster Steve Yadlowsky Taedong Yun Xiaohua Zhai and D. Sculley. 2020. Underspecification presents challenges for credibility in modern machine learning. arXiv:2011.03395 (November 2020). Retrieved August 24 2021 from http:\/\/arxiv.org\/abs\/2011.03395."},{"key":"e_1_3_1_22_2","first-page":"9","article-title":"Responsible artificial intelligence: Designing AI for human values","volume":"1","author":"Dignum Virginia","year":"2017","unstructured":"Virginia Dignum. 2017. Responsible artificial intelligence: Designing AI for human values. ITU Journal 1 (2017), 9.","journal-title":"ITU Journal"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359786"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3392496"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2020.100016"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3518482"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISBI.2018.8363576"},{"key":"e_1_3_1_28_2","first-page":"1","volume-title":"Proceedings of Machine Learning Research","author":"Friedler Sorelle A.","year":"2018","unstructured":"Sorelle A. Friedler and Christo Wilson. 2018. Conference on fairness, accountability and transparency. In Proceedings of Machine Learning Research. 1\u20132. Retrieved August 24, 2021 from https:\/\/proceedings.mlr.press\/v81\/friedler18a.html."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1287\/educ.2020.0215"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-18918-3"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2015.12.006"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458723"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-00257-z"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.5465\/amj.2014.4002"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3314117"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2018.3763"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2015.02.005"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/564946.564949"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aay7120"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2019.11.146"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0307-0"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222250111"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1312"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.giq.2020.101493"},{"key":"e_1_3_1_46_2","unstructured":"Matthias Jarke and Yannis Vassiliou. Data Warehouse Quality: A Review of the DWQ Project. 14."},{"key":"e_1_3_1_47_2","article-title":"Amazon scraps secret AI recruiting tool that showed bias against women","author":"Dastin Jeffrey","year":"2018","unstructured":"Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. Retrieved August 25, 2021 from https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G.","journal-title":"Reuters"},{"key":"e_1_3_1_48_2","article-title":"AI recruiting tools aim to reduce bias in the hiring process","author":"Hsu Jeremy","year":"2020","unstructured":"Jeremy Hsu. 2020. AI recruiting tools aim to reduce bias in the hiring process. IEEE Spectrum. Retrieved August 25, 2021 from https:\/\/spectrum.ieee.org\/ai-tools-bias-hiring.","journal-title":"IEEE Spectrum"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0088-2"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10551-014-2180-1"},{"key":"e_1_3_1_51_2","article-title":"This hot startup is now valued at $1 billion for its A.I. skills","author":"Vanian Jonathan","year":"2021","unstructured":"Jonathan Vanian. 2021. This hot startup is now valued at $1 billion for its A.I. skills. Fortune. Retrieved September 9, 2021 from https:\/\/fortune.com\/2021\/08\/09\/snorkel-ai-funding-data-labeling-startup\/.","journal-title":"Fortune"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1929916.1929920"},{"key":"e_1_3_1_53_2","unstructured":"Katherine Anne Long. New Amazon Data Shows Black Latino and Female Employees are Underrepresented in Best-paid Jobs. The Seattle Times. Retrieved August 25 2021 from https:\/\/www.seattletimes.com\/business\/amazon\/new-amazon-data-shows-black-latino-and-female-employees-are-underrepresented-in-best-paid-jobs\/."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-019-1426-2"},{"key":"e_1_3_1_55_2","article-title":"We need more women in tech: The data prove it","author":"Khanna Derek","year":"2013","unstructured":"Derek Khanna. 2013. We need more women in tech: The data prove it. The Atlantic. Retrieved September 5, 2021 from https:\/\/www.theatlantic.com\/technology\/archive\/2013\/10\/we-need-more-women-in-tech-the-data-prove-it\/280964\/.","journal-title":"The Atlantic"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.07.014"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328526.3329621"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118266588"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314229"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2696365"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCGRID.2017.8"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.2196\/16607"},{"key":"e_1_3_1_63_2","unstructured":"Thomas Macaulay. 2020. Flawed algorithm used to determine U.K. welfare payments Is \u201cPushing People Into Poverty.\u201d (2020). Retrieved from https:\/\/thenextweb.com\/neural\/2020\/09\/29\/flawed-algorithm-used-to-determine-uk-welfare-payments-is-pushing-people-into-poverty\/http:\/\/cacm.acm.org\/news\/247831-flawed-algorithm-used-to-determine-u-k-welfare-payments-is-pushing-people-into-poverty."},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-004-3422-1"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2020.0363"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445887"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1214\/18-AOAS1161SF"},{"key":"e_1_3_1_68_2","article-title":"Sepsis Accounts for 1 in 5 Deaths, Leading Cause of Death in Hospitals","author":"Ksiazek Michael C.","unstructured":"Michael C. Ksiazek. Sepsis Accounts for 1 in 5 Deaths, Leading Cause of Death in Hospitals. The National Law Review. Retrieved September 5, 2021 from https:\/\/www.natlawreview.com\/article\/sepsis-accounts-1-5-deaths-leading-cause-death-hospitals.","journal-title":"The National Law Review"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.5555\/1267359.1267363"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1866835.1866845"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-0372-6_38"},{"key":"e_1_3_1_72_2","unstructured":"B. Nestor Matthew B. A. McDermott Geeticka Chauhan Tristan Naumann Michael C. Hughes A. Goldenberg and M. Ghassemi. 2018. Rethinking clinical prediction: Why machine learning must consider year of care and feature aggregation. ArXiv:1811.12583. Retrieved from https:\/\/arxiv.org\/abs\/1811.12583."},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1038\/506150a"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33019780"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.2514\/6.2018-2482"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.895989"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056442"},{"key":"e_1_3_1_78_2","article-title":"Hand labeling is the past. The future is #NoLabel AI","author":"Jurney Russell","unstructured":"Russell Jurney. Hand labeling is the past. The future is #NoLabel AI. KDnuggets. Retrieved September 5, 2021 from https:\/\/www.kdnuggets.com\/hand-labeling-is-the-past-the-future-is-nolabel-ai.html\/.","journal-title":"KDnuggets"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3230713"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-83555-1_5"},{"key":"e_1_3_1_81_2","doi-asserted-by":"publisher","DOI":"10.1080\/12460125.2020.1819094"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.2196\/13659"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2020.102551"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2019.04.019"},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1002\/bs.3830190603"},{"key":"e_1_3_1_86_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.07.008"},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2010-2064"},{"key":"e_1_3_1_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3027314"},{"key":"e_1_3_1_89_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0300-7"},{"issue":"2","key":"e_1_3_1_90_2","first-page":"56","article-title":"An analysis of multi-criteria decision making methods","volume":"10","author":"Velasquez Mark","year":"2013","unstructured":"Mark Velasquez and Patrick T. Hester. 2013. An analysis of multi-criteria decision making methods. International Journal of Operations Research 10, 2 (2013), 56\u201366.","journal-title":"International Journal of Operations Research"},{"key":"e_1_3_1_91_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aan7059"},{"key":"e_1_3_1_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01155"},{"key":"e_1_3_1_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372833"},{"key":"e_1_3_1_94_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamapediatrics.2013.186"},{"key":"e_1_3_1_95_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2021.2626"},{"key":"e_1_3_1_96_2","unstructured":"Kyra Yee Uthaipon Tantipongpipat and Shubhanshu Mishra. 2021. Image cropping on twitter: Fairness metrics their limitations and the importance of representation design and agency. arXiv:2105.08667. Retrieved August 25 2021 from http:\/\/arxiv.org\/abs\/2105.08667."},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.15.2.148.14449"},{"key":"e_1_3_1_98_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72188-6_2"},{"key":"e_1_3_1_99_2","unstructured":"2018. What is GDPR the EU's new data protection law? GDPR.eu. Retrieved August 25 2021 from https:\/\/gdpr.eu\/what-is-gdpr\/."},{"key":"e_1_3_1_100_2","unstructured":"Shining a light on dark data. Accenture. Retrieved August 25 2021 from https:\/\/www.accenture.com\/us-en\/insights\/financial-services\/technology-advisory-dark-data."},{"key":"e_1_3_1_101_2","unstructured":"Reducing Sepsis Mortality by One-Fifth with Epic. Retrieved September 11 2021 from https:\/\/www.epic.com\/epic\/post\/reducing-sepsis-mortality-epic."}],"container-title":["ACM Transactions on Management Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503488","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3503488","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:18Z","timestamp":1750186938000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503488"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,10]]},"references-count":100,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3503488"],"URL":"https:\/\/doi.org\/10.1145\/3503488","relation":{},"ISSN":["2158-656X","2158-6578"],"issn-type":[{"value":"2158-656X","type":"print"},{"value":"2158-6578","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,10]]},"assertion":[{"value":"2021-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}