{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T16:33:31Z","timestamp":1780331611751,"version":"3.54.1"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>The use of real-world data (RWD) in artificial intelligence (AI) applications for healthcare offers unique opportunities but also poses complex challenges related to interpretability, transparency, safety, efficacy, bias, equity, privacy, ethics, accountability, and stakeholder engagement.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A multi-stakeholder expert panel comprising healthcare professionals, AI developers, policymakers, and other stakeholders was assembled. Their task was to identify critical issues and formulate consensus recommendations, focusing on the responsible use of RWD in healthcare AI. The panel\u2019s work involved an in-person conference and workshop and extensive deliberations over several months.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The panel\u2019s findings revealed several critical challenges, including the necessity for data literacy and documentation, the identification and mitigation of bias, privacy and ethics considerations, and the absence of an accountability structure for stakeholder management. To address these, the panel proposed a series of recommendations, such as the adoption of metadata standards for RWD sources, the development of transparency frameworks and instructional labels likened to \u201cnutrition labels\u201d for AI applications, the provision of cross-disciplinary training materials, the implementation of bias detection and mitigation strategies, and the establishment of ongoing monitoring and update processes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Guidelines and resources focused on the responsible use of RWD in healthcare AI are essential for developing safe, effective, equitable, and trustworthy applications. The proposed recommendations provide a foundation for a comprehensive framework addressing the entire lifecycle of healthcare AI, emphasizing the importance of documentation, training, transparency, accountability, and multi-stakeholder engagement.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf133","type":"journal-article","created":{"date-parts":[[2025,7,26]],"date-time":"2025-07-26T11:48:55Z","timestamp":1753530535000},"page":"1746-1755","source":"Crossref","is-referenced-by-count":7,"title":["Towards responsible artificial intelligence in healthcare\u2014getting real about real-world data and evidence"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3621-9613","authenticated-orcid":false,"given":"Eileen","family":"Koski","sequence":"first","affiliation":[{"name":"IBM Research , Yorktown Heights, NY 10598,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3556-0844","authenticated-orcid":false,"given":"Amar","family":"Das","sequence":"additional","affiliation":[{"name":"Real World Evidence, Guardant Health , Palo Alto, CA 94304,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0737-4983","authenticated-orcid":false,"given":"Pei-Yun Sabrina","family":"Hsueh","sequence":"additional","affiliation":[{"name":"Department of Ethical AI and External Innovation, Pfizer Inc. , New York, NY 10001,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2117-2461","authenticated-orcid":false,"given":"Anthony","family":"Solomonides","sequence":"additional","affiliation":[{"name":"Research Institute, Endeavor Health , Evanston, IL 60201,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5869-037X","authenticated-orcid":false,"given":"Amanda L","family":"Joseph","sequence":"additional","affiliation":[{"name":"School of Health Information Science, University of Victoria , Victoria, BC V8P 5C2,","place":["Canada"]},{"name":"Homewood Research Institute , Guelph, ON N1E 6K9,","place":["Canada"]},{"name":"Division of Clinical Informatics, Beth Israel Deaconess Medical Center , Boston, MA 02215,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7414-8062","authenticated-orcid":false,"given":"Gyana","family":"Srivastava","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics, Beth Israel Deaconess Medical Center , Boston, MA 02215,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2462-0813","authenticated-orcid":false,"given":"Carl Erwin","family":"Johnson","sequence":"additional","affiliation":[{"name":"Outcomes Research, Merck & Co., Inc. , North Wales, PA 19454,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6089-1488","authenticated-orcid":false,"given":"Joseph","family":"Kannry","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Department of Medicine, Icahn School of Medicine at Mount Sinai , New York, NY 10029,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7349-3350","authenticated-orcid":false,"given":"Bilikis","family":"Oladimeji","sequence":"additional","affiliation":[{"name":"Clinical Technology Strategy, UnitedHealth Group , Eden Prairie, MN 55344,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5200-7322","authenticated-orcid":false,"given":"Amy","family":"Price","sequence":"additional","affiliation":[{"name":"The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth , Hanover, NH 03755,","place":["United States"]},{"name":"BMJ , London, WC1H 9JP,","place":["United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2337-5185","authenticated-orcid":false,"given":"Steven","family":"Labkoff","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics, Beth Israel Deaconess Medical Center , Boston, MA 02215,","place":["United States"]},{"name":"Quantori , Cambridge, MA 02142,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8384-9509","authenticated-orcid":false,"given":"Gnana","family":"Bharathy","sequence":"additional","affiliation":[{"name":"Faculty of Engineering and IT, University of Technology Sydney , Sydney, NSW 2007,","place":["Australia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7979-5509","authenticated-orcid":false,"given":"Baihan","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of AI, Hasso Plattner Institute for Digital Health, Icahn School of Medicine at Mount Sinai , New York, NY 10029-5674,","place":["United States"]},{"name":"Department of Psychiatry, Mount Sinai , New York, NY 10029,","place":["United States"]},{"name":"Department of Neuroscience, Icahn School of Medicine at Mount Sinai , New York, NY 10029,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1767-3264","authenticated-orcid":false,"given":"Douglas","family":"Fridsma","sequence":"additional","affiliation":[{"name":"Health Universe , San Francisco, CA 94114,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2899-2294","authenticated-orcid":false,"given":"Lee A","family":"Fleisher","sequence":"additional","affiliation":[{"name":"Anesthesiology and Critical Care, University of Pennsylvania Perelman School of Medicine , Philadelphia, PA 19104,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9153-6852","authenticated-orcid":false,"given":"Monica","family":"Lopez-Gonzalez","sequence":"additional","affiliation":[{"name":"Cognitive Insights for Artificial Intelligence , Baltimore, MD 21202,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1168-0983","authenticated-orcid":false,"given":"Reva","family":"Singh","sequence":"additional","affiliation":[{"name":"Public Policy, American Medical Informatics Association , Washington, DC 20011,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5586-9940","authenticated-orcid":false,"given":"Mark G","family":"Weiner","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences, Weill Cornell Medicine , New York, NY 10065,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0154-5948","authenticated-orcid":false,"given":"Robert","family":"Stolper","sequence":"additional","affiliation":[{"name":"Strategy Consulting, IQVIA , Boston, MA 02210,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7661-1412","authenticated-orcid":false,"given":"Russell","family":"Baris","sequence":"additional","affiliation":[{"name":"eLumindata , Westport, CT,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3584-4831","authenticated-orcid":false,"given":"Suzanne","family":"Sincavage","sequence":"additional","affiliation":[{"name":"Foundation for Biodefense Research , Fallbrook, CA,","place":["United States"]},{"name":"IDIQ Inc. , Fallbrook, CA 92028,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2150-1747","authenticated-orcid":false,"given":"Tristan","family":"Naumann","sequence":"additional","affiliation":[{"name":"Microsoft New England Research and Development Center, Microsoft Research , Cambridge, MA 02142,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-4482-6556","authenticated-orcid":false,"given":"Tayler","family":"Williams","sequence":"additional","affiliation":[{"name":"Public Policy, American Medical Informatics Association , Washington, DC 20011,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5283-8494","authenticated-orcid":false,"given":"Tien Thi Thuy","family":"Bui","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics, Beth Israel Deaconess Medical Center , Boston, MA 02215,","place":["United States"]},{"name":"Massachusetts College of Pharmacy and Health Sciences , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6583-0257","authenticated-orcid":false,"given":"Yuri","family":"Quintana","sequence":"additional","affiliation":[{"name":"Homewood Research Institute , Guelph, ON N1E 6K9,","place":["Canada"]},{"name":"Division of Clinical Informatics, Beth Israel Deaconess Medical Center , Boston, MA 02215,","place":["United States"]},{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,9,26]]},"reference":[{"key":"2025111813565159100_ocaf133-B1","author":"OECD AI Principles overview [Internet","year":"2024"},{"key":"2025111813565159100_ocaf133-B2","author":"FDA","year":"."},{"key":"2025111813565159100_ocaf133-B3","author":"FDA","year":"2024"},{"key":"2025111813565159100_ocaf133-B4","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1126\/science.aax2342","article-title":"Dissecting racial bias in an algorithm used to manage the health of populations","volume":"366","author":"Obermeyer","year":"2019","journal-title":"Science"},{"key":"2025111813565159100_ocaf133-B5","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"2025111813565159100_ocaf133-B6","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1016\/S0140-6736(13)61752-3","article-title":"The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective","volume":"383","author":"Mahmood","year":"2014","journal-title":"Lancet"},{"key":"2025111813565159100_ocaf133-B7","doi-asserted-by":"crossref","first-page":"e0132321","DOI":"10.1371\/journal.pone.0132321","article-title":"Race\/ethnic differences in the associations of the Framingham risk factors with carotid IMT and cardiovascular events","volume":"10","author":"Gijsberts","year":"2015","journal-title":"PLoS One"},{"key":"2025111813565159100_ocaf133-B8","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1093\/jamia\/ocac006","article-title":"Defining AMIA's artificial intelligence principles","volume":"29","author":"Solomonides","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"2025111813565159100_ocaf133-B9","author":"American Medical Informatics Association (AMIA)","year":"2024"},{"key":"2025111813565159100_ocaf133-B10","author":"DCI Network","year":"2023"},{"key":"2025111813565159100_ocaf133-B11","doi-asserted-by":"publisher","first-page":"2730","DOI":"10.1093\/jamia\/ocae209","article-title":"Toward a responsible future: recommendations for AI-enabled clinical decision support","volume":"31","author":"Labkoff","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2025111813565159100_ocaf133-B12","doi-asserted-by":"crossref","first-page":"160018","DOI":"10.1038\/sdata.2016.18","article-title":"The FAIR Guiding Principles for scientific data management and stewardship","volume":"3","author":"Wilkinson","year":"2016","journal-title":"Sci Data"},{"key":"2025111813565159100_ocaf133-B13","doi-asserted-by":"crossref","DOI":"10.6028\/NIST.SP.1270","volume-title":"Towards a Standard for Identifying and Managing Bias in Artificial Intelligence","author":"Schwartz Schwartz","year":"2022"},{"key":"2025111813565159100_ocaf133-B14","doi-asserted-by":"crossref","first-page":"2377","DOI":"10.1001\/jama.2019.18058","article-title":"Addressing bias in artificial intelligence in health care","volume":"322","author":"Parikh","year":"2019","journal-title":"JAMA"},{"key":"2025111813565159100_ocaf133-B15","doi-asserted-by":"publisher","first-page":"e2402267121","DOI":"10.1073\/pnas.2402267121","article-title":"Race adjustments in clinical algorithms can help correct for racial disparities in data quality","volume":"121","author":"Zink","year":"2024","journal-title":"Proc Natl Acad Sci USA"},{"key":"2025111813565159100_ocaf133-B16","doi-asserted-by":"publisher","first-page":"ooab045","DOI":"10.1093\/jamiaopen\/ooab045","article-title":"Illustrating potential effects of alternate control populations on real-world evidence-based statistical analyses","volume":"4","author":"Huang","year":"2021","journal-title":"JAMIA Open"},{"key":"2025111813565159100_ocaf133-B17","doi-asserted-by":"publisher","first-page":"ocae317","DOI":"10.1093\/jamia\/ocae317","article-title":"Objective study validity diagnostics: a framework requiring pre-specified, empirical verification to increase trust in the reliability of real-world evidence","volume":"32","author":"Conover","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"2025111813565159100_ocaf133-B18","author":"Wartella"},{"key":"2025111813565159100_ocaf133-B19","volume-title":"Front-of-Package Nutrition Rating Systems and Symbols: Phase I Report","author":"History of Nutrition Labeling","year":"2010"},{"key":"2025111813565159100_ocaf133-B20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-020-0253-3","article-title":"Presenting machine learning model information to clinical end users with model facts labels","volume":"3","author":"Sendak","year":"2020","journal-title":"NPJ Digital Med"},{"key":"2025111813565159100_ocaf133-B21","doi-asserted-by":"publisher","first-page":"105713","DOI":"10.1016\/j.ijmedinf.2024.105713","article-title":"Towards a Multi-Stakeholder process for developing responsible AI governance in consumer health","volume":"195","author":"Rozenblit","year":"2025","journal-title":"Int J Med Inform"},{"key":"2025111813565159100_ocaf133-B22","doi-asserted-by":"publisher","first-page":"1721","DOI":"10.1145\/2783258.2788613","volume-title":"Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '15)","author":"Caruana","year":"2015"},{"key":"2025111813565159100_ocaf133-B23","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/978-3-031-07912-2_15","volume-title":"Healthcare Information Management Systems: Cases, Strategies & Solutions","author":"Park","year":"2022","edition":"5th edn"},{"key":"2025111813565159100_ocaf133-B24","author":"Standardized Data: The OMOP Common Data Model","year":"2025"},{"key":"2025111813565159100_ocaf133-B25","author":"FDA","year":"2023"},{"key":"2025111813565159100_ocaf133-B26","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-30371-6","volume-title":"Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way","author":"Dignum","year":"2019"},{"key":"2025111813565159100_ocaf133-B27","doi-asserted-by":"publisher","first-page":"1680","DOI":"10.1056\/NEJMp2200089","article-title":"Real-world evidence\u2014where are we now?","volume":"386","author":"Concato","year":"2022","journal-title":"N Engl J Med"},{"key":"2025111813565159100_ocaf133-B28","volume-title":"The Learning Healthcare System: Workshop Summary","author":"Institute of medicine (IOM)","year":"2007"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/11\/1746\/64398142\/ocaf133.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/11\/1746\/64398142\/ocaf133.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T18:56:59Z","timestamp":1763492219000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/11\/1746\/8264332"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,26]]},"references-count":28,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,9,26]]},"published-print":{"date-parts":[[2025,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf133","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,11]]},"published":{"date-parts":[[2025,9,26]]}}}