{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T03:18:44Z","timestamp":1771298324037,"version":"3.50.1"},"reference-count":64,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2019,9,12]],"date-time":"2019-09-12T00:00:00Z","timestamp":1568246400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>We sought to demonstrate applicability of user stories, progressively elaborated by testable acceptance criteria, as lightweight requirements for agile development of clinical decision support (CDS).<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>User stories employed the template: As a [type of user], I want [some goal] so that [some reason]. From the \u201cso that\u201d section, CDS benefit measures were derived. Detailed acceptance criteria were elaborated through ensuing conversations. We estimated user story size with \u201cstory points,\u201d and depicted multiple user stories with a use case diagram or feature breakdown structure. Large user stories were split to fit into 2-week iterations.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>One example user story was: As a rheumatologist, I want to be advised if my patient with rheumatoid arthritis is not on a disease-modifying anti-rheumatic drug (DMARD), so that they receive optimal therapy and can experience symptom improvement. This yielded a process measure (DMARD use), and an outcome measure (Clinical Disease Activity Index). Following implementation, the DMARD nonuse rate decreased from 3.7% to 1.4%. Patients with a high Clinical Disease Activity Index improved from 13.7% to 7%. For a thromboembolism prevention CDS project, diagrams organized multiple user stories.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>User stories written in the clinician\u2019s voice aid CDS governance and lead naturally to measures of CDS effectiveness. Estimation of relative story size helps plan CDS delivery dates. User stories prove to be practical even on larger projects.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>User stories concisely communicate the who, what, and why of a CDS request, and serve as lightweight requirements for agile development to meet the demand for increasingly diverse CDS.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocz123","type":"journal-article","created":{"date-parts":[[2019,7,2]],"date-time":"2019-07-02T03:09:41Z","timestamp":1562036981000},"page":"1344-1354","source":"Crossref","is-referenced-by-count":28,"title":["User stories as lightweight requirements for agile clinical decision support development"],"prefix":"10.1093","volume":"26","author":[{"given":"Vaishnavi","family":"Kannan","sequence":"first","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"},{"name":"Health System Information Resources Department, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Mujeeb A","family":"Basit","sequence":"additional","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"},{"name":"Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Puneet","family":"Bajaj","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Angela R","family":"Carrington","sequence":"additional","affiliation":[{"name":"Health System Information Resources Department, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Irma B","family":"Donahue","sequence":"additional","affiliation":[{"name":"Health System Information Resources Department, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Emily L","family":"Flahaven","sequence":"additional","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"}]},{"given":"Richard","family":"Medford","sequence":"additional","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"},{"name":"Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Tsedey","family":"Melaku","sequence":"additional","affiliation":[{"name":"Clinical Informatics, Parkland Health and Hospital System, Dallas, Texas, USA"}]},{"given":"Brett A","family":"Moran","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA"},{"name":"Clinical Informatics, Parkland Health and Hospital System, Dallas, Texas, USA"}]},{"given":"Luis E","family":"Saldana","sequence":"additional","affiliation":[{"name":"Clinical Informatics, Texas Health Resources, Arlington, Texas, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3087-4193","authenticated-orcid":false,"given":"Duwayne L","family":"Willett","sequence":"additional","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"},{"name":"Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Josh E","family":"Youngblood","sequence":"additional","affiliation":[{"name":"Health System Information Resources Department, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]},{"given":"Seth M","family":"Toomay","sequence":"additional","affiliation":[{"name":"Clinical Informatics, University of Texas Southwestern Health System, Dallas, Texas, USA"},{"name":"Department of Radiology, University of Texas Southwestern Medical Center, Dallas, Texas, USA"}]}],"member":"286","published-online":{"date-parts":[[2019,9,12]]},"reference":[{"key":"2020110613055535600_ocz123-B1","author":"Hastie","year":"2015"},{"key":"2020110613055535600_ocz123-B2","volume-title":"Agile and Iterative Development: A Manager's Guide","author":"Larman","year":"2003"},{"issue":"2","key":"2020110613055535600_ocz123-B3","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/j.jbi.2007.09.003","article-title":"Grand challenges in clinical decision support","volume":"41","author":"Sittig","year":"2008","journal-title":"J Biomed Inform"},{"key":"2020110613055535600_ocz123-B4","first-page":"S103\u201316.","article-title":"Clinical decision support: a 25 year retrospective and a 25 year vision","volume":"25(Suppl 1)","author":"Middleton","year":"2016","journal-title":"Yearb Med Inform"},{"key":"2020110613055535600_ocz123-B5","doi-asserted-by":"crossref","DOI":"10.4324\/9781498757461","volume-title":"Improving Outcomes with Clinical Decision Support: An Implementer's Guide","author":"Osheroff","year":"2012"},{"key":"2020110613055535600_ocz123-B6","doi-asserted-by":"crossref","first-page":"68.","DOI":"10.3389\/fpubh.2018.00068","article-title":"Big data's role in precision public health","volume":"6","author":"Dolley","year":"2018","journal-title":"Front Public Health"},{"issue":"4","key":"2020110613055535600_ocz123-B7","doi-asserted-by":"crossref","first-page":"e38.","DOI":"10.2196\/medinform.5359","article-title":"Challenges and opportunities of big data in health care: a systematic review","volume":"4","author":"Kruse","year":"2016","journal-title":"JMIR Med Inform"},{"issue":"4","key":"2020110613055535600_ocz123-B8","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1093\/jamia\/ocw053","article-title":"Precision medicine informatics","volume":"23","author":"Frey","year":"2016","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613055535600_ocz123-B9","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1002\/cpt.707","article-title":"Electronic medical record-integrated pharmacogenomics and related clinical decision support concepts","volume":"102","author":"Caraballo","year":"2017","journal-title":"Clin Pharmacol Ther"},{"issue":"3","key":"2020110613055535600_ocz123-B10","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1002\/psp4.12173","article-title":"Implementing genomic clinical decision support for drug-based precision medicine","volume":"6","author":"Freimuth","year":"2017","journal-title":"CPT Pharmacometrics Syst Pharmacol"},{"issue":"3","key":"2020110613055535600_ocz123-B11","first-page":"e1378","article-title":"Integrated precision medicine: the role of electronic health records in delivering personalized treatment","volume":"9","author":"Sitapati","year":"2017","journal-title":"Syst Biol Med"},{"issue":"4","key":"2020110613055535600_ocz123-B12","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1093\/jamia\/ocw015","article-title":"SMART precision cancer medicine: a FHIR-based app to provide genomic information at the point of care","volume":"23","author":"Warner","year":"2016","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613055535600_ocz123-B13","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.jbi.2016.07.011","article-title":"Publication, discovery and interoperability of clinical decision support systems: a linked data approach","volume":"62","author":"Marco-Ruiz","year":"2016","journal-title":"J Biomed Inform"},{"key":"2020110613055535600_ocz123-B14","author":"Agency for Healthcare Research and Quality"},{"key":"2020110613055535600_ocz123-B15","author":"HL7"},{"key":"2020110613055535600_ocz123-B16","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-30370-3","volume-title":"Principles of Health Interoperability: SNOMED CT, HL7, and FHIR","author":"Benson","year":"2016","edition":"3rd ed."},{"key":"2020110613055535600_ocz123-B17","author":"CDS Hooks","year":"2018"},{"issue":"1","key":"2020110613055535600_ocz123-B18","doi-asserted-by":"crossref","first-page":"e20.","DOI":"10.2196\/mhealth.3335","article-title":"Behavioral functionality of mobile apps in health interventions: a systematic review of the literature","volume":"3","author":"Payne","year":"2015","journal-title":"JMIR mHealth uHealth"},{"issue":"57","key":"2020110613055535600_ocz123-B19","doi-asserted-by":"crossref","first-page":"57cm29.","DOI":"10.1126\/scitranslmed.3001456","article-title":"Achieving a nationwide learning health system","volume":"2","author":"Friedman","year":"2010","journal-title":"Sci Transl Med"},{"issue":"14","key":"2020110613055535600_ocz123-B20","doi-asserted-by":"crossref","first-page":"e826","DOI":"10.1161\/CIR.0000000000000480","article-title":"The learning healthcare system and cardiovascular care: a scientific statement from the American Heart Association","volume":"135","author":"Maddox","year":"2017","journal-title":"Circulation"},{"issue":"5","key":"2020110613055535600_ocz123-B21","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1093\/jamia\/ocx016","article-title":"Pragmatic (trial) informatics: a perspective from the NIH Health Care Systems Research Collaboratory","volume":"24","author":"Richesson","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613055535600_ocz123-B22","doi-asserted-by":"crossref","first-page":"S6.","DOI":"10.1186\/1475-925X-14-S2-S6","article-title":"Design, implementation and validation of a novel open framework for agile development of mobile health applications","volume":"14 Suppl 2","author":"Banos","year":"2015","journal-title":"Biomed Eng Online"},{"key":"2020110613055535600_ocz123-B23","author":"Kannan","year":"2016"},{"key":"2020110613055535600_ocz123-B24","first-page":"103","volume-title":"AMIA 2017 Annual Symposium","author":"Kannan","year":"2017"},{"key":"2020110613055535600_ocz123-B25","first-page":"56","author":"Kannan","year":"2017"},{"key":"2020110613055535600_ocz123-B26","doi-asserted-by":"crossref","first-page":"e74","DOI":"10.3414\/ME16-02-0031","article-title":"Rapid development of specialty population registries and quality measures from electronic health record data: an agile framework","volume":"56","author":"Kannan","year":"2017","journal-title":"Methods Inf Med"},{"issue":"5","key":"2020110613055535600_ocz123-B27","doi-asserted-by":"crossref","first-page":"e119.","DOI":"10.2196\/resprot.9258","article-title":"The development of delta: using agile to develop a decision aid for pediatric oncology clinical trial enrollment","volume":"7","author":"Robertson","year":"2018","journal-title":"JMIR Res Protoc"},{"issue":"4","key":"2020110613055535600_ocz123-B28","doi-asserted-by":"crossref","first-page":"e99.","DOI":"10.2196\/mhealth.9661","article-title":"Lessons from the dot contraceptive efficacy study: analysis of the use of agile development to improve recruitment and enrollment for mHealth research","volume":"6","author":"Shattuck","year":"2018","journal-title":"JMIR mHealth uHealth"},{"key":"2020110613055535600_ocz123-B29","volume-title":"Agile Modeling: Effective Practices for Extreme Programming and the Unified Process","author":"Ambler","year":"2002"},{"key":"2020110613055535600_ocz123-B30","author":"Agile vs","year":"2017"},{"key":"2020110613055535600_ocz123-B31","volume-title":"Essential Scrum: A Practical Guide to the Most Popular Agile Process","author":"Rubin","year":"2012"},{"key":"2020110613055535600_ocz123-B32","volume-title":"User Stories Applied: For Agile Software Development","author":"Cohn","year":"2004"},{"key":"2020110613055535600_ocz123-B33","author":"Cohn","year":"2018"},{"key":"2020110613055535600_ocz123-B34","volume-title":"Agile Estimating and Planning","author":"Cohn","year":"2005"},{"key":"2020110613055535600_ocz123-B35","author":"Agile Estimating and Planning by","year":"2018"},{"key":"2020110613055535600_ocz123-B36","author":"Usman","year":"2014"},{"key":"2020110613055535600_ocz123-B37","volume-title":"Use Case Driven Object Modeling with UML: Theory and Practice","author":"Rosenberg D","year":"2013","edition":"2nd ed."},{"key":"2020110613055535600_ocz123-B38","volume-title":"Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and Iterative Development","author":"Larman","year":"2004","edition":"3rd ed"},{"key":"2020110613055535600_ocz123-B39","volume-title":"Specification by Example: How Successful Teams Deliver the Right Software","author":"Adzic","year":"2011"},{"key":"2020110613055535600_ocz123-B40","volume-title":"More Agile Testing: Learning Journeys for the Whole Team","author":"Gregory","year":"2014"},{"key":"2020110613055535600_ocz123-B41","volume-title":"User Story Mapping: Discover the Whole Story, Build the Right Product","author":"Patton","year":"2014"},{"key":"2020110613055535600_ocz123-B42","volume-title":"Test Driven NET Development with FitNesse","author":"Adzic","year":"2008"},{"key":"2020110613055535600_ocz123-B43"},{"key":"2020110613055535600_ocz123-B44","author":"Agile Alliance. INVEST","year":"2018"},{"key":"2020110613055535600_ocz123-B45","author":"Lattenkamp","year":"2017"},{"key":"2020110613055535600_ocz123-B46","author":"Nikolov","year":"2016"},{"key":"2020110613055535600_ocz123-B47","volume-title":"AMIA Clinical Informatics Conference","author":"Keshvani","year":"2018"},{"key":"2020110613055535600_ocz123-B48","first-page":"465","author":"Kannan","year":"2016"},{"key":"2020110613055535600_ocz123-B49","doi-asserted-by":"crossref","first-page":"e74","DOI":"10.3414\/ME16-02-0031","article-title":"Rapid development of specialty population registries and quality measures from electronic health record data: supplementary material","volume":"56","author":"Kannan","year":"2017","journal-title":"Methods Inf Med"},{"key":"2020110613055535600_ocz123-B50","author":"Youngblood","year":"2017"},{"issue":"2","key":"2020110613055535600_ocz123-B51","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1136\/jamia.2009.002030","article-title":"Governance for clinical decision support: case studies and recommended practices from leading institutions","volume":"18","author":"Wright","year":"2011","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2020110613055535600_ocz123-B52","doi-asserted-by":"crossref","first-page":"1068","DOI":"10.1093\/jamia\/ocw005","article-title":"Analysis of clinical decision support system malfunctions: a case series and survey","volume":"23","author":"Wright","year":"2016","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613055535600_ocz123-B53","first-page":"910\u201323.","article-title":"Clinical decision support malfunctions in a commercial electronic health record","volume":"8","author":"Kassakian","year":"2017","journal-title":"Appl Clin Inform"},{"issue":"7","key":"2020110613055535600_ocz123-B54","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1093\/jamia\/ocy041","article-title":"Using statistical anomaly detection models to find clinical decision support malfunctions","volume":"25","author":"Ray","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613055535600_ocz123-B55","doi-asserted-by":"crossref","first-page":"e23.","DOI":"10.2196\/medinform.9679","article-title":"Agile acceptance test-driven development of clinical decision support advisories: feasibility of using open source software","volume":"6","author":"Basit","year":"2018","journal-title":"JMIR Med Inform"},{"issue":"10","key":"2020110613055535600_ocz123-B56","first-page":"540","article-title":"Toward more proactive approaches to safety in the electronic health record era","volume":"43","author":"Sittig","year":"2017","journal-title":"Jt Comm J Qual Patient Saf"},{"key":"2020110613055535600_ocz123-B57","author":"Johnson","year":"2002"},{"key":"2020110613055535600_ocz123-B58","author":"Standish","year":"2014"},{"key":"2020110613055535600_ocz123-B59","author":"Ambler","year":"2017"},{"key":"2020110613055535600_ocz123-B60","author":"Ambler","year":"2017"},{"issue":"6","key":"2020110613055535600_ocz123-B61","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1016\/j.jbi.2012.09.002","article-title":"Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions","volume":"45","author":"Horsky","year":"2012","journal-title":"J Biomed Inform"},{"issue":"6","key":"2020110613055535600_ocz123-B62","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1016\/j.ijmedinf.2013.02.003","article-title":"Design of decision support interventions for medication prescribing","volume":"82","author":"Horsky","year":"2013","journal-title":"Int J Med Inform"},{"issue":"6","key":"2020110613055535600_ocz123-B63","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1093\/jamia\/ocx060","article-title":"Participatory design of probability-based decision support tools for in-hospital nurses","volume":"24","author":"Jeffery","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110613055535600_ocz123-B64","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1093\/jamia\/ocx118","article-title":"Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support","volume":"25","author":"Miller","year":"2018","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/11\/1344\/34152027\/ocz123.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/11\/1344\/34152027\/ocz123.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,9]],"date-time":"2021-01-09T02:07:40Z","timestamp":1610158060000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/26\/11\/1344\/5568259"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,12]]},"references-count":64,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2019,9,12]]},"published-print":{"date-parts":[[2019,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz123","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,11]]},"published":{"date-parts":[[2019,9,12]]}}}