{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T14:25:04Z","timestamp":1781360704012,"version":"3.54.1"},"reference-count":36,"publisher":"Georg Thieme Verlag KG","issue":"01","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2016,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Older adults are at risk for inadequate emergency department (ED) pain care. Unrelieved acute pain is associated with poor outcomes. Clinical decision support systems (CDSS) hold promise to improve patient care, but CDSS quality varies widely, particularly when usability evaluation is not employed.<\/jats:p><jats:p>To conduct an iterative usability and redesign process of a novel geriatric abdominal pain care CDSS. We hypothesized this process would result in the creation of more usable and favorable pain care interventions.<\/jats:p><jats:p>Thirteen emergency physicians familiar with the Electronic Health Record (EHR) in use at the study site were recruited. Over a 10-week period, 17 1-hour usability test sessions were conducted across 3 rounds of testing. Participants were given 3 patient scenarios and provided simulated clinical care using the EHR, while interacting with the CDSS interventions. Quantitative System Usability Scores (SUS), favorability scores and qualitative narrative feedback were collected for each session. Using a multi-step review process by an interdisciplinary team, positive and negative usability issues in effectiveness, efficiency, and satisfaction were considered, prioritized and incorporated in the iterative redesign process of the CDSS. Video analysis was used to determine the appropriateness of the CDS appearances during simulated clinical care.<\/jats:p><jats:p>Over the 3 rounds of usability evaluations and subsequent redesign processes, mean SUS progressively improved from 74.8 to 81.2 to 88.9; mean favorability scores improved from 3.23 to 4.29 (1 worst, 5 best). Video analysis revealed that, in the course of the iterative redesign processes, rates of physicians\u2019 acknowledgment of CDS interventions increased, however most rates of desired actions by physicians (such as more frequent pain score updates) decreased.<\/jats:p><jats:p>The iterative usability redesign process was instrumental in improving the usability of the CDSS; if implemented in practice, it could improve geriatric pain care. The usability evaluation process led to improved acknowledgement and favorability. Incorporating usability testing when designing CDSS interventions for studies may be effective to enhance clinician use.<\/jats:p>","DOI":"10.4338\/aci-2015-08-ra-0108","type":"journal-article","created":{"date-parts":[[2016,3,9]],"date-time":"2016-03-09T03:11:12Z","timestamp":1457493072000},"page":"128-142","source":"Crossref","is-referenced-by-count":20,"title":["Usability Evaluation of a Clinical Decision Support System for Geriatric ED Pain Treatment"],"prefix":"10.4338","volume":"07","author":[{"given":"Min","family":"Kim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frederick","family":"Thum","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Rivera","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rosemary","family":"Beato","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carolyn","family":"Song","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jared","family":"Soriano","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph","family":"Kannry","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"Baumlin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ula","family":"Hwang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Genes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2017,12,16]]},"reference":[{"key":"10.4338\/ACI-2015-08-RA-0108-1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1532-5415.2010.03152.x"},{"key":"10.4338\/ACI-2015-08-RA-0108-2","doi-asserted-by":"publisher","DOI":"10.1016\/j.annemergmed.2011.09.014"},{"key":"10.4338\/ACI-2015-08-RA-0108-3","doi-asserted-by":"publisher","DOI":"10.1016\/j.emc.2004.12.003"},{"key":"10.4338\/ACI-2015-08-RA-0108-4","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3959(02)00458-X"},{"key":"10.4338\/ACI-2015-08-RA-0108-5","doi-asserted-by":"publisher","DOI":"10.1016\/S1082-3174(97)70009-6"},{"key":"10.4338\/ACI-2015-08-RA-0108-6","doi-asserted-by":"publisher","DOI":"10.1016\/0885-3924(94)90142-2"},{"key":"10.4338\/ACI-2015-08-RA-0108-7","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1213\/00000539-199804000-00019","volume":"86","author":"Lynch","year":"1998","journal-title":"Anesth Analg"},{"key":"10.4338\/ACI-2015-08-RA-0108-8","unstructured":"The AGS guideline on the management of persistent pain in older persons. 2002; http:\/\/www.americangeriatrics.org\/products\/positionpapers\/persistent_pain_guide.shtml. Accessed January 16, 2007"},{"key":"10.4338\/ACI-2015-08-RA-0108-9","doi-asserted-by":"publisher","DOI":"10.1111\/j.1526-4637.2009.00699.x"},{"key":"10.4338\/ACI-2015-08-RA-0108-10","doi-asserted-by":"publisher","DOI":"10.1503\/cmaj.141183"},{"key":"10.4338\/ACI-2015-08-RA-0108-11","doi-asserted-by":"crossref","unstructured":"Gupta A, Ip IK, Raja AS. Effect of clinical decision support on documented guideline adherence for head CT in emergency department patients with mild traumatic brain injury. J Am Med Inform Assoc, 21 (2014), pp. e347-e351","DOI":"10.1136\/amiajnl-2013-002536"},{"key":"10.4338\/ACI-2015-08-RA-0108-12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jemermed.2012.06.016"},{"key":"10.4338\/ACI-2015-08-RA-0108-13","unstructured":"Dashboard.HealthIT.gov [Internet]. Washington, D.C.: Health IT Adoption & Use Dashboard from U.S. HHS\/ONC; [updated 2015 Apr; cited 2015 Aug 15]. Available from: http:\/\/dashboard.healthit.gov\/HITAdoption\/?view=0"},{"key":"10.4338\/ACI-2015-08-RA-0108-14","unstructured":"Institute of Medicine (U.S.), and Committee on Patient Safety and Health Information Technology. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, D.C.: National Academies Press; 2012"},{"issue":"10","key":"10.4338\/ACI-2015-08-RA-0108-15","first-page":"897","volume":"171","author":"Romano","year":"2015","journal-title":"Arch Intern Med"},{"key":"10.4338\/ACI-2015-08-RA-0108-16","doi-asserted-by":"publisher","DOI":"10.7326\/M13-1531"},{"key":"10.4338\/ACI-2015-08-RA-0108-17","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-144-10-200605160-00125"},{"key":"10.4338\/ACI-2015-08-RA-0108-18","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2009.000695"},{"key":"10.4338\/ACI-2015-08-RA-0108-19","doi-asserted-by":"publisher","DOI":"10.1001\/archinternmed.2010.324"},{"key":"10.4338\/ACI-2015-08-RA-0108-20","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X09342752"},{"key":"10.4338\/ACI-2015-08-RA-0108-21","doi-asserted-by":"publisher","DOI":"10.1111\/j.1532-5415.2008.02004.x"},{"key":"10.4338\/ACI-2015-08-RA-0108-22","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1370"},{"key":"10.4338\/ACI-2015-08-RA-0108-23","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.38398.500764.8F"},{"key":"10.4338\/ACI-2015-08-RA-0108-24","unstructured":"Gold M, Hossain M, Charles DR, Furukawa MF. Evolving vendor market for HITECH-certified ambulatory EHR products. Am J Manag Care 2013; 11; 353-361"},{"key":"10.4338\/ACI-2015-08-RA-0108-25","unstructured":"Nielson, J. Usability 101: Introduction to Usability [Internet]. Freemont: Nielsen Norman Group. 2012 Jan [cited 2015 Aug 15]. Available from: http:\/\/www.nngroup.com\/articles\/usability-101-introduction-to-usability\/"},{"key":"10.4338\/ACI-2015-08-RA-0108-26","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2012.09.002"},{"key":"10.4338\/ACI-2015-08-RA-0108-27","doi-asserted-by":"publisher","DOI":"10.1016\/j.pain.2014.09.017"},{"key":"10.4338\/ACI-2015-08-RA-0108-28","unstructured":"Techsmith. Morae - usability testing software. http:\/\/www.techsmith.com\/morae.asp. Accessed March 27, 2011"},{"key":"10.4338\/ACI-2015-08-RA-0108-29","unstructured":"Brooke, J. SUS: a \u2019quick and dirty\u2019 usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, and McClelland IL editors. Usability Evaluation in Industry. London: Taylor and Francis. p. 189-194"},{"key":"10.4338\/ACI-2015-08-RA-0108-30","unstructured":"Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean; adding an Aadjective rating scale. J Usability Studies 2009; 4: 114-123"},{"issue":"2","key":"10.4338\/ACI-2015-08-RA-0108-31","first-page":"195","volume":"14","author":"McCoy","year":"2014","journal-title":"Ochsner J"},{"key":"10.4338\/ACI-2015-08-RA-0108-32","unstructured":"Ong, M-S, Coiera, E. Evaluating the effectiveness of clinical alerts: a signal detection approach. AMIA Annu Symp Proc 2011; 1036-1044"},{"key":"10.4338\/ACI-2015-08-RA-0108-33","doi-asserted-by":"crossref","unstructured":"Lewis JR, Sauro J. The factor structure of the system usability scale. In: HCI International 2009: Proceedings of the 1st International Conference on Human Centered Design; 2009; San Diego:Springer-Verlag; 94-103","DOI":"10.1007\/978-3-642-02806-9_12"},{"key":"10.4338\/ACI-2015-08-RA-0108-34","doi-asserted-by":"publisher","DOI":"10.4338\/ACI-2014-05-RA-0050"},{"key":"10.4338\/ACI-2015-08-RA-0108-35","unstructured":"Bukata, R. EHRs are Inevitable, Yet Studies Still Pose Serious Questions. Emer Phys Monthly. 2014;10:4. Available from: http:\/\/epmonthly.com\/article\/ehrs-are-inevitable-yet-studies-still-pose-serious-questions\/"},{"key":"10.4338\/ACI-2015-08-RA-0108-36","doi-asserted-by":"publisher","DOI":"10.1016\/j.annemergmed.2013.12.019"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.4338\/ACI-2015-08-RA-0108.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2018,3,21]],"date-time":"2018-03-21T02:54:25Z","timestamp":1521600865000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.4338\/ACI-2015-08-RA-0108"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,1]]},"references-count":36,"journal-issue":{"issue":"01","published-online":{"date-parts":[[2017,12,16]]},"published-print":{"date-parts":[[2016]]}},"URL":"https:\/\/doi.org\/10.4338\/aci-2015-08-ra-0108","relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,1]]}}}