{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T08:16:31Z","timestamp":1770279391844,"version":"3.49.0"},"reference-count":0,"publisher":"Georg Thieme Verlag KG","issue":"03","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2018,7]]},"abstract":"<jats:p>\n            Background\u2003The assessment of user preferences for performance characteristics of patient-oriented clinical prediction models is lacking. It is unknown if complex statistical aspects of prediction models are readily understandable by a general audience.<\/jats:p><jats:p>\n            Objective\u2003A pilot study was conducted among nonclinical audiences to determine the feasibility of interpreting statistical concepts that describe the performance of prediction models.<\/jats:p><jats:p>\n            Methods\u2003We conducted a cross-sectional electronic survey using the Amazon Mechanical Turk platform. The survey instrument included educational modules about predictive models, sensitivity, specificity, and confidence intervals (CIs). Follow-up questions tested participants' abilities to interpret these characteristics with both verbatim and gist knowledge. Objective and subjective numeracy were assessed using previously validated instruments. We also tested understanding of these concepts when embedded in a sample discrete choice experiment task to establish feasibility for future elicitation of preferences using a discrete choice experiment design. Multivariable linear regression was used to identify factors associated with correct interpretation of statistical concepts.<\/jats:p><jats:p>\n            Results\u2003Among 534 respondents who answered all nine questions, the mean correct responses was 95.9% (95% CI, 93.8\u201397.4) for sensitivity, 93.1% (95% CI, 90.5\u201395.0) for specificity, and 86.6% (95% CI, 83.3\u201389.3) for CIs. Verbatim interpretation was high for all concepts, but significantly higher than gist only for CIs (p\u2009&lt;\u20090.001). Scores on each discrete choice experiment tasks were slightly lower in each category. Both objective and subjective numeracy were positively associated with an increased proportion of correct responses (p\u2009&lt;\u20090.001).<\/jats:p><jats:p>\n            Conclusion\u2003These results suggest that a nonclinical audience can interpret quantitative performance measures of predictive models with very high accuracy. Future development of patient-facing clinical prediction models can feasibly incorporate patient preferences for model features into their development.<\/jats:p>","DOI":"10.1055\/s-0038-1669457","type":"journal-article","created":{"date-parts":[[2018,8,29]],"date-time":"2018-08-29T23:27:20Z","timestamp":1535585240000},"page":"683-692","source":"Crossref","is-referenced-by-count":6,"title":["Numeracy and Understanding of Quantitative Aspects of Predictive Models: A Pilot Study"],"prefix":"10.1055","volume":"09","author":[{"given":"Gary","family":"Weissman","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kuldeep","family":"Yadav","sequence":"additional","affiliation":[{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vanessa","family":"Madden","sequence":"additional","affiliation":[{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katherine","family":"Courtright","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joanna","family":"Hart","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Asch","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Center for Health Care Innovation, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"The Center for Health Equity Research and Promotion, Philadelphia VA Medical Center, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marilyn","family":"Schapira","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"The Center for Health Equity Research and Promotion, Philadelphia VA Medical Center, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Scott","family":"Halpern","sequence":"additional","affiliation":[{"name":"Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Department of Medicine, Palliative and Advanced Illness Research Center, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Fostering Improvement in End-of-Life Decision Science Program, University of Pennsylvania, Philadelphia, Pennsylvania, United States"},{"name":"Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"194","published-online":{"date-parts":[[2018,8,29]]},"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0038-1669457.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,5,6]],"date-time":"2019-05-06T20:46:03Z","timestamp":1557175563000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/s-0038-1669457"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7]]},"references-count":0,"journal-issue":{"issue":"03","published-online":{"date-parts":[[2018,7,3]]},"published-print":{"date-parts":[[2018,7]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0038-1669457","relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7]]}}}