{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T00:56:08Z","timestamp":1772585768759,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-2106710"],"award-info":[{"award-number":["CCF-2106710"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,9,27]]},"abstract":"<jats:p>Mobile medical score calculator apps are widely used among practitioners to help make decisions regarding patient treatment and diagnosis. Errors in score definition, input, or calculations can result in severe and potentially life-threatening situations. Despite these high stakes, there has been no systematic or rigorous effort to examine and verify score calculator apps. We address these issues via a novel, interval-based score checking approach. Based on our observation that medical reference tables themselves may contain errors (which can propagate to apps) we first introduce automated correctness checking of reference tables. Specifically, we reduce score correctness checking to partition checking (coverage and non-overlap) over score parameters' ranges. We checked 12 scoring systems used in emergency, intensive, and acute care. Surprisingly, though some of these scores have been used for decades, we found errors in 5 score specifications: 8 coverage violations and 3 non-overlap violations. Second, we design and implement an automatic, dynamic analysis-based approach for verifying score correctness in a given Android app; the approach combines efficient, automatic GUI extraction and app exploration with partition\/consistency checking to expose app errors. We applied the approach to 90 Android apps that implement medical score calculators. We found 23 coverage violations in 11 apps; 32 non-overlap violations in 12 apps, and 16 incorrect score calculations in 16 apps. We reported all findings to developers, which so far has led to fixes in 6 apps.<\/jats:p>","DOI":"10.1145\/3610912","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T15:45:03Z","timestamp":1695829503000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Diagnosing Medical Score Calculator Apps"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6041-629X","authenticated-orcid":false,"given":"Sydur","family":"Rahaman","sequence":"first","affiliation":[{"name":"New Jersey Institute of Technology, Newark, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0672-6705","authenticated-orcid":false,"given":"Raina","family":"Samuel","sequence":"additional","affiliation":[{"name":"New Jersey Institute of Technology, Newark, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9730-9650","authenticated-orcid":false,"given":"Iulian","family":"Neamtiu","sequence":"additional","affiliation":[{"name":"New Jersey Institute of Technology, Newark, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2021. FTC cracks down on marketers of \"melanoma detection\" apps. https:\/\/www.ftc.gov\/news-events\/news\/press-releases\/2015\/02\/ftc-cracks-down-marketers-melanoma-detection-apps."},{"key":"e_1_2_1_2_1","unstructured":"2021. National Early Warning Score (NEWS) 2. https:\/\/www.rcplondon.ac.uk\/projects\/outputs\/national-early-warning-score-news-2"},{"key":"e_1_2_1_3_1","unstructured":"2022. IELTS Band Score. https:\/\/play.google.com\/store\/apps\/details?id=com.samir.ieltsmarkstoband. Accessed: 2022-09-01."},{"key":"e_1_2_1_4_1","unstructured":"2022. MDApp SOFA Score. https:\/\/www.mdapp.co\/sequential-organ-failure-assessment-sofa-score-calculator-184\/. Accessed: 2022-09-01."},{"key":"e_1_2_1_5_1","unstructured":"2022. MEWS Brasil. https:\/\/play.google.com\/store\/apps\/details?id=appinventor.ai_blinkeado.InformaticasaudeMEWS."},{"key":"e_1_2_1_6_1","unstructured":"2022. Policy for device software functions and mobile medical applications guidance for industry and food and drug administration staff."},{"key":"e_1_2_1_7_1","unstructured":"2022. SOFA. https:\/\/www.rccc.eu\/ppc\/indicadores\/sofa.html. Accessed: 2022-09-01."},{"key":"e_1_2_1_8_1","unstructured":"2023. RangeSlider. https:\/\/developer.android.com\/reference\/com\/google\/android\/material\/slider\/RangeSlider. Accessed: 2023-07-01."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3328661"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz175"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz175"},{"key":"e_1_2_1_12_1","unstructured":"Wayne M. Alves Brett E. Skolnick Brett E. Skolnick and Shamik Chakraborty. 2018. Chapter 5 - Traumatic Brain Injury. Academic Press."},{"key":"e_1_2_1_13_1","unstructured":"Android Developers. 2023. UI\/Application Exerciser Monkey. https:\/\/developer.android.com\/studio\/test\/other-testing-tools\/monkey."},{"key":"e_1_2_1_14_1","volume-title":"Guigle: A GUI Search Engine for Android Apps. CoRR abs\/1901.00891","author":"Bernal-C\u00e1rdenas Carlos","year":"2019","unstructured":"Carlos Bernal-C\u00e1rdenas, Kevin Moran, Michele Tufano, Zichang Liu, Linyong Nan, Zhehan Shi, and Denys Poshyvanyk. 2019. Guigle: A GUI Search Engine for Android Apps. CoRR abs\/1901.00891 (2019). arXiv:1901.00891 http:\/\/arxiv.org\/abs\/1901.00891"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40264-013-0015-0"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1378\/chest.11-0330"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.mpaic.2018.11.011"},{"key":"e_1_2_1_18_1","volume-title":"ACM\/IEEE International Conference on 0 (07 2006","author":"Bultan Tevfik","year":"2006","unstructured":"Tevfik Bultan and Constance Heitmeyer. 2006. Analyzing tabular requirements specifications using infinite state model checking. Formal Methods and Models for Co-Design, ACM\/IEEE International Conference on 0 (07 2006), 7--16. https:\/\/doi.org\/10.1109\/ MEMCOD.2006.1695895"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.89"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2935334.2935349"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-78800-3_24"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pec.2018.10.014"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.286.14.1754"},{"key":"e_1_2_1_24_1","first-page":"3","article-title":"Smartphone use by nurses in acute care settings","volume":"36","author":"Huck Flynn Greir Ander","year":"2018","unstructured":"Greir Ander Huck Flynn, Barbara Polivka, and Jodi Herron Behr. 2018. Smartphone use by nurses in acute care settings. Comput. Inform. Nurs. 36, 3 (March 2018), 120--126.","journal-title":"Comput. Inform. Nurs."},{"key":"e_1_2_1_25_1","unstructured":"FTC. 2019. MOBILE HEALTH APPS INTERACTIVE TOOL. https:\/\/www.ftc.gov\/tips-advice\/business-center\/guidance\/mobile-health-apps-interactive-tool"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1308\/003588406x130615"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2019.105002"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40264-013-0015-0"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594368.2594390"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3390\/jcm10225426"},{"key":"e_1_2_1_31_1","volume-title":"Health Information Laws. https:\/\/www.hhs.gov\/hipaa\/for-professionals\/security\/index.htmlas of","author":"HHS.","year":"2019","unstructured":"HHS. 2019. Health Information Laws. https:\/\/www.hhs.gov\/hipaa\/for-professionals\/security\/index.htmlas of December 2019."},{"key":"e_1_2_1_32_1","volume-title":"Summary of Privacy Rule. https:\/\/www.hhs.gov\/hipaa\/for-professionals\/privacy\/laws-regulations\/index.htmlas of","author":"HHS.","year":"2019","unstructured":"HHS. 2019. Summary of Privacy Rule. https:\/\/www.hhs.gov\/hipaa\/for-professionals\/privacy\/laws-regulations\/index.htmlas of December 2019."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-81278-5"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-015-0314-7"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1097\/00003246-198510000-00009"},{"key":"e_1_2_1_36_1","volume-title":"Application of tabular methods to the specification and verification of a nuclear reactor shutdown system. Formal Methods in System Design - FMSD (09","author":"Lawford Mark","year":"2001","unstructured":"Mark Lawford and P Froebel. 2001. Application of tabular methods to the specification and verification of a nuclear reactor shutdown system. Formal Methods in System Design - FMSD (09 2001)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462456.2464459"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-C.2017.8"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacc.2010.09.024"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2015.4369"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242650"},{"key":"e_1_2_1_42_1","volume-title":"National Early Warning Score (NEWS) 2: standardising the assessment of acute-illness severity in the NHS. Updated report of a working party","author":"Royal College of Physicians. 2021.","year":"2017","unstructured":"Royal College of Physicians. 2021. National Early Warning Score (NEWS) 2: standardising the assessment of acute-illness severity in the NHS. Updated report of a working party 2017. (2021)."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2796.2004.01321.x"},{"key":"e_1_2_1_44_1","unstructured":"Ortholive. 2019. 32 Statistics on mHealth. https:\/\/www.ortholive.com\/blog\/32-statistics-on-mhealth\/."},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the 27th USENIX Conference on Security Symposium","author":"Pan Xiang","year":"2018","unstructured":"Xiang Pan, Yinzhi Cao, Xuechao Du, Boyuan He, Gan Fang, and Yan Chen. 2018. FlowCog: Context-Aware Semantics Extraction and Analysis of Information Flow Leaks in Android Apps. In Proceedings of the 27th USENIX Conference on Security Symposium (Baltimore, MD, USA) (SEC'18). USENIX Association, USA, 1669--1685."},{"key":"e_1_2_1_46_1","first-page":"2","article-title":"A UK perspective on smartphone use amongst doctors within the surgical profession","volume":"4","author":"Patel Rikesh K","year":"2015","unstructured":"Rikesh K Patel, Adele E Sayers, Nina L Patrick, Kaylie Hughes, Jonathan Armitage, and Iain Andrew Hunter. 2015. A UK perspective on smartphone use amongst doctors within the surgical profession. Ann. Med. Surg. (Lond.) 4, 2 (June 2015), 107--112.","journal-title":"Ann. Med. Surg. (Lond.)"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.46747\/cfp.6804e127"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594368.2594377"},{"key":"e_1_2_1_49_1","first-page":"4","article-title":"The Rapid Acute Physiology","volume":"5","author":"Rhee K J","year":"1987","unstructured":"K J Rhee, C J Fisher, Jr, and N H Willitis. 1987. The Rapid Acute Physiology Score. Am. J. Emerg. Med. 5, 4 (July 1987), 278--282.","journal-title":"Score. Am. J. Emerg. Med."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/bf03086144"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1197\/aemj.9.2.99"},{"key":"e_1_2_1_52_1","first-page":"7872","article-title":"Assessment of coma and impaired consciousness. A practical scale","volume":"2","author":"Teasdale G","year":"1974","unstructured":"G Teasdale and B Jennett. 1974. Assessment of coma and impaired consciousness. A practical scale. Lancet 2, 7872 (July 1974), 81--84.","journal-title":"Lancet"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","unstructured":"Jean-Louis Vincent Rui Moreno Jukka Takala S Willatts A Mendon\u00e7a H Bruining C Reinhart Peter Suter and L Thijs. 1996. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction\/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive care medicine 22 (08 1996) 707--10. https:\/\/doi.org\/10.1007\/BF01709751","DOI":"10.1007\/BF01709751"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2012-001099"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1159\/000018578"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610912","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610912","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610912","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,28]],"date-time":"2025-07-28T16:28:03Z","timestamp":1753720083000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610912"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,9,27]]}},"alternative-id":["10.1145\/3610912"],"URL":"https:\/\/doi.org\/10.1145\/3610912","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,27]]},"assertion":[{"value":"2023-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}