{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T04:42:24Z","timestamp":1787892144415,"version":"build-2784847793"},"reference-count":45,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Horizon 2020","award":["643736"],"award-info":[{"award-number":["643736"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>In this article, we described a new mobile-Health (mHealth) supported clinical pathway of care for people living with medically stable HIV in terms of platform acceptability, usability and technical feasibility. The EmERGE mHealth platform was codesigned with clinicians and the community, developed using Scrum agile methodology, integrated with hospital information systems and validated in a large prospective cohort study of 2251 participants. The evaluation of this new paradigm of care was conducted using a tailored Health Technology Assessment: the Model for Assessment of Telemedicine Applications. Usability and acceptability were assessed through the System Usability Score and a Patient Reported Experience Measure. The EmERGE platform was successfully deployed across diverse care settings in five European countries and used by 2251 patients and more than 20 clinicians for up to 30 months. Results from the formal evaluation demonstrated that the EmERGE platform is feasible and acceptable, with a high level of usability (median System Usability Score (SUS) 85.0%) and very positive patient-reported experiences (94.2% would recommend to a friend). The EmERGE platform is a secure and General Data Protection Regulation (GDPR)-compliant system with a complete set of functionalities that could be easily adapted to other clinical conditions, clinical sites and health systems thanks to its modular technical architecture.<\/jats:p>","DOI":"10.3390\/ijerph18063156","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T11:41:06Z","timestamp":1616067666000},"page":"3156","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["EmERGE mHealth Platform: Implementation and Technical Evaluation of a Digital Supported Pathway of Care for Medically Stable HIV"],"prefix":"10.3390","volume":"18","author":[{"given":"Francisco","family":"G\u00e1rate","sequence":"first","affiliation":[{"name":"Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicaci\u00f3n, Center for Biomedical Technology, Universidad Polit\u00e9cnica de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6740-657X","authenticated-orcid":false,"given":"Paloma","family":"Chausa","sequence":"additional","affiliation":[{"name":"Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicaci\u00f3n, Center for Biomedical Technology, Universidad Polit\u00e9cnica de Madrid, 28040 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Whetham","sequence":"additional","affiliation":[{"name":"Department of Sexual Health and HIV Medicine, Brighton and Sussex University Hospitals NHS Trust, BN2 5BE Brighton, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7065-1157","authenticated-orcid":false,"given":"Christopher","family":"Jones","sequence":"additional","affiliation":[{"name":"Brighton and Sussex Medical School, BN1 9PH Falmer, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7658-5832","authenticated-orcid":false,"given":"Felipe","family":"Garc\u00eda","sequence":"additional","affiliation":[{"name":"Infectious Diseases Department, Fundacio Privada Clinic per a la Recerca Biomedica\u2014IDIBAPS, University of Barcelona, 08036 Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C\u00e9sar","family":"C\u00e1ceres","sequence":"additional","affiliation":[{"name":"Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicaci\u00f3n, Center for Biomedical Technology, Universidad Polit\u00e9cnica de Madrid, 28040 Madrid, Spain"},{"name":"Computer Science Department, Universidad Rey Juan Carlos, 28933 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9871-0884","authenticated-orcid":false,"given":"Patricia","family":"S\u00e1nchez-Gonz\u00e1lez","sequence":"additional","affiliation":[{"name":"Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicaci\u00f3n, Center for Biomedical Technology, Universidad Polit\u00e9cnica de Madrid, 28040 Madrid, Spain"},{"name":"Centro de Investigaci\u00f3n Biom\u00e9dica en Red, Biomateriales y Nanomedicina (CIBER-BBN), 28029 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward","family":"Wallitt","sequence":"additional","affiliation":[{"name":"Podmedics, HA6 2UE Northwood, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enrique","family":"G\u00f3mez","sequence":"additional","affiliation":[{"name":"Biomedical Engineering and Telemedicine Centre, ETSI Telecomunicaci\u00f3n, Center for Biomedical Technology, Universidad Polit\u00e9cnica de Madrid, 28040 Madrid, Spain"},{"name":"Centro de Investigaci\u00f3n Biom\u00e9dica en Red, Biomateriales y Nanomedicina (CIBER-BBN), 28029 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"family":"on behalf of the EmERGE Consortium","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,18]]},"reference":[{"key":"ref_1","unstructured":"(2021, February 09). eHealth Action Plan 2012\u20132020-Innovative Healthcare for the 21st Century. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Available online: https:\/\/ec.europa.eu\/digital-single-market\/en\/news\/ehealth-action-plan-2012-2020-innovative-healthcare-21st-century#:~:text=The%20European%20Commission\u2019s%20eHealth%20Action,and%20patient%2Dcentred%20health%20services."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1056\/NEJMra1806949","article-title":"Mobile Devices and Health","volume":"381","author":"Sim","year":"2019","journal-title":"N. Engl. J. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e97","DOI":"10.2196\/jmir.4883","article-title":"The Effectiveness of Self-Management Mobile Phone and Tablet Apps in Long-term Condition Management: A Systematic Review","volume":"18","author":"Whitehead","year":"2016","journal-title":"J. Med. Internet Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lee, J.A., Choi, M., Lee, S.A., and Jiang, N. (2018). Effective behavioral intervention strategies using mobile health applications for chronic disease management: A systematic review. BMC Med. Inform. Decis. Mak., 18.","DOI":"10.1186\/s12911-018-0591-0"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1093\/cid\/ciw773","article-title":"Infectious Diseases Society of America Position Statement on Telehealth and Telemedicine as Applied to the Practice of Infectious Diseases","volume":"64","author":"Siddiqui","year":"2017","journal-title":"Clin. Infect. Dis."},{"key":"ref_6","unstructured":"WHO (2016). Global Diffusion of eHealth: Making Universal Health Coverage Achievable. Report of the Third Global Survey on eHealth, World Health Organization."},{"key":"ref_7","unstructured":"(2021, February 09). EU4Health 2021\u20132027\u2013a Vision for a Healthier European Union. Available online: https:\/\/ec.europa.eu\/health\/funding\/eu4health_en."},{"key":"ref_8","unstructured":"(2021, February 09). Europe mHealth Market Research Report. Available online: https:\/\/www.marketdataforecast.com\/market-reports\/europe-mobile-health-market."},{"key":"ref_9","unstructured":"(2021, February 09). Retention Rate on Day 1 and Day 30 of Mobile App Installs Worldwide as of August 2020, by Category. Available online: https:\/\/www.statista.com\/statistics\/259329\/ios-and-android-app-user-retention-rate\/."},{"key":"ref_10","unstructured":"Patient Adoption of mHealth (2015). Use, Evidence and Remaining Barriers to Mainstream Acceptance, IMS Institute for Healthcare Informatics."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e58","DOI":"10.1016\/S2589-7500(19)30025-1","article-title":"The role of health-care providers in mHealth adoption","volume":"1","author":"Leigh","year":"2019","journal-title":"Lancet Digit. Health"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e15935","DOI":"10.2196\/15935","article-title":"Social, Organizational, and Technological Factors Impacting Clinicians\u2019 Adoption of Mobile Health Tools: Systematic Literature Review","volume":"8","author":"Jacob","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1755","DOI":"10.1007\/s10461-016-1537-0","article-title":"Barriers and Facilitators to Interventions Improving Retention in HIV Care: A Qualitative Evidence Meta-Synthesis","volume":"21","author":"Hall","year":"2017","journal-title":"AIDS Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.ijnurstu.2018.04.009","article-title":"Self-management assessment in multiple chronic conditions: A narrative review of literature","volume":"83","author":"Ko","year":"2018","journal-title":"Int. J. Nurs. Stud."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1007\/s10461-016-1644-y","article-title":"Predictors of Adult Retention in HIV Care: A Systematic Review","volume":"22","author":"Bulsara","year":"2018","journal-title":"AIDS Behav."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1097\/QAD.0000000000000243","article-title":"Impact on life expectancy of HIV-1 positive individuals of CD4+ cell count and viral load response to antiretroviral therapy: UK cohort study","volume":"28","author":"May","year":"2014","journal-title":"AIDS"},{"key":"ref_17","unstructured":"Global HIV & AIDS Statistics\u20142019 Fact Sheet (2020, February 18). UNAIDS. Available online: https:\/\/www.unaids.org\/en\/resources\/fact-sheet."},{"key":"ref_18","unstructured":"EACS Guidelines Version 10.0 (2020, February 18). November 2019. Available online: https:\/\/www.eacsociety.org\/files\/2019_guidelines-10.0_final.pdf."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1089\/apc.2017.0303","article-title":"PositiveLinks: A Mobile Health Intervention for Retention in HIV Care and Clinical Outcomes with 12-Month Follow-Up","volume":"32","author":"Dillingham","year":"2018","journal-title":"AIDS Patient Care STDS"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e155","DOI":"10.2196\/mhealth.8376","article-title":"Improving Linkage to HIV Care Through Mobile Phone Apps: Randomized Controlled Trial","volume":"6","author":"Venter","year":"2018","journal-title":"JMIR MHealth UHealth"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e12652","DOI":"10.2196\/12652","article-title":"Improving Linkage to and Retention in Care in Newly Diagnosed HIV-Positive Patients Using Smartphones in South Africa: Randomized Controlled Trial","volume":"7","author":"Venter","year":"2019","journal-title":"JMIR MHealth UHealth"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"119","DOI":"10.2174\/1874613601711010119","article-title":"mHealth Interventions to Support Self-Management in HIV: A Systematic Review","volume":"11","author":"Cooper","year":"2017","journal-title":"Open AIDS J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s11904-014-0239-3","article-title":"A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care","volume":"12","author":"Muessig","year":"2015","journal-title":"Curr. HIV AIDS Rep."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1016\/j.ijmedinf.2006.04.010","article-title":"An integral care telemedicine system for HIV\/AIDS patients","volume":"75","author":"Caceres","year":"2006","journal-title":"Int. J. Med. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Le\u00f3n, A., C\u00e1ceres, C., Fern\u00e1ndez, E., Chausa, P., Martin, M., Codina, C., Rousaud, A., Blanch, J., Mallolas, J., and Martinez, E. (2011). A New Multidisciplinary Home Care Telemedicine System to Monitor Stable Chronic Human Immunodeficiency Virus-Infected Patients: A Randomized Study. PLoS ONE, 21.","DOI":"10.1371\/journal.pone.0014515"},{"key":"ref_26","first-page":"95","article-title":"EmERGE: Feasibility and uptake of a co-designed digital health supported pathway of care for people living with medically stable HIV. P111","volume":"23","author":"Whetham","year":"2020","journal-title":"J. Int. AIDS Soc."},{"key":"ref_27","unstructured":"(2021, February 09). EmERGE Project Website. Available online: https:\/\/www.emergeproject.eu\/."},{"key":"ref_28","unstructured":"(2021, February 09). EmERGE Project on the Community Research and Development Information Service Website. Available online: https:\/\/cordis.europa.eu\/project\/id\/643736."},{"key":"ref_29","unstructured":"(2021, February 09). Ruby on Rails Website. Available online: https:\/\/rubyonrails.org\/."},{"key":"ref_30","unstructured":"(2021, February 09). Ruby Website. Available online: https:\/\/www.ruby-lang.org\/en\/."},{"key":"ref_31","unstructured":"(2021, February 09). Ionic website. Available online: https:\/\/ionicframework.com\/."},{"key":"ref_32","unstructured":"(2021, February 09). Heroku Website. Available online: https:\/\/www.heroku.com\/."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Santos, A.R., Kroll, J., Sales, A., Fernandes, P., and Wildt, D. (2016). Investigating the Adoption of Agile Practices in Mobile Application Development. ICEIS.","DOI":"10.5220\/0005835404900497"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"9","DOI":"10.7815\/ijorcs.34.2013.068","article-title":"A review and analysis on mobile application development processes using agile methodologies","volume":"3","author":"Flora","year":"2013","journal-title":"Int. J. Res. Comput. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.socscimed.2018.09.003","article-title":"Ambivalence in digital health: Co-designing an mHealth platform for HIV care","volume":"215","author":"Marent","year":"2018","journal-title":"Soc. Sci. Med."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e184","DOI":"10.2196\/mhealth.9856","article-title":"Development of an mHealth platform for HIV Care: Gathering User Perspectives Through Co-Design Workshops and Interviews","volume":"6","author":"Marent","year":"2018","journal-title":"JMIR Mhealth Uhealth"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1017\/S0266462311000638","article-title":"A model for assessment of telemedicine applications: Mast","volume":"28","author":"Kidholm","year":"2012","journal-title":"Int. J. Technol. Assess Health Care"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jordan, P., Thomas, B., Weerdmeester, B., and McLelland, I. (1996). SUS-a quick and dirty usability scale. Usability Evaluation in Industry, Taylor and Francis.","DOI":"10.1201\/9781498710411"},{"key":"ref_39","unstructured":"McClelland, L., Jordan, W., Thomas, B., and Weerdmeester, A. (1996). SUS: A \u2018quick and dirty\u2019 usability scale. Usability Evaluation in Industry, Taylor & Francis. [1st ed.]."},{"key":"ref_40","first-page":"114","article-title":"Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale","volume":"4","author":"Bangor","year":"2009","journal-title":"J. Usability Stud."},{"key":"ref_41","unstructured":"World Health Organization (2021, February 09). People-Centred and Integrated Health Services: An Overview of the Evidence. Available online: https:\/\/apps.who.int\/iris\/bitstream\/handle\/10665\/155004\/WHO_HIS_SDS_2015.7_eng.pdf?sequence=1."},{"key":"ref_42","unstructured":"(2021, February 09). WHO Global Strategy on People-Centred and Integrated Health Services. World Health Organization. Available online: https:\/\/apps.who.int\/iris\/bitstream\/handle\/10665\/155002\/WHO_HIS_SDS_2015.6_eng.pdf?sequence=1."},{"key":"ref_43","unstructured":"(2021, March 18). Regulation (EU) 2016\/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of such Data, and Repealing Directive 95\/46\/EC (General Data Protection Regulation). Available online: https:\/\/eur-lex.europa.eu\/eli\/reg\/2016\/679\/oj."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2","DOI":"10.15761\/JMT.1000130","article-title":"mHealth: A cost-effective solution to chronic problems?","volume":"2","author":"Hill","year":"2018","journal-title":"J. Med. Therap."},{"key":"ref_45","first-page":"e8","article-title":"Improving HIV-related care through eHealth","volume":"7","author":"Leoni","year":"2019","journal-title":"Lancet HIV"}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/6\/3156\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:37:47Z","timestamp":1760161067000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/6\/3156"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,18]]},"references-count":45,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["ijerph18063156"],"URL":"https:\/\/doi.org\/10.3390\/ijerph18063156","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,18]]}}}