{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T14:54:30Z","timestamp":1793199270732,"version":"4.5.0"},"reference-count":35,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T00:00:00Z","timestamp":1778544000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Health Serv."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Tracking implementation strategies and their adaptations over time remains a persistent methodological challenge in health services research. The Longitudinal Implementation Strategy Tracking System (LISTS) was developed to support structured, longitudinal documentation of strategies, modifications, and discontinuations aligned with established implementation science frameworks using timeline follow-back procedures and an open-access web-based platform. The study objective was to evaluate the usability, feasibility, and methodological fit of the open-access web-based LISTS platform using a structured think-aloud protocol with implementation researchers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Nine implementation researchers completed structured think-aloud usability sessions while performing core LISTS tasks, including project setup, strategy entry, bundling, modification logging, discontinuation tracking, and dashboard exploration. Sessions were transcribed and analyzed using hybrid deductive-inductive thematic analysis. Participants also completed the System Usability Scale (SUS) and NASA Task Load Index (NASA-TLX).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The mean SUS score was 76.94, indicating good perceived usability, and mean NASA-TLX workload was low (1.98 on a 1\u20137 scale). Qualitative findings revealed consistent endorsement of LISTS' structured longitudinal tracking and theoretical alignment with well-known frameworks. Usability barriers included terminology density, multi-level unit hierarchy confusion, and ambiguity in modification-tracking workflows. A cross-cutting tension emerged between theoretical fidelity and accessibility for non-expert users. Despite conceptual friction, participants completed tasks successfully and rated the system positively overall.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>LISTS demonstrates strong conceptual rigor and functional usability among implementation researchers. However, refinements to terminology scaffolding, hierarchical logic, and modification workflows are needed to enhance accessibility and scalability beyond expert users. Findings highlight broad challenges and considerations for translating implementation science theory and terminology into practical digital infrastructure.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frhs.2026.1837215","type":"journal-article","created":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T05:43:32Z","timestamp":1778564612000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Usability and feasibility of the longitudinal implementation strategy tracking system: a think-aloud study with implementation researchers"],"prefix":"10.3389","volume":"6","author":[{"given":"James L.","family":"Merle","sequence":"first","affiliation":[{"name":"Department of Population Health Sciences, Division of Health System Innovation and Research, Spencer Fox Eccles School of Medicine, University of Utah","place":["Salt Lake City, UT, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin D.","family":"Smith","sequence":"additional","affiliation":[{"name":"Department of Population Health Sciences, Division of Health System Innovation and Research, Spencer Fox Eccles School of Medicine, University of Utah","place":["Salt Lake City, UT, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13012-015-0209-1","article-title":"A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project","volume":"10","author":"Powell","year":"2015","journal-title":"Implement Sci"},{"key":"B2","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1186\/1748-5908-8-139","article-title":"Implementation strategies: recommendations for specifying and reporting","volume":"8","author":"Proctor","year":"2013","journal-title":"Implement Sci"},{"key":"B3","doi-asserted-by":"publisher","first-page":"829","DOI":"10.1001\/jamapsychiatry.2022.1462","article-title":"Implementation strategies used and reported in brief suicide prevention intervention studies","volume":"79","author":"Rudd","year":"2022","journal-title":"JAMA Psychiatry"},{"key":"B4","doi-asserted-by":"publisher","first-page":"101269","DOI":"10.1016\/j.conctc.2024.101269","article-title":"Tracking activities and adaptations in a multi-site stepped wedge pragmatic trial of a cancer symptom management intervention","volume":"38","author":"Ridgeway","year":"2024","journal-title":"Contemp Clin Trials Commun"},{"key":"B5","doi-asserted-by":"publisher","first-page":"983217","DOI":"10.3389\/frhs.2022.983217","article-title":"Tracking dynamic changes in implementation strategies over time within a hybrid type 2 trial of an electronic patient-reported oncology symptom and needs monitoring program","volume":"2","author":"Smith","year":"2022","journal-title":"Frontiers in Health Services"},{"key":"B6","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1016\/j.beth.2017.11.012","article-title":"A method for tracking implementation strategies: an exemplar implementing measurement-based care in community behavioral health clinics","volume":"49","author":"Boyd","year":"2018","journal-title":"Behav Ther"},{"key":"B7","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s12961-017-0175-y","article-title":"Tracking implementation strategies: a description of a practical approach and early findings","volume":"15","author":"Bunger","year":"2017","journal-title":"Health Res Policy Syst"},{"key":"B8","doi-asserted-by":"publisher","first-page":"26334895211016028","DOI":"10.1177\/26334895211016028","article-title":"A pilot study comparing tools for tracking implementation strategies and treatment adaptations","volume":"2","author":"Walsh-Bailey","year":"2021","journal-title":"Implement Res Pract"},{"key":"B9","doi-asserted-by":"publisher","first-page":"349325","DOI":"10.3389\/fpubh.2018.00102","article-title":"Systematic, multimethod assessment of adaptations across four diverse health systems interventions","volume":"6","author":"Rabin","year":"2018","journal-title":"Front Public Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1186\/s12874-021-01326-6","article-title":"Strengthening methods for tracking adaptations and modifications to implementation strategies","volume":"21","author":"Haley","year":"2021","journal-title":"BMC Med Res Methodol"},{"key":"B11","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1186\/s43058-023-00529-w","article-title":"The longitudinal implementation strategy tracking system (LISTS): feasibility, usability, and pilot testing of a novel method","volume":"4","author":"Smith","year":"2023","journal-title":"Implement Sci Commun"},{"key":"B12","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1186\/s13012-021-01105-3","article-title":"The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare","volume":"16","author":"Miller","year":"2021","journal-title":"Implement Sci"},{"key":"B13","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1007\/978-1-4612-0357-5_3","article-title":"Timeline follow-back","volume-title":"Measuring Alcohol Consumption: Psychosocial and Biochemical Methods","author":"Sobell","year":"1992"},{"key":"B14","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/1748-5908-4-50","article-title":"Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science","volume":"4","author":"Damschroder","year":"2009","journal-title":"Implement Sci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1186\/s13012-022-01245-0","article-title":"The updated consolidated framework for implementation research based on user feedback","volume":"17","author":"Damschroder","year":"2022","journal-title":"Implement Sci"},{"key":"B16","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1186\/s13012-015-0295-0","article-title":"Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the expert recommendations for implementing change (ERIC) study","volume":"10","author":"Waltz","year":"2015","journal-title":"Implement Sci"},{"key":"B17","year":""},{"key":"B18","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1109\/47.867942","article-title":"Thinking aloud: reconciling theory and practice","volume":"43","author":"Boren","year":"2000","journal-title":"IEEE Trans Prof Commun"},{"key":"B19","volume-title":"Usability Engineering","author":"Nielsen","year":"1994"},{"key":"B20","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1080\/10447318.2018.1455307","article-title":"The system usability scale: past, present, and future","volume":"34","author":"Lewis","year":"2018","journal-title":"Int J Hum Comput Interact"},{"key":"B21","first-page":"904","article-title":"Nasa-Task Load Index (NASA-TLX); 20 Years Later","volume-title":"Proc Hum Factors Ergon Soc Annu Meet","author":"Hart","year":"2006"},{"key":"B22","first-page":"173","volume-title":"Designing for People","author":"Lee","year":"2017"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1053496","DOI":"10.3389\/frhs.2022.1053496","article-title":"Reflections on 10 years of effectiveness-implementation hybrid studies","volume":"2","author":"Curran","year":"2022","journal-title":"Front Health Serv"},{"key":"B24","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1186\/s13012-021-01181-5","article-title":"Conceptualizing outcomes for use with the consolidated framework for implementation research (CFIR): the CFIR outcomes addendum","volume":"17","author":"Damschroder","year":"2022","journal-title":"Implement Sci"},{"key":"B25","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1177\/1049731513505778","article-title":"A systematic review of strategies for implementing empirically supported mental health interventions","volume":"24","author":"Powell","year":"2014","journal-title":"Res Soc Work Pract"},{"key":"B26","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s10488-010-0327-7","article-title":"Advancing a conceptual model of evidence-based practice implementation in public service sectors","volume":"38","author":"Aarons","year":"2011","journal-title":"Adm Policy Ment Health"},{"key":"B27","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/S1553-7250(08)34030-6","article-title":"A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice","volume":"34","author":"Feldstein","year":"2008","journal-title":"Jt Comm J Qual Patient Saf"},{"key":"B28","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3389\/fpubh.2019.00064","article-title":"RE-AIM Planning and evaluation framework: adapting to new science and practice with a 20-year review","volume":"7","author":"Glasgow","year":"2019","journal-title":"Front Public Health"},{"key":"B29","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda","volume":"38","author":"Proctor","year":"2011","journal-title":"Adm Policy Ment Health"},{"key":"B30","first-page":"114","article-title":"Determining what individual SUS scores mean: adding an adjective rating scale","volume":"4","author":"Bangor","year":"2009","journal-title":"J Usability Stud"},{"key":"B31","first-page":"1727","article-title":"How high is high? a meta-analysis of NASA-TLX global workload scores","volume-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet","author":"Grier","year":"2015"},{"key":"B32","volume-title":"Designing and Conducting Mixed Methods Research","author":"Creswell","year":"2017"},{"key":"B33","doi-asserted-by":"publisher","first-page":"2134","DOI":"10.1111\/1475-6773.12117","article-title":"Achieving integration in mixed methods designs-principles and practices","volume":"48","author":"Fetters","year":"2013","journal-title":"Health Serv Res"},{"key":"B34","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1080\/10447310802205776","article-title":"An empirical evaluation of the system usability scale","volume":"24","author":"Bangor","year":"2008","journal-title":"Int J Hum Comput Interact"},{"key":"B35","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1186\/s13012-023-01286-z","article-title":"Ten years of implementation outcomes research: a scoping review","volume":"18","author":"Proctor","year":"2023","journal-title":"Implement Sci"}],"container-title":["Frontiers in Health Services"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frhs.2026.1837215\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T05:43:35Z","timestamp":1778564615000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frhs.2026.1837215\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,12]]},"references-count":35,"alternative-id":["10.3389\/frhs.2026.1837215"],"URL":"https:\/\/doi.org\/10.3389\/frhs.2026.1837215","relation":{},"ISSN":["2813-0146"],"issn-type":[{"value":"2813-0146","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,12]]},"article-number":"1837215"}}