{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T04:36:41Z","timestamp":1785299801730,"version":"3.55.0"},"reference-count":61,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2021,1,28]],"date-time":"2021-01-28T00:00:00Z","timestamp":1611792000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["P30DA027828"],"award-info":[{"award-number":["P30DA027828"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA037222-05S1"],"award-info":[{"award-number":["R01DA037222-05S1"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA037222"],"award-info":[{"award-number":["R01DA037222"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["P30DA027828-10S1"],"award-info":[{"award-number":["P30DA027828-10S1"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Between January 2016 and June 2020, the Substance Abuse and Mental Health Services Administration rapidly distributed $7.5 billion in response to the U.S. opioid crisis. These funds are designed to increase access to medications for addiction treatment, reduce unmet treatment need, reduce overdose death rates, and provide and sustain effective prevention, treatment and recovery activities. It is unclear whether or not the services developed using these funds will be sustained beyond the start-up period. Based on 34 (64%) State Opioid Response (SOR) applications, we assessed the states\u2019 sustainability plans focusing on potential funding sources, policies, and quality monitoring. We found variable commitment to sustainability across response plans with less than half the states adequately describing sustainability plans. States with higher proportions of opioid prescribing, opioid misuse, and poverty had somewhat higher scores on sustainment. A text mining\/machine learning approach automatically rated sustainability in SOR applications with an 82% accuracy compared to human ratings. Because life saving evidence-based programs and services may be lost, intentional commitment to sustainment beyond the bolus of start-up funding is essential.<\/jats:p>","DOI":"10.1371\/journal.pone.0245920","type":"journal-article","created":{"date-parts":[[2021,1,28]],"date-time":"2021-01-28T19:33:22Z","timestamp":1611862402000},"page":"e0245920","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":6,"title":["Sustainability planning in the US response to the opioid crisis: An examination using expert and text mining approaches"],"prefix":"10.1371","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3270-8786","authenticated-orcid":true,"given":"Carlos","family":"Gallo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karen","family":"Abram","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nanette","family":"Hannah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4376-1251","authenticated-orcid":true,"given":"Lauren","family":"Caton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barbara","family":"Cimaglio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9736-573X","authenticated-orcid":true,"given":"Mark","family":"McGovern","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C. Hendricks","family":"Brown","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,1,28]]},"reference":[{"key":"pone.0245920.ref001","volume-title":"12 Month-ending Provisional Counts of Drug Overdose Deaths","author":"Statistics NCfH","year":"2019"},{"key":"pone.0245920.ref002","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.drugalcdep.2018.08.029","article-title":"Twin epidemics: The surging rise of methamphetamine use in chronic opioid users","volume":"193","author":"MS Ellis","year":"2018","journal-title":"Drug Alcohol Depend"},{"key":"pone.0245920.ref003","volume-title":"Opioid Summaries by State","author":"National Institute on Drug Abuse"},{"key":"pone.0245920.ref004","volume-title":"Provisional Drug Overdose Death Counts","author":"F Ahmad","year":"2019"},{"key":"pone.0245920.ref005","unstructured":"Center for Disease Control and Prevention. Synthetic Overdose Data [Internet]. 2020 [Available from: https:\/\/www.cdc.gov\/drugoverdose\/data\/fentanyl.html]."},{"key":"pone.0245920.ref006","unstructured":"National Association of State Alcohol and Drug Abuse Directors. Timeline of Funding for the State Targeted Response (STR) and State Opioid Response (SOR) Grant Programs. 2019."},{"key":"pone.0245920.ref007","volume-title":"Substance Abuse and Mental Health Administration","author":"Substance Abuse and Mental Health Services Administration","year":"2020"},{"issue":"1","key":"pone.0245920.ref008","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1093\/tbm\/ibz170","article-title":"Conceptualizing and measuring sustainability of prevention programs, policies, and practices","volume":"10","author":"LA Palinkas","year":"2020","journal-title":"Translational Behavioral Medicine"},{"key":"pone.0245920.ref009","first-page":"19","article-title":"Terminology for dissemination and implementation research","author":"BA Rabin","year":"2017","journal-title":"Dissemination and implementation research in health: Translating science to practice"},{"issue":"1","key":"pone.0245920.ref010","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1186\/s13012-017-0637-1","article-title":"Developing a comprehensive definition of sustainability","volume":"12","author":"JE Moore","year":"2017","journal-title":"Implementation Science"},{"key":"pone.0245920.ref011","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1186\/1748-5908-7-17","article-title":"The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research","volume":"7","author":"S Wiltsey Stirman","year":"2012","journal-title":"Implemention Science"},{"issue":"1","key":"pone.0245920.ref012","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1186\/s13012-019-0910-6","article-title":"Evidence-based intervention sustainability strategies: a systematic review","volume":"14","author":"M Hailemariam","year":"2019","journal-title":"Implementation Science"},{"key":"pone.0245920.ref013","article-title":"Conceptualizing and measuring sustainability of prevention programs and initiatives","author":"L Palinkas","year":"2018","journal-title":"PSMG Virtual Grand Rounds"},{"issue":"11","key":"pone.0245920.ref014","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.2105\/AJPH.2011.300193","article-title":"An agenda for research on the sustainability of public health programs","volume":"101","author":"MA Scheirer","year":"2011","journal-title":"American Journal of Public Health"},{"issue":"1","key":"pone.0245920.ref015","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1186\/s13012-020-01030-x","article-title":"Measurement of sustainment of prevention programs and initiatives: the sustainment measurement system scale","volume":"15","author":"LA Palinkas","year":"2020","journal-title":"Implementation Science"},{"issue":"5","key":"pone.0245920.ref016","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1007\/s10488-014-0538-4","article-title":"Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention.","volume":"42","author":"CG Gallo","year":"2015","journal-title":"Administration and Policy in Mental Health and Mental Health Services Research"},{"issue":"1","key":"pone.0245920.ref017","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1186\/s13012-016-0483-6","article-title":"Automatic classification of communication logs into implementation stages via text analysis","volume":"11","author":"D Wang","year":"2016","journal-title":"Implementation Science"},{"key":"pone.0245920.ref018","unstructured":"Substance Abuse and Mental Health Services Administration. Directory of Single State Agencies (SSA) for Substance Abuse Services [Internet]. samhsa.gov 2016 [30 Mar 2020]. Available from: https:\/\/www.samhsa.gov\/sites\/default\/files\/ssadirectory.pdf]."},{"key":"pone.0245920.ref019","article-title":"The Prospects for Sustaining Evidence-based Responses to the US Opioid Epidemic State Leadership Perspectives","author":"L Caton","year":"2020","journal-title":"To Appear in Substance Abuse Treatment, Prevention, and Policy"},{"issue":"4","key":"pone.0245920.ref020","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1525\/maq.1990.4.4.02a00020","article-title":"A Systematic Approach for Using Qualitative Methods in Primary Prevention Research","volume":"4","author":"DG Willms","year":"1990","journal-title":"Medical Anthropology Quarterly"},{"key":"pone.0245920.ref021","unstructured":"United States Census Bureau. Annual Estimates of the Resident Population for the United States, Regions, States, and Puerto Rico: April 1, 2010 to July 1, 2019."},{"key":"pone.0245920.ref022","unstructured":"United States Census Bureau. US Census QuickFacts [Internet]. census.gov [Available from: https:\/\/www.census.gov\/quickfacts\/US]."},{"key":"pone.0245920.ref023","unstructured":"Governing the future of states and localities. State Population By Race, Ethnicity Data [Internet]. 2017 [Available from: https:\/\/www.governing.com\/gov-data\/census\/state-minority-population-data-estimates.html]."},{"key":"pone.0245920.ref024","unstructured":"List of U.S. states and territoris by poverty rate US Census Bureau2020 [Available from: https:\/\/www.census.gov\/quickfacts\/fact\/table\/US\/PST045219."},{"key":"pone.0245920.ref025","unstructured":"Kaiser Family Foundation. Health Insurance Coverage of the Total Population [Internet]. KFF.org 2018 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/total-population\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"key":"pone.0245920.ref026","unstructured":"Kaiser Family Foundation. Individuals Reporting Past Year Opioid Use Disorder [Internet]. KFF.org 2017\u20132018 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/past-year-opioid-use-disorder\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"key":"pone.0245920.ref027","unstructured":"Substance Abuse and Mental Health Services Administration CfBHSaQ. Treatment Episode Data Set (TEDS): 2017. Admissions to and Discharges from Publicly-Funded Substance Use Treatment. Substance Abuse and Mental Health Services Administration. 2017."},{"key":"pone.0245920.ref028","unstructured":"2018 N-SSATS State Profiles. Substance Abuse and Mental Health Services Administration [Internet]. 2018 [Available from: https:\/\/www.samhsa.gov\/data\/sites\/default\/files\/reports\/rpt23232\/2018_NSSATS_StPro_combined.pdf]."},{"key":"pone.0245920.ref029","unstructured":"Center for Disease Control and Prevention. Nonfatal Overdoses: All Opioids [Internet]. 2020 [updated 2020 Apr 21. Available from: https:\/\/www.cdc.gov\/drugoverdose\/data\/nonfatal\/nonfatal-opioids.html]."},{"key":"pone.0245920.ref030","unstructured":"Kaiser Family Foundation. Opioid Overdose Death Rates and All Drug Overdose Death Rates per 100,000 Population (Age-Adjusted) [Internet]. KFF.org 2020 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/opioid-overdose-death-rates\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"issue":"4","key":"pone.0245920.ref031","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1097\/ADM.0000000000000324","article-title":"Misuse of Novel Synthetic Opioids: A Deadly New Trend","volume":"11","author":"MP Prekupec","year":"2017","journal-title":"Journal of Addiction Medicine"},{"key":"pone.0245920.ref032","unstructured":"Kaiser Family Foundation. Individuals Reporting Needing but Not Receiving Treatment for Illicit Drug Use in the Past Year [Internet]. KFF.org 2017\u20132018 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/individuals-reporting-needing-but-not-receiving-treatment-for-illicit-drug-use-in-the-past-year\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"key":"pone.0245920.ref033","unstructured":"Kaiser Family Foundation. Medicaid Behavioral Health Services: Naloxone Coverage Provided for Family Members or Friends Obtaining a Naloxone Prescription on Enrollee\u2019s Behalf [Internet]. KFF.org 2018 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/medicaid-behavioral-health-services-naloxone-coverage-provided-for-family-members-or-friends-obtaining-naloxone-prescription-on-enrollees-behalf\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"key":"pone.0245920.ref034","unstructured":"Substance Abuse and Mental Health Services Administration. Opioid Treatment Program Directory [Internet]. samhsa.gov [Available from: https:\/\/dpt2.samhsa.gov\/treatment\/directory.aspx]."},{"key":"pone.0245920.ref035","unstructured":"Substance Abuse and Mental Health Services Administration. Buprenorphine Practitioner Locator [Internet]. samhsa.gov [Available from: https:\/\/www.samhsa.gov\/medication-assisted-treatment\/practitioner-program-data\/treatment-practitioner-locator]."},{"key":"pone.0245920.ref036","unstructured":"Healthinsurance.org. What is Medicaid expansion? [Internet]. Healthinsurance.org [updated 21 Feb 2020. Available from: https:\/\/www.healthinsurance.org\/glossary\/medicaid-expansion\/]."},{"key":"pone.0245920.ref037","unstructured":"Kaiser Family Foundation. Medicaid Behavioral Health Services: Methadone for Medication Assisted Treatment (MAT) [Available from: https:\/\/www.kff.org\/other\/state-indicator\/medicaid-behavioral-health-services-methadone-for-medication-assisted-treatment-mat\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D."},{"key":"pone.0245920.ref038","unstructured":"Substance Abuse and Mental Health Services Administration. Supplemental NSDUH Opioid Tables. 2018."},{"key":"pone.0245920.ref039","unstructured":"Kaiser Family Foundation. Medicaid Behavioral Health Services: Methadone for Medication Assisted Treatment (MAT) [Internet]. KFF.org 2018 [Available from: https:\/\/www.kff.org\/other\/state-indicator\/medicaid-behavioral-health-services-methadone-for-medication-assisted-treatment-mat\/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D]."},{"key":"pone.0245920.ref040","unstructured":"Iowa State University Iowa Community Indicators Program. Urban Percentage of the Population for States, Historical [Internet]. [Available from: https:\/\/www.icip.iastate.edu\/tables\/population\/urban-pct-states]."},{"key":"pone.0245920.ref041","unstructured":"National Conference of State Legislatures. Medicaid 1115 Waivers by State [Internet]. ncsl.org 2019 [updated 2019 Aug 30. Available from: https:\/\/www.ncsl.org\/research\/health\/medicaid-1115-waivers-by-state.aspx]."},{"key":"pone.0245920.ref042","unstructured":"KFF. Medicaid Waiver Tracker: Approved and Pending Section 1115 Waivers by State 2020 [Available from: https:\/\/www.kff.org\/medicaid\/issue-brief\/medicaid-waiver-tracker-approved-and-pending-section-1115-waivers-by-state\/."},{"key":"pone.0245920.ref043","unstructured":"Kaiser Family Foundation. Medicaid Waiver Tracker: Approved and Pending Section 1115 Waivers by State [Internet]. KFF.org 2020 [Available from: https:\/\/www.kff.org\/medicaid\/issue-brief\/medicaid-waiver-tracker-approved-and-pending-section-1115-waivers-by-state\/]."},{"key":"pone.0245920.ref044","unstructured":"Mitchell AB, Sara. Overview of the ACA Medicaid Expansion [Internet]. 2018 [updated 2018 Dec 3. Available from: https:\/\/fas.org\/sgp\/crs\/misc\/IF10399.pdf]."},{"issue":"1","key":"pone.0245920.ref045","first-page":"1","article-title":"Probability Plotting Methods for the Analysis of Data","volume":"55","author":"MB Wilk","year":"1968","journal-title":"Biometrika"},{"issue":"17\u201318","key":"pone.0245920.ref046","doi-asserted-by":"crossref","first-page":"2191","DOI":"10.1002\/(SICI)1097-0258(19990915\/30)18:17\/18<2191::AID-SIM249>3.0.CO;2-M","article-title":"Modelling ordered categorical data: recent advances and future challenges","volume":"18","author":"A Agresti","year":"1999","journal-title":"Statistics in Medicine"},{"issue":"1","key":"pone.0245920.ref047","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/s13012-015-0209-1","article-title":"A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project","volume":"10","author":"BJ Powell","year":"2015","journal-title":"Implementation Science"},{"issue":"2","key":"pone.0245920.ref048","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.jbusvent.2016.11.001","article-title":"Linguistic style and crowdfunding success among social and commercial entrepreneurs","volume":"32","author":"A Parhankangas","year":"2017","journal-title":"Journal of Business Venturing"},{"issue":"12","key":"pone.0245920.ref049","doi-asserted-by":"crossref","first-page":"e115844","DOI":"10.1371\/journal.pone.0115844","article-title":"When small words foretell academic success: the case of college admissions essays","volume":"9","author":"JW Pennebaker","year":"2014","journal-title":"PLoS One"},{"issue":"9","key":"pone.0245920.ref050","first-page":"28","volume":"15","author":"A Karatzoglou","year":"2006","journal-title":"Support Vector Machines in R"},{"issue":"6","key":"pone.0245920.ref051","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1109\/TPAMI.2016.2578326","article-title":"Cross Validation Through Two-Dimensional Solution Surface for Cost-Sensitive SVM","volume":"39","author":"B Gu","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"pone.0245920.ref052","doi-asserted-by":"crossref","first-page":"281","DOI":"10.5194\/isprsarchives-XL-2-W3-281-2014","article-title":"A Comparison Study of Different Kernel Functions for SVM-Based Classification of Multi-Temporal Polarimetry SAR Data","author":"B Yekkehkhany","year":"2014","journal-title":"International Archives of the Photogrammetry Remote Sensing and Spatial Information Science"},{"key":"pone.0245920.ref053","volume-title":"An Optimized Cost-Sensitive SVM for Imbalanced Data Learning","author":"P Cao","year":"2013"},{"key":"pone.0245920.ref054","first-page":"826","article-title":"Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA","volume":"18","author":"L Kotthoff","year":"2017","journal-title":"Journal of Machine Learning Research"},{"issue":"1","key":"pone.0245920.ref055","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1093\/tbm\/ibz170","article-title":"Conceptualizing and measuring sustainability of prevention programs and initiatives","volume":"10","author":"LA Palinkas","year":"2019","journal-title":"Translational and Behavioral Medicine"},{"issue":"6","key":"pone.0245920.ref056","doi-asserted-by":"crossref","first-page":"926","DOI":"10.1016\/j.amepre.2018.07.024","article-title":"A Machine Learning Approach to Identify NIH-Funded Applied Prevention Research","volume":"55","author":"J Villani","year":"2018","journal-title":"Am J Prev Med"},{"issue":"2","key":"pone.0245920.ref057","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1007\/s11414-015-9475-6","article-title":"Methods to Improve the Selection and Tailoring of Implementation Strategies","volume":"44","author":"BJ Powell","year":"2017","journal-title":"The Journal of Behavioral Health Services & Research"},{"key":"pone.0245920.ref058","first-page":"572","volume-title":"Government inter-organizational information sharing initiatives: Understanding the main determinants of success","author":"JR Gil-Garcia","year":"2016"},{"issue":"1","key":"pone.0245920.ref059","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13012-018-0842-6","article-title":"Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework","volume":"14","author":"JC Moullin","year":"2019","journal-title":"Implementation Science"},{"issue":"1","key":"pone.0245920.ref060","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1186\/s13012-020-00999-9","article-title":"Contracting as a bridging factor linking outer and inner contexts during EBP implementation and sustainment: a prospective study across multiple U.S. public sector service systems","volume":"15","author":"R Lengnick-Hall","year":"2020","journal-title":"Implementation Science"},{"key":"pone.0245920.ref061","article-title":"The prospects for sustaining eidence-based responses to the US opioid epidemic: state leadership perspectives","author":"L Caton","year":"2020","journal-title":"Under Review"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0245920","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,28]],"date-time":"2021-01-28T19:34:05Z","timestamp":1611862445000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0245920"}},"subtitle":[],"editor":[{"given":"Giuseppe","family":"Carr\u00e0","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,1,28]]},"references-count":61,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1,28]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0245920","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,28]]}}}