{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T17:49:26Z","timestamp":1782409766052,"version":"3.54.5"},"reference-count":24,"publisher":"American Speech Language Hearing Association","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Speech Lang Hear Res"],"published-print":{"date-parts":[[2026,6,10]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Meta-therapy (MT) is a powerful dialogue-based element of voice therapy that scaffolds patients' cognitive models of treatment. MT dialogues have not historically been taught in an explicit manner, and substantial variability in identifying MT exists within and across clinicians. This low reliability is problematic for empirical research and educational transfer. Harnessing contemporary natural language processing technologies to stress test the concept of MT and its clinical instances may enhance our theoretical and empirical grasp of the construct.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method:<\/jats:title>\n                    <jats:p>To capture crucial therapeutic component parts and MT, 10,443 clinician utterances stemming from conversation training therapy sessions delivered by six expert voice-specialized speech-language pathologists were transcribed and analyzed with a refined annotation framework. Two independent raters annotated each session and reconciled disagreements through adjudication, and the resulting consensus was compared with an expert gold standard. Time distribution was analyzed. Linguistic profiles were derived from bigram frequencies in utterances reaching expert\u2013annotator consensus. Reliability and ambiguity in identification were assessed with multilabel confusion matrices, percentage agreement, Cohen's kappa, and Gwet's agreement coefficient (AC1).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>Gwet's AC1 indicated substantial-to-almost-perfect agreement for MT in most sessions, outperforming \u03ba and mitigating base-rate artifacts (AC1: up to .98). Mean stand-alone MT duration ranged from 0.32 to 3.88 min per session. Distinctive MT bigrams contrasted with motor practice or psychosocial collocations that typified direct and counseling content, although lexical overlap produced systematic confusions with MT blended with direct and education\/indirect labels.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions:<\/jats:title>\n                    <jats:p>The revised annotation model markedly improved the reproducibility of MT identification and revealed its linguistic signature. The model confirmed MT's role as a cross-cutting discourse that integrates with other therapeutic modalities. These advances provide an empirical foundation for machine learning classifiers, formal curriculum content, and further investigation into MT's contribution to treatment efficacy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplemental Material:<\/jats:title>\n                    <jats:p>\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"doi\" xlink:href=\"https:\/\/doi.org\/10.23641\/asha.32159277\">https:\/\/doi.org\/10.23641\/asha.32159277<\/jats:ext-link>\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1044\/2026_jslhr-25-00697","type":"journal-article","created":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T17:11:41Z","timestamp":1778260301000},"page":"2481-2497","source":"Crossref","is-referenced-by-count":0,"title":["Advances in the Identification and Agreement About Meta-Therapy for Voice Disorders: A Methodological Paper"],"prefix":"10.1044","volume":"69","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7444-4649","authenticated-orcid":false,"given":"Sarah","family":"Martineau","sequence":"first","affiliation":[{"name":"\u00c9cole d'orthophonie et d'audiologie, Universit\u00e9 de Montr\u00e9al, Quebec, Canada"},{"name":"Centre de recherche du Centre int\u00e9gr\u00e9 universitaire de sant\u00e9 et services sociaux de l'Est-de-l'\u00eele-de-Montr\u00e9al, H\u00f4pital Maisonneuve\u2013Rosemont, Quebec, Canada"},{"name":"University of Pittsburgh, PA"},{"name":"\u00c9cole de technologie sup\u00e9rieure, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pierre Andr\u00e9","family":"M\u00e9nard","sequence":"additional","affiliation":[{"name":"\u00c9cole de technologie sup\u00e9rieure, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0018-245X","authenticated-orcid":false,"given":"Jackie","family":"Gartner-Schmidt","sequence":"additional","affiliation":[{"name":"Carlow University, Pittsburgh, PA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5840-0160","authenticated-orcid":false,"given":"Leah Bernadette","family":"Helou","sequence":"additional","affiliation":[{"name":"University of Pittsburgh, PA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine Murphy","family":"Estes","sequence":"additional","affiliation":[{"name":"Voice and Swallow Program, Westchester Medical Center, Valhalla, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brianna K.","family":"Hammerle","sequence":"additional","affiliation":[{"name":"Department of Communication Sciences and Disorders, Syracuse University, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juliana K.","family":"Litts","sequence":"additional","affiliation":[{"name":"Grabscheid Voice and Swallowing Center, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marci","family":"Rosenberg","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2013Head &amp; Neck Surgery, Division of Laryngology, Michigan Medicine, Ann Arbor"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erin","family":"Schmura","sequence":"additional","affiliation":[{"name":"Strong Memorial Hospital, University of Rochester Medical Center, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sylvie","family":"Ratt\u00e9","sequence":"additional","affiliation":[{"name":"\u00c9cole de technologie sup\u00e9rieure, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1310","reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1044\/2022_PERSP-22-00082"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvoice.2015.06.007"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.1055\/s-0040-1722751"},{"key":"e_1_3_2_5_1","volume-title":"The it factor: Strategies for developing the therapeutic alliance","author":"Gartner-Schmidt J.","year":"2025","unstructured":"Gartner-Schmidt, J., & Hapner, E. (2025). The it factor: Strategies for developing the therapeutic alliance. MedBridge. https:\/\/www.medbridge.com\/educate\/courses\/the-it-factor-strategies-for-developing-the-therapeutic-alliance-edie-hapner-jackie-gartner-schmidt"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1044\/2019_JSLHR-S-19-0136"},{"key":"e_1_3_2_7_1","volume-title":"Voice therapy: Clinical case studies","author":"Helou L. B.","year":"2019","unstructured":"Helou, L. B. (2019). Beyond successful voice therapy techniques and tasks: The concept of meta-therapy. In J. C. Stemple & E. R. Hapner (Ed.), Voice therapy: Clinical case studies (5th ed.). Plural. https:\/\/www.pluralpublishing.com\/publications\/voice-therapy-clinical-case-studies","edition":"5"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.1055\/s-0040-1722756"},{"key":"e_1_3_2_9_1","article-title":"Toward a consensus on the definition and nature of meta-therapy in voice therapy: Current state and future directions","author":"Helou L. B.","unstructured":"Helou, L. B., Martineau, S., Gartner Schmidt, J. L., Almand, K. B., Battipaglia, J., Becker, D. R., Blanca, A., Fritz, R., Hammerle, B. K., Hapner, E., Jett\u00e9, M. E., Krusemark, C., Littks, J. K., McDowell, S. K., Merrick, G., Estes, C. M., Nicastro, G., Norotsky, R. L., Petty, B. E., \u2026 Tripp, R. M. (in press). Toward a consensus on the definition and nature of meta-therapy in voice therapy: Current state and future directions. International Journal of Speech-Language Pathology.","journal-title":"International Journal of Speech-Language Pathology"},{"key":"e_1_3_2_10_1","doi-asserted-by":"publisher","DOI":"10.3109\/14015439.2014.949302"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1093\/ptj\/pzaa078"},{"issue":"8","key":"e_1_3_2_12_1","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions, and reversals","volume":"10","author":"Levenshtein V. I.","year":"1966","unstructured":"Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8), 707\u2013710.","journal-title":"Soviet Physics Doklady"},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvoice.2019.03.014"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1044\/2025_AJSLP-24-00308"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1044\/2024_PERSP-23-00241"},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2021.09.004"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1044\/2022_PERSP-22-00092"},{"key":"e_1_3_2_18_1","volume-title":"Natural language annotation for machine learning","author":"Pustejovsky J.","year":"2013","unstructured":"Pustejovsky, J., & Stubbs, A. (2013). Natural language annotation for machine learning. O'Reilly Media."},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1044\/2022_PERSP-22-00072"},{"key":"e_1_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.apmr.2018.09.112"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1044\/2015_AJSLP-14-0030"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1044\/2021_AJSLP-21-00076"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.apmr.2020.09.383"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1044\/2023_AJSLP-23-00283"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-13-61"}],"container-title":["Journal of Speech, Language, and Hearing Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.asha.org\/doi\/pdf\/10.1044\/2026_JSLHR-25-00697","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T17:33:20Z","timestamp":1782408800000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.asha.org\/doi\/10.1044\/2026_JSLHR-25-00697"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,10]]},"references-count":24,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,10]]}},"alternative-id":["10.1044\/2026_JSLHR-25-00697"],"URL":"https:\/\/doi.org\/10.1044\/2026_jslhr-25-00697","relation":{},"ISSN":["1092-4388","1558-9102"],"issn-type":[{"value":"1092-4388","type":"print"},{"value":"1558-9102","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,10]]}}}