{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T00:24:59Z","timestamp":1775780699757,"version":"3.50.1"},"reference-count":29,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T00:00:00Z","timestamp":1711411200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["sigmapubs.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J of Nursing Scholarship"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Introduction<\/jats:title><jats:p>Common goals for procedural sedation are to control pain and ensure the patient is not moving to an extent that is impeding safe progress or completion of the procedure. Clinicians perform regular assessments of the adequacy of procedural sedation in accordance with these goals to inform their decision\u2010making around sedation titration and also for documentation of the care provided. Natural language processing could be applied to real\u2010time transcriptions of audio recordings made during procedures in order to classify sedation states that involve movement and pain, which could then be integrated into clinical documentation systems. The aim of this study was to determine whether natural language processing algorithms will work with sufficient accuracy to detect sedation states during procedural sedation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Design<\/jats:title><jats:p>A prospective observational study was conducted.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Audio recordings from consenting participants undergoing elective procedures performed in the interventional radiology suite at a large academic hospital were transcribed using an automated speech recognition model. Sentences of transcribed text were used to train and evaluate several different NLP pipelines for a text classification task. The NLP pipelines we evaluated included a simple Bag\u2010of\u2010Words (BOW) model, an ensemble architecture combining a linear BOW model and a \u201ctoken\u2010to\u2010vector\u201d (Tok2Vec) component, and a transformer\u2010based architecture using the RoBERTa pre\u2010trained model.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>A total of 15,936 sentences from transcriptions of 82 procedures was included in the analysis. The RoBERTa model achieved the highest performance among the three models with an area under the ROC curve (AUC\u2010ROC) of 0.97, an F1 score of 0.87, a precision of 0.86, and a recall of 0.89. The Ensemble model showed a similarly high AUC\u2010ROC of 0.96, but lower F1 score of 0.79, precision of 0.83, and recall of 0.77. The BOW approach achieved an AUC\u2010ROC of 0.97 and the F1 score was 0.7, precision was 0.83 and recall was 0.66.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The transformer\u2010based architecture using the RoBERTa pre\u2010trained model achieved the best classification performance. Further research is required to confirm the that this natural language processing pipeline can accurately perform text classifications with real\u2010time audio data to allow for automated sedation state assessments.<\/jats:p><\/jats:sec><jats:sec><jats:title>Clinical Relevance<\/jats:title><jats:p>Automating sedation state assessments using natural language processing pipelines would allow for more timely documentation of the care received by sedated patients, and, at the same time, decrease documentation burden for clinicians. Downstream applications can also be generated from the classifications, including for example real\u2010time visualizations of sedation state, which may facilitate improved communication of the adequacy of the sedation between clinicians, who may be performing supervision remotely. Also, accumulation of sedation state assessments from multiple procedures may reveal insights into the efficacy of particular sedative medications or identify procedures where the current approach for sedation and analgesia is not optimal (i.e. a significant amount of time spent in \u201cpain\u201d or \u201cmovement\u201d sedation states).<\/jats:p><\/jats:sec>","DOI":"10.1111\/jnu.12968","type":"journal-article","created":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T06:44:46Z","timestamp":1711521886000},"page":"17-27","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Automating sedation state assessments using natural language processing"],"prefix":"10.1111","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9583-8636","authenticated-orcid":false,"given":"Aaron","family":"Conway","sequence":"first","affiliation":[{"name":"School of Nursing QUT (Queensland University of Technology)  Brisbane Queensland Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jack","family":"Li","sequence":"additional","affiliation":[{"name":"Lawrence S. Bloomberg Faculty of Nursing University of Toronto  Toronto Ontario Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammad Goudarzi","family":"Rad","sequence":"additional","affiliation":[{"name":"Lawrence S. Bloomberg Faculty of Nursing University of Toronto  Toronto Ontario Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0832-6358","authenticated-orcid":false,"given":"Blair","family":"Warren","sequence":"additional","affiliation":[{"name":"Joint Department of Medical Imaging University Health Network  Toronto Ontario Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebastian","family":"Mafeld","sequence":"additional","affiliation":[{"name":"Joint Department of Medical Imaging University Health Network  Toronto Ontario Canada"},{"name":"Department of Medical Imaging University of Toronto  Toronto Ontario Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Babak","family":"Taati","sequence":"additional","affiliation":[{"name":"KITE Research Institute \u2013 Toronto Rehabilitation Institute University Health Network  Toronto Ontario Canada"},{"name":"Department of Computer Science University of Toronto  Toronto Ontario Canada"},{"name":"Institute of Biomaterials and Biomedical Engineering University of Toronto  Toronto Ontario Canada"},{"name":"Vector Institute for Artificial Intelligence  Toronto Ontario Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2024,3,26]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"crossref","unstructured":"Alsentzer E. Murphy J. R. Boag W. Weng W.\u2010H. Jin D. Naumann T. &McDermott M.(2019).Publicly available clinical BERT embeddings.arXiv Preprint arXiv:1904.03323.","DOI":"10.18653\/v1\/W19-1909"},{"key":"e_1_2_10_3_1","doi-asserted-by":"crossref","unstructured":"Bain M. Huh J. Han T. &Zisserman A.(2023).WhisperX: Time\u2010accurate speech transcription of long\u2010form audio.INTERSPEECH 2023.","DOI":"10.21437\/Interspeech.2023-78"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jopan.2013.05.017"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/jocn.12147"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aucc.2013.05.003"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2016-2897"},{"key":"e_1_2_10_8_1","unstructured":"D\u00e9fossez A.(2021).Hybrid spectrogram and waveform source separation.Proceedings of the ISMIR 2021 Workshop on Music Source Separation."},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12630-018-1231-y"},{"key":"e_1_2_10_10_1","unstructured":"Goodfellow I. Warde\u2010Farley D. Mirza M. Courville A. &Bengio Y.(2013).Maxout networks.International Conference on Machine Learning 1319\u20131327."},{"key":"e_1_2_10_11_1","unstructured":"Honnibal M. Montani I. Van Landeghem S. &Boyd A.(2020).spaCy: Industrial\u2010strength natural language processing in python[Article].https:\/\/doi.org\/10.5281\/zenodo.1212303"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2018.09.002"},{"key":"e_1_2_10_13_1","unstructured":"Kingma D. P. &Ba J.(2014).Adam: A method for stochastic optimization.arXiv Preprint arXiv:1412.6980."},{"key":"e_1_2_10_14_1","unstructured":"Kuchaiev O. Li J. Nguyen H. Hrinchuk O. Leary R. Ginsburg B. Kriman S. Beliaev S. Lavrukhin V. Cook J. Castonguay P. Popova M. Huang J. &Cohen J. M.(2019).Nemo: A toolkit for building ai applications using neural modules.arXiv Preprint arXiv:1909.09577."},{"key":"e_1_2_10_15_1","unstructured":"Liu Y. Ott M. Goyal N. Du J. Joshi M. Chen D. Levy O. Lewis M. Zettlemoyer L. &Stoyanov V.(2019).RoBERTa: A robustly optimized BERT pretraining approach.CoRR abs\/1907.11692.http:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmn.2022.01.007"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(20)30275-2"},{"key":"e_1_2_10_18_1","unstructured":"Miranda L. J. K\u00e1d\u00e1r \u00c1. Boyd A. Van Landeghem S. S\u00f8gaard A. &Honnibal M.(2022).Multi hash embeddings in spaCy.arXiv Preprint arXiv:2212.09255."},{"key":"e_1_2_10_19_1","unstructured":"Montani I. &Honnibal M.(2022).Prodigy: A modern and scriptable annotation tool for creating training data for machine learning models.Prodigy. Explosion.https:\/\/prodi.gy\/"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2019-033199"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/204946370800100203"},{"key":"e_1_2_10_22_1","unstructured":"Radford A. Kim J. W. Xu T. Brockman G. McLeavey C. &Sutskever I.(2022).Robust speech recognition via large\u2010scale weak supervision.arXivhttps:\/\/doi.org\/10.48550\/ARXIV.2212.04356"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.1097\/ACO.0000000000000792"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/1744987120913590"},{"key":"e_1_2_10_25_1","first-page":"1","article-title":"Hybrid transformers for music source separation","author":"Rouard S.","year":"2023","journal-title":"ICASSP 2023 \u2010 2023 IEEE International Conference on Accousstics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023"},{"key":"e_1_2_10_26_1","first-page":"179","volume-title":"Pediatric sedation outside of the operating room: A multispecialty international collaboration","author":"Thomas J. J.","year":"2011"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1259\/bjr.20220769"},{"key":"e_1_2_10_28_1","unstructured":"Ye J. Chen X. Xu N. Zu C. Shao Z. Liu S. Cui Y. Zhou Z. Gong C. Shen Y. et\u00a0al. (2023).A comprehensive capability analysis of gpt\u20103 and gpt\u20103.5 series models.arXiv Preprint arXiv:2303.10420."},{"key":"e_1_2_10_29_1","unstructured":"Yu H. Yang Z. Pelrine K. Godbout J. F. &Rabbany R.(2023).Open closed or small language models for text classification?arXiv Preprint arXiv:2308.10092."},{"key":"e_1_2_10_30_1","doi-asserted-by":"crossref","unstructured":"Zarghami Y. Mafeld S. Conway A. &Taati B.(2023).Pain detection in masked faces during procedural sedation.2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG) 1\u20136.","DOI":"10.1109\/FG57933.2023.10042502"}],"container-title":["Journal of Nursing Scholarship"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/sigmapubs.onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jnu.12968","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,29]],"date-time":"2025-01-29T13:24:11Z","timestamp":1738157051000},"score":1,"resource":{"primary":{"URL":"https:\/\/sigmapubs.onlinelibrary.wiley.com\/doi\/10.1111\/jnu.12968"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,26]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1111\/jnu.12968"],"URL":"https:\/\/doi.org\/10.1111\/jnu.12968","archive":["Portico"],"relation":{},"ISSN":["1527-6546","1547-5069"],"issn-type":[{"value":"1527-6546","type":"print"},{"value":"1547-5069","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,26]]},"assertion":[{"value":"2024-01-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}