{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T04:05:00Z","timestamp":1789790700287,"version":"4.0.1"},"reference-count":26,"publisher":"Wiley","license":[{"start":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T00:00:00Z","timestamp":1789689600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T00:00:00Z","timestamp":1789689600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["P50DC01990"],"award-info":[{"award-number":["P50DC01990"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["The Laryngoscope"],"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Develop a real\u2010time prediction model to detect laryngeal behaviors from videolaryngoscopy\u2010based tracking data during clinical examinations. This will improve data collection quality by providing immediate feedback to clinicians and enable future automation pipelines for efficient large\u2010scale analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We trained a stateful residual Gated Recurrent Unit (GRU) model that analyzed the tracking data of 39 laryngeal keypoints, derived from our published keypoint detection model, to predict the patient's laryngeal task state. These included \u201cphonation,\u201d \u201csustained phonation,\u201d \u201cswallowing,\u201d \u201cidle,\u201d \u201ccoughing,\u201d \u201csniffing,\u201d and \u201cout of view.\u201d Model development used 916 state segments comprising 222,065 video frames from 72 laryngoscopy videos. The resulting model was evaluated on an independent dataset comprising 8 videos, 123 segments, and 49,770 frames. Performance was assessed using classification metrics and temporal intersection over union (mIoU).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The model achieved a mean accuracy score of 92% and an mIoU of 0.82 when evaluated on the independent test dataset, indicating agreement with manual annotations and accurate temporal identification of laryngeal task states. At the class level, the model performed consistently across most state categories, with per\u2010class F1\u2010scores on the test set ranging from 83% for phonation to 97% for sniffing. The lowest validation F1 was observed for cough (82% [70%\u201392%]).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Real\u2010time classification of laryngeal states from videolaryngoscopy\u2010based pose tracking during clinical examinations is feasible. With reliable automated detection of states such as swallowing and phonation, this work establishes a foundation for further automated analysis, interpretation, and documentation of laryngeal pathology.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Level of Evidence<\/jats:title>\n                    <jats:p>4.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/lary.70927","type":"journal-article","created":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T03:54:07Z","timestamp":1789790047000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Real\u2010Time Classification of Functional Laryngeal Behaviors From Vocal Fold Kinematics"],"prefix":"10.1002","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4116-344X","authenticated-orcid":false,"given":"Aki","family":"Koivu","sequence":"first","affiliation":[{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Massachusetts Eye &amp; Ear  Boston Massachusetts USA"},{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pin\u2010Yu","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Massachusetts Eye &amp; Ear  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7444-0437","authenticated-orcid":false,"given":"Kristina","family":"Simonyan","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Massachusetts Eye &amp; Ear  Boston Massachusetts USA"},{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3927-3984","authenticated-orcid":false,"given":"Matthew R.","family":"Naunheim","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Massachusetts Eye &amp; Ear  Boston Massachusetts USA"},{"name":"Department of Otolaryngology\u2010Head &amp; Neck Surgery Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,9,18]]},"reference":[{"issue":"1","key":"e_1_2_10_2_1","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1002\/lary.23630","article-title":"Diagnostic Accuracy of History, Laryngoscopy, and Stroboscopy","volume":"123","author":"Paul B. C.","year":"2013","journal-title":"Laryngoscope"},{"issue":"7","key":"e_1_2_10_3_1","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1111\/anae.13837","article-title":"Accuracy, Intra\u2010and Inter\u2010Rater Reliability of Three Scoring Systems for the Glottic View at Videolaryngoscopy","volume":"72","author":"O'Loughlin E. J.","year":"2017","journal-title":"Anaesthesia"},{"issue":"1","key":"e_1_2_10_4_1","doi-asserted-by":"crossref","first-page":"13760","DOI":"10.1038\/s41598-021-93149-0","article-title":"OpenHSV: An Open Platform for Laryngeal High\u2010Speed Videoendoscopy","volume":"11","author":"Kist A. M.","year":"2021","journal-title":"Scientific Reports"},{"issue":"6","key":"e_1_2_10_5_1","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1044\/2021_JSLHR-20-00498","article-title":"A Deep Learning Enhanced Novel Software Tool for Laryngeal Dynamics Analysis","volume":"64","author":"Kist A. M.","year":"2021","journal-title":"Journal of Speech, Language, and Hearing Research"},{"issue":"6","key":"e_1_2_10_6_1","doi-asserted-by":"crossref","first-page":"2098","DOI":"10.1044\/2022_JSLHR-21-00540","article-title":"A Deep Learning Approach for Quantifying Vocal Fold Dynamics During Connected Speech Using Laryngeal High\u2010Speed Videoendoscopy","volume":"65","author":"Yousef A. M.","year":"2022","journal-title":"Journal of Speech, Language, and Hearing Research"},{"issue":"2","key":"e_1_2_10_7_1","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0227791","article-title":"Fully Automatic Segmentation of Glottis and Vocal Folds in Endoscopic Laryngeal High\u2010Speed Videos Using a Deep Convolutional LSTM Network","volume":"15","author":"Fehling M. K.","year":"2020","journal-title":"PLoS One"},{"key":"e_1_2_10_8_1","doi-asserted-by":"crossref","first-page":"79545","DOI":"10.1109\/ACCESS.2019.2923002","article-title":"Prediction of the Location of the Glottis in Laryngeal Images by Using a Novel Deep\u2010Learning Algorithm","volume":"7","author":"Kim J. S.","year":"2019","journal-title":"IEEE Access"},{"key":"e_1_2_10_9_1","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.ebiom.2019.08.075","article-title":"Computer\u2010Aided Diagnosis of Laryngeal Cancer via Deep Learning Based on Laryngoscopic Images","volume":"48","author":"Xiong H.","year":"2019","journal-title":"eBioMedicine"},{"issue":"1","key":"e_1_2_10_10_1","doi-asserted-by":"crossref","first-page":"E219","DOI":"10.1002\/lary.28669","article-title":"An Open\u2010Source Computer Vision Tool for Automated Vocal Fold Tracking From Videoendoscopy","volume":"131","author":"Adamian N.","year":"2021","journal-title":"Laryngoscope"},{"issue":"9","key":"e_1_2_10_11_1","doi-asserted-by":"crossref","first-page":"2285","DOI":"10.1002\/lary.30473","article-title":"Predictive Outcomes of Deep Learning Measurement of the Anterior Glottic Angle in Bilateral Vocal Fold Immobility","volume":"133","author":"DeVore E. K.","year":"2023","journal-title":"Laryngoscope"},{"issue":"2","key":"e_1_2_10_12_1","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1002\/lary.70104","article-title":"Feasibility of Real\u2010Time Automated Vocal Fold Motion Tracking for\u00a0In\u2010Office Laryngoscopy","volume":"136","author":"Koivu A.","year":"2025","journal-title":"Laryngoscope"},{"issue":"4","key":"e_1_2_10_13_1","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1177\/0194599821989608","article-title":"Application of a Computer Vision Tool for Automated Glottic Tracking to Vocal Fold Paralysis Patients","volume":"165","author":"Wang T. V.","year":"2021","journal-title":"Otolaryngology\u2014Head and Neck Surgery: Official Journal of the American Academy of Otolaryngology\u2010Head and Neck Surgery"},{"issue":"2","key":"e_1_2_10_14_1","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1093\/jamia\/ocw112","article-title":"Using Recurrent Neural Network Models for Early Detection of Heart Failure Onset","volume":"24","author":"Choi E.","year":"2017","journal-title":"Journal of the American Medical Informatics Association"},{"issue":"1","key":"e_1_2_10_15_1","doi-asserted-by":"crossref","first-page":"6085","DOI":"10.1038\/s41598-018-24271-9","article-title":"Recurrent Neural Networks for Multivariate Time Series With Missing Values","volume":"8","author":"Che Z.","year":"2018","journal-title":"Scientific Reports"},{"key":"e_1_2_10_16_1","first-page":"551","volume-title":"International Conference on Medical Image Computing and Computer\u2010Assisted Intervention","author":"DiPietro R.","year":"2016"},{"key":"e_1_2_10_17_1","first-page":"6105","volume-title":"International Conference on Machine Learning","author":"Tan M.","year":"2019"},{"issue":"9","key":"e_1_2_10_18_1","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1038\/s41593-018-0209-y","article-title":"DeepLabCut: Markerless Pose Estimation of User\u2010Defined Body Parts With Deep Learning","volume":"21","author":"Mathis A.","year":"2018","journal-title":"Nature Neuroscience"},{"key":"e_1_2_10_19_1","unstructured":"M.Tkachenko M.Malyuk A.Holmanyuk andN.Liubimov \u201cLabel Studio: Data Labeling Software \u201dhttps:\/\/github.com\/HumanSignal\/label\u2010studio."},{"key":"e_1_2_10_20_1","doi-asserted-by":"crossref","unstructured":"K.Cho B.Van Merri\u00ebnboer \u00c7.Gul\u00e7ehre et\u00a0al. \u201cLearning Phrase Representations Using RNN Encoder\u2013Decoder for Statistical Machine Translation \u201dProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) 1724\u20131734.","DOI":"10.3115\/v1\/D14-1179"},{"key":"e_1_2_10_21_1","unstructured":"A.Graves \u201cGenerating Sequences With Recurrent Neural Networks \u201d(2013) arXiv Preprint arXiv:13080850."},{"key":"e_1_2_10_22_1","doi-asserted-by":"crossref","DOI":"10.1016\/j.neucom.2025.129396","article-title":"Residual\u2010Time Gated Recurrent Unit","volume":"624","author":"Wu Y.","year":"2025","journal-title":"Neurocomputing"},{"issue":"1","key":"e_1_2_10_23_1","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks From Overfitting","volume":"15","author":"Srivastava N.","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_10_24_1","unstructured":"A.Vaswani N.Shazeer N.Parmar et\u00a0al. \u201cAttention is All You\u00a0Need \u201d(2017) Advances in Neural Information Processing Systems 30."},{"key":"e_1_2_10_25_1","unstructured":"A.Paszke S.Gross F.Massa et\u00a0al. \u201cPyTorch: An Open Source Machine Learning Framework Version 2.5.0 \u201dhttps:\/\/pytorch.org."},{"key":"e_1_2_10_26_1","first-page":"961","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Caba Heilbron F.","year":"2015"},{"key":"e_1_2_10_27_1","first-page":"3131","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Richard A.","year":"2016"}],"container-title":["The Laryngoscope"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/lary.70927","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/lary.70927","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/lary.70927","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T03:54:11Z","timestamp":1789790051000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/lary.70927"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,18]]},"references-count":26,"alternative-id":["10.1002\/lary.70927"],"URL":"https:\/\/doi.org\/10.1002\/lary.70927","archive":["Portico"],"relation":{},"ISSN":["0023-852X","1531-4995"],"issn-type":[{"value":"0023-852X","type":"print"},{"value":"1531-4995","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,18]]},"assertion":[{"value":"2026-05-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"lary.70927"}}