{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:33:09Z","timestamp":1787027589837,"version":"build-2736575974"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,5,22]],"date-time":"2023-05-22T00:00:00Z","timestamp":1684713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institute of Health (NIH)","doi-asserted-by":"publisher","award":["1S10OD025025-01"],"award-info":[{"award-number":["1S10OD025025-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Dynamic magnetic resonance imaging has emerged as a powerful modality for investigating upper-airway function during speech production. Analyzing the changes in the vocal tract airspace, including the position of soft-tissue articulators (e.g., the tongue and velum), enhances our understanding of speech production. The advent of various fast speech MRI protocols based on sparse sampling and constrained reconstruction has led to the creation of dynamic speech MRI datasets on the order of 80\u2013100 image frames\/second. In this paper, we propose a stacked transfer learning U-NET model to segment the deforming vocal tract in 2D mid-sagittal slices of dynamic speech MRI. Our approach leverages (a) low- and mid-level features and (b) high-level features. The low- and mid-level features are derived from models pre-trained on labeled open-source brain tumor MR and lung CT datasets, and an in-house airway labeled dataset. The high-level features are derived from labeled protocol-specific MR images. The applicability of our approach to segmenting dynamic datasets is demonstrated in data acquired from three fast speech MRI protocols: Protocol 1: 3 T-based radial acquisition scheme coupled with a non-linear temporal regularizer, where speakers were producing French speech tokens; Protocol 2: 1.5 T-based uniform density spiral acquisition scheme coupled with a temporal finite difference (FD) sparsity regularization, where speakers were producing fluent speech tokens in English, and Protocol 3: 3 T-based variable density spiral acquisition scheme coupled with manifold regularization, where speakers were producing various speech tokens from the International Phonetic Alphabetic (IPA). Segments from our approach were compared to those from an expert human user (a vocologist), and the conventional U-NET model without transfer learning. Segmentations from a second expert human user (a radiologist) were used as ground truth. Evaluations were performed using the quantitative DICE similarity metric, the Hausdorff distance metric, and segmentation count metric. This approach was successfully adapted to different speech MRI protocols with only a handful of protocol-specific images (e.g., of the order of 20 images), and provided accurate segmentations similar to those of an expert human.<\/jats:p>","DOI":"10.3390\/bioengineering10050623","type":"journal-article","created":{"date-parts":[[2023,5,22]],"date-time":"2023-05-22T05:04:44Z","timestamp":1684731884000},"page":"623","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Automatic Multiple Articulator Segmentation in Dynamic Speech MRI Using a Protocol Adaptive Stacked Transfer Learning U-NET Model"],"prefix":"10.3390","volume":"10","author":[{"given":"Subin","family":"Erattakulangara","sequence":"first","affiliation":[{"name":"Roy J. Carver Department of Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karthika","family":"Kelat","sequence":"additional","affiliation":[{"name":"Roy J. Carver Department of Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9514-0457","authenticated-orcid":false,"given":"David","family":"Meyer","sequence":"additional","affiliation":[{"name":"Janette Ogg Voice Research Center, Shenandoah University, Winchester, VA 22601, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarv","family":"Priya","sequence":"additional","affiliation":[{"name":"Department of Radiology, University of Iowa, Iowa City, IA 52242, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1812-6256","authenticated-orcid":false,"given":"Sajan Goud","family":"Lingala","sequence":"additional","affiliation":[{"name":"Roy J. Carver Department of Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USA"},{"name":"Department of Radiology, University of Iowa, Iowa City, IA 52242, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1109\/MSP.2008.918034","article-title":"Seeing speech: Capturing vocal tract shaping using real-time magnetic resonance imaging","volume":"25","author":"Bresch","year":"2008","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1016\/j.ejmp.2014.05.001","article-title":"Speech MRI: Morphology and function","volume":"30","author":"Scott","year":"2014","journal-title":"Physica Medica"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1002\/jmri.24997","article-title":"Recommendations for real-time speech MRI","volume":"43","author":"Lingala","year":"2016","journal-title":"J. Magn. Reson. Imaging"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.wocn.2008.10.002","article-title":"Timing effects of syllable structure and stress on nasals: A real-time MRI examination","volume":"37","author":"Byrd","year":"2009","journal-title":"J. Phon."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sorensen, T., Skordilis, Z., Toutios, A., Kim, Y.-C., Zhu, Y., Kim, J., Lammert, A., Ramanarayanan, V., Goldstein, L., and Byrd, D. (2017, January 20\u201324). Database of volumetric and real-time vocal tract MRI for speech science. Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech 2017), Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-608"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1080\/14015430701875653","article-title":"Vocal tract and register changes analysed by real-time MRI in male professional singers\u2014A pilot study","volume":"33","author":"Echternach","year":"2008","journal-title":"Logop. Phoniatr. Vocol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1827","DOI":"10.1121\/1.5068052","article-title":"How beatboxers produce percussion sounds: A real-time magnetic resonance imaging investigation","volume":"144","author":"Greer","year":"2018","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1038\/s41597-021-00976-x","article-title":"A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric images","volume":"8","author":"Lim","year":"2021","journal-title":"Sci. Data"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1597\/10-255","article-title":"MRI with synchronized audio to evaluate velopharyngeal insufficiency","volume":"49","author":"Maturo","year":"2012","journal-title":"Cleft Palate Craniofac. J."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1007\/s00247-014-3230-7","article-title":"Evaluating velopharyngeal closure with real-time MRI","volume":"45","author":"Miquel","year":"2015","journal-title":"Pediatr. Radiol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1044\/1092-4388(2013\/13-0085)","article-title":"Tongue motion patterns in post-glossectomy and typical speakers: A principal components analysis","volume":"57","author":"Stone","year":"2014","journal-title":"J. Speech Lang. Hear. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4205","DOI":"10.1121\/1.4831435","article-title":"Characterizing post-glossectomy speech using real-time magnetic resonance imaging","volume":"134","author":"Hagedorn","year":"2013","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e35","DOI":"10.1597\/10-189","article-title":"Using magnetic resonance imaging for early assessment of submucous cleft palate: A case report","volume":"49","author":"Perry","year":"2012","journal-title":"Cleft Palate Craniofac. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1002\/mrm.26090","article-title":"A fast and flexible MRI system for the study of dynamic vocal tract shaping","volume":"77","author":"Lingala","year":"2017","journal-title":"Magn. Reson. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1002\/mrm.24276","article-title":"Real-time MRI of speaking at a resolution of 33 ms: Undersampled radial FLASH with nonlinear inverse reconstruction","volume":"69","author":"Niebergall","year":"2013","journal-title":"Magn. Reson. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1619","DOI":"10.1002\/mrm.26248","article-title":"High-frame-rate full-vocal-tract 3D dynamic speech imaging","volume":"77","author":"Fu","year":"2017","journal-title":"Magn. Reson. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1467","DOI":"10.1002\/mrm.27139","article-title":"Assessment of velopharyngeal function with dual-planar high-resolution real-time spiral dynamic MRI","volume":"80","author":"Feng","year":"2018","journal-title":"Magn. Reson. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1002\/jmri.24857","article-title":"Acceleration of MRI of the vocal tract provides additional insight into articulator modifications","volume":"42","author":"Burdumy","year":"2015","journal-title":"J. Magn. Reson. Imaging"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1038\/s41597-021-01041-3","article-title":"Multimodal dataset of real-time 2D and static 3D MRI of healthy French speakers","volume":"8","author":"Isaieva","year":"2021","journal-title":"Sci. Data"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1109\/TMI.2008.928920","article-title":"Region segmentation in the frequency domain applied to upper airway real-time magnetic resonance images","volume":"28","author":"Bresch","year":"2009","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_21","unstructured":"Kim, J., Kumar, N., Lee, S., and Narayanan, S. (2014, January 5\u20138). Enhanced airway-tissue boundary segmentation for real-time magnetic resonance imaging data. Proceedings of the 10th International Seminar on Speech Production (ISSP 2014), Cologne, Germany."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1109\/TBME.2015.2462750","article-title":"Dynamic 3-D MR visualization and detection of upper airway obstruction during sleep using region-growing segmentation","volume":"63","author":"Javed","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Skordilis, Z.I., Ramanarayanan, V., Goldstein, L., and Narayanan, S.S. (2015, January 6\u201310). Experimental assessment of the tongue incompressibility hypothesis during speech production. Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech 2015), Dresden, Germany.","DOI":"10.21437\/Interspeech.2015-160"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Somandepalli, K., Toutios, A., and Narayanan, S.S. (2017, January 20\u201324). Semantic edge detection for tracking vocal tract air-Tissue boundaries in real-Time magnetic resonance images. Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech 2017), Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-1580"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Valliappan, C.A., Kumar, A., Mannem, R., Karthik, G.R., and Ghosh, P.K. (2019, January 12\u201317). An Improved Air Tissue Boundary Segmentation Technique for Real Time Magnetic Resonance Imaging Video Using Segnet. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal. Proceedings, Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683153"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Erattakulangara, S., and Lingala, S.G. (2020, January 3\u20137). Airway segmentation in speech MRI using the U-net architecture. Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI 2020), Iowa City, IA, USA.","DOI":"10.1109\/ISBI45749.2020.9098536"},{"key":"ref_27","unstructured":"Valliappan, C.A., Mannem, R., and Kumar Ghosh, P. (2018, January 2\u20136). Air-tissue boundary segmentation in real-time magnetic resonance imaging video using semantic segmentation with fully convolutional networks. Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech 2018), Hyderabad, India."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105814","DOI":"10.1016\/j.cmpb.2020.105814","article-title":"Deep-learning-based segmentation of the vocal tract and articulators in real-time magnetic resonance images of speech","volume":"198","author":"Ruthven","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1002\/mp.15345","article-title":"Automatic upper airway segmentation in static and dynamic MRI via anatomy-guided convolutional neural networks","volume":"49","author":"Xie","year":"2022","journal-title":"Med. Phys."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1993","DOI":"10.1109\/TMI.2014.2377694","article-title":"The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)","volume":"34","author":"Menze","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_31","unstructured":"Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., and Rozycki, M. (2018). Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge. Sandra Gonzlez-Vill., 124."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3250","DOI":"10.1002\/mp.14107","article-title":"CT images with expert manual contours of thoracic cancer for benchmarking auto-segmentation accuracy","volume":"47","author":"Yang","year":"2020","journal-title":"Med. Phys."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1109\/34.232073","article-title":"Comparing Images Using the Hausdorff Distance","volume":"15","author":"Huttenlocher","year":"1993","journal-title":"Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2514","DOI":"10.1109\/TMI.2018.2837502","article-title":"Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?","volume":"37","author":"Bernard","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_35","unstructured":"Li, L., Zimmer, V.A., Schnabel, J.A., and Zhuang, X. (October, January 27). AtrialGeneral: Domain Generalization for Left Atrial Segmentation of Multi-Center LGE MRIs. Proceedings of the MICCAI, Strasbourg, France."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"102360","DOI":"10.1016\/j.media.2022.102360","article-title":"Medical image analysis on left atrial LGE MRI for atrial fibrillation studies: A review","volume":"77","author":"Li","year":"2022","journal-title":"Med. Image Anal."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"102303","DOI":"10.1016\/j.media.2021.102303","article-title":"AtrialJSQnet: A New framework for joint segmentation and quantification of left atrium and scars incorporating spatial and shape information","volume":"76","author":"Li","year":"2022","journal-title":"Med. Image Anal."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1590\/1678-775720150421","article-title":"Analysis of speech and tongue motion in normal and post-glossectomy speaker using cine MRI","volume":"24","author":"Ha","year":"2016","journal-title":"J. Appl. Oral Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1109\/TMI.2004.828354","article-title":"Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation","volume":"23","author":"Warfield","year":"2004","journal-title":"IEEE Trans. Med. Imaging"}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/10\/5\/623\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:39:48Z","timestamp":1760125188000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/10\/5\/623"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,22]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["bioengineering10050623"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering10050623","relation":{},"ISSN":["2306-5354"],"issn-type":[{"value":"2306-5354","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,22]]}}}