{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T08:23:27Z","timestamp":1782289407970,"version":"3.54.5"},"reference-count":138,"publisher":"Acoustical Society of America (ASA)","issue":"3","content-domain":{"domain":["pubs.aip.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:p>B-mode ultrasound has emerged as a prevalent tool for observing tongue motion in speech production, gaining traction in speech therapy applications. However, the effective analysis of ultrasound tongue image frame sequences (UTIFs) encounters many challenges, such as the presence of high levels of speckle noise and obscured views. Recently, the application of machine learning, especially deep learning techniques, to UTIF interpretation has shown promise in overcoming these hurdles. This paper presents a thorough examination of the existing literature, focusing on UTIF analysis. The scope of our work encompasses four key areas: a foundational introduction to deep learning principles, an exploration of motion tracking methodologies, a discussion of feature extraction techniques, and an examination of cross-modality mapping. The paper concludes with a detailed discussion of insights gleaned from the comprehensive literature review, outlining potential trends and challenges that lie ahead in the field.<\/jats:p>","DOI":"10.1121\/10.0028610","type":"journal-article","created":{"date-parts":[[2024,9,17]],"date-time":"2024-09-17T18:18:48Z","timestamp":1726597128000},"page":"1796-1819","update-policy":"https:\/\/doi.org\/10.1063\/aip-crossmark-policy-page","source":"Crossref","is-referenced-by-count":7,"title":["A systematic review of the application of machine learning techniques to ultrasound tongue imaging analysis"],"prefix":"10.1121","volume":"156","author":[{"given":"Zhen","family":"Xia","sequence":"first","affiliation":[{"name":"National Key Lab of Parallel and Distributed Processing, National University of Defense Technology 1 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5009-5173","authenticated-orcid":false,"given":"Ruicheng","family":"Yuan","sequence":"additional","affiliation":[{"name":"College of Computer Science and Electronic Engineering, Hunan University 2 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuan","family":"Cao","sequence":"additional","affiliation":[{"name":"National Key Lab of Parallel and Distributed Processing, National University of Defense Technology 1 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tao","family":"Sun","sequence":"additional","affiliation":[{"name":"National Key Lab of Parallel and Distributed Processing, National University of Defense Technology 1 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunsheng","family":"Xiong","sequence":"additional","affiliation":[{"name":"National Key Lab of Parallel and Distributed Processing, National University of Defense Technology 1 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kele","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Lab of Parallel and Distributed Processing, National University of Defense Technology 1 , Changsha, Hunan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"231","published-online":{"date-parts":[[2024,9,17]]},"reference":[{"key":"2024091718184219000_c1","first-page":"126","article-title":"Automatic motion analysis of the tongue surface from ultrasound image sequences","year":"1998"},{"key":"2024091718184219000_c2","first-page":"298","article-title":"Extraction and tracking of the tongue surface from ultrasound image sequences","year":"1998"},{"issue":"10","key":"2024091718184219000_c3","doi-asserted-by":"publisher","first-page":"1035","DOI":"10.1109\/42.811315","article-title":"Automatic extraction and tracking of the tongue contours","volume":"18","year":"1999","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2024091718184219000_c4","doi-asserted-by":"crossref","unstructured":"Al Ani, \nS. (2023). \u201c\nSystematic review of deep learning models in ultrasound tongue imaging for the detection of speech disorders,\u201d Technical Report (TechRxiv) (Authorea), https:\/\/www.techrxiv.org\/users\/691092\/articles\/681920-systematic-review-of-deep-learning-models-in-ultrasound-tongue-imaging-for-the-detection-of-speech-disorders.","DOI":"10.36227\/techrxiv.22699291"},{"key":"2024091718184219000_c5","first-page":"326","article-title":"Automated classification of phonetic segments in child speech using raw ultrasound imaging","year":"2024"},{"key":"2024091718184219000_c6","unstructured":"Al-Hammuri, \nK. (2019). \u201c\nComputer vision-based tracking and feature extraction for lingual ultrasound,\u201d Ph.D. thesis, \nUniversity of Victoria, \nVictoria, Australia."},{"issue":"11","key":"2024091718184219000_c7","doi-asserted-by":"publisher","first-page":"2811","DOI":"10.3390\/diagnostics12112811","article-title":"Tongue contour tracking and segmentation in lingual ultrasound for speech recognition: A review","volume":"12","year":"2022","journal-title":"Diagnostics"},{"key":"2024091718184219000_c8","first-page":"173","article-title":"Deep speech 2: End-to-end speech recognition in English and Mandarin","volume-title":"Proceedings of the International Conference on Machine Learning","year":"2016"},{"key":"2024091718184219000_c9","article-title":"Automatic classification of tongue gestures in ultrasound images","year":"2010"},{"key":"2024091718184219000_c10","article-title":"Recognition and real time performances of a lightweight ultrasound based silent speech interface employing a language model","year":"2011"},{"key":"2024091718184219000_c11","first-page":"676","article-title":"Vocal tract imaging system for post-laryngectomy voice replacement","year":"2013"},{"key":"2024091718184219000_c12","article-title":"Extending text-to-speech synthesis with articulatory movement prediction using ultrasound tongue imaging","year":"2021"},{"key":"2024091718184219000_c13","article-title":"Ultrasound-based silent speech interface built on a continuous vocoder","year":"2019"},{"key":"2024091718184219000_c14","article-title":"Comparison of acoustic-to-articulatory and brain-to-articulatory mapping during speech production using ultrasound tongue imaging and EEG","year":"2023"},{"key":"2024091718184219000_c15","first-page":"1164","article-title":"Towards Ultrasound Tongue Image prediction from EEG during speech production","year":"2023"},{"issue":"22","key":"2024091718184219000_c16","doi-asserted-by":"publisher","first-page":"8601","DOI":"10.3390\/s22228601","article-title":"Optimizing the ultrasound tongue image representation for residual network-based articulatory-to-acoustic mapping","volume":"22","year":"2022","journal-title":"Sensors"},{"key":"2024091718184219000_c17","first-page":"3672","article-title":"Dnn-based ultrasound-to-speech conversion for a silent speech interface","year":"2017"},{"key":"2024091718184219000_c18","article-title":"Ultrasound-based articulatory-to-acoustic mapping with waveglow speech synthesis","year":"2020"},{"key":"2024091718184219000_c19","doi-asserted-by":"publisher","first-page":"159","DOI":"10.12700\/APH.18.2.2021.2.9","article-title":"Impact of preprocessing features on the performance of ultrasound tongue contour tracking, via dynamic programming","volume":"18","year":"2021","journal-title":"Acta Polytech. Hung."},{"issue":"5","key":"2024091718184219000_c20","doi-asserted-by":"publisher","first-page":"3646","DOI":"10.1121\/1.4987881","article-title":"Recent results in silent speech interfaces","volume":"141","year":"2017","journal-title":"J. Acoust. Soc. Am."},{"issue":"4","key":"2024091718184219000_c21","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.specom.2009.08.002","article-title":"Silent speech interfaces","volume":"52","year":"2010","journal-title":"Speech Commun."},{"key":"2024091718184219000_c22","first-page":"I","article-title":"Speech synthesis from real time ultrasound images of the tongue","year":"2004"},{"key":"2024091718184219000_c23","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","year":"2019"},{"issue":"10","key":"2024091718184219000_c24","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.1109\/LSP.2014.2332186","article-title":"Maximum voiced frequency estimation: Exploiting amplitude and phase spectra","volume":"21","year":"2014","journal-title":"IEEE Signal Process. Lett."},{"key":"2024091718184219000_c25","first-page":"1888","article-title":"UltraSuite: A Repository of Ultrasound and Acoustic Data from Child Speech Therapy Sessions","year":"2018"},{"key":"2024091718184219000_c26","article-title":"Tongue tracking in ultrasound images using eigentongue decomposition and artificial neural networks","year":"2015"},{"key":"2024091718184219000_c27","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1016\/j.specom.2017.08.002","article-title":"Automatic animation of an articulatory tongue model from ultrasound images of the vocal tract","volume":"93","year":"2017","journal-title":"Speech Commun."},{"key":"2024091718184219000_c28","first-page":"1493","article-title":"Deep belief networks for real-time extraction of tongue contours from ultrasound during speech","year":"2010"},{"key":"2024091718184219000_c29","first-page":"6443","article-title":"Improving ultrasound tongue contour extraction using u-net and shape consistency-based regularizer","year":"2021"},{"key":"2024091718184219000_c30","first-page":"7053","article-title":"Self-supervised moving vehicle tracking with stereo sound","year":"2019"},{"issue":"1","key":"2024091718184219000_c31","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1109\/LSP.2012.2231675","article-title":"A simple continuous pitch estimation algorithm","volume":"20","year":"2012","journal-title":"IEEE Signal Process. Lett."},{"key":"2024091718184219000_c32","first-page":"1046","article-title":"Interactive segmentation of tongue contours in ultrasound video sequences using quality maps","year":"2014"},{"key":"2024091718184219000_c33","first-page":"241","article-title":"Medical image denoising using convolutional denoising autoencoders","year":"2016"},{"key":"2024091718184219000_c34","volume-title":"Deep Learning","year":"2016"},{"key":"2024091718184219000_c35","first-page":"1","article-title":"Autoencoder-based articulatory-to-acoustic mapping for ultrasound silent speech interfaces","year":"2019"},{"key":"2024091718184219000_c36","first-page":"6645","article-title":"Speech recognition with deep recurrent neural networks","year":"2013"},{"key":"2024091718184219000_c37","first-page":"291","article-title":"F0 estimation for dnn-based ultrasound silent speech interfaces","year":"2018"},{"issue":"6","key":"2024091718184219000_c38","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition","volume":"29","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"issue":"9","key":"2024091718184219000_c39","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1113\/expphysiol.2008.042424","article-title":"Breathing rhythms and emotions","volume":"93","year":"2008","journal-title":"Exp. Physiol."},{"issue":"07","key":"2024091718184219000_c40","doi-asserted-by":"publisher","first-page":"10973","DOI":"10.1609\/aaai.v34i07.6731","article-title":"Joint commonsense and relation reasoning for image and video captioning","volume":"34","year":"2020","journal-title":"AAAI"},{"key":"2024091718184219000_c41","doi-asserted-by":"crossref","unstructured":"Hu, \nS., \nXie, \nX., \nGeng, \nM., \nCui, \nM., \nDeng, \nJ., \nWang, \nT., \nLiu, \nX., and \nMeng, \nH. (2022). \u201c\nExploiting cross-domain and cross-lingual ultrasound tongue imaging features for elderly and dysarthric speech recognition,\u201d arXiv:2206.07327.","DOI":"10.21437\/Interspeech.2023-297"},{"key":"2024091718184219000_c43","doi-asserted-by":"publisher","first-page":"1203","DOI":"10.1109\/TMI.2023.3332614","article-title":"Self-supervised deep unrolled reconstruction using regularization by denoising","volume":"43","year":"2023","journal-title":"IEEE Trans. Med. Imaging"},{"key":"2024091718184219000_c42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2017\/6027029","article-title":"A review on real-time 3D ultrasound imaging technology","volume":"2017","year":"2017","journal-title":"BioMed research international"},{"key":"2024091718184219000_c44","first-page":"I","article-title":"Eigentongue feature extraction for an ultrasound-based silent speech interface","year":"2007"},{"key":"2024091718184219000_c45","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1016\/j.csl.2015.03.005","article-title":"Statistical conversion of silent articulation into audible speech using full-covariance hmm","volume":"36","year":"2016","journal-title":"Comput. Speech Lang."},{"key":"2024091718184219000_c46","first-page":"P3c","article-title":"Continuous articulatory-to-acoustic mapping using phone-based trajectory hmm for a silent speech interface","year":"2012"},{"issue":"4","key":"2024091718184219000_c47","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.specom.2009.11.004","article-title":"Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips","volume":"52","year":"2010","journal-title":"Speech Commun."},{"key":"2024091718184219000_c48","article-title":"Statistical mapping between articulatory and acoustic data for an ultrasound-based silent speech interface","year":"2011"},{"key":"2024091718184219000_c49","article-title":"Data augmentation methods on ultrasound tongue images for articulation-to-speech synthesis","year":"2023"},{"issue":"6\u20137","key":"2024091718184219000_c50","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1080\/02699200500113871","article-title":"Detecting the edge of the tongue: A tutorial","volume":"19","year":"2005","journal-title":"Clin. Ling. Phon."},{"key":"2024091718184219000_c51","doi-asserted-by":"publisher","first-page":"47628","DOI":"10.1109\/ACCESS.2022.3170836","article-title":"A text-to-speech pipeline, evaluation methodology, and initial fine-tuning results for child speech synthesis","volume":"10","year":"2022","journal-title":"IEEE Access"},{"key":"2024091718184219000_c52","article-title":"Tongue contour extraction from ultrasound images based on deep neural network","year":"2016"},{"issue":"2","key":"2024091718184219000_c53","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1007\/s10484-015-9279-8","article-title":"Self-regulation of breathing as a primary treatment for anxiety","volume":"40","year":"2015","journal-title":"Appl. Psychophysiol. Biofeedback"},{"key":"2024091718184219000_c54","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.specom.2018.02.002","article-title":"Updating the silent speech challenge benchmark with deep learning","volume":"98","year":"2018","journal-title":"Speech Commun."},{"key":"2024091718184219000_c55","doi-asserted-by":"publisher","first-page":"102878","DOI":"10.1016\/j.media.2023.102878","article-title":"Robotic ultrasound imaging: State-of-the-art and future perspectives","volume":"89","year":"2023","journal-title":"Medical Image Anal."},{"issue":"5","key":"2024091718184219000_c56","doi-asserted-by":"publisher","first-page":"EL425","DOI":"10.1121\/1.5039726","article-title":"Visualizing the mechanical wave of vocal fold tissue during phonation using electroglottogram-triggered ultrasonography","volume":"143","year":"2018","journal-title":"J. Acoust. Soc. Am."},{"issue":"11","key":"2024091718184219000_c57","doi-asserted-by":"publisher","first-page":"4037","DOI":"10.1109\/TPAMI.2020.2992393","article-title":"Self-supervised visual feature learning with deep neural networks: A survey","volume":"43","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"4","key":"2024091718184219000_c58","doi-asserted-by":"publisher","first-page":"587","DOI":"10.3813\/AAA.919339","article-title":"Ultrasound-based silent speech interface using convolutional and recurrent neural networks","volume":"105","year":"2019","journal-title":"Acta Acust. united Ac."},{"key":"2024091718184219000_c59","doi-asserted-by":"publisher","first-page":"2268","DOI":"10.1121\/1.408637","article-title":"An ultrasonic method for monitoring tongue shape and the position of a fixed point on the tongue surface","volume":"95","year":"1994","journal-title":"J. Acoust. Soc. Am."},{"key":"2024091718184219000_c60","doi-asserted-by":"publisher","first-page":"103335","DOI":"10.1016\/j.compbiomed.2019.103335","article-title":"Fully-automated tongue detection in ultrasound images","volume":"111","year":"2019","journal-title":"Comput. Biol. Med."},{"issue":"4","key":"2024091718184219000_c61","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1007\/BF00133570","article-title":"Snakes: Active contour models","volume":"1","year":"1988","journal-title":"Int. J. Comput. Vision"},{"key":"2024091718184219000_c62","first-page":"1","article-title":"Sottovoce: An ultrasound imaging-based silent speech interaction using deep neural networks","year":"2019"},{"key":"2024091718184219000_c63","first-page":"6866","article-title":"Ssr7000: A synchronized corpus of ultrasound tongue imaging for end-to-end silent speech recognition","year":"2022"},{"key":"2024091718184219000_c64","article-title":"Robust tongue tracking in ultrasound images: A multi-hypothesis approach","year":"2015"},{"key":"2024091718184219000_c65","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1016\/j.media.2017.12.003","article-title":"Multi-hypothesis tracking of the tongue surface in ultrasound video recordings of normal and impaired speech","volume":"44","year":"2018","journal-title":"Med. Image Anal."},{"issue":"7553","key":"2024091718184219000_c66","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","year":"2015","journal-title":"Nature"},{"key":"2024091718184219000_c69","first-page":"7130","article-title":"Denoising convolutional autoencoder based b-mode ultrasound tongue image feature extraction","year":"2019"},{"key":"2024091718184219000_c67","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1016\/j.specom.2022.05.004","article-title":"wUnet: A new network used for ultrasonic tongue contour extraction","volume":"141","year":"2022","journal-title":"Speech Commun."},{"issue":"6\u20137","key":"2024091718184219000_c68","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1080\/02699200500113616","article-title":"Automatic contour tracking in ultrasound images","volume":"19","year":"2005","journal-title":"Clin. Ling. Phon."},{"key":"2024091718184219000_c70","first-page":"16743","article-title":"Swintrack: A simple and strong baseline for transformer tracking","volume":"35","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"1","key":"2024091718184219000_c71","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1002\/jmri.24997","article-title":"Recommendations for real-time speech mri","volume":"43","year":"2016","journal-title":"J. Magn. Reson. Imaging"},{"issue":"16\u201318","key":"2024091718184219000_c72","doi-asserted-by":"publisher","first-page":"3150","DOI":"10.1016\/j.neucom.2008.04.030","article-title":"Modeling word perception using the elman network","volume":"71","year":"2008","journal-title":"Neurocomputing"},{"key":"2024091718184219000_c78","unstructured":"Liu, \nH., and \nZhang, \nJ. (2021). \u201c\nImproving ultrasound tongue image reconstruction from lip images using self-supervised learning and attention mechanism,\u201d arXiv:2106.11769."},{"key":"2024091718184219000_c74","first-page":"1","article-title":"Comparison of DCT and autoencoder-based features for DNN-HMM multimodal silent speech recognition","year":"2016"},{"key":"2024091718184219000_c73","doi-asserted-by":"publisher","first-page":"2267","DOI":"10.1109\/TASLP.2021.3091805","article-title":"Recent progress in the cuhk dysarthric speech recognition system","volume":"29","year":"2021","journal-title":"IEEE\/ACM Trans. Audio. Speech. Lang. Process."},{"key":"2024091718184219000_c75","first-page":"10012","article-title":"Swin transformer: Hierarchical vision transformer using shifted windows","year":"2021"},{"issue":"7","key":"2024091718184219000_c76","doi-asserted-by":"publisher","first-page":"4486","DOI":"10.1109\/TCSVT.2021.3127149","article-title":"Swinnet: Swin transformer drives edge-aware RGB-D and RGB-T salient object detection","volume":"32","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"2024091718184219000_c79","first-page":"67","article-title":"Using a biomechanical model for tongue tracking in ultrasound images","year":"2014"},{"key":"2024091718184219000_c80","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1044\/2019_PERS-SIG19-2019-0001","article-title":"Three-\/four-dimensional ultrasound technology in speech research","volume":"4","year":"2019","journal-title":"Perspectives"},{"key":"2024091718184219000_c81","first-page":"52","article-title":"Stacked convolutional auto-encoders for hierarchical feature extraction","year":"2011"},{"issue":"9","key":"2024091718184219000_c82","doi-asserted-by":"publisher","first-page":"1281","DOI":"10.1038\/s41593-018-0209-y","article-title":"Deeplabcut: Markerless pose estimation of user-defined body parts with deep learning","volume":"21","year":"2018","journal-title":"Nat. Neurosci."},{"key":"2024091718184219000_c83","first-page":"1","article-title":"Guided learning of pronunciation by visualizing tongue articulation in ultrasound image sequences","year":"2018"},{"key":"2024091718184219000_c84","doi-asserted-by":"publisher","first-page":"2940","DOI":"10.1121\/1.5137211","article-title":"Transfer learning for ultrasound tongue contour extraction with different domains","volume":"146","year":"2019","journal-title":"J. Acoust. Soc. Am."},{"key":"2024091718184219000_c85","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1016\/j.ymeth.2020.05.011","article-title":"Encoder-decoder cnn models for automatic tracking of tongue contours in real-time ultrasound data","volume":"179","year":"2020","journal-title":"Methods"},{"issue":"3","key":"2024091718184219000_c86","doi-asserted-by":"publisher","first-page":"1623","DOI":"10.1121\/10.0000891","article-title":"Automatic tongue surface extraction from three-dimensional ultrasound vocal tract images","volume":"147","year":"2020","journal-title":"J. Acoust. Soc. Am."},{"issue":"4","key":"2024091718184219000_c87","doi-asserted-by":"publisher","first-page":"1771","DOI":"10.1121\/1.1652588","article-title":"An approach to real-time magnetic resonance imaging for speech production","volume":"115","year":"2004","journal-title":"J. Acoust. Soc. Am."},{"issue":"3","key":"2024091718184219000_c88","doi-asserted-by":"publisher","first-page":"1307","DOI":"10.1121\/1.4890284","article-title":"Real-time magnetic resonance imaging and electromagnetic articulography database for speech production research (TC)","volume":"136","year":"2014","journal-title":"J. Acoust. Soc. Am."},{"key":"2024091718184219000_c89","first-page":"129","article-title":"Multimodal learning analytics","volume":"1","year":"2017","journal-title":"Soc. Learn. Analytics Res. Beaumont, AB, Canada"},{"key":"2024091718184219000_c90","article-title":"Deep contextualized word representations","year":"2018"},{"key":"2024091718184219000_c91","first-page":"1","article-title":"Dnn-based acoustic-to-articulatory inversion using ultrasound tongue imaging","year":"2019"},{"key":"2024091718184219000_c92","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1016\/j.specom.2021.02.001","article-title":"Exploiting ultrasound tongue imaging for the automatic detection of speech articulation errors","volume":"128","year":"2021","journal-title":"Speech Commun."},{"key":"2024091718184219000_c93","first-page":"1328","article-title":"Speaker-independent classification of phonetic segments from raw ultrasound in child speech","year":"2019"},{"key":"2024091718184219000_c94","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation","year":"2015"},{"issue":"4","key":"2024091718184219000_c95","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1109\/TASL.2010.2072499","article-title":"Articulatory knowledge in the recognition of dysarthric speech","volume":"19","year":"2011","journal-title":"IEEE Trans. Audio. Speech. Lang. Process."},{"key":"2024091718184219000_c96","first-page":"473","article-title":"Ultra2speech-a deep learning framework for formant frequency estimation and tracking from ultrasound tongue images","year":"2020"},{"key":"2024091718184219000_c97","first-page":"430","article-title":"Improving neural silent speech interface models by adversarial training","year":"2021"},{"key":"2024091718184219000_c98","first-page":"924","article-title":"Estimation of vocal tract area function from volumetric magnetic resonance imaging","year":"2017"},{"key":"2024091718184219000_c99","first-page":"1","article-title":"Laboratory techniques for investigating speech articulation","volume":"1","year":"1997","journal-title":"Handbook Phon. Sci."},{"issue":"6\u20137","key":"2024091718184219000_c100","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1080\/02699200500113558","article-title":"A guide to analysing tongue motion from ultrasound images","volume":"19","year":"2005","journal-title":"Clin. Ling. Phon."},{"issue":"8","key":"2024091718184219000_c101","doi-asserted-by":"publisher","first-page":"1503","DOI":"10.1016\/j.media.2012.07.001","article-title":"Tongue contour tracking in dynamic ultrasound via higher-order mrfs and efficient fusion moves","volume":"16","year":"2012","journal-title":"Med. Image Anal."},{"key":"2024091718184219000_c102","first-page":"154","article-title":"Graph-based tracking of the tongue contour in ultrasound sequences with adaptive temporal regularization","year":"2010"},{"key":"2024091718184219000_c103","first-page":"2971","article-title":"Feature extraction using multimodal convolutional neural networks for visual speech recognition","year":"2017"},{"key":"2024091718184219000_c104","first-page":"159","article-title":"3d convolutional neural networks for ultrasound-based silent speech interfaces","year":"2020"},{"key":"2024091718184219000_c105","doi-asserted-by":"crossref","unstructured":"T\u00f3th, \nL., \nShandiz, \nA. H., \nGosztolya, \nG., and \nG\u00e1bor, \nC. T. (2023). \u201c\nAdaptation of tongue ultrasound-based silent speech interfaces using spatial transformer networks,\u201d arXiv:2305.19130.","DOI":"10.21437\/Interspeech.2023-1607"},{"key":"2024091718184219000_c106","first-page":"5998","article-title":"Attention is all you need","volume":"30","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"2024091718184219000_c107","doi-asserted-by":"publisher","first-page":"57511","DOI":"10.1007\/s11042-023-17813-5","article-title":"Automated tongue contour extraction from ultrasound sequences using signal enhancing neural network and energy minimized spline","volume":"83","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"2024091718184219000_c108","first-page":"3371","article-title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","volume":"11","year":"2010","journal-title":"J. Mach. Learn. Res."},{"issue":"4","key":"2024091718184219000_c111","doi-asserted-by":"publisher","first-page":"3087","DOI":"10.1121\/1.5137727","article-title":"Learning a representation of tongue dynamics from unlabeled ultrasound videos","volume":"146","year":"2019","journal-title":"J. Acoust. Soc. Am."},{"issue":"12","key":"2024091718184219000_c112","doi-asserted-by":"publisher","first-page":"2209","DOI":"10.1587\/transinf.2021EDP7090","article-title":"Representation learning of tongue dynamics for a silent speech interface","volume":"E104.D","year":"2021","journal-title":"IEICE Trans. Inf. Syst."},{"key":"2024091718184219000_c113","first-page":"200","article-title":"Automatic tongue contour tracking in ultrasound sequences without manual initialization","year":"2015"},{"issue":"4","key":"2024091718184219000_c109","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","year":"2004","journal-title":"IEEE Trans. Image Process."},{"issue":"11","key":"2024091718184219000_c110","doi-asserted-by":"crossref","first-page":"2385","DOI":"10.1109\/TIP.2009.2025923","article-title":"CWS: A new structural similarity index for image quality assessment","volume":"18","year":"2009","journal-title":"IEEE Trans. Image Process."},{"key":"2024091718184219000_c114","doi-asserted-by":"publisher","first-page":"5223","DOI":"10.1007\/s11042-015-3038-y","article-title":"Mapping ultrasound-based articulatory images and vowel sounds with a deep neural network framework","volume":"75","year":"2016","journal-title":"Multimed. Tools Appl."},{"key":"2024091718184219000_c115","unstructured":"Wen, \nS. (2018). \u201c\nAutomatic tongue contour segmentation using deep learning,\u201d M.S. thesis, \nUniversity of Ottawa, \nOttawa, Canada."},{"key":"2024091718184219000_c116","article-title":"Face reconstruction from voice using generative adversarial networks","year":"2019"},{"key":"2024091718184219000_c117","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1044\/1092-4388(2005\/037)","article-title":"The Haskins optically corrected ultrasound system (hocus)","volume":"48","year":"2005","journal-title":"J. Speech. Lang. Hear. Res."},{"key":"2024091718184219000_c118","article-title":"A multichannel articulatory database and its application for automatic speech recognition","year":"2000"},{"issue":"3","key":"2024091718184219000_c119","doi-asserted-by":"publisher","first-page":"1133","DOI":"10.3390\/s22031133","article-title":"Beyond the edge: Markerless pose estimation of speech articulators from ultrasound and camera images using deeplabcut","volume":"22","year":"2022","journal-title":"Sensors"},{"key":"2024091718184219000_c120","first-page":"5764","article-title":"Predicting tongue motion in unlabeled ultrasound video using 3d convolutional neural networks","year":"2018"},{"key":"2024091718184219000_c121","first-page":"8262","article-title":"Improving the classification of phonetic segments from raw ultrasound using self-supervised learning and hard example mining","year":"2022"},{"key":"2024091718184219000_c122","unstructured":"Xu, \nK. (2016). \u201c\n3D tongue motion visualization based on the b-mode ultrasound tongue images,\u201d Ph.D. thesis, \nUniversit\u00e9 Pierre et Marie Curie-Paris VI, \nParis."},{"issue":"5","key":"2024091718184219000_c123","doi-asserted-by":"publisher","first-page":"EL154","DOI":"10.1121\/1.4951024","article-title":"A comparative study on the contour tracking algorithms in ultrasound tongue images with automatic re-initialization","volume":"139","year":"2016","journal-title":"J. Acoust. Soc. Am."},{"issue":"6","key":"2024091718184219000_c124","doi-asserted-by":"publisher","first-page":"EL531","DOI":"10.1121\/1.4984122","article-title":"Convolutional neural network-based automatic classification of midsagittal tongue gestural targets using b-mode ultrasound images","volume":"141","year":"2017","journal-title":"J. Acoust. Soc. Am."},{"issue":"3","key":"2024091718184219000_c125","doi-asserted-by":"publisher","first-page":"365","DOI":"10.3813\/AAA.919065","article-title":"Is speckle tracking feasible for ultrasound tongue images?","volume":"103","year":"2017","journal-title":"Acta Acust. united Ac."},{"key":"2024091718184219000_c126","first-page":"2194","article-title":"Ultrasound-based silent speech interface using sequential convolutional auto-encoder,\u201d in Proceedings of","year":"2019"},{"issue":"3\u20135","key":"2024091718184219000_c127","doi-asserted-by":"publisher","first-page":"313","DOI":"10.3109\/02699206.2015.1110714","article-title":"Robust contour tracking in ultrasound tongue image sequences","volume":"30","year":"2016","journal-title":"Clin. Ling. Phon."},{"issue":"6","key":"2024091718184219000_c128","doi-asserted-by":"publisher","first-page":"EL441","DOI":"10.1121\/10.0001328","article-title":"Predicting ultrasound tongue image from lip images using sequence to sequence learning","volume":"147","year":"2020","journal-title":"J. Acoust. Soc. Am."},{"key":"2024091718184219000_c129","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.csl.2019.05.002","article-title":"Articulatory and bottleneck features for speaker-independent ASR of dysarthric speech","volume":"58","year":"2019","journal-title":"Comput. Speech Lang."},{"key":"2024091718184219000_c130","first-page":"1","article-title":"Raw ultrasound-based phonetic segments classification via mask modeling","year":"2023"},{"key":"2024091718184219000_c131","doi-asserted-by":"crossref","unstructured":"Zaink\u00f3, \nC., \nT\u00f3th, \nL., \nShandiz, \nA. H., \nGosztolya, \nG., \nMark\u00f3, \nA., \nN\u00e9meth, \nG., and \nCsap\u00f3, \nT. G. (2021). \u201c\nAdaptation of tacotron2-based text-to-speech for articulatory-to-acoustic mapping using ultrasound tongue imaging,\u201d arXiv:2107.12051.","DOI":"10.21437\/SSW.2021-10"},{"key":"2024091718184219000_c133","doi-asserted-by":"publisher","first-page":"13076","DOI":"10.1109\/ACCESS.2021.3050843","article-title":"Creating song from lip and tongue videos with a convolutional vocoder","volume":"9","year":"2021","journal-title":"IEEE Access"},{"key":"2024091718184219000_c132","doi-asserted-by":"publisher","first-page":"14402","DOI":"10.1609\/aaai.v35i16.17693","article-title":"TaLNet: Voice reconstruction from tongue and lip articulation with transfer learning from text-to-speech synthesis","volume":"35","year":"2021","journal-title":"AAAI"},{"key":"2024091718184219000_c136","first-page":"5926","article-title":"Predicting tongue motion in unlabeled ultrasound videos using convolutional LSTM neural networks","year":"2019"},{"issue":"6","key":"2024091718184219000_c135","doi-asserted-by":"publisher","first-page":"3182","DOI":"10.1002\/mrm.28651","article-title":"Improved 3D real-time MRI of speech production","volume":"85","year":"2021","journal-title":"Magn. Resonance Med."},{"issue":"2","key":"2024091718184219000_c137","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1016\/j.wocn.2008.10.005","article-title":"Measuring lingual coarticulation from midsagittal tongue contours: Description and example calculations using English","volume":"37","year":"2009","journal-title":"J. Phon."},{"issue":"2","key":"2024091718184219000_c138","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1017\/S0025100312000060","article-title":"An ultrasound study of lingual coarticulation in\/sv\/syllables produced by adults and typically developing children","volume":"42","year":"2012","journal-title":"J. Int. Phon. Assoc."},{"key":"2024091718184219000_c139","doi-asserted-by":"crossref","unstructured":"Zheng, \nR.-C., \nAi, \nY., and \nLing, \nZ.-H. (2023). \u201c\nIncorporating ultrasound tongue images for audio-visual speech enhancement through knowledge distillation,\u201d arXiv:2305.14933.","DOI":"10.21437\/Interspeech.2023-780"},{"key":"2024091718184219000_c140","unstructured":"Zhu, \nJ., \nStyler, \nW., and \nCalloway, \nI. (2019). \u201c\nA CNN-based tool for automatic tongue contour tracking in ultrasound images,\u201d arXiv:1907.10210."}],"container-title":["The Journal of the Acoustical Society of America"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.aip.org\/asa\/jasa\/article-pdf\/156\/3\/1796\/20161609\/1796_1_10.0028610.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.aip.org\/asa\/jasa\/article-pdf\/156\/3\/1796\/20161609\/1796_1_10.0028610.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,17]],"date-time":"2024-09-17T18:18:54Z","timestamp":1726597134000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.aip.org\/jasa\/article\/156\/3\/1796\/3313026\/A-systematic-review-of-the-application-of-machine"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,1]]},"references-count":138,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1121\/10.0028610","relation":{},"ISSN":["0001-4966"],"issn-type":[{"value":"0001-4966","type":"print"}],"subject":[],"published-other":{"date-parts":[[2024,9]]},"published":{"date-parts":[[2024,9,1]]}}}