{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T00:56:52Z","timestamp":1792803412228,"version":"4.5.0"},"reference-count":99,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,5,22]],"date-time":"2023-05-22T00:00:00Z","timestamp":1684713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Simulator Pilot Operator in support to Real-Time Simulations","award":["20-220319-A"],"award-info":[{"award-number":["20-220319-A"]}]},{"name":"Simulator Pilot Operator in support to Real-Time Simulations","award":["884287"],"award-info":[{"award-number":["884287"]}]},{"name":"SESAR Joint Undertaking under Grant Agreement","award":["20-220319-A"],"award-info":[{"award-number":["20-220319-A"]}],"id":[{"id":"https:\/\/ror.org\/042r3zc18","id-type":"ROR","asserted-by":"crossref"}]},{"name":"SESAR Joint Undertaking under Grant Agreement","award":["884287"],"award-info":[{"award-number":["884287"]}],"id":[{"id":"https:\/\/ror.org\/042r3zc18","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Aerospace"],"abstract":"<jats:p>In this paper we propose a novel virtual simulation-pilot engine for speeding up air traffic controller (ATCo) training by integrating different state-of-the-art artificial intelligence (AI)-based tools. The virtual simulation-pilot engine receives spoken communications from ATCo trainees, and it performs automatic speech recognition and understanding. Thus, it goes beyond only transcribing the communication and can also understand its meaning. The output is subsequently sent to a response generator system, which resembles the spoken read-back that pilots give to the ATCo trainees. The overall pipeline is composed of the following submodules: (i) an automatic speech recognition (ASR) system that transforms audio into a sequence of words; (ii) a high-level air traffic control (ATC)-related entity parser that understands the transcribed voice communication; and (iii) a text-to-speech submodule that generates a spoken utterance that resembles a pilot based on the situation of the dialogue. Our system employs state-of-the-art AI-based tools such as Wav2Vec 2.0, Conformer, BERT and Tacotron models. To the best of our knowledge, this is the first work fully based on open-source ATC resources and AI tools. In addition, we develop a robust and modular system with optional submodules that can enhance the system\u2019s performance by incorporating real-time surveillance data, metadata related to exercises (such as sectors or runways), or even a deliberate read-back error to train ATCo trainees to identify them. Our ASR system can reach as low as 5.5% and 15.9% absolute word error rates (WER) on high- and low-quality ATC audio. We also demonstrate that adding surveillance data into the ASR can yield a callsign detection accuracy of more than 96%.<\/jats:p>","DOI":"10.3390\/aerospace10050490","type":"journal-article","created":{"date-parts":[[2023,5,22]],"date-time":"2023-05-22T03:57:32Z","timestamp":1684727852000},"page":"490","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6947-2706","authenticated-orcid":false,"given":"Juan","family":"Zuluaga-Gomez","sequence":"first","affiliation":[{"name":"Speech & Audio Processing Group, Idiap Research Institute, 1920 Martigny, Switzerland"},{"name":"LIDIAP, Ecole Polytechnique F\u00e9d\u00e9rale de Lausanne, 1015 Lausanne, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amrutha","family":"Prasad","sequence":"additional","affiliation":[{"name":"Speech & Audio Processing Group, Idiap Research Institute, 1920 Martigny, Switzerland"},{"name":"Faculty of Information Technology, Brno University of Technology, 60190 Brno, Czech Republic"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5111-1873","authenticated-orcid":false,"given":"Iuliia","family":"Nigmatulina","sequence":"additional","affiliation":[{"name":"Speech & Audio Processing Group, Idiap Research Institute, 1920 Martigny, Switzerland"},{"name":"Institute of Computational Linguistics, University of Zurich, 8006 Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6467-1119","authenticated-orcid":false,"given":"Petr","family":"Motlicek","sequence":"additional","affiliation":[{"name":"Speech & Audio Processing Group, Idiap Research Institute, 1920 Martigny, Switzerland"},{"name":"Institute of Computational Linguistics, University of Zurich, 8006 Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0782-4147","authenticated-orcid":false,"given":"Matthias","family":"Kleinert","sequence":"additional","affiliation":[{"name":"Institute of Flight Guidance, German Aerospace Center (DLR), 38108 Braunschweig, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"19143","DOI":"10.1109\/ACCESS.2019.2896880","article-title":"Speech recognition using deep neural networks: A systematic review","volume":"7","author":"Nassif","year":"2019","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1109\/TNNLS.2020.2979670","article-title":"A survey of the usages of deep learning for natural language processing","volume":"32","author":"Otter","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e1253","DOI":"10.1002\/widm.1253","article-title":"Deep learning for sentiment analysis: A survey","volume":"8","author":"Zhang","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lugosch, L., Ravanelli, M., Ignoto, P., Tomar, V.S., and Bengio, Y. (2019, January 15\u201319). Speech Model Pre-Training for End-to-End Spoken Language Understanding. Proceedings of the Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria.","DOI":"10.21437\/Interspeech.2019-2396"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1109\/TASSP.1977.1162968","article-title":"An assessment of the technology of automatic speech recognition for military applications","volume":"25","author":"Beek","year":"1977","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_6","unstructured":"Hamel, C.J., Kotick, D., and Layton, M. (1989). Microcomputer System Integration for Air Control Training, Naval Training Systems Center. Technical Report."},{"key":"ref_7","unstructured":"Matrouf, K., Gauvain, J., Neel, F., and Mariani, J. (1990, January 3\u20136). Adapting probability-transitions in DP matching processing for an oral task-oriented dialogue. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Albuquerque, NM, USA."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Helmke, H., Ohneiser, O., M\u00fchlhausen, T., and Wies, M. (2016, January 25\u201329). Reducing controller workload with automatic speech recognition. Proceedings of the 2016 IEEE\/AIAA 35th Digital Avionics Systems Conference (DASC), Sacramento, CA, USA.","DOI":"10.1109\/DASC.2016.7778024"},{"key":"ref_9","unstructured":"Helmke, H., Ohneiser, O., Buxbaum, J., and Kern, C. (2017, January 27\u201330). Increasing ATM efficiency with assistant based speech recognition. Proceedings of the 13th USA\/Europe Air Traffic Management Research and Development Seminar, Seattle, WA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nigmatulina, I., Zuluaga-Gomez, J., Prasad, A., Sarfjoo, S.S., and Motlicek, P. (2022, January 23\u201327). A two-step approach to leverage contextual data: Speech recognition in air-traffic communications. Proceedings of the ICASSP, Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746563"},{"key":"ref_11","unstructured":"Zuluaga-Gomez, J., Vesel\u1ef3, K., Sz\u00f6ke, I., Motlicek, P., Kocour, M., Rigault, M., Choukri, K., Prasad, A., Sarfjoo, S.S., and Nigmatulina, I. (2022). ATCO2 corpus: A Large-Scale Dataset for Research on Automatic Speech Recognition and Natural Language Understanding of Air Traffic Control Communications. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kocour, M., Vesel\u00fd, K., Blatt, A., Zuluaga-Gomez, J., Sz\u00f6ke, I., Cernock\u00fd, J., Klakow, D., and Motl\u00edcek, P. (September, January 30). Boosting of Contextual Information in ASR for Air-Traffic Call-Sign Recognition. Proceedings of the Interspeech 2021, 22nd Annual Conference of the International Speech Communication Association, Brno, Czechia.","DOI":"10.21437\/Interspeech.2021-1619"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zuluaga-Gomez, J., Sarfjoo, S.S., Prasad, A., Nigmatulina, I., Motlicek, P., Ondre, K., Ohneiser, O., and Helmke, H. (2023, January 9\u201312). BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communications. Proceedings of the 2022 IEEE Spoken Language Technology Workshop (SLT), Doha, Qatar.","DOI":"10.1109\/SLT54892.2023.10022718"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.neucom.2020.08.092","article-title":"Improving speech recognition models with small samples for air traffic control systems","volume":"445","author":"Lin","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_15","first-page":"14","article-title":"Automatic call sign detection: Matching air surveillance data with air traffic spoken communications","volume":"59","author":"Blatt","year":"2020","journal-title":"Proceedings"},{"key":"ref_16","unstructured":"Fan, P., Guo, D., Lin, Y., Yang, B., and Zhang, J. (2021). Speech recognition for air traffic control via feature learning and end-to-end training. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Helmke, H., Slotty, M., Poiger, M., Herrer, D.F., Ohneiser, O., Vink, N., Cerna, A., Hartikainen, P., Josefsson, B., and Langr, D. (2018, January 23\u201327). Ontology for transcription of ATC speech commands of SESAR 2020 solution PJ. 16-04. Proceedings of the IEEE\/AIAA 37th Digital Avionics Systems Conference (DASC), London, UK.","DOI":"10.1109\/DASC.2018.8569238"},{"key":"ref_18","unstructured":"Guo, D., Zhang, Z., Yang, B., Zhang, J., and Lin, Y. (2023). IEEE Transactions on Circuits and Systems II: Express Briefs, IEEE."},{"key":"ref_19","unstructured":"Fan, P., Guo, D., Zhang, J., Yang, B., and Lin, Y. (2023). Enhancing multilingual speech recognition in air traffic control by sentence-level language identification. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Guo, D., Zhang, Z., Fan, P., Zhang, J., and Yang, B. (2021). A context-aware language model to improve the speech recognition in air traffic control. Aerospace, 8.","DOI":"10.3390\/aerospace8110348"},{"key":"ref_21","unstructured":"International Civil Aviation Organization (2020). ICAO Phraseology Reference Guide, ICAO."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bouchal, A., Had, P., and Bouchaudon, P. (2022, January 26\u201327). The Design and Implementation of Upgraded ESCAPE Light ATC Simulator Platform at the CTU in Prague. Proceedings of the 2022 New Trends in Civil Aviation (NTCA), Prague, Czech Republic.","DOI":"10.23919\/NTCA55899.2022.9934771"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lin, Y. (2021). Spoken instruction understanding in air traffic control: Challenge, technique, and application. Aerospace, 8.","DOI":"10.3390\/aerospace8030065"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1109\/THMS.2021.3102827","article-title":"A deep learning framework of autonomous pilot agent for air traffic controller training","volume":"51","author":"Lin","year":"2021","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_25","unstructured":"Prasad, A., Zuluaga-Gomez, J., Motlicek, P., Sarfjoo, S., Nigmatulina, I., and Vesely, K. (2022, January 5\u20138). Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator. Proceedings of the 12th SESAR Innovation Days, Budapest, Hungary."},{"key":"ref_26","unstructured":"Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. (2020, January 6\u201312). wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Virtual."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Conneau, A., Baevski, A., Collobert, R., Mohamed, A., and Auli, M. (2021). Unsupervised cross-lingual representation learning for speech recognition. arXiv.","DOI":"10.21437\/Interspeech.2021-329"},{"key":"ref_28","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics."},{"key":"ref_29","unstructured":"Ren, Y., Hu, C., Tan, X., Qin, T., Zhao, S., Zhao, Z., and Liu, T. (2021, January 3\u20137). FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria."},{"key":"ref_30","unstructured":"Godfrey, J. (1994). The Air Traffic Control Corpus (ATC0)\u2014LDC94S14A, Linguistic Data Consortium."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1007\/s10579-019-09449-5","article-title":"Air traffic control communication (ATCC) speech corpora and their use for ASR and TTS development","volume":"53","author":"Tihelka","year":"2019","journal-title":"Lang. Resour. Eval."},{"key":"ref_32","unstructured":"Hofbauer, K., Petrik, S., and Hering, H. (2008). Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC\u201908), European Language Resources Association (ELRA)."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Pavlinovi\u0107, M., Juri\u010di\u0107, B., and Antulov-Fantulin, B. (2017, January 22\u201326). Air traffic controllers\u2019 practical part of basic training on computer based simulation device. Proceedings of the International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia.","DOI":"10.23919\/MIPRO.2017.7973554"},{"key":"ref_34","unstructured":"Juri\u010di\u0107, B., Vare\u0161ak, I., and Bo\u017ei\u0107, D. (2011, January 26\u201328). The role of the simulation devices in air traffic controller training. Proceedings of the International Symposium on Electronics in Traffic, ISEP 2011 Proceedings, Berlin, Germany."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chhaya, B., Jafer, S., Coyne, W.B., Thigpen, N.C., and Durak, U. (2018, January 8\u201312). Enhancing scenario-centric air traffic control training. Proceedings of the 2018 AIAA modeling and Simulation Technologies Conference, Kissimmee, FL, USA.","DOI":"10.2514\/6.2018-1399"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Updegrove, J.A., and Jafer, S. (2017). Optimization of air traffic control training at the Federal Aviation Administration Academy. Aerospace, 4.","DOI":"10.3390\/aerospace4040050"},{"key":"ref_37","unstructured":"Eide, A.W., \u00d8deg\u00e5rd, S.S., and Karahasanovi\u0107, A. (2014). Human Interface and the Management of Information. Information and Knowledge Design and Evaluation, Springer."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"N\u00e9methov\u00e1, H., B\u00e1lint, J., and Vagner, J. (2019, January 21\u201322). The education and training methodology of the air traffic controllers in training. Proceedings of the International Conference on Emerging eLearning Technologies and Applications (ICETA), Star\u00fd Smokovec, Slovakia.","DOI":"10.1109\/ICETA48886.2019.9040125"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"108232","DOI":"10.1016\/j.knosys.2022.108232","article-title":"Automatic repetition instruction generation for air traffic control training using multi-task learning with an improved copy network","volume":"241","author":"Zhang","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_40","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is All you Need. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1006\/csla.2001.0184","article-title":"Weighted finite-state transducers in speech recognition","volume":"16","author":"Mohri","year":"2002","journal-title":"Comput. Speech Lang."},{"key":"ref_42","unstructured":"Mohri, M., Pereira, F., and Riley, M. (2008). Springer Handbook of Speech Processing, Springer."},{"key":"ref_43","unstructured":"Riley, M., Allauzen, C., and Jansche, M. (2009). Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Tutorial Abstracts, Association for Computational Linguistics."},{"key":"ref_44","first-page":"2345","article-title":"Sequence-discriminative training of deep neural networks","volume":"2013","author":"Ghoshal","year":"2013","journal-title":"Interspeech"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Bourlard, H.A., and Morgan, N. (1993). Connectionist Speech Recognition: A Hybrid Approach, Springer Science & Business Media.","DOI":"10.1007\/978-1-4615-3210-1"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Povey, D., Cheng, G., Wang, Y., Li, K., Xu, H., Yarmohammadi, M., and Khudanpur, S. (2018, January 2\u20136). Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks. Proceedings of the Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India.","DOI":"10.21437\/Interspeech.2018-1417"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zuluaga-Gomez, J., Motlicek, P., Zhan, Q., Vesel\u1ef3, K., and Braun, R. (2020). Automatic Speech Recognition Benchmark for Air-Traffic Communications. Proc. Interspeech, 2297\u20132301.","DOI":"10.21437\/Interspeech.2020-2173"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Srinivasamurthy, A., Motl\u00edcek, P., Himawan, I., Szasz\u00e1k, G., Oualil, Y., and Helmke, H. (2017, January 20\u201324). Semi-Supervised Learning with Semantic Knowledge Extraction for Improved Speech Recognition in Air Traffic Control. Proceedings of the Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-1446"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zuluaga-Gomez, J., Nigmatulina, I., Prasad, A., Motlicek, P., Vesel\u1ef3, K., Kocour, M., and Sz\u00f6ke, I. (2021). Contextual Semi-Supervised Learning: An Approach to Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems. Proc. Interspeech, 3296\u20133300.","DOI":"10.21437\/Interspeech.2021-1373"},{"key":"ref_50","first-page":"8","article-title":"Automatic processing pipeline for collecting and annotating air-traffic voice communication data","volume":"13","author":"Kocour","year":"2021","journal-title":"Eng. Proc."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Chen, S., Kopald, H., Avjian, B., and Fronzak, M. (2022, January 18\u201322). Automatic Pilot Report Extraction from Radio Communications. Proceedings of the 2022 IEEE\/AIAA 41st Digital Avionics Systems Conference (DASC), Portsmouth, VA, USA.","DOI":"10.1109\/DASC55683.2022.9925803"},{"key":"ref_52","unstructured":"Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., and Schwarz, P. (2011, January 11\u201315). The Kaldi speech recognition toolkit. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding, Waikoloa, HI, USA."},{"key":"ref_53","unstructured":"Madikeri, S., Tong, S., Zuluaga-Gomez, J., Vyas, A., Motlicek, P., and Bourlard, H. (2020). Pkwrap: A pytorch package for lf-mmi training of acoustic models. arXiv."},{"key":"ref_54","unstructured":"Graves, A., and Jaitly, N. (2014, January 21\u201326). Towards End-To-End Speech Recognition with Recurrent Neural Networks. Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China."},{"key":"ref_55","unstructured":"Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2015, January 7\u201312). Attention-Based Models for Speech Recognition. Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, Montreal, QC, Canada."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Schneider, S., Baevski, A., Collobert, R., and Auli, M. (2019, January 15\u201319). wav2vec: Unsupervised Pre-Training for Speech Recognition. Proceedings of the Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria.","DOI":"10.21437\/Interspeech.2019-1873"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Baevski, A., and Mohamed, A. (2020, January 4\u20138). Effectiveness of Self-Supervised Pre-Training for ASR. Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054224"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Zhang, Z.Q., Song, Y., Wu, M.H., Fang, X., and Dai, L.R. (2021). Xlst: Cross-lingual self-training to learn multilingual representation for low resource speech recognition. arXiv.","DOI":"10.1007\/s00034-022-02075-7"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., and Xiao, X. (2021). Wavlm: Large-scale self-supervised pre-training for full stack speech processing. arXiv.","DOI":"10.1109\/JSTSP.2022.3188113"},{"key":"ref_60","unstructured":"Oord, A.v.d., Li, Y., and Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv."},{"key":"ref_61","unstructured":"Baevski, A., Schneider, S., and Auli, M. (2020, January 26\u201330). vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations. Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Zuluaga-Gomez, J., Prasad, A., Nigmatulina, I., Sarfjoo, S., Motlicek, P., Kleinert, M., Helmke, H., Ohneiser, O., and Zhan, Q. (2023, January 9\u201312). How Does Pre-trained Wav2Vec2.0 Perform on Domain Shifted ASR? An Extensive Benchmark on Air Traffic Control Communications. Proceedings of the IEEE Spoken Language Technology Workshop (SLT), Doha, Qatar.","DOI":"10.1109\/SLT54892.2023.10022724"},{"key":"ref_63","unstructured":"Yadav, V., and Bethard, S. (2018, January 20\u201326). A Survey on Recent Advances in Named Entity Recognition from Deep Learning models. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Grishman, R., and Sundheim, B. (1996, January 5\u20139). Message Understanding Conference- 6: A Brief History. Proceedings of the COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics, Copenhagen, Denmark.","DOI":"10.3115\/992628.992709"},{"key":"ref_65","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_66","unstructured":"Piskorski, J., Pivovarova, L., \u0160najder, J., Steinberger, J., and Yangarber, R. (2017). Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing, Association for Computational Linguistics."},{"key":"ref_67","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv."},{"key":"ref_68","unstructured":"He, P., Liu, X., Gao, J., and Chen, W. (2021, January 3\u20137). Deberta: Decoding-Enhanced Bert with Disentangled Attention. Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1121\/1.395275","article-title":"Review of text-to-speech conversion for English","volume":"82","author":"Klatt","year":"1987","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/S0167-6393(96)00046-5","article-title":"Emotional stress in synthetic speech: Progress and future directions","volume":"20","author":"Murray","year":"1996","journal-title":"Speech Commun."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1109\/JPROC.2013.2251852","article-title":"Speech synthesis based on hidden Markov models","volume":"101","author":"Tokuda","year":"2013","journal-title":"Proc. IEEE"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Wang, Y., Skerry-Ryan, R.J., Stanton, D., Wu, Y., Weiss, R.J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., and Bengio, S. (2017, January 20\u201324). Tacotron: Towards End-to-End Speech Synthesis. Proceedings of the Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-1452"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Shen, J., Pang, R., Weiss, R.J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., and Ryan, R. (2018, January 15\u201320). Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8461368"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Kaur, N., and Singh, P. (2022). Conventional and contemporary approaches used in text to speech synthesis: A review. Artif. Intell. Rev., 1\u201344.","DOI":"10.1007\/s10462-022-10315-0"},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Jeong, M., Kim, H., Cheon, S.J., Choi, B.J., and Kim, N.S. (September, January 30). Diff-TTS: A Denoising Diffusion Model for Text-to-Speech. Proceedings of the Interspeech 2021, 22nd Annual Conference of the International Speech Communication Association, Brno, Czechia.","DOI":"10.21437\/Interspeech.2021-469"},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Sarfjoo, S.S., Madikeri, S.R., and Motl\u00edcek, P. (September, January 30). Speech Activity Detection Based on Multilingual Speech Recognition System. Proceedings of the Interspeech 2021, 22nd Annual Conference of the International Speech Communication Association, Brno, Czechia.","DOI":"10.21437\/Interspeech.2021-1058"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/JSTSP.2019.2901195","article-title":"An end-to-end multimodal voice activity detection using wavenet encoder and residual networks","volume":"13","author":"Ariav","year":"2019","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Ding, S., Wang, Q., Chang, S.y., Wan, L., and Moreno, I.L. (2019). Personal VAD: Speaker-conditioned voice activity detection. arXiv.","DOI":"10.21437\/Odyssey.2020-62"},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Medennikov, I., Korenevsky, M., Prisyach, T., Khokhlov, Y.Y., Korenevskaya, M., Sorokin, I., Timofeeva, T., Mitrofanov, A., Andrusenko, A., and Podluzhny, I. (2020, January 25\u201329). Target-Speaker Voice Activity Detection: A Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.","DOI":"10.21437\/Interspeech.2020-1602"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Ng, T., Zhang, B., Nguyen, L., Matsoukas, S., Zhou, X., Mesgarani, N., Vesel\u1ef3, K., and Mat\u011bjka, P. (2012, January 9\u201313). Developing a speech activity detection system for the DARPA RATS program. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.","DOI":"10.21437\/Interspeech.2012-527"},{"key":"ref_81","unstructured":"Helmke, H., Ond\u0159ej, K., Shetty, S., Aril\u00edusson, H., Simiganoschi, T., Kleinert, M., Ohneiser, O., Ehr, H., Zuluaga-Gomez, J., and Smrz, P. (2022, January 5\u20138). Readback Error Detection by Automatic Speech Recognition and Understanding\u2014Results of HAAWAII Project for Isavia\u2019s Enroute Airspace. Proceedings of the 12th SESAR Innovation Days, Budapest, Hungary."},{"key":"ref_82","unstructured":"Cordero, J.M., Dorado, M., and de Pablo, J.M. (2012, January 29\u201331). Automated speech recognition in ATC environment. Proceedings of the 2nd International Conference on Application and Theory of Automation in Command and Control Systems, London, UK."},{"key":"ref_83","unstructured":"Delpech, E., Laignelet, M., Pimm, C., Raynal, C., Trzos, M., Arnold, A., and Pronto, D. (2018). Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), European Language Resources Association (ELRA)."},{"key":"ref_84","unstructured":"Segura, J., Ehrette, T., Potamianos, A., Fohr, D., Illina, I., Breton, P., Clot, V., Gemello, R., Matassoni, M., and Maragos, P. (2023, May 12). The HIWIRE Database, a Noisy and Non-Native English Speech Corpus for Cockpit Communication. Available online: http:\/\/www.hiwire.org."},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Gulati, A., Qin, J., Chiu, C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., and Wu, Y. (2020, January 25\u201329). Conformer: Convolution-augmented Transformer for Speech Recognition. Proceedings of the Interspeech 2020, 21st Annual Conference of the International Speech Communication Association, Virtual Event, Shanghai, China.","DOI":"10.21437\/Interspeech.2020-3015"},{"key":"ref_86","first-page":"1995","article-title":"Convolutional networks for images, speech, and time series","volume":"3361","author":"LeCun","year":"1995","journal-title":"Handb. Brain Theory Neural Netw."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Vyas, A., Madikeri, S., and Bourlard, H. (2021, January 6\u201311). Lattice-Free Mmi Adaptation of Self-Supervised Pretrained Acoustic Models. Proceedings of the ICASSP 2021\u20142021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9414741"},{"key":"ref_88","unstructured":"Ravanelli, M., Parcollet, T., Plantinga, P., Rouhe, A., Cornell, S., Lugosch, L., Subakan, C., Dawalatabad, N., Heba, A., and Zhong, J. (2021). SpeechBrain: A general-purpose speech toolkit. arXiv."},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19\u201324). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015, South Brisbane, Australia.","DOI":"10.1109\/ICASSP.2015.7178964"},{"key":"ref_90","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_91","unstructured":"Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv."},{"key":"ref_92","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Karita, S., Chen, N., Hayashi, T., Hori, T., Inaguma, H., Jiang, Z., Someki, M., Soplin, N.E.Y., Yamamoto, R., and Wang, X. (2019, January 14\u201318). A comparative study on transformer vs rnn in speech applications. Proceedings of the 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Singapore.","DOI":"10.1109\/ASRU46091.2019.9003750"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Graves, A., and Graves, A. (2012). Supervised Sequence Labelling with Recurrent Neural Networks, Springer.","DOI":"10.1007\/978-3-642-24797-2"},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Chen, Z., Jain, M., Wang, Y., Seltzer, M.L., and Fuegen, C. (, January 12\u201317). End-to-end Contextual Speech Recognition Using Class Language Models and a Token Passing Decoder. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683573"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2020, January 16\u201320). Transformers: State-of-the-Art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Lhoest, Q., Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., and Tunstall, L. (2021, January 7\u201311). Datasets: A Community Library for Natural Language Processing. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online.","DOI":"10.18653\/v1\/2021.emnlp-demo.21"},{"key":"ref_98","unstructured":"Pascanu, R., Mikolov, T., and Bengio, Y. (2013, January 16\u201321). On the difficulty of training recurrent neural networks. Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA."},{"key":"ref_99","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA."}],"container-title":["Aerospace"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2226-4310\/10\/5\/490\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:39:43Z","timestamp":1760125183000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2226-4310\/10\/5\/490"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,22]]},"references-count":99,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["aerospace10050490"],"URL":"https:\/\/doi.org\/10.3390\/aerospace10050490","relation":{},"ISSN":["2226-4310"],"issn-type":[{"value":"2226-4310","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,22]]}}}