{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T16:04:51Z","timestamp":1780675491489,"version":"3.54.1"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T00:00:00Z","timestamp":1696982400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Using computers to replace pilot seats in air traffic control (ATC) simulators is an effective way to improve controller training efficiency and reduce training costs. To achieve this, we propose a deep reinforcement learning model, RoBERTa-RL (RoBERTa with Reinforcement Learning), for generating pilot repetitions. RoBERTa-RL is based on the pre-trained language model RoBERTa and is optimized through transfer learning and reinforcement learning. Transfer learning is used to address the issue of scarce data in the ATC domain, while reinforcement learning algorithms are employed to optimize the RoBERTa model and overcome the limitations in model generalization caused by transfer learning. We selected a real-world area control dataset as the target task training and testing dataset, and a tower control dataset generated based on civil aviation radio land-air communication rules as the test dataset for evaluating model generalization. In terms of the ROUGE evaluation metrics, RoBERTa-RL achieved significant results on the area control dataset with ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.9962, 0.992, and 0.996, respectively. On the tower control dataset, the scores were 0.982, 0.954, and 0.982, respectively. To overcome the limitations of ROUGE in this field, we conducted a detailed evaluation of the proposed model architecture using keyword-based evaluation criteria for the generated repetition instructions. This evaluation criterion calculates various keyword-based metrics based on the segmented results of the repetition instruction text. In the keyword-based evaluation criteria, the constructed model achieved an overall accuracy of 98.8% on the area control dataset and 81.8% on the tower control dataset. In terms of generalization, RoBERTa-RL improved accuracy by 56% compared to the model before improvement and achieved a 47.5% improvement compared to various comparative models. These results indicate that employing reinforcement learning strategies to enhance deep learning algorithms can effectively mitigate the issue of poor generalization in text generation tasks, and this approach holds promise for future application in other related domains.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1285831","type":"journal-article","created":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T09:26:16Z","timestamp":1697016376000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Research on automatic pilot repetition generation method based on deep reinforcement learning"],"prefix":"10.3389","volume":"17","author":[{"given":"Weijun","family":"Pan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peiyuan","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yukun","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhuang","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junxiang","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,11]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1007\/978-3-030-90321-3_61","article-title":"Fine-tuning GPT-3 for Russian text summarization","volume-title":"Data Science and Intelligent Systems: Proceedings of 5th Computational Methods in Systems and Software 2021","author":"Alexandr","year":"2021"},{"key":"B2","doi-asserted-by":"crossref","first-page":"233","DOI":"10.18653\/v1\/W18-6530","article-title":"Generating e-commerce product titles and predicting their quality","volume-title":"Proceedings of the 11th International Conference on Natural Language Generation","author":"de Souza","year":"2018"},{"key":"B3","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"B4","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.esp.2022.10.001","article-title":"The development, evaluation and application of an aviation radiotelephony specialised technical vocabulary list","volume":"69","author":"Drayton","year":"2023","journal-title":"English Specific Purposes"},{"key":"B5","article-title":"Bert fine-tuning for Arabic text summarization","author":"Elmadani","year":"2020","journal-title":"arXiv preprint arXiv:2004.14135"},{"key":"B6","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1007\/978-1-4614-4803-7_7","article-title":"Metrics and evaluation of spoken dialogue systems","volume-title":"Data-Driven Methods for Adaptive Spoken Dialogue Systems: Computational Learning for Conversational Interfaces","author":"Hastie","year":"2012"},{"key":"B7","article-title":"Increasing atm efficiency with assistant based speech recognition","volume-title":"Proc. of the 13th USA\/Europe Air Traffic Management Research and Development Seminar","author":"Helmke","year":"2017"},{"key":"B8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/DASC.2016.7778024","article-title":"Reducing controlle workload with automatic speech recognition","volume-title":"2016 IEEE\/AIAA 35th Digital Avionics Systems Conference (DASC)","author":"Helmke","year":"2016"},{"key":"B9","first-page":"132","article-title":"Bluesky atc simulator project: an open data and open source approach","author":"Hoekstra","year":"2016","journal-title":"Proceedings of the 7th International Conference on Research in Air Transportation"},{"key":"B10","first-page":"1933","article-title":"Possibilities, challenges and the state of the art of automatic speech recognition in air traffic control","volume":"9","author":"Holone","year":"2015","journal-title":"Int. J. Comput. Inform. Eng."},{"key":"B11","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1137\/1.9781611975673.76","article-title":"Deep transfer reinforcement learning for text summarization","volume-title":"Proceedings of the 2019 SIAM International Conference on Data Mining","author":"Keneshloo","year":"2019"},{"key":"B12","article-title":"Machine learning of controller command prediction models from recorded radar data and controller speech utterances","volume-title":"7th SESAR Innovation Days","author":"Kleinert","year":"2017"},{"key":"B13","article-title":"Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","author":"Lewis","year":"2019","journal-title":"arXiv preprint arXiv:1910.13461"},{"key":"B14","article-title":"Looking for a few good metrics: rouge and its evaluation","author":"Lin","year":"2004","journal-title":"NTCIR Workshop"},{"key":"B15","doi-asserted-by":"publisher","first-page":"65","DOI":"10.3390\/aerospace8030065","article-title":"Spoken instruction understanding in air traffic control: challenge, technique, and application","volume":"8","author":"Lin","year":"2015","journal-title":"Aerospace"},{"key":"B16","doi-asserted-by":"publisher","first-page":"4572","DOI":"10.1109\/TITS.2019.2940992","article-title":"A real-time ATC safety monitoring framework using a deep learning approach","volume":"21","author":"Lin","year":"2019","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"B17","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1109\/THMS.2021.3102827","article-title":"A deep learning framework of autonomous pilot agent for air traffic controller training","volume":"51","author":"Lin","year":"2021","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"B18","doi-asserted-by":"publisher","first-page":"012213","DOI":"10.1088\/1742-6596\/1754\/1\/012213","article-title":"Automatic Chinese text summarization for emergency domain","volume":"1754","author":"Liu","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"B19","article-title":"RoBERTa: a robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv preprint arXiv:1907.11692"},{"key":"B20","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1016\/j.ipm.2018.11.006","article-title":"Knowledge empowered prominent aspect extraction from product reviews","volume":"56","author":"Luo","year":"2019","journal-title":"Inform. Process. Manage."},{"key":"B21","article-title":"Product title generation for conversational systems using BERT","author":"Mane","year":"2020","journal-title":"arXiv preprint arXiv:2007.11768"},{"key":"B22","doi-asserted-by":"publisher","first-page":"102123","DOI":"10.1016\/j.ipm.2019.102123","article-title":"Multi-modal product title compression","volume":"57","author":"Miao","year":"2020","journal-title":"Inform. Process. Manage."},{"key":"B23","article-title":"Sequence-to-sequence rnns for text summarization","author":"Nallapati","year":"","journal-title":"Workshop Track - ICLR 2016"},{"key":"B24","article-title":"Abstractive text summarization using sequence-to-sequence RNNs and beyond","author":"Nallapati","year":"","journal-title":"arXiv preprint arXiv:1602.06023"},{"key":"B25","first-page":"121","article-title":"A survey on automatic text summarization","volume":"7","author":"Nazari","year":"2019","journal-title":"J. AI Data Mining"},{"key":"B26","doi-asserted-by":"publisher","first-page":"349","DOI":"10.3390\/aerospace10040349","article-title":"Ernie-gram biGRU attention: an improved multi-intention recognition model for air traffic control","volume":"10","author":"Pan","year":"2023","journal-title":"Aerospace"},{"key":"B27","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"B28","article-title":"A neural attention model for abstractive sentence summarization","author":"Rush","year":"2015","journal-title":"arXiv preprint arXiv:1509.00685"},{"key":"B29","article-title":"Get to the point: Summarization with pointer-generator networks","author":"See","year":"2017","journal-title":"arXiv preprint arXiv:1704.04368"},{"key":"B30","doi-asserted-by":"publisher","first-page":"857","DOI":"10.1007\/s11042-018-5749-3","article-title":"Abstractive text summarization using LSTM-CNN based deep learning","volume":"78","author":"Song","year":"2019","journal-title":"Multimedia Tools Appl."},{"key":"B31","doi-asserted-by":"crossref","first-page":"2406","DOI":"10.21437\/Interspeech.2017-1446","article-title":"Semi-supervised learning with semantic knowledge extraction for improved speech recognition in air traffic control","author":"Srinivasamurthy","year":"2017","journal-title":"Proceedings of Interspeech 2017"},{"key":"B32","doi-asserted-by":"publisher","first-page":"458","DOI":"10.5755\/j01.itc.50.3.28047","article-title":"Automatic text summarization using deep reinforcement learning and beyond","volume":"50","author":"Sun","year":"2021","journal-title":"Inform. Technol. Control"},{"key":"B33","article-title":"ATCspeech: a multilingual pilot-controller speech corpus from real air traffic control environment","author":"Yang","year":"2019","journal-title":"arXiv preprint arXiv:1911.11365"},{"key":"B34","doi-asserted-by":"publisher","first-page":"10923","DOI":"10.3390\/app112210923","article-title":"A digital twin-based platform towards intelligent automation with virtual counterparts of flight and air traffic control operations","volume":"11","author":"Yiu","year":"2021","journal-title":"Appl. Sci."},{"key":"B35","doi-asserted-by":"publisher","first-page":"108232","DOI":"10.1016\/j.knosys.2022.108232","article-title":"Automatic repetition instruction generation for air traffic control training using multi-task learning with an improved copy network","volume":"241","author":"Zhang","year":"","journal-title":"Knowledge Based Syst."},{"key":"B36","doi-asserted-by":"publisher","first-page":"7132226","DOI":"10.1155\/2022\/7132226","article-title":"A comprehensive survey of abstractive text summarization based on deep learning","volume":"2022","author":"Zhang","year":"","journal-title":"Comput. Intell. Neurosci."},{"key":"B37","article-title":"Optimizing the factual correctness of a summary: a study of summarizing radiology reports","author":"Zhang","year":"2019","journal-title":"arXiv preprint arXiv:1911.02541"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.1201\/b12207","volume-title":"Ensemble Methods: Foundations and Algorithms","author":"Zhou","year":"2012"},{"key":"B39","doi-asserted-by":"publisher","first-page":"490","DOI":"10.3390\/aerospace10050490","article-title":"A virtual simulation-pilot agent for training of air traffic controllers","volume":"10","author":"Zuluaga-Gomez","year":"2023","journal-title":"Aerospace"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1285831\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T09:26:31Z","timestamp":1697016391000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1285831\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,11]]},"references-count":39,"alternative-id":["10.3389\/fnbot.2023.1285831"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1285831","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,11]]},"article-number":"1285831"}}