{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,8]],"date-time":"2025-12-08T22:46:57Z","timestamp":1765234017936,"version":"3.41.2"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T00:00:00Z","timestamp":1749168000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Recent advances in few-shot learning have demonstrated the potential of prompt-based techniques with pre-trained models, eliminating the need for extensive fine-tuning. However, challenges such as obtaining optimal prompts and addressing data scarcity in specialized domains remain challenging. We introduce a novel framework incorporating a Global Attention Mechanism (GAM) that effectively integrates features from multiple layers of pre-trained language models, enhanced by Latent Dirichlet Allocation (LDA) generated topic features for prompt optimization. Extensive experiments on four datasets consistently show that our approach outperforms state of-the-art baselines. The strategic integration of GAM with layer-specific features and LDA topics proves particularly effective in extracting valuable latent information for few-shot learning scenarios, yielding significant improvements in specialized domains, as evidenced by enhanced performance in therapeutic dialogue classification within a Applied Behavior Analysis clinical dataset.<\/jats:p>","DOI":"10.3389\/frobt.2025.1579990","type":"journal-article","created":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T04:10:45Z","timestamp":1749183045000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Improving optimal prompt learning through multilayer fusion and latent dirichlet allocation"],"prefix":"10.3389","volume":"12","author":[{"given":"Qinghua","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica","family":"Korneder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Osamah A.","family":"Rawashdeh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanfeng","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wing-Yue Geoffrey","family":"Louie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,6,6]]},"reference":[{"article-title":"Palm 2 technical report","year":"2023","author":"Anil","key":"B1"},{"article-title":"Large language models for human-robot interaction: opportunities and risks","year":"2024","author":"Atuhurra","key":"B2"},{"key":"B3","doi-asserted-by":"crossref","first-page":"93","DOI":"10.18653\/v1\/2022.acl-demo.9","article-title":"PromptSource: an integrated development environment and repository for natural language prompts","volume-title":"Proceedings of the 60th annual meeting of the association for computational linguistics: system demonstrations","author":"Bach","year":"2022"},{"key":"B4","doi-asserted-by":"publisher","first-page":"05862","DOI":"10.48550\/ARXIV.2204.05862","article-title":"Training a helpful and harmless assistant with reinforcement learning from human feedback","author":"Bai","year":"2022","journal-title":"Corr. abs\/2204"},{"key":"B5","doi-asserted-by":"publisher","first-page":"15787","DOI":"10.48550\/arXiv.2107.07170","article-title":"Flex: unifying evaluation for few-shot nlp","volume":"34","author":"Bragg","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B6","first-page":"1498","article-title":"A sample efficiency improved method via hierarchical reinforcement learning networks","volume-title":"2022 31st IEEE international conference on robot and human interactive communication","author":"Chen","year":"2022"},{"key":"B7","doi-asserted-by":"crossref","first-page":"3369","DOI":"10.18653\/v1\/2022.emnlp-main.222","article-title":"RLPrompt: optimizing discrete text prompts with reinforcement learning","volume-title":"Proceedings of the 2022 conference on empirical methods in natural language processing","author":"Deng","year":"2022"},{"key":"B8","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding","volume-title":"North American chapter of the association for computational linguistics","author":"Devlin","year":"2019"},{"key":"B9","doi-asserted-by":"crossref","first-page":"4339","DOI":"10.18653\/v1\/2020.findings-emnlp.389","article-title":"What\u2018s so special about BERT\u2018s layers? a closer look at the NLP pipeline in monolingual and multilingual models","volume-title":"Findings of the association for computational linguistics: emnlp 2020","author":"de Vries","year":"2020"},{"key":"B10","article-title":"Promptbreeder: self-referential self-improvement via prompt evolution","volume-title":"Proceedings of the 41st international conference on machine learning","author":"Fernando","year":"2024"},{"key":"B11","doi-asserted-by":"crossref","first-page":"8410","DOI":"10.18653\/v1\/2022.acl-long.576","article-title":"PPT: pre-trained prompt tuning for few-shot learning","volume-title":"Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers)","author":"Gu","year":"2022"},{"key":"B12","article-title":"Optimizing prompts for text-to-image generation","volume-title":"Thirty-seventh conference on neural information processing systems","author":"Hao","year":"2023"},{"key":"B13","first-page":"2014","article-title":"Few-shot preference learning for human-in-the-loop rl","volume-title":"Conference on robot learning","author":"Hejna III","year":"2023"},{"key":"B14","first-page":"709","article-title":"Visual prompt tuning","volume-title":"European conference on computer vision","author":"Jia","year":"2022"},{"key":"B15","doi-asserted-by":"publisher","first-page":"72578","DOI":"10.1109\/access.2024.3403426","article-title":"Discrete prompt compression with reinforcement learning","volume":"12","author":"Jung","year":"2024","journal-title":"IEEE Access"},{"key":"B16","first-page":"08189","article-title":"Prewrite: prompt rewriting with reinforcement learning","author":"Kong","year":"2024","journal-title":"Corr. abs\/2401"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.20944\/preprints202304.0827.v2","article-title":"Rosgpt: next-generation human-robot interaction with chatgpt and ros","author":"Koubaa","year":"2023"},{"article-title":"Layer normalization","year":"2016","author":"Lei Ba","key":"B18"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.07602","article-title":"P-tuning v2: prompt tuning can be comparable to fine-tuning universally across scales and tasks","author":"Liu","year":"2021","journal-title":"arXiv Prepr. arXiv:2110.07602"},{"key":"B20","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.aiopen.2023.08.012","article-title":"Gpt understands, too","volume":"5","author":"Liu","year":"2024","journal-title":"AI Open"},{"key":"B21","doi-asserted-by":"crossref","first-page":"11048","DOI":"10.18653\/v1\/2022.emnlp-main.759","article-title":"Rethinking the role of demonstrations: what makes in-context learning work?","volume-title":"Proceedings of the 2022 conference on empirical methods in natural language processing","author":"Min","year":"2022"},{"volume-title":"Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales","year":"2005","author":"Pang","key":"B22"},{"key":"B23","doi-asserted-by":"crossref","first-page":"7047","DOI":"10.18653\/v1\/2020.acl-main.630","article-title":"tbert: topic models and bert joining forces for semantic similarity detection","volume-title":"Proceedings of the 58th annual meeting of the association for computational linguistics","author":"Peinelt","year":"2020"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1709.10087","article-title":"Learning complex dexterous manipulation with deep reinforcement learning and demonstrations","author":"Rajeswaran","year":"2017"},{"key":"B25","first-page":"399","article-title":"Exploring the space of topic coherence measures","volume-title":"Wsdm 2015 - proceedings of the 8th ACM international conference on web search and data mining","author":"R\u00f6der","year":"2015"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/.arXiv.2010.15980","article-title":"Autoprompt: eliciting knowledge from language models with automatically generated prompts","author":"Shin","year":"2020"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1631","DOI":"10.18653\/v1\/d13-1170","article-title":"Recursive deep models for semantic compositionality over a sentiment treebank","volume":"1631","author":"Socher","year":"2013","journal-title":"EMNLP"},{"key":"B28","first-page":"2754","article-title":"All the attention you need: global-local, spatial-channel attention for image retrieval","volume-title":"Proceedings of the IEEE\/CVF winter conference on applications of computer vision","author":"Song","year":"2022"},{"key":"B29","doi-asserted-by":"publisher","first-page":"06553","DOI":"10.48550\/.ARXIV.2309.06553","article-title":"Query-dependent prompt evaluation and optimization with offline inverse RL","author":"Sun","year":"2023","journal-title":"Corr. abs\/2309"},{"key":"B30","doi-asserted-by":"publisher","first-page":"111736","DOI":"10.1016\/j.knosys.2024.111736","article-title":"Hierarchical reinforcement learning from imperfect demonstrations through reachable coverage-based subgoal filtering","volume":"294","author":"Tang","year":"2024","journal-title":"Knowledge-Based Syst."},{"key":"B31","article-title":"Instance normalization: the missing ingredient for fast stylization","author":"Ulyanov","year":"2016","journal-title":"arXiv Prepr. arXiv:1607.08022"},{"key":"B32","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"B33","doi-asserted-by":"publisher","first-page":"108668","DOI":"10.1016\/j.knosys.2022.108668","article-title":"An enhanced guided lda model augmented with bert based semantic strength for aspect term extraction in sentiment analysis","volume":"246","author":"Venugopalan","year":"2022","journal-title":"Knowledge-Based Syst."},{"key":"B34","doi-asserted-by":"crossref","first-page":"1080","DOI":"10.18653\/v1\/2023.acl-short.93","article-title":"Let me check the examples: enhancing demonstration learning via explicit imitation","volume-title":"Proceedings of the 61st annual meeting of the association for computational linguistics (volume 2: short papers)","author":"Wang","year":"2023"},{"key":"B35","doi-asserted-by":"crossref","first-page":"5085","DOI":"10.18653\/v1\/2022.emnlp-main.340","article-title":"Super-NaturalInstructions: generalization via declarative instructions on 1600+ NLP tasks","volume-title":"Proceedings of the 2022 conference on empirical methods in natural language processing","author":"Wang","year":"2022"},{"key":"B36","article-title":"Finetuned language models are zero-shot learners","volume-title":"International conference on learning representations","author":"Wei","year":"2022"},{"key":"B37","article-title":"Huggingface\u2019s transformers: state-of-the-art natural language processing","author":"Wolf","year":"2019","journal-title":"arXiv Prepr. arXiv:1910.03771"},{"key":"B38","doi-asserted-by":"publisher","first-page":"2142","DOI":"10.3390\/.healthcare11152142","article-title":"A study on online health community users\u2019 information demands based on the bert-lda model","volume":"11","author":"Xiang","year":"2023","journal-title":"Healthcare"},{"key":"B39","doi-asserted-by":"crossref","first-page":"3839","DOI":"10.1145\/3442381.3450039","article-title":"Personalized approximate pareto-efficient recommendation","volume-title":"Proceedings of the web conference 2021","author":"Xie","year":"2021"},{"key":"B40","first-page":"24631","article-title":"Prompting decision transformer for few-shot policy generalization","volume-title":"Proceedings of the 39th international conference on machine learningPMLR), vol. 162 of","author":"Xu","year":"2022"},{"key":"B41","article-title":"Large language models as optimizers","volume-title":"The twelfth international conference on learning representations","author":"Yang","year":"2024"},{"key":"B42","doi-asserted-by":"crossref","first-page":"93","DOI":"10.18653\/v1\/D18-1009","article-title":"SWAG: a large-scale adversarial dataset for grounded commonsense inference","volume-title":"Proceedings of the 2018 conference on empirical methods in natural language processing","author":"Zellers","year":"2018"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.05286","article-title":"Learning from ambiguous demonstrations with self-explanation guided reinforcement learning","author":"Zha","year":"2021"},{"key":"B44","article-title":"Differentiable prompt makes pre-trained language models better few-shot learners","volume-title":"International conference on learning representations","author":"Zhang","year":""},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-2305862\/.v1","article-title":"Fusing lda topic features for bert-based text classification","author":"Zhang","year":"","journal-title":"Res. Square"},{"key":"B46","article-title":"TEMPERA: test-time prompt editing via reinforcement learning","volume-title":"The eleventh international conference on learning representations","author":"Zhang","year":"2023"},{"key":"B47","first-page":"649","article-title":"Character-level convolutional networks for text classification","author":"Zhang","year":"2015","journal-title":"NIPS"},{"key":"B48","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1016\/j.neucom.2021.07.002","article-title":"Bert-jam: maximizing the utilization of bert for neural machine translation","volume":"460","author":"Zhang","year":"2021","journal-title":"Neurocomputing"},{"key":"B49","doi-asserted-by":"crossref","first-page":"16795","DOI":"10.1109\/CVPR52688.2022.01631","article-title":"Conditional prompt learning for vision-language models","volume-title":"2022 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Zhou","year":"2022"},{"key":"B50","article-title":"Large language models are human-level prompt engineers","volume-title":"The eleventh international conference on learning representations","author":"Zhou","year":"2023"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1579990\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T04:10:56Z","timestamp":1749183056000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1579990\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,6]]},"references-count":50,"alternative-id":["10.3389\/frobt.2025.1579990"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1579990","relation":{},"ISSN":["2296-9144"],"issn-type":[{"type":"electronic","value":"2296-9144"}],"subject":[],"published":{"date-parts":[[2025,6,6]]},"article-number":"1579990"}}