{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T11:05:42Z","timestamp":1784977542483,"version":"3.55.0"},"reference-count":99,"publisher":"Association for Computing Machinery (ACM)","issue":"11","funder":[{"name":"Ministry of Culture and Science of the State of North Rhine-Westphalia","award":["NW21-059A"],"award-info":[{"award-number":["NW21-059A"]}]},{"name":"Natural Language Learning & Generation (NLLG) Lab at TU N\u00fcrnberg"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,8,30]]},"abstract":"<jats:p>\n                    Large language model (LLM) research has grown rapidly, along with increasing concern about their limitations. In this survey, we conduct a data-driven, semi-automated review of research on limitations of LLMs (\n                    <jats:monospace>LLLMs<\/jats:monospace>\n                    ) from 2022 to early 2025 using a bottom-up approach. From a corpus of 250,000 ACL and arXiv papers, we identify 14,648 relevant papers using keyword filtering, LLM-based classification, validated against expert labels, and topic clustering (via two approaches, HDBSCAN+BERTopic and LlooM). We find that the share of LLM-related papers increases over fivefold in ACL and nearly eightfold in arXiv between 2022 and 2025. Since 2022,\n                    <jats:monospace>LLLMs<\/jats:monospace>\n                    research grows even faster, reaching over 30% of LLM papers by 2025.\n                    <jats:italic toggle=\"yes\">Reasoning<\/jats:italic>\n                    remains the most studied limitation, followed by\n                    <jats:italic toggle=\"yes\">generalization<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">hallucination<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">bias<\/jats:italic>\n                    , and\n                    <jats:italic toggle=\"yes\">security<\/jats:italic>\n                    . The distribution of topics in the ACL dataset stays relatively stable over time, while arXiv shifts toward\n                    <jats:italic toggle=\"yes\">security risks<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">alignment<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">hallucinations<\/jats:italic>\n                    ,\n                    <jats:italic toggle=\"yes\">knowledge editing<\/jats:italic>\n                    , and\n                    <jats:italic toggle=\"yes\">multimodality<\/jats:italic>\n                    . We offer a quantitative view of trends in\n                    <jats:monospace>LLLMs<\/jats:monospace>\n                    research and release a dataset of annotated abstracts and a validated methodology, available at: github.com\/a-kostikova\/LLLMs-Survey.\n                  <\/jats:p>","DOI":"10.1145\/3801096","type":"journal-article","created":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T20:40:23Z","timestamp":1772916023000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Models"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-6138-8707","authenticated-orcid":false,"given":"Aida","family":"Kostikova","sequence":"first","affiliation":[{"name":"CITEC, Bielefeld University Faculty of Technology","place":["Bielefeld, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1390-4709","authenticated-orcid":false,"given":"Zhipin","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Technology Nuremberg","place":["Nuremberg, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0516-2760","authenticated-orcid":false,"given":"Deidamea","family":"Bajri","sequence":"additional","affiliation":[{"name":"University of Mannheim","place":["Mannheim, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5286-9571","authenticated-orcid":false,"given":"Ole","family":"P\u00fctz","sequence":"additional","affiliation":[{"name":"CITEC, Bielefeld University Faculty of Technology","place":["Bielefeld, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3899-2450","authenticated-orcid":false,"given":"Benjamin","family":"Paa\u00dfen","sequence":"additional","affiliation":[{"name":"CITEC, Bielefeld University Faculty of Technology","place":["Bielefeld, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4663-8336","authenticated-orcid":false,"given":"Steffen","family":"Eger","sequence":"additional","affiliation":[{"name":"University of Technology Nuremberg","place":["Nuremberg, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,4,17]]},"reference":[{"key":"e_1_3_4_2_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman et\u00a0al. 2023. Gpt-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_4_3_2","unstructured":"Shubham Agarwal Gaurav Sahu Abhay Puri Issam H. Laradji Krishnamurthy D. J. Dvijotham Jason Stanley Laurent Charlin and Christopher Pal. 2024. Litllm: A toolkit for scientific literature review. arXiv:2402.01788. Retrieved from https:\/\/arxiv.org\/abs\/2402.01788"},{"key":"e_1_3_4_4_2","unstructured":"Janice Ahn Rishu Verma Renze Lou Di Liu Rui Zhang and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv:2402.00157. Retrieved from https:\/\/arxiv.org\/abs\/2402.00157"},{"key":"e_1_3_4_5_2","doi-asserted-by":"crossref","unstructured":"Ibrahim Al Azher Miftahul Jannat Mokarrama Zhishuai Guo Sagnik Ray Choudhury and Hamed Alhoori. 2025. BAGELS: Benchmarking the automated generation and extraction of limitations from scholarly text. arXiv:2505.18207. Retrieved from https:\/\/arxiv.org\/abs\/2505.18207","DOI":"10.18653\/v1\/2025.findings-emnlp.1050"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/metrics2020005"},{"key":"e_1_3_4_7_2","first-page":"1","volume-title":"Proceedings of the 24th ACM\/IEEE Joint Conference on Digital Libraries","author":"Azher Ibrahim Al","year":"2024","unstructured":"Ibrahim Al Azher, Venkata Devesh Reddy Seethi, Akhil Pandey Akella, and Hamed Alhoori. 2024. Limtopic: Llm-based topic modeling and text summarization for analyzing scientific articles limitations. In Proceedings of the 24th ACM\/IEEE Joint Conference on Digital Libraries. 1\u201312."},{"key":"e_1_3_4_8_2","unstructured":"Sourav Banerjee Ayushi Agarwal and Saloni Singla. 2024. Llms will always hallucinate and we need to live with this. arXiv:2409.05746. Retrieved from https:\/\/arxiv.org\/abs\/2409.05746"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","unstructured":"Nitesh V. Chawla Kevin W. Bowyer Lawrence O. Hall and W. Philip Kegelmeyer. 2002. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16 1 (2002) 321\u2013357.","DOI":"10.1613\/jair.953"},{"key":"e_1_3_4_11_2","unstructured":"Qiguang Chen Libo Qin Jinhao Liu Dengyun Peng Jiannan Guan Peng Wang Mengkang Hu Yuhang Zhou Te Gao and Wangxiang Che. 2025. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv:2503.09567. Retrieved from https:\/\/arxiv.org\/abs\/2503.09567"},{"key":"e_1_3_4_12_2","unstructured":"Zhiyu Zoey Chen Jing Ma Xinlu Zhang Nan Hao An Yan Armineh Nourbakhsh Xianjun Yang Julian McAuley Linda Petzold and William Yang Wang. 2024. A survey on large language models for critical societal domains: Finance healthcare and law. arXiv:2405.01769. Retrieved from https:\/\/arxiv.org\/abs\/2405.01769"},{"key":"e_1_3_4_13_2","unstructured":"Wei-Lin Chiang Lianmin Zheng Ying Sheng Anastasios Nikolas Angelopoulos Tianle Li Dacheng Li Banghua Zhu Hao Zhang Michael I. Jordan Joseph E. Gonzalez and Ion Stoica. 2024. Chatbot arena: An open Platform for evaluating LLMs by human preference. In Proceedings of the 41st International Conference on Machine Learning 235 (2024) 8359\u20138388."},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/3648699.3648939"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3712001"},{"key":"e_1_3_4_16_2","unstructured":"Jay DeYoung Sarthak Jain Nazneen Fatema Rajani Eric Lehman Caiming Xiong Richard Socher and Byron C. Wallace. 2019. ERASER: A benchmark to evaluate rationalized NLP models. arXiv:1911.03429. Retrieved from https:\/\/arxiv.org\/abs\/1911.03429"},{"key":"e_1_3_4_17_2","unstructured":"Jairo Diaz-Rodriguez. 2025. k-LLMmeans: Summaries as centroids for interpretable and scalable LLM-based text clustering. arXiv:2502.09667. Retrieved from https:\/\/arxiv.org\/abs\/2502.09667"},{"issue":"1","key":"e_1_3_4_18_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1108\/JEBDE-08-2023-0015","article-title":"Unraveling the landscape of large language models: A systematic review and future perspectives","volume":"3","author":"Ding Qinxu","year":"2024","unstructured":"Qinxu Ding, Ding Ding, Yue Wang, Chong Guan, and Bosheng Ding. 2024. Unraveling the landscape of large language models: A systematic review and future perspectives. Journal of Electronic Business & Digital Economics 3, 1 (2024), 3\u201319.","journal-title":"Journal of Electronic Business & Digital Economics"},{"key":"e_1_3_4_19_2","unstructured":"Steffen Eger Yong Cao Jennifer D\u2019Souza Andreas Geiger Christian Greisinger Stephanie Gross Yufang Hou Brigitte Krenn Anne Lauscher Yizhi Li et\u00a0al. 2025. Transforming science with large language models: A survey on AI-assisted scientific discovery experimentation content generation and evaluation. arXiv:2502.05151. Retrieved from https:\/\/arxiv.org\/abs\/2502.05151"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.1025"},{"key":"e_1_3_4_21_2","unstructured":"Yuyou Gan Yong Yang Zhe Ma Ping He Rui Zeng Yiming Wang Qingming Li Chunyi Zhou Songze Li Ting Wang et\u00a0al. 2024. Navigating the risks: A survey of security privacy and ethics threats in llm-based agents. arXiv:2411.09523. Retrieved from https:\/\/arxiv.org\/abs\/2411.09523"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.3390\/app14177675"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-024-02164-w"},{"key":"e_1_3_4_24_2","unstructured":"Aaron Grattafiori Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Alex Vaughan et\u00a0al. 2024. The llama 3 herd of models. arXiv:2407.21783. Retrieved from https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_4_25_2","unstructured":"Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv:2203.05794. Retrieved from https:\/\/arxiv.org\/abs\/2203.05794"},{"key":"e_1_3_4_26_2","unstructured":"Zishan Guo Renren Jin Chuang Liu Yufei Huang Dan Shi Supryadi Linhao Yu Yan Liu Jiaxuan Li Bojian Xiong et\u00a0al. 2023. Evaluating large language models: A comprehensive survey. arXiv:2310.19736. Retrieved from https:\/\/arxiv.org\/abs\/2310.19736"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dim.2024.100066"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","unstructured":"Muhammad Usman Hadi Qasem Al Tashi Rizwan Qureshi Abbas Shah Amgad Muneer Muhammad Irfan Anas Zafar Muhammad Bilal Shaikh Naveed Akhtar Syed Zohaib Hassan et\u00a0al. 2023. Large language models: A comprehensive survey of its applications challenges limitations and future prospects. TechRxiv. DOI:10.36227\/techrxiv.23589741.v8","DOI":"10.36227\/techrxiv.23589741.v8"},{"key":"e_1_3_4_29_2","unstructured":"Yichen He Guanhua Huang Peiyuan Feng Yuan Lin Yuchen Zhang Hang Li and Weinan E. 2025. PaSa: An LLM agent for comprehensive academic paper search. arXiv:2501.10120. Retrieved from https:\/\/arxiv.org\/abs\/2501.10120"},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3695988"},{"key":"e_1_3_4_31_2","unstructured":"Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv:2212.10403. Retrieved from https:\/\/arxiv.org\/abs\/2212.10403"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3703155"},{"key":"e_1_3_4_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_3_4_34_2","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier et\u00a0al. 2023. Mistral 7B. arxiv:2310.06825 [cs.CL]. Retrieved from https:\/\/arxiv.org\/abs\/2310.06825"},{"key":"e_1_3_4_35_2","unstructured":"Weipeng Jiang Zhenting Wang Juan Zhai Shiqing Ma Zhengyu Zhao and Chao Shen. 2024. Unlocking adversarial suffix optimization without affirmative phrases: Efficient black-box jailbreaking via llm as optimizer. arXiv:2408.11313. Retrieved from https:\/\/arxiv.org\/abs\/2408.11313"},{"key":"e_1_3_4_36_2","unstructured":"Junfeng Jiao Saleh Afroogh Yiming Xu and Connor Phillips. 2024. Navigating llm ethics: Advancements challenges and future directions. arXiv:2406.18841. Retrieved from https:\/\/arxiv.org\/abs\/2406.18841"},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3469578"},{"key":"e_1_3_4_38_2","unstructured":"Maurice G. Kendall. 1948. Rank Correlation Methods. Charles Griffin London."},{"key":"e_1_3_4_39_2","unstructured":"Jonathan Kim Anna Podlasek Kie Shidara Feng Liu Ahmed Alaa and Danilo Bernardo. 2025. Limitations of large language models in clinical problem-solving arising from inflexible reasoning. arXiv:2502.04381. Retrieved from https:\/\/arxiv.org\/abs\/2502.04381"},{"key":"e_1_3_4_40_2","unstructured":"Ashutosh Kumar Shiv Vignesh Murthy Sagarika Singh and Swathy Ragupathy. 2024. The ethics of interaction: Mitigating security threats in llms. arXiv:2401.12273. Retrieved from https:\/\/arxiv.org\/abs\/2401.12273"},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642830"},{"key":"e_1_3_4_42_2","doi-asserted-by":"crossref","unstructured":"Christoph Leiter Jonas Belouadi Yanran Chen Ran Zhang Daniil Larionov Aida Kostikova and Steffen Eger. 2024. NLLG quarterly arXiv report 09\/24: What are the most influential current AI papers? arXiv:2412.12121. Retrieved from https:\/\/arxiv.org\/abs\/2412.12121","DOI":"10.2139\/ssrn.5045225"},{"key":"e_1_3_4_43_2","doi-asserted-by":"crossref","unstructured":"Benjamin A. Levinstein and Daniel A. Herrmann. 2025. Still no lie detector for language models: Probing empirical and conceptual roadblocks. Philosophical Studies 182 7 (2025) 1539\u20131565.","DOI":"10.1007\/s11098-023-02094-3"},{"key":"e_1_3_4_44_2","first-page":"30850","article-title":"Should we really edit language models? On the evaluation of edited language models","volume":"37","author":"Li Qi","year":"2024","unstructured":"Qi Li, Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Xinglin Pan, and Xiaowen Chu. 2024. Should we really edit language models? On the evaluation of edited language models. Advances in Neural Information Processing Systems 37 (2024), 30850\u201330885.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_45_2","unstructured":"Yucheng Li Bo Dong Chenghua Lin and Frank Guerin. 2023. Compressing context to enhance inference efficiency of large language models. arXiv:2310.06201. Retrieved from https:\/\/arxiv.org\/abs\/2310.06201"},{"key":"e_1_3_4_46_2","unstructured":"Zhong-Zhi Li Duzhen Zhang Ming-Liang Zhang Jiaxin Zhang Zengyan Liu Yuxuan Yao Haotian Xu Junhao Zheng Pei-Jie Wang Xiuyi Chen et\u00a0al. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv:2502.17419. Retrieved from https:\/\/arxiv.org\/abs\/2502.17419"},{"key":"e_1_3_4_47_2","unstructured":"Xun Liang Jiawei Yang Yezhaohui Wang Chen Tang Zifan Zheng Shichao Song Zehao Lin Yebin Yang Simin Niu Hanyu Wang et\u00a0al. 2025. Surveyx: Academic survey automation via large language models. arXiv:2502.14776. Retrieved from https:\/\/arxiv.org\/abs\/2502.14776"},{"key":"e_1_3_4_48_2","unstructured":"Inna Wanyin Lin Lucille Njoo Anjalie Field Ashish Sharma Katharina Reinecke Tim Althoff and Yulia Tsvetkov. 2022. Gendered mental health stigma in masked language models. arXiv:2210.15144. Retrieved from https:\/\/arxiv.org\/abs\/2210.15144"},{"key":"e_1_3_4_49_2","first-page":"34892","article-title":"Visual instruction tuning","volume":"36","author":"Liu Haotian","year":"2023","unstructured":"Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in Neural Information Processing Systems 36 (2023), 34892\u201334916.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_50_2","unstructured":"Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rongjun Li and Wei Peng. 2024. A survey on hallucination in large vision-language models. arXiv:2402.00253. Retrieved from https:\/\/arxiv.org\/abs\/2402.00253"},{"key":"e_1_3_4_51_2","doi-asserted-by":"crossref","first-page":"6525","DOI":"10.18653\/v1\/2023.emnlp-main.402","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Long Quanyu","year":"2023","unstructured":"Quanyu Long, Wenya Wang, and Sinno Pan. 2023. Adapt in contexts: Retrieval-augmented domain adaptation via in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 6525\u20136542."},{"key":"e_1_3_4_52_2","unstructured":"Ziming Luo Zonglin Yang Zexin Xu Wei Yang and Xinya Du. 2025. LLM4SR: A survey on large language models for scientific research. arXiv:2501.04306. Retrieved from https:\/\/arxiv.org\/abs\/2501.04306"},{"key":"e_1_3_4_53_2","unstructured":"Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang and Feng Wu. 2024. Coarse-to-fine highlighting: Reducing knowledge hallucination in large language models. arXiv:2410.15116. Retrieved from https:\/\/arxiv.org\/abs\/2410.15116"},{"key":"e_1_3_4_54_2","doi-asserted-by":"crossref","unstructured":"Henry B. Mann. 1945. Nonparametric tests against trend. Econometrica 13 3 (July 1945) 245\u2013259.","DOI":"10.2307\/1907187"},{"key":"e_1_3_4_55_2","first-page":"3659","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Marchal Marian","year":"2022","unstructured":"Marian Marchal, Merel Scholman, Frances Yung, and Vera Demberg. 2022. Establishing annotation quality in multi-label annotations. In Proceedings of the 29th International Conference on Computational Linguistics. 3659\u20133668."},{"key":"e_1_3_4_56_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324921000127"},{"key":"e_1_3_4_57_2","unstructured":"Andrea Matarazzo and Riccardo Torlone. 2025. A survey on large language models with some insights on their capabilities and limitations. arXiv:2501.04040. Retrieved from https:\/\/arxiv.org\/abs\/2501.04040"},{"key":"e_1_3_4_58_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00205"},{"key":"e_1_3_4_59_2","unstructured":"Leland McInnes John Healy and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426. Retrieved from https:\/\/arxiv.org\/abs\/1802.03426"},{"key":"e_1_3_4_60_2","doi-asserted-by":"crossref","unstructured":"Rajiv Movva Sidhika Balachandar Kenny Peng Gabriel Agostini Nikhil Garg and Emma Pierson. 2023. Topics authors and institutions in Large Language Model research: Trends from 17K arXiv papers. arXiv:2307.10700. Retrieved from https:\/\/arxiv.org\/abs\/2307.10700","DOI":"10.18653\/v1\/2024.naacl-long.67"},{"key":"e_1_3_4_61_2","unstructured":"Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes and Ajmal Mian. 2023. A comprehensive overview of large language models. arXiv:2307.06435. Retrieved from https:\/\/arxiv.org\/abs\/2307.06435"},{"key":"e_1_3_4_62_2","unstructured":"Gabriel Nicholas and Aliya Bhatia. 2023. Lost in translation: Large language models in non-English content analysis. arXiv:2306.07377. Retrieved from https:\/\/arxiv.org\/abs\/2306.07377"},{"key":"e_1_3_4_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3352100"},{"key":"e_1_3_4_64_2","doi-asserted-by":"crossref","unstructured":"Brendan Park Madeline Janecek Naser Ezzati-Jivan Yifeng Li and Ali Emami. 2024. Picturing ambiguity: A visual twist on the winograd schema challenge. arXiv:2405.16277. Retrieved from https:\/\/arxiv.org\/abs\/2405.16277","DOI":"10.18653\/v1\/2024.acl-long.22"},{"key":"e_1_3_4_65_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-industry.54"},{"key":"e_1_3_4_66_2","unstructured":"Duy Khoa Pham and Bao Quoc Vo. 2024. Towards reliable medical question answering: Techniques and challenges in mitigating hallucinations in language models. arXiv:2408.13808. Retrieved from https:\/\/arxiv.org\/abs\/2408.13808"},{"key":"e_1_3_4_67_2","unstructured":"Aske Plaat Annie Wong Suzan Verberne Joost Broekens Niki van Stein and Thomas Back. 2024. Reasoning with large language models a survey. arXiv:2407.11511. Retrieved from https:\/\/arxiv.org\/abs\/2407.11511"},{"key":"e_1_3_4_68_2","unstructured":"Xiangyu Qi Yi Zeng Tinghao Xie Pin-Yu Chen Ruoxi Jia Prateek Mittal and Peter Henderson. 2023. Fine-tuning aligned language models compromises safety even when users do not intend to! arXiv:2310.03693. Retrieved from https:\/\/arxiv.org\/abs\/2310.03693"},{"key":"e_1_3_4_69_2","doi-asserted-by":"crossref","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv:1908.10084. Retrieved from https:\/\/arxiv.org\/abs\/1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_4_70_2","doi-asserted-by":"publisher","unstructured":"Anna Rogers Olga Kovaleva and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. TACL 8 (2020) 842\u2013866. DOI:10.1162\/tacl_a_00349","DOI":"10.1162\/tacl_a_00349"},{"key":"e_1_3_4_71_2","first-page":"187","volume-title":"Proceedings of the 14th International Workshop on Semantic Evaluation","author":"Rother David","year":"2020","unstructured":"David Rother, Thomas Haider, and Steffen Eger. 2020. CMCE at SemEval-2020 task 1: Clustering on manifolds of contextualized embeddings to detect historical meaning shifts. In Proceedings of the 14th International Workshop on Semantic Evaluation. 187\u2013193."},{"key":"e_1_3_4_72_2","doi-asserted-by":"publisher","unstructured":"Pranab Sahoo Prabhash Meharia Akash Ghosh Sriparna Saha Vinija Jain and Aman Chadha. 2024. A comprehensive survey of hallucination in large language image video and audio foundation models. In Findings of the Association for Computational Linguistics: EMNLP 2024 Yaser Al-Onaizan Mohit Bansal and Yun-Nung Chen (Eds.). Association for Computational Linguistics 11709\u201311724. DOI:10.18653\/v1\/2024.findings-emnlp.685","DOI":"10.18653\/v1\/2024.findings-emnlp.685"},{"key":"e_1_3_4_73_2","doi-asserted-by":"crossref","unstructured":"Pranab Sahoo Prabhash Meharia Akash Ghosh Sriparna Saha Vinija Jain and Aman Chadha. 2024. A comprehensive survey of hallucination in large language image video and audio foundation models. .arXiv:2405.09589. Retrieved from https:\/\/arxiv.org\/abs\/2405.09589","DOI":"10.18653\/v1\/2024.findings-emnlp.685"},{"key":"e_1_3_4_74_2","unstructured":"Omar Shaikh Hongxin Zhang William Held Michael Bernstein and Diyi Yang. 2022. On second thought let\u2019s not think step by step! bias and toxicity in zero-shot reasoning. arXiv:2212.08061. Retrieved from https:\/\/arxiv.org\/abs\/2212.08061"},{"key":"e_1_3_4_75_2","unstructured":"Siqi Shen Lajanugen Logeswaran Moontae Lee Honglak Lee Soujanya Poria and Rada Mihalcea. 2024. Understanding the capabilities and limitations of large language models for cultural commonsense. arXiv:2405.04655. Retrieved from https:\/\/arxiv.org\/abs\/2405.04655"},{"key":"e_1_3_4_76_2","unstructured":"Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu and Deyi Xiong. 2023. Large language model alignment: A survey. arXiv:2309.15025. Retrieved from https:\/\/arxiv.org\/abs\/2309.15025"},{"key":"e_1_3_4_77_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00639"},{"key":"e_1_3_4_78_2","unstructured":"Aarohi Srivastava Abhinav Rastogi Abhishek Rao Abu Awal Md Shoeb Abubakar Abid Adam Fisch Adam R. Brown Adam Santoro Aditya Gupta and Garriga-Alonso Adri\u00e0. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv:2206.04615. Retrieved from https:\/\/arxiv.org\/abs\/2206.04615"},{"key":"e_1_3_4_79_2","unstructured":"Lichao Sun Yue Huang Haoran Wang Siyuan Wu Qihui Zhang Chujie Gao Yixin Huang Wenhan Lyu Yixuan Zhang et\u00a0al. 2024. Trustllm: Trustworthiness in large language models. arXiv:2401.05561. Retrieved from https:\/\/arxiv.org\/abs\/2401.05561"},{"key":"e_1_3_4_80_2","unstructured":"Yingshui Tan Yilei Jiang Yanshi Li Jiaheng Liu Xingyuan Bu Wenbo Su Xiangyu Yue Xiaoyong Zhu and Bo Zheng. 2025. Equilibrate RLHF: Towards balancing helpfulness-safety trade-off in large language models. arXiv:2502.11555. Retrieved from https:\/\/arxiv.org\/abs\/2502.11555"},{"key":"e_1_3_4_81_2","unstructured":"S. M. Tonmoy S. M. Zaman Vinija Jain Anku Rani Vipula Rawte Aman Chadha and Amitava Das. 2024. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv:2401.01313. Retrieved from https:\/\/arxiv.org\/abs\/2401.01313"},{"key":"e_1_3_4_82_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re et\u00a0al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_4_83_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-02855-5"},{"key":"e_1_3_4_84_2","doi-asserted-by":"crossref","unstructured":"Vijay Viswanathan Kiril Gashteovski Carolin Lawrence Tongshuang Wu and Graham Neubig. 2024. Large language models enable few-shot clustering. TACL 12 (2024) 321\u2013333.","DOI":"10.1162\/tacl_a_00648"},{"key":"e_1_3_4_85_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-024-40231-1"},{"key":"e_1_3_4_86_2","first-page":"115119","article-title":"Autosurvey: Large language models can automatically write surveys","volume":"37","author":"Wang Yidong","year":"2024","unstructured":"Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, et\u00a0al. 2024. Autosurvey: Large language models can automatically write surveys. Advances in Neural Information Processing Systems 37 (2024), 115119\u2013115145.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_87_2","unstructured":"Zhichao Wang Bin Bi Shiva Kumar Pentyala Kiran Ramnath Sougata Chaudhuri Shubham Mehrotra Zixu Zhu Xiang-Bo Mao Sitaram Asur et\u00a0al. 2024. A comprehensive survey of LLM alignment techniques: RLHF RLAIF PPO DPO and more. arXiv:2407.16216. Retrieved from https:\/\/arxiv.org\/abs\/2407.16216"},{"key":"e_1_3_4_88_2","unstructured":"Jason Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph Sebastian Borgeaud Dani Yogatama Maarten Bosma Denny Zhou Donald Metzler Ed H. Chi Tatsunori Hashimoto Oriol Vinyals Percy Liang Jeff Dean and William Fedus. 2022. Emergent abilities of large language models. Transactions on Machine Learning Research 2022 (2022). Available at https:\/\/openreview.net\/forum?id=yzkSU5zdwD"},{"key":"e_1_3_4_89_2","unstructured":"Chaoyi Wu Jiayu Lei Qiaoyu Zheng Weike Zhao Weixiong Lin Xiaoman Zhang Xiao Zhou Ziheng Zhao Ya Zhang Yanfeng Wang et\u00a0al. 2023. Can gpt-4v (ision) serve medical applications? Case studies on gpt-4v for multimodal medical diagnosis. arXiv:2310.09909. Retrieved from https:\/\/arxiv.org\/abs\/2310.09909"},{"key":"e_1_3_4_90_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-024-01291-2"},{"key":"e_1_3_4_91_2","unstructured":"Fengli Xu Qianyue Hao Zefang Zong Jingwei Wang Yunke Zhang Jingyi Wang Xiaochong Lan Jiahui Gong Tianjian Ouyang Fanjin Meng et\u00a0al. 2025. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv:2501.09686. Retrieved from https:\/\/arxiv.org\/abs\/2501.09686"},{"key":"e_1_3_4_92_2","unstructured":"Ziwei Xu Sanjay Jain and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models. arXiv:2401.11817. Retrieved from https:\/\/arxiv.org\/abs\/2401.11817"},{"key":"e_1_3_4_93_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.acl-long.1009"},{"key":"e_1_3_4_94_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.hcc.2024.100211"},{"key":"e_1_3_4_95_2","doi-asserted-by":"crossref","unstructured":"Ron Yosef Yonatan Bitton and Dafna Shahaf. 2023. Irfl: Image recognition of figurative language. arXiv:2303.15445. Retrieved from https:\/\/arxiv.org\/abs\/2303.15445","DOI":"10.18653\/v1\/2023.findings-emnlp.74"},{"key":"e_1_3_4_96_2","doi-asserted-by":"crossref","unstructured":"Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu and Dong Yu. 2024. Mm-llms: Recent advances in multimodal large language models. arXiv:2401.13601. Retrieved from https:\/\/arxiv.org\/abs\/2401.13601","DOI":"10.18653\/v1\/2024.findings-acl.738"},{"key":"e_1_3_4_97_2","unstructured":"Longteng Zhang Xiang Liu Zeyu Li Xinglin Pan Peijie Dong Ruibo Fan Rui Guo Xin Wang Qiong Luo Shaohuai Shi and et\u00a0al. 2023. Dissecting the runtime performance of the training fine-tuning and inference of large language models. arXiv:2311.03687. Retrieved from https:\/\/arxiv.org\/abs\/2311.03687"},{"key":"e_1_3_4_98_2","doi-asserted-by":"crossref","unstructured":"Yuwei Zhang Zihan Wang and Jingbo Shang. 2023. Clusterllm: Large language models as a guide for text clustering. arXiv:2305.14871. Retrieved from https:\/\/arxiv.org\/abs\/2305.14871","DOI":"10.18653\/v1\/2023.emnlp-main.858"},{"key":"e_1_3_4_99_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et\u00a0al. 2023. A survey of large language models. arXiv:2303.18223. Retrieved from https:\/\/arxiv.org\/abs\/2303.18223"},{"key":"e_1_3_4_100_2","unstructured":"Yutao Zhu Huaying Yuan Shuting Wang Jiongnan Liu Wenhan Liu Chenlong Deng Haonan Chen Zheng Liu Zhicheng Dou and Ji-Rong Wen. 2023. Large language models for information retrieval: A survey. arXiv:2308.07107. Retrieved from https:\/\/arxiv.org\/abs\/2308.07107"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3801096","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:20:23Z","timestamp":1776442823000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3801096"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,17]]},"references-count":99,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2026,8,30]]}},"alternative-id":["10.1145\/3801096"],"URL":"https:\/\/doi.org\/10.1145\/3801096","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,17]]},"assertion":[{"value":"2025-05-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}