{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T05:13:44Z","timestamp":1781673224516,"version":"3.54.5"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T00:00:00Z","timestamp":1730246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2024,10,30]]},"abstract":"<jats:p>Large Language Models (LLMs) have demonstrated impressive abilities in tackling tasks across numerous domains. The capabilities of LLMs could potentially be applied to various computer networking tasks, including network synthesis, management, debugging, security, and education. However, LLMs can be unreliable: they are prone to reasoning errors and may hallucinate incorrect information. Their effectiveness and limitations in computer networking tasks remain unclear. In this paper, we attempt to understand the capabilities and limitations of LLMs in network applications. We evaluate misunderstandings regarding networking related concepts across 3 LLMs over 500 questions. We assess the reliability, explain-ability, and stability of LLM responses to networking questions. Furthermore, we investigate errors made, analyzing their cause, detectability, effects, and potential mitigation strategies.<\/jats:p>","DOI":"10.1145\/3717512.3717515","type":"journal-article","created":{"date-parts":[[2025,2,11]],"date-time":"2025-02-11T20:41:41Z","timestamp":1739306501000},"page":"14-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Understanding Misunderstandings: Evaluating LLMs on Networking Questions"],"prefix":"10.1145","volume":"54","author":[{"given":"Mubashir","family":"Anwar","sequence":"first","affiliation":[{"name":"University of Illinois Urbana-Champaign, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Caesar","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,2,11]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Introducing the next generation of Claude. (Mar","year":"2024","unstructured":"Anthropic. 2024. Introducing the next generation of Claude. (Mar 2024). https:\/\/www.anthropic.com\/news\/claude-3-family"},{"key":"e_1_2_1_2_1","volume-title":"Himanshu Gaurav Singh, and Mausam","author":"Arora Daman","year":"2023","unstructured":"Daman Arora, Himanshu Gaurav Singh, and Mausam. 2023. Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models. (2023). arXiv:cs.CL\/2305.15074 https:\/\/arxiv.org\/abs\/2305.15074"},{"key":"e_1_2_1_3_1","unstructured":"Yejin Bang Samuel Cahyawijaya Nayeon Lee Wenliang Dai Dan Su Bryan Wilie Holy Lovenia Ziwei Ji Tiezheng Yu Willy Chung Quyet V. Do Yan Xu and Pascale Fung. 2023. A Multitask Multilingual Multimodal Evaluation of ChatGPT on Reasoning Hallucination and Interactivity. (2023). arXiv:cs.CL\/2302.04023 https:\/\/arxiv.org\/abs\/2302.04023"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. Association for Computing Machinery, Canada, 610--623","author":"Bender Emily M","year":"2021","unstructured":"Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. Association for Computing Machinery, Canada, 610--623."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2218523120"},{"key":"e_1_2_1_6_1","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. (2020). arXiv:cs.CL\/2005.14165 https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_2_1_7_1","volume-title":"Survey: Massive retooling around large language models underway. (Apr","author":"Burch David","year":"2023","unstructured":"David Burch. 2023. Survey: Massive retooling around large language models underway. (Apr 2023). https:\/\/arize.com\/blog\/survey-massive-retooling-around-large-language-models-underway\/"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jcim.3c00285"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.03.002"},{"key":"e_1_2_1_10_1","volume-title":"Service-Oriented Computing-ICSOC 2007 Workshops: ICSOC 2007, International Workshops","author":"May Chan KS","year":"2009","unstructured":"KS May Chan, Judith Bishop, Johan Steyn, Luciano Baresi, and Sam Guinea. 2009. A fault taxonomy for web service composition. In Service-Oriented Computing-ICSOC 2007 Workshops: ICSOC 2007, International Workshops, Vienna, Austria, September 17, 2007, Revised Selected Papers 5. Springer, Vienna, 363--375."},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Yinfang Chen Huaibing Xie Minghua Ma Yu Kang Xin Gao Liu Shi Yunjie Cao Xuedong Gao Hao Fan Ming Wen Jun Zeng Supriyo Ghosh Xuchao Zhang Chaoyun Zhang Qingwei Lin Saravan Rajmohan Dongmei Zhang and Tianyin Xu. 2023. Automatic Root Cause Analysis via Large Language Models for Cloud Incidents. (2023). arXiv:cs.SE\/2305.15778 https:\/\/arxiv.org\/abs\/2305.15778","DOI":"10.1145\/3627703.3629553"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.fertnstert.2023.05.151"},{"key":"e_1_2_1_13_1","volume-title":"Cisco Certified Network Associate. (Apr","year":"2024","unstructured":"Cisco. 2024. Cisco Certified Network Associate. (Apr 2024). https:\/\/www.cisco.com\/c\/en\/us\/training-events\/training-certifications\/exams\/current-list\/ccna-200-301.html"},{"key":"e_1_2_1_14_1","volume-title":"Network Security. (May","year":"2024","unstructured":"Cisco. 2024. Network Security. (May 2024). https:\/\/www.coursera.org\/learn\/network-security"},{"key":"e_1_2_1_15_1","unstructured":"Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick and Oyvind Tafjord. 2018. Think you have Solved Question Answering? Try ARC the AI2 Reasoning Challenge. (2018). arXiv:cs.AI\/1803.05457 https:\/\/arxiv.org\/abs\/1803.05457"},{"key":"e_1_2_1_16_1","unstructured":"Katherine M. Collins Albert Q. Jiang Simon Frieder Lionel Wong Miri Zilka Umang Bhatt Thomas Lukasiewicz Yuhuai Wu Joshua B. Tenenbaum William Hart Timothy Gowers Wenda Li Adrian Weller and Mateja Jamnik. 2023. Evaluating Language Models for Mathematics through Interactions. (2023). arXiv:cs.LG\/2306.01694 https:\/\/arxiv.org\/abs\/2306.01694"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT58122.2023.00100"},{"key":"e_1_2_1_18_1","unstructured":"Xuan-Quy Dao and Ngoc-Bich Le. 2023. Investigating the Effectiveness of ChatGPT in Mathematical Reasoning and Problem Solving: Evidence from the Vietnamese National High School Graduation Examination. (2023). arXiv:cs.CL\/2306.06331 https:\/\/arxiv.org\/abs\/2306.06331"},{"key":"e_1_2_1_19_1","volume-title":"How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?","author":"Deroy Aniket","year":"2023","unstructured":"Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh. 2023. How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization? (2023). arXiv:cs.CL\/2306.01248 https:\/\/arxiv.org\/abs\/2306.01248"},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1002\/jaba.390","article-title":"Effects of programmed teaching errors on acquisition and durability of self-care skills","volume":"50","author":"Donnelly Maeve G","year":"2017","unstructured":"Maeve G Donnelly and Amanda M Karsten. 2017. Effects of programmed teaching errors on acquisition and durability of self-care skills. Journal of Applied Behavior Analysis 50, 3 (2017), 511--528.","journal-title":"Journal of Applied Behavior Analysis"},{"key":"e_1_2_1_21_1","unstructured":"examsdigest. 2024. Study Learn and Pass the CCNA 200-301. (May 2024). https:\/\/examsdigest.com\/courses\/cisco-ccna-200-301\/"},{"key":"e_1_2_1_22_1","volume-title":"Cisco 200-301 Exam Actual Questions. (Sept","year":"2024","unstructured":"ExamTopics. 2024. Cisco 200-301 Exam Actual Questions. (Sept 2024). https:\/\/www.examtopics.com\/exams\/cisco\/200-301\/view\/1\/"},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/a0028649","article-title":"Creating illusions of knowledge: learning errors that contradict prior knowledge","volume":"142","author":"Fazio Lisa K","year":"2013","unstructured":"Lisa K Fazio, Sarah J Barber, Suparna Rajaram, Peter A Ornstein, and Elizabeth J Marsh. 2013. Creating illusions of knowledge: learning errors that contradict prior knowledge. Journal of Experimental Psychology: General 142, 1 (2013), 1.","journal-title":"Journal of Experimental Psychology: General"},{"key":"e_1_2_1_24_1","volume-title":"Coursera - Online Courses and Credentials From Top Educators. Join for Free. (May","author":"Feamster Nick","year":"2024","unstructured":"Nick Feamster. 2024. Coursera - Online Courses and Credentials From Top Educators. Join for Free. (May 2024). https:\/\/www.coursera.org\/learn\/sdn\/"},{"key":"e_1_2_1_25_1","volume-title":"Processing Inaccurate Information: Theoretical and Applied Perspectives from Cognitive Science and the Educational Sciences","author":"Ghazali-Mohammed Zayba","unstructured":"Zayba Ghazali-Mohammed. 2015. Processing Inaccurate Information: Theoretical and Applied Perspectives from Cognitive Science and the Educational Sciences. The MIT Press, Cambridge, MA."},{"key":"e_1_2_1_26_1","volume-title":"The Bits and Bytes of Computer Networking. (May","year":"2024","unstructured":"Google. 2024. The Bits and Bytes of Computer Networking. (May 2024). https:\/\/www.coursera.org\/learn\/computer-networking"},{"key":"e_1_2_1_27_1","volume-title":"Weinberger","author":"Guo Chuan","year":"2017","unstructured":"Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On Calibration of Modern Neural Networks. (2017). arXiv:cs.LG\/1706.04599 https:\/\/arxiv.org\/abs\/1706.04599"},{"key":"e_1_2_1_28_1","unstructured":"Thilo Hagendorff Ishita Dasgupta Marcel Binz Stephanie C. Y. Chan Andrew Lampinen Jane X. Wang Zeynep Akata and Eric Schulz. 2024. Machine Psychology. (2024). arXiv:cs.CL\/2303.13988 https:\/\/arxiv.org\/abs\/2303.13988"},{"key":"e_1_2_1_29_1","volume-title":"ChatGPT sets record for fastest-growing user base - analyst note. (Feb","author":"Krystal Hu.","year":"2023","unstructured":"Krystal Hu. 2023. ChatGPT sets record for fastest-growing user base - analyst note. (Feb 2023). https:\/\/www.reut6ers.com\/technology\/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01\/"},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Jie Huang and Kevin Chen-Chuan Chang. 2023. Towards Reasoning in Large Language Models: A Survey. (2023). arXiv:cs.CL\/2212.10403 https:\/\/arxiv.org\/abs\/2212.10403","DOI":"10.18653\/v1\/2023.findings-acl.67"},{"key":"e_1_2_1_31_1","volume-title":"Adams Wei Yu, Xinying Song, and Denny Zhou.","author":"Huang Jie","year":"2024","unstructured":"Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot Self-Correct Reasoning Yet. (2024). arXiv:cs.CL\/2310.01798 https:\/\/arxiv.org\/abs\/2310.01798"},{"key":"e_1_2_1_32_1","unstructured":"Yuzhen Huang Yuzhuo Bai Zhihao Zhu Junlei Zhang Jinghan Zhang Tangjun Su Junteng Liu Chuancheng Lv Yikai Zhang Jiayi Lei Yao Fu Maosong Sun and Junxian He. 2023. C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models. (2023). arXiv:cs.CL\/2305.08322 https:\/\/arxiv.org\/abs\/2305.08322"},{"key":"e_1_2_1_33_1","volume-title":"Enabling Techniques, and Challenges.","author":"Huang Yudong","year":"2023","unstructured":"Yudong Huang, Hongyang Du, Xinyuan Zhang, Dusit Niyato, Jiawen Kang, Zehui Xiong, Shuo Wang, and Tao Huang. 2023. Large Language Models for Networking: Applications, Enabling Techniques, and Challenges. (2023). arXiv:cs.NI\/2311.17474 https:\/\/arxiv.org\/abs\/2311.17474"},{"key":"e_1_2_1_34_1","volume-title":"Chun Peng, and Jimmy Huang.","author":"Jahan Israt","year":"2023","unstructured":"Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, and Jimmy Huang. 2023. Evaluation of ChatGPT on Biomedical Tasks: A Zero-Shot Comparison with Fine-Tuned Generative Transformers. (2023). arXiv:cs.CL\/2306.04504 https:\/\/arxiv.org\/abs\/2306.04504"},{"key":"e_1_2_1_35_1","unstructured":"Dongfu Jiang Xiang Ren and Bill Yuchen Lin. 2023. LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion. (2023). arXiv:cs.CL\/2306.02561 https:\/\/arxiv.org\/abs\/2306.02561"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341525.3387392"},{"key":"e_1_2_1_37_1","volume-title":"Tim Rockt\u00e4schel, Sebastian Riedel, and Douwe Kiela.","author":"Lewis Patrick","year":"2021","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen tau Yih, Tim Rockt\u00e4schel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. (2021). arXiv:cs.CL\/2005.11401 https:\/\/arxiv.org\/abs\/2005.11401"},{"key":"e_1_2_1_39_1","volume-title":"Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda.","author":"Liang Percy","year":"2023","unstructured":"Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher R\u00e9, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2023. Holistic Evaluation of Language Models. (2023). arXiv:cs.CL\/2211.09110 https:\/\/arxiv.org\/abs\/2211.09110"},{"key":"e_1_2_1_40_1","unstructured":"Chuang Liu Renren Jin Yuqi Ren Linhao Yu Tianyu Dong Xiaohan Peng Shuting Zhang Jianxiang Peng Peiyi Zhang Qingqing Lyu Xiaowen Su Qun Liu and Deyi Xiong. 2023. M3KE: A Massive Multi-Level Multi-Subject Knowledge Evaluation Benchmark for Chinese Large Language Models. (2023). arXiv:cs.CL\/2305.10263 https:\/\/arxiv.org\/abs\/2305.10263"},{"key":"e_1_2_1_41_1","volume-title":"Yuyao Wang, and Lingming Zhang.","author":"Liu Jiawei","year":"2023","unstructured":"Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. (2023). arXiv:cs.SE\/2305.01210 https:\/\/arxiv.org\/abs\/2305.01210"},{"key":"e_1_2_1_42_1","volume-title":"Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark.","author":"Madaan Aman","year":"2023","unstructured":"Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. (2023). arXiv:cs.CL\/2303.17651 https:\/\/arxiv.org\/abs\/2303.17651"},{"key":"e_1_2_1_43_1","unstructured":"Sathiya Kumaran Mani Yajie Zhou Kevin Hsieh Santiago Segarra Ranveer Chandra and Srikanth Kandula. 2023. Enhancing Network Management Using Code Generated by Large Language Models. (2023). arXiv:cs.NI\/2308.06261 https:\/\/arxiv.org\/abs\/2308.06261"},{"key":"e_1_2_1_44_1","unstructured":"Yukai Miao Yu Bai Li Chen Dan Li Haifeng Sun Xizheng Wang Ziqiu Luo Yanyu Ren Dapeng Sun Xiuting Xu Qi Zhang Chao Xiang and Xinchi Li. 2023. An Empirical Study of NetOps Capability of Pre-Trained Large Language Models. (2023). arXiv:cs.CL\/2309.05557 https:\/\/arxiv.org\/abs\/2309.05557"},{"key":"e_1_2_1_45_1","volume-title":"What do LLMs need to Synthesize Correct Router Configurations?","author":"Mondal Rajdeep","year":"2023","unstructured":"Rajdeep Mondal, Alan Tang, Ryan Beckett, Todd Millstein, and George Varghese. 2023. What do LLMs need to Synthesize Correct Router Configurations? (2023). arXiv:cs.NI\/2307.04945 https:\/\/arxiv.org\/abs\/2307.04945"},{"key":"e_1_2_1_46_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.2307\/749532","article-title":"An empirical classification model for errors in high school mathematics","volume":"18","author":"Movshovitz-Hadar Nitsa","year":"1987","unstructured":"Nitsa Movshovitz-Hadar, Orit Zaslavsky, and Shlomo Inbar. 1987. An empirical classification model for errors in high school mathematics. Journal for research in mathematics Education 18, 1 (1987), 3--14.","journal-title":"Journal for research in mathematics Education"},{"key":"e_1_2_1_47_1","volume-title":"Jonathan H. Choi, and Jungo Kasai.","author":"Nay John J.","year":"2023","unstructured":"John J. Nay, David Karamardian, Sarah B. Lawsky, Wenting Tao, Meghana Bhat, Raghav Jain, Aaron Travis Lee, Jonathan H. Choi, and Jungo Kasai. 2023. Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence. (2023). arXiv:cs.CL\/2306.07075 https:\/\/arxiv.org\/abs\/2306.07075"},{"key":"e_1_2_1_49_1","unstructured":"Vipula Rawte Amit Sheth and Amitava Das. 2023. A Survey of Hallucination in Large Foundation Models. (2023). arXiv:cs.AI\/2309.05922 https:\/\/arxiv.org\/abs\/2309.05922"},{"key":"e_1_2_1_50_1","volume-title":"Computer Networks Questions and Answers. (Sept","year":"2024","unstructured":"Sanfoundry. 2024. Computer Networks Questions and Answers. (Sept 2024). https:\/\/www.sanfoundry.com\/computer-networks-mcqs-basics\/"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9737.003.0006"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626111.3628205"},{"key":"e_1_2_1_53_1","unstructured":"Karan Singhal Shekoofeh Azizi Tao Tu S. Sara Mahdavi Jason Wei Hyung Won Chung Nathan Scales Ajay Tanwani Heather Cole-Lewis Stephen Pfohl Perry Payne Martin Seneviratne Paul Gamble Chris Kelly Nathaneal Scharli Aakanksha Chowdhery Philip Mansfield Blaise Aguera y Arcas Dale Webster Greg S. Corrado Yossi Matias Katherine Chou Juraj Gottweis Nenad Tomasev Yun Liu Alvin Rajkomar Joelle Barral Christopher Semturs Alan Karthikesalingam and Vivek Natarajan. 2022. Large Language Models Encode Clinical Knowledge. (2022). arXiv:cs.CL\/2212.13138 https:\/\/arxiv.org\/abs\/2212.13138"},{"key":"e_1_2_1_54_1","unstructured":"Giriprasad Sridhara Ranjani H. G. and Sourav Mazumdar. 2023. ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Tasks. (2023). arXiv:cs.SE\/2305.16837 https:\/\/arxiv.org\/abs\/2305.16837"},{"key":"e_1_2_1_55_1","unstructured":"Jing Su Chufeng Jiang Xin Jin Yuxin Qiao Tingsong Xiao Hongda Ma Rong Wei Zhi Jing Jiajun Xu and Junhong Lin. 2024. Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review. (2024). arXiv:cs.LG\/2402.10350 https:\/\/arxiv.org\/abs\/2402.10350"},{"key":"e_1_2_1_56_1","volume-title":"Approaches to biology teaching and learning: understanding the wrong answers---teaching toward conceptual change. Cell biology education 4, 2","author":"Tanner Kimberly","year":"2005","unstructured":"Kimberly Tanner and Deborah Allen. 2005. Approaches to biology teaching and learning: understanding the wrong answers---teaching toward conceptual change. Cell biology education 4, 2 (2005), 112--117."},{"key":"e_1_2_1_57_1","volume-title":"Youcheng Sun, and Lucas C. Cordeiro.","author":"Tihanyi Norbert","year":"2024","unstructured":"Norbert Tihanyi, Ridhi Jain, Yiannis Charalambous, Mohamed Amine Ferrag, Youcheng Sun, and Lucas C. Cordeiro. 2024. A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification. (2024). arXiv:cs.SE\/2305.14752 https:\/\/arxiv.org\/abs\/2305.14752"},{"key":"e_1_2_1_58_1","first-page":"67","article-title":"Classification of middle school students' mistakes: mistake types","volume":"46","author":"T\u00fcrkdo\u011fan Ali","year":"2013","unstructured":"Ali T\u00fcrkdo\u011fan and Adnan Baki. 2013. Classification of middle school students' mistakes: mistake types. Ankara University Journal of Faculty of Educational Sciences (JFES) 46, 1 (2013), 67--88.","journal-title":"Ankara University Journal of Faculty of Educational Sciences (JFES)"},{"key":"e_1_2_1_59_1","volume-title":"Computer Networking. (May","author":"Vaccaro Kevin","year":"2024","unstructured":"Kevin Vaccaro. 2024. Computer Networking. (May 2024). https:\/\/www.coursera.org\/learn\/illinois-tech-computer-networking"},{"key":"e_1_2_1_60_1","first-page":"324","article-title":"Types of student errors in mathematical symbols, graphs and problem-solving","volume":"11","author":"Veloo AKHN","year":"2015","unstructured":"AKHN Veloo, Hariharan N Krishnasamy, and Wan Shahida Wan Abdullah. 2015. Types of student errors in mathematical symbols, graphs and problem-solving. Asian Social Science 11, 15 (2015), 324--334.","journal-title":"Asian Social Science"},{"key":"e_1_2_1_61_1","unstructured":"Cunxiang Wang Sirui Cheng Qipeng Guo Yuanhao Yue Bowen Ding Zhikun Xu Yidong Wang Xiangkun Hu Zheng Zhang and Yue Zhang. 2023. Evaluating Open-QA Evaluation. (2023). arXiv:cs.CL\/2305.12421 https:\/\/arxiv.org\/abs\/2305.12421"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3656296"},{"key":"e_1_2_1_63_1","volume-title":"Wang and Dorottya Demszky","author":"Rose","year":"2023","unstructured":"Rose E. Wang and Dorottya Demszky. 2023. Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction. (2023). arXiv:cs.CL\/2306.03090 https:\/\/arxiv.org\/abs\/2306.03090"},{"key":"e_1_2_1_64_1","volume-title":"Aakanksha Chowdhery, and Denny Zhou.","author":"Wang Xuezhi","year":"2023","unstructured":"Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. (2023). arXiv:cs.CL\/2203.11171 https:\/\/arxiv.org\/abs\/2203.11171"},{"key":"e_1_2_1_65_1","volume-title":"Chi, Quoc Le, and Denny Zhou","author":"Wei Jason","year":"2023","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. (2023). arXiv:cs.CL\/2201.11903 https:\/\/arxiv.org\/abs\/2201.11903"},{"key":"e_1_2_1_66_1","volume-title":"CMATH: Can Your Language Model Pass Chinese Elementary School Math Test?","author":"Wei Tianwen","year":"2023","unstructured":"Tianwen Wei, Jian Luan, Wei Liu, Shuang Dong, and Bin Wang. 2023. CMATH: Can Your Language Model Pass Chinese Elementary School Math Test? (2023). arXiv:cs.CL\/2306.16636 https:\/\/arxiv.org\/abs\/2306.16636"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3651890.3672268"},{"key":"e_1_2_1_68_1","volume-title":"LIMA: Less Is More for Alignment.","author":"Zhou Chunting","year":"2023","unstructured":"Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. LIMA: Less Is More for Alignment. (2023). arXiv:cs.CL\/2305.11206 https:\/\/arxiv.org\/abs\/2305.11206"},{"key":"e_1_2_1_69_1","unstructured":"Xiaobo Zhou. 2024. TCP\/IP and Advanced Topics. (May 2024). https:\/\/www.coursera.org\/learn\/tcp-ip-advanced"},{"key":"e_1_2_1_70_1","unstructured":"Yan Zhuang Qi Liu Yuting Ning Weizhe Huang Zachary A. Pardos Patrick C. Kyllonen Jiyun Zu Qingyang Mao Rui Lv Zhenya Huang Guanhao Zhao Zheng Zhang Shijin Wang and Enhong Chen. 2024. From Static Benchmarks to Adaptive Testing: Psychometrics in AI Evaluation. (2024). arXiv:cs.CL\/2306.10512 https:\/\/arxiv.org\/abs\/2306.10512"}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3717512.3717515","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3717512.3717515","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:15Z","timestamp":1750295835000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3717512.3717515"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":68,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,10,30]]}},"alternative-id":["10.1145\/3717512.3717515"],"URL":"https:\/\/doi.org\/10.1145\/3717512.3717515","relation":{},"ISSN":["0146-4833"],"issn-type":[{"value":"0146-4833","type":"print"}],"subject":[],"published":{"date-parts":[[2024,10,30]]},"assertion":[{"value":"2025-02-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}