{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T15:30:30Z","timestamp":1777390230358,"version":"3.51.4"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"CoNEXT2","license":[{"start":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T00:00:00Z","timestamp":1718236800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006374","name":"European Research Council","doi-asserted-by":"publisher","award":["770889"],"award-info":[{"award-number":["770889"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004359","name":"Swedish Research Council","doi-asserted-by":"crossref","award":["2021-04212"],"award-info":[{"award-number":["2021-04212"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100030798","name":"KTH Digital Futures","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100030798","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001858","name":"Vinnova","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Netw."],"published-print":{"date-parts":[[2024,6,13]]},"abstract":"<jats:p>This paper explores opportunities to utilize Large Language Models (LLMs) to make network configuration human-friendly, simplifying the configuration of network devices &amp; development of routing algorithms and minimizing errors. We design a set of benchmarks (NetConfEval) to examine the effectiveness of different models in facilitating and automating network configuration. More specifically, we focus on the scenarios where LLMs translate high-level policies, requirements, and descriptions (i.e., specified in natural language) into low-level network configurations &amp; Python code. NetConfEval considers four tasks that could potentially facilitate network configuration, such as (i) generating high-level requirements into a formal specification format, (ii) generating API\/function calls from high-level requirements, (iii) developing routing algorithms based on high-level descriptions, and (iv) generating low-level configuration for existing and new protocols based on input documentation. Learning from the results of our study, we propose a set of principles to design LLM-based systems to configure networks. Finally, we present two GPT-4-based prototypes to (i) automatically configure P4-enabled devices from a set of high-level requirements and (ii) integrate LLMs into existing network synthesizers.<\/jats:p>","DOI":"10.1145\/3656296","type":"journal-article","created":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T09:54:11Z","timestamp":1718272451000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":47,"title":["NetConfEval: Can LLMs Facilitate Network Configuration?"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-4604-1180","authenticated-orcid":false,"given":"Changjie","family":"Wang","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9780-873X","authenticated-orcid":false,"given":"Mariano","family":"Scazzariello","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5083-4052","authenticated-orcid":false,"given":"Alireza","family":"Farshin","sequence":"additional","affiliation":[{"name":"NVIDIA, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0722-2656","authenticated-orcid":false,"given":"Simone","family":"Ferlin","sequence":"additional","affiliation":[{"name":"Red Hat, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1256-1070","authenticated-orcid":false,"given":"Dejan","family":"Kosti\u0107","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9675-9729","authenticated-orcid":false,"given":"Marco","family":"Chiesa","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Stockholm, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2024,6,13]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Tiramisu: Fast Multilayer Network Verification. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Abhashkumar Anubhavnidhi","year":"2020","unstructured":"Anubhavnidhi Abhashkumar, Aaron Gember-Jacobson, and Aditya Akella. 2020. Tiramisu: Fast Multilayer Network Verification. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 201--219. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/abhashkumar"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544216.3544220"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2535838.2535862"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.001.2300313"},{"key":"e_1_2_1_5_1","unstructured":"Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin Alexandre Passos Siamak Shakeri Emanuel Taropa Paige Bailey Zhifeng Chen Eric Chu Jonathan H. Clark Laurent El Shafey Yanping Huang Kathy Meier-Hellstern Gaurav Mishra Erica Moreira Mark Omernick Kevin Robinson Sebastian Ruder Yi Tay Kefan Xiao Yuanzhong Xu Yujing Zhang Gustavo Hernandez Abrego Junwhan Ahn Jacob Austin Paul Barham Jan Botha James Bradbury Siddhartha Brahma Kevin Brooks Michele Catasta Yong Cheng Colin Cherry Christopher A. Choquette-Choo Aakanksha Chowdhery Cl\u00e9ment Crepy Shachi Dave Mostafa Dehghani Sunipa Dev Jacob Devlin Mark D\u00edaz Nan Du Ethan Dyer Vlad Feinberg Fangxiaoyu Feng Vlad Fienber Markus Freitag Xavier Garcia Sebastian Gehrmann Lucas Gonzalez Guy Gur-Ari Steven Hand Hadi Hashemi Le Hou Joshua Howland Andrea Hu Jeffrey Hui Jeremy Hurwitz Michael Isard Abe Ittycheriah Matthew Jagielski Wenhao Jia Kathleen Kenealy Maxim Krikun Sneha Kudugunta Chang Lan Katherine Lee Benjamin Lee Eric Li Music Li Wei Li YaGuang Li Jian Li Hyeontaek Lim Hanzhao Lin Zhongtao Liu Frederick Liu Marcello Maggioni Aroma Mahendru Joshua Maynez Vedant Misra Maysam Moussalem Zachary Nado John Nham Eric Ni Andrew Nystrom Alicia Parrish Marie Pellat Martin Polacek Alex Polozov Reiner Pope Siyuan Qiao Emily Reif Bryan Richter Parker Riley Alex Castro Ros Aurko Roy Brennan Saeta Rajkumar Samuel Renee Shelby Ambrose Slone Daniel Smilkov David R. So Daniel Sohn Simon Tokumine Dasha Valter Vijay Vasudevan Kiran Vodrahalli Xuezhi Wang Pidong Wang Zirui Wang Tao Wang John Wieting Yuhuai Wu Kelvin Xu Yunhan Xu Linting Xue Pengcheng Yin Jiahui Yu Qiao Zhang Steven Zheng Ce Zheng Weikang Zhou Denny Zhou Slav Petrov and Yonghui Wu. 2023. PaLM 2 Technical Report. arxiv: 2305.10403 [cs.CL]"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098834"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371110"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062367"},{"key":"e_1_2_1_9_1","volume-title":"Network Control Plane Synthesis and Verification. Ph.,D. Dissertation","author":"Beckett Ryan Andrew","unstructured":"Ryan Andrew Beckett. 2018. Network Control Plane Synthesis and Verification. Ph.,D. Dissertation. Princeton University. http:\/\/arks.princeton.edu\/ark:\/88435\/dsp01d217qs28v"},{"key":"e_1_2_1_10_1","volume-title":"Advances in Neural Information Processing Systems 35, , Sanmi Koyejo, Shakir Mohamed, Alekh Agarwal, Danielle Belgrave, Kyunghyun Cho, and Alice Oh (Eds.). Curran","author":"Beurer-Kellner Luca","year":"2022","unstructured":"Luca Beurer-Kellner, Martin Vechev, Laurent Vanbever, and Petar Velickovic. 2022. Learning to Configure Computer Networks with Neural Algorithmic Reasoning. In Advances in Neural Information Processing Systems 35, , Sanmi Koyejo, Shakir Mohamed, Alekh Agarwal, Danielle Belgrave, Kyunghyun Cho, and Alice Oh (Eds.). Curran, Red Hook, NY, 730 -- 742. 36th Annual Conference on Neural Information Processing Systems (NeurIPS 2022); Conference Location: New Orleans, LA, USA; Conference Date: November 28 - December 9, 2022; Poster presentation on December 1, 2022.."},{"key":"e_1_2_1_11_1","volume-title":"17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Birkner Rudiger","year":"2020","unstructured":"Rudiger Birkner, Dana Drachsler-Cohen, Laurent Vanbever, and Martin Vechev. 2020. Config2Spec: Mining Network Specifications from Network Configurations. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 969--984. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/birkner"},{"key":"e_1_2_1_12_1","volume-title":"Language Models are Few-Shot Learners. arxiv","author":"Brown Tom B.","year":"2005","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv: 2005.14165 [cs.CL]"},{"key":"e_1_2_1_13_1","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Josh Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arxiv: 2107.03374 [cs.LG]"},{"key":"e_1_2_1_14_1","unstructured":"Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra Adam Roberts Paul Barham Hyung Won Chung Charles Sutton Sebastian Gehrmann Parker Schuh Kensen Shi Sasha Tsvyashchenko Joshua Maynez Abhishek Rao Parker Barnes Yi Tay Noam Shazeer Vinodkumar Prabhakaran Emily Reif Nan Du Ben Hutchinson Reiner Pope James Bradbury Jacob Austin Michael Isard Guy Gur-Ari Pengcheng Yin Toju Duke Anselm Levskaya Sanjay Ghemawat Sunipa Dev Henryk Michalewski Xavier Garcia Vedant Misra Kevin Robinson Liam Fedus Denny Zhou Daphne Ippolito David Luan Hyeontaek Lim Barret Zoph Alexander Spiridonov Ryan Sepassi David Dohan Shivani Agrawal Mark Omernick Andrew M. Dai Thanumalayan Sankaranarayana Pillai Marie Pellat Aitor Lewkowycz Erica Moreira Rewon Child Oleksandr Polozov Katherine Lee Zongwei Zhou Xuezhi Wang Brennan Saeta Mark Diaz Orhan Firat Michele Catasta Jason Wei Kathy Meier-Hellstern Douglas Eck Jeff Dean Slav Petrov and Noah Fiedel. 2022. PaLM: Scaling Language Modeling with Pathways. arxiv: 2204.02311 [cs.CL]"},{"key":"e_1_2_1_15_1","volume-title":"Computer Aided Verification, , Rupak Majumdar and Viktor Kunvc ak (Eds.)","author":"El-Hassany Ahmed","unstructured":"Ahmed El-Hassany, Petar Tsankov, Laurent Vanbever, and Martin Vechev. 2017. Network-Wide Configuration Synthesis. In Computer Aided Verification, , Rupak Majumdar and Viktor Kunvc ak (Eds.). Springer International Publishing, Cham, 261--281."},{"key":"e_1_2_1_16_1","volume-title":"NetComplete: Practical Network-Wide Configuration Synthesis with Autocompletion. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18)","author":"El-Hassany Ahmed","year":"2018","unstructured":"Ahmed El-Hassany, Petar Tsankov, Laurent Vanbever, and Martin Vechev. 2018. NetComplete: Practical Network-Wide Configuration Synthesis with Autocompletion. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA, 579--594. https:\/\/www.usenix.org\/conference\/nsdi18\/presentation\/el-hassany"},{"key":"e_1_2_1_17_1","volume-title":"12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15)","author":"Fogel Ari","year":"2015","unstructured":"Ari Fogel, Stanley Fung, Luis Pedrosa, Meg Walraed-Sullivan, Ramesh Govindan, Ratul Mahajan, and Todd Millstein. 2015. A General Approach to Network Configuration Analysis. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). USENIX Association, Oakland, CA, 469--483. https:\/\/www.usenix.org\/conference\/nsdi15\/technical-sessions\/presentation\/fogel"},{"key":"e_1_2_1_18_1","unstructured":"kathara Framework. 2023. Kathara-Labs: Collection of kathara Network Scenarios. https:\/\/github.com\/KatharaFramework\/Kathara-Labs Accessed 2023--11--29."},{"key":"e_1_2_1_19_1","unstructured":"FRRouting. 2023. FRRouting. https:\/\/frrouting.org\/ Accessed 2023--11--29."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934872.2934876"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626111.3628176"},{"key":"e_1_2_1_22_1","unstructured":"Chase Harrison. 2022. LangChain. https:\/\/github.com\/langchain-ai\/langchain Accessed 2023--11--29."},{"key":"e_1_2_1_23_1","volume-title":"Ansible: Up and Running: Automating configuration management and deployment the easy way","author":"Hochstein Lorin","year":"2017","unstructured":"Lorin Hochstein and Rene Moser. 2017. Ansible: Up and Running: Automating configuration management and deployment the easy way. O'Reilly Media, Inc., Sebastopol, California."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2022.06.035"},{"key":"e_1_2_1_25_1","volume-title":"Contra: A Programmable System for Performance-aware Routing. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Hsu Kuo-Feng","year":"2020","unstructured":"Kuo-Feng Hsu, Ryan Beckett, Ang Chen, Jennifer Rexford, and David Walker. 2020. Contra: A Programmable System for Performance-aware Routing. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 701--721. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/hsu"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3229584.3229590"},{"key":"e_1_2_1_27_1","volume-title":"Tim Rockt\u00e4schel, Sebastian Riedel, and Douwe Kiela.","author":"Lewis Patrick","year":"2021","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen tau Yih, Tim Rockt\u00e4schel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arxiv: 2005.11401 [cs.CL]"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3230543.3230582"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626111.3628183"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626111.3628194"},{"key":"e_1_2_1_31_1","unstructured":"Networked Systems Group (NSG) - ETH Z\u00fcrich. 2017. GitHub - SyNET. https:\/\/github.com\/nsg-ethz\/synet Accessed 2023--12-02."},{"key":"e_1_2_1_32_1","unstructured":"Nokia. 2023. SR Linux and ChatGPT combine for network AIOps. https:\/\/www.nokia.com\/blog\/sr-linux-and-chatgpt-combine-for-network-aiops Accessed 2023--11--29."},{"key":"e_1_2_1_33_1","unstructured":"OpenAI. 2023. Function calling. https:\/\/platform.openai.com\/docs\/guides\/function-calling\/parallel-function-calling Accessed 2023--11--29."},{"key":"e_1_2_1_34_1","unstructured":"OpenAI. 2023. GPT-4 Technical Report. arxiv: 2303.08774 [cs.CL]"},{"key":"e_1_2_1_35_1","unstructured":"OpenAI. 2023. Models. https:\/\/platform.openai.com\/docs\/models\/gpt-4-and-gpt-4-turbo https:\/\/platform.openai.com\/docs\/models\/gpt-4-and-gpt-4-turbo Accessed 2023--11--29."},{"key":"e_1_2_1_36_1","first-page":"11","volume":"202","author":"Behavioral Model","unstructured":"p4lang. 2023. Behavioral Model (bmv2). https:\/\/github.com\/p4lang\/behavioral-model Accessed 2023--11--29.","journal-title":"Accessed"},{"key":"e_1_2_1_37_1","unstructured":"Russell A Poldrack Thomas Lu and Ga?per Begu?. 2023. AI-assisted coding: Experiments with GPT-4. arxiv: 2304.13187 [cs.AI]"},{"key":"e_1_2_1_38_1","volume-title":"17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Prabhu Santhosh","year":"2020","unstructured":"Santhosh Prabhu, Kuan Yen Chou, Ali Kheradmand, Brighten Godfrey, and Matthew Caesar. 2020. Plankton: Scalable network configuration verification through model checking. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 953--967. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/prabhu"},{"key":"e_1_2_1_39_1","volume-title":"RIFT: Routing in Fat Trees. Internet-Draft draft-ietf-rift-rift-21. Internet Engineering Task Force. https:\/\/datatracker.ietf.org\/doc\/draft-ietf-rift-rift\/21\/ Work in Progress.","author":"Przygienda Tony","year":"2023","unstructured":"Tony Przygienda, Alankar Sharma, Pascal Thubert, Bruno Rijsman, Dmitry Afanasiev, and Jordan Head. 2023. RIFT: Routing in Fat Trees. Internet-Draft draft-ietf-rift-rift-21. Internet Engineering Task Force. https:\/\/datatracker.ietf.org\/doc\/draft-ietf-rift-rift\/21\/ Work in Progress."},{"key":"e_1_2_1_40_1","volume-title":"Practical Intent-driven Routing Configuration Synthesis. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23)","author":"Ramanathan Sivaramakrishnan","year":"2023","unstructured":"Sivaramakrishnan Ramanathan, Ying Zhang, Mohab Gawish, Yogesh Mundada, Zhaodong Wang, Sangki Yun, Eric Lippert, Walid Taha, Minlan Yu, and Jelena Mirkovic. 2023. Practical Intent-driven Routing Configuration Synthesis. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). USENIX Association, Boston, MA, 629--644. https:\/\/www.usenix.org\/conference\/nsdi23\/presentation\/ramanathan"},{"key":"e_1_2_1_41_1","unstructured":"Baptiste Rozi\u00e8re Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Tal Remez J\u00e9r\u00e9my Rapin Artyom Kozhevnikov Ivan Evtimov Joanna Bitton Manish Bhatt Cristian Canton Ferrer Aaron Grattafiori Wenhan Xiong Alexandre D\u00e9fossez Jade Copet Faisal Azhar Hugo Touvron Louis Martin Nicolas Usunier Thomas Scialom and Gabriel Synnaeve. 2023. Code Llama: Open Foundation Models for Code. arxiv: 2308.12950 [cs.CL]"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/NOMS47738.2020.9110351"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626111.3628205"},{"key":"e_1_2_1_44_1","unstructured":"Kumar Shridhar Koustuv Sinha Andrew Cohen Tianlu Wang Ping Yu Ram Pasunuru Mrinmaya Sachan Jason Weston and Asli Celikyilmaz. 2023. The ART of LLM Refinement: Ask Refine and Trust. arxiv: 2311.07961 [cs.CL]"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3009837.3009845"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341302.3342088"},{"key":"e_1_2_1_47_1","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave and Guillaume Lample. 2023 a. LLaMA: Open and Efficient Foundation Language Models. arxiv: 2302.13971 [cs.CL]"},{"key":"e_1_2_1_48_1","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023 b. Llama 2: Open Foundation and Fine-Tuned Chat Models. arxiv: 2307.09288 [cs.CL]"},{"key":"e_1_2_1_49_1","unstructured":"Changjie Wang Mariano Scazzariello Alireza Farshin Simone Ferlin Dejan Kosti\u0107 and Marco Chiesa. 2024 a. benchmarkname GitHub Repository. https:\/\/github.com\/NetConfEval\/NetConfEval Accessed 2024-04--23."},{"key":"e_1_2_1_50_1","unstructured":"Changjie Wang Mariano Scazzariello Alireza Farshin Simone Ferlin Dejan Kosti\u0107 and Marco Chiesa. 2024 b. benchmarkname HuggingFace Repository. https:\/\/huggingface.co\/datasets\/NetConfEval\/NetConfEval Accessed 2024-04--23."},{"key":"e_1_2_1_51_1","unstructured":"Changjie Wang Mariano Scazzariello Alireza Farshin Dejan Kostic and Marco Chiesa. 2023. Making Network Configuration Human Friendly. arxiv: 2309.06342 [cs.NI]"},{"key":"e_1_2_1_52_1","unstructured":"Will Knight. 2023. OpenAI's CEO Says the Age of Giant AI Models Is Already Over. https:\/\/www.wired.com\/story\/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over\/ Accessed 2023--12-05."},{"key":"e_1_2_1_53_1","unstructured":"Shirley Wu. 2023. Move Naturally and Rule the Network with ChatGPT. https:\/\/blogs.juniper.net\/en-us\/enterprise-cloud-and-transformation\/move-naturally-and-rule-the-network-with-chatgpt Accessed 2023--11--29."},{"key":"e_1_2_1_54_1","unstructured":"Shijie Xia Xuefeng Li Yixin Liu Tongshuang Wu and Pengfei Liu. 2024. Evaluating Mathematical Reasoning Beyond Accuracy. arxiv: 2404.05692 [cs.CL]"},{"key":"e_1_2_1_55_1","unstructured":"Yu Zhou Xingyu Wu Beicheng Huang Jibin Wu Liang Feng and Kay Chen Tan. 2024. CausalBench: A Comprehensive Benchmark for Causal Learning Capability of Large Language Models. arxiv: 2404.06349 [cs.LG]"}],"container-title":["Proceedings of the ACM on Networking"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656296","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3656296","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T20:13:03Z","timestamp":1755979983000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,13]]},"references-count":55,"journal-issue":{"issue":"CoNEXT2","published-print":{"date-parts":[[2024,6,13]]}},"alternative-id":["10.1145\/3656296"],"URL":"https:\/\/doi.org\/10.1145\/3656296","relation":{},"ISSN":["2834-5509"],"issn-type":[{"value":"2834-5509","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,13]]},"assertion":[{"value":"2024-06-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}