{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,28]],"date-time":"2025-08-28T12:50:38Z","timestamp":1756385438750,"version":"3.41.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,1,30]],"date-time":"2025-01-30T00:00:00Z","timestamp":1738195200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCOMM Comput. Commun. Rev."],"published-print":{"date-parts":[[2025,1,30]]},"abstract":"<jats:p>The purpose of this editorial note is to raise awareness about a deeply concerning and yet much-overlooked development in the use of Artificial Intelligence (AI) and Machine Learning (ML) for solving problems in science in general and in networking in particular. To put it simply, in today's age of AI\/ML, the much-publicized and well-documented \"reproducibility crisis\" in science is further compounded by an inconspicuous and rarely mentioned \"credibility crisis.\" More to the point, by focusing on the area of networking research, we provide evidence that among the already small number of reproducible scientific publications that describe AI\/ML-based solutions, even fewer, and often none, describe trained AI\/ML models that are \"credible;\" that is, can be trusted to not only perform well in their original training domain but also in new and untested environments. We elaborate on the root cause of this credibility crisis, discuss why the credibility of AI\/ML models is of paramount importance for their successful use in practice, and put forward an aggressive but imminently practical proposal for addressing this crisis head-on so as to pave the way for a future where networking research can reap the full benefits of AI\/ML.<\/jats:p>","DOI":"10.1145\/3727063.3727067","type":"journal-article","created":{"date-parts":[[2025,4,10]],"date-time":"2025-04-10T14:56:51Z","timestamp":1744297011000},"page":"10-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["When Something Looks too Good to be True, it Usually is! AI is Causing a Credibility Crisis in Networking"],"prefix":"10.1145","volume":"55","author":[{"given":"Walter","family":"Willinger","sequence":"first","affiliation":[{"name":"NIKSUN, Inc., USA"}]},{"given":"Ronaldo A.","family":"Ferreira","sequence":"additional","affiliation":[{"name":"UFMS, Brazil"}]},{"given":"Arpit","family":"Gupta","sequence":"additional","affiliation":[{"name":"UCSB, USA"}]},{"given":"Roman (Sylee)","family":"Beltiukov","sequence":"additional","affiliation":[{"name":"UCSB, USA"}]},{"given":"Satyandra","family":"Guthula","sequence":"additional","affiliation":[{"name":"UCSB, USA"}]},{"given":"Lisandro Z.","family":"Granville","sequence":"additional","affiliation":[{"name":"UFRGS, Brazil"}]},{"given":"Arthur S.","family":"Jacobs","sequence":"additional","affiliation":[{"name":"UFRGS, Brazil"}]}],"member":"320","published-online":{"date-parts":[[2025,4,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314212.3314217"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1038\/533452a"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623075"},{"key":"e_1_2_1_4_1","unstructured":"L. Breiman J. H. Friedman R. A. Olshen and C. J. Stone. 1984. Classification and Regression Trees. Wadsworth and Brooks Monterey CA."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2812803"},{"key":"e_1_2_1_6_1","unstructured":"Jon Crowcroft. 2024. 10 000 maniacs and AI is destroying Computer Science one topic at a time ... http:\/\/paravirtualization.blogspot.com\/2024\/06\/10000-manacs-and-ai-is-destroying.html"},{"key":"e_1_2_1_7_1","first-page":"1","article-title":"Under-specification Presents Challenges for Credibility in Modern Machine Learning","volume":"23","author":"D'Amour Alexander","year":"2022","unstructured":"Alexander D'Amour, Katherine Heller, Dan Moldovan, and et al. 2022. Under-specification Presents Challenges for Credibility in Modern Machine Learning. Journal of Machine Learning Research 23, 226 (2022), 1--61. https:\/\/www.jmlr.org\/papers\/volume23\/20-1335\/20-1335.pdf","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_8_1","volume-title":"AI Will Become Mathematicians' 'Co-Pilot'. Scientific American (June","author":"Dr\u00f6sser Christoph","year":"2024","unstructured":"Christoph Dr\u00f6sser. 2024. AI Will Become Mathematicians' 'Co-Pilot'. Scientific American (June 2024). https:\/\/www.scientificamerican.com\/article\/ai-will-become-mathematicians-co-pilot\/"},{"key":"e_1_2_1_9_1","volume-title":"Proc. 7th ICML Workshop on Automated Machine Learning. PMLR, Virtual, 16 pages. https:\/\/www.automl.org\/wp-content\/uploads\/2020\/07\/AutoML_2020_paper_7.pdf","author":"Erickson Nick","year":"2020","unstructured":"Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. 2020. AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. In Proc. 7th ICML Workshop on Automated Machine Learning. PMLR, Virtual, 16 pages. https:\/\/www.automl.org\/wp-content\/uploads\/2020\/07\/AutoML_2020_paper_7.pdf"},{"key":"e_1_2_1_10_1","volume-title":"Aug. 24, 2020. ACM. https:\/\/www.acm.org\/publications\/policies\/artifact-review-and-badging-current","author":"Association for Computing Machinery.","year":"2020","unstructured":"Association for Computing Machinery. 2020. ACM Artifact Review and Badging Version 1.1, Aug. 24, 2020. ACM. https:\/\/www.acm.org\/publications\/policies\/artifact-review-and-badging-current"},{"key":"e_1_2_1_11_1","unstructured":"Canadian Institute for Cybersecurity. 2016. VPN-nonVPN dataset (ISCXVPN2016). CIC. https:\/\/www.unb.ca\/cic\/datasets\/vpn.html"},{"key":"e_1_2_1_12_1","unstructured":"Canadian Institute for Cybersecurity. 2017. Intrusion detection evaluation dataset (CIC-IDS2017). CIC. https:\/\/www.unb.ca\/cic\/datasets\/ids-2017.html"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243792"},{"key":"e_1_2_1_14_1","unstructured":"Satyandra Guthula Navya Battula Roman Beltiukov Wenbo Guo and Arpit Gupta. 2025. NetFound: Foundation Model for Network Security. arXiv:2310.17025v4 [cs.OS] https:\/\/arxiv.org\/pdf\/2310.17025v4"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460120.3484758"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3548606.3560609"},{"key":"e_1_2_1_17_1","volume-title":"Machine Learning is Contributing to a Reproducibility Crisis within Science . MIT Technology Review (February","author":"Jee Charlotte","year":"2019","unstructured":"Charlotte Jee. 2019. Machine Learning is Contributing to a Reproducibility Crisis within Science . MIT Technology Review (February 2019), 1 pages. https:\/\/www.technologyreview.com\/2019\/02\/18\/137357\/machine-learning-is-contributing-to-a-reproducibility-crisis-within-science"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512217"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098843"},{"key":"e_1_2_1_20_1","unstructured":"Yisroel Mirsky Tomer Doitshman Yuval Elovici and Asaf Shabtai. 2018. KitNET. https:\/\/github.com\/ymirsky\/KitNET-py"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.1802.09089"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623130"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3696348.3696894"},{"key":"e_1_2_1_24_1","first-page":"1","article-title":"Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)","volume":"22","author":"Pineau Joelle","year":"2021","unstructured":"Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivi\u00e8re, Alina Beygelzimer, Florence d'Alch\u00e9 Buc, Emily Fox, and Hugo Larochelle. 2021. Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program). Journal of Machine Learning 22, 164 (2021), 1--20. https:\/\/jmlr2020.csail.mit.edu\/papers\/volume22\/20-303\/20-303.pdf","journal-title":"Journal of Machine Learning"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-3020"},{"key":"e_1_2_1_26_1","unstructured":"Haakon Riiser et al. 2012. HSDPA-bandwidth Logs for Mobile HTTP Streaming Scenarios. UMASS. https:\/\/skulddata.cs.umass.edu\/traces\/mmsys\/2013\/pathbandwidth\/"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2483977.2483991"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1515\/9781400881970-018"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5220\/0006639801080116"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2018.2866249"},{"key":"e_1_2_1_32_1","volume-title":"Data collected for our paper at IEEE TMC","author":"Sydney UNSW","year":"2018","unstructured":"UNSW Sydney. 2018. Data collected for our paper at IEEE TMC 2018. UNSW. https:\/\/iotanalytics.unsw.edu.au\/iottraces.html"},{"key":"e_1_2_1_33_1","unstructured":"Terence Tao. 2023. Embracing change and resetting expectations. Microsoft. https:\/\/unlocked.microsoft.com\/ai-anthology\/terence-tao"},{"key":"e_1_2_1_34_1","unstructured":"Microsoft Unlocked. 2023. AI Anthology: A Collection of Essays on the Future of AI. Microsoft. https:\/\/unlocked.microsoft.com\/ai-anthology\/"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2020.24412"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISI.2017.8004872"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3365609.3365864"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","unstructured":"Ruijie Zhao Mingwei Zhan Xianwen Deng Yanhao Wang Yijun Wang Guan Gui and Zhi Xue. 2023. Yet Another Traffic Classifier: A Masked Autoencoder Based Traffic Transformer with Multi-level Flow Representation. In Proc. of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence (AAAI'23\/IAAI'23\/EAAI'23). AAAI Washington DC Article 605 8 pages. 10.1609\/aaai.v37i4.25674","DOI":"10.1609\/aaai.v37i4.25674"}],"container-title":["ACM SIGCOMM Computer Communication Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3727063.3727067","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3727063.3727067","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:39Z","timestamp":1750295919000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3727063.3727067"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,30]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1,30]]}},"alternative-id":["10.1145\/3727063.3727067"],"URL":"https:\/\/doi.org\/10.1145\/3727063.3727067","relation":{},"ISSN":["0146-4833"],"issn-type":[{"type":"print","value":"0146-4833"}],"subject":[],"published":{"date-parts":[[2025,1,30]]},"assertion":[{"value":"2025-04-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}