{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T14:04:24Z","timestamp":1768485864961,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":42,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T00:00:00Z","timestamp":1691107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["CIF-1937357"],"award-info":[{"award-number":["CIF-1937357"]}]},{"name":"IARPA","award":["W911NF20C0035"],"award-info":[{"award-number":["W911NF20C0035"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,6]]},"DOI":"10.1145\/3580305.3599518","type":"proceedings-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T18:10:58Z","timestamp":1691172658000},"page":"3011-3021","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Test Accuracy vs. Generalization Gap: Model Selection in NLP without Accessing Training or Testing Data"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9908-5531","authenticated-orcid":false,"given":"Yaoqing","family":"Yang","sequence":"first","affiliation":[{"name":"Dartmouth College, Hanover, NH, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7542-5921","authenticated-orcid":false,"given":"Ryan","family":"Theisen","sequence":"additional","affiliation":[{"name":"University of California Berkeley, Berkeley, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4595-0347","authenticated-orcid":false,"given":"Liam","family":"Hodgkinson","sequence":"additional","affiliation":[{"name":"University of Melbourne, Melbourne, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2921-956X","authenticated-orcid":false,"given":"Joseph E.","family":"Gonzalez","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4567-328X","authenticated-orcid":false,"given":"Kannan","family":"Ramchandran","sequence":"additional","affiliation":[{"name":"University of California Berkeley, Berkeley, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5320-9972","authenticated-orcid":false,"given":"Charles H.","family":"Martin","sequence":"additional","affiliation":[{"name":"Calculation Consulting, San Francisco, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7920-4652","authenticated-orcid":false,"given":"Michael W.","family":"Mahoney","sequence":"additional","affiliation":[{"name":"University of California Berkeley, Berkeley, CA, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,8,4]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0085777"},{"key":"e_1_3_2_2_2_1","volume-title":"International Conference on Artificial Intelligence and Statistics. 2938--2948","author":"Bagdasaryan Eugene","year":"2020","unstructured":"Eugene Bagdasaryan , Andreas Veit , Yiqing Hua , Deborah Estrin , and Vitaly Shmatikov . 2020 . How to backdoor federated learning . In International Conference on Artificial Intelligence and Statistics. 2938--2948 . Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. 2020. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics. 2938--2948."},{"key":"e_1_3_2_2_3_1","first-page":"6241","article-title":"Spectrally-normalized margin bounds for neural networks","volume":"30","author":"Bartlett Peter","year":"2017","unstructured":"Peter Bartlett , Dylan Foster , and Matus Telgarsky . 2017 . Spectrally-normalized margin bounds for neural networks . Advances in Neural Information Processing Systems , Vol. 30 (2017), 6241 -- 6250 . Peter Bartlett, Dylan Foster, and Matus Telgarsky. 2017. Spectrally-normalized margin bounds for neural networks. Advances in Neural Information Processing Systems, Vol. 30 (2017), 6241--6250.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_4_1","volume-title":"Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine translation. 12--58","author":"Bojar Ondvr","year":"2014","unstructured":"Ondvr ej Bojar , Christian Buck , Christian Federmann , Barry Haddow , Philipp Koehn , Johannes Leveling , Christof Monz , Pavel Pecina , Matt Post , Herve Saint-Amand , 2014 . Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine translation. 12--58 . Ondvr ej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, et al. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine translation. 12--58."},{"key":"e_1_3_2_2_5_1","volume-title":"Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective. In International Conference on Learning Representations.","author":"Chen Wuyang","year":"2020","unstructured":"Wuyang Chen , Xinyu Gong , and Zhangyang Wang . 2020 . Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective. In International Conference on Learning Representations. Wuyang Chen, Xinyu Gong, and Zhangyang Wang. 2020. Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_6_1","volume-title":"Cosma Rohilla Shalizi, and Mark EJ Newman","author":"Clauset Aaron","year":"2009","unstructured":"Aaron Clauset , Cosma Rohilla Shalizi, and Mark EJ Newman . 2009 . Power-law distributions in empirical data. SIAM review, Vol. 51 , 4 (2009), 661--703. Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. 2009. Power-law distributions in empirical data. SIAM review, Vol. 51, 4 (2009), 661--703."},{"key":"e_1_3_2_2_7_1","volume-title":"Funnel-transformer: Filtering out sequential redundancy for efficient language processing. Advances in neural information processing systems","author":"Dai Zihang","year":"2020","unstructured":"Zihang Dai , Guokun Lai , Yiming Yang , and Quoc Le . 2020 . Funnel-transformer: Filtering out sequential redundancy for efficient language processing. Advances in neural information processing systems , Vol. 33 (2020), 4271--4282. Zihang Dai, Guokun Lai, Yiming Yang, and Quoc Le. 2020. Funnel-transformer: Filtering out sequential redundancy for efficient language processing. Advances in neural information processing systems, Vol. 33 (2020), 4271--4282."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00038"},{"key":"e_1_3_2_2_9_1","volume-title":"Advances in Neural Information Processing Systems","volume":"33","author":"Dziugaite Gintare Karolina","year":"2020","unstructured":"Gintare Karolina Dziugaite , Alexandre Drouin , Brady Neal , Nitarshan Rajkumar , Ethan Caballero , Linbo Wang , Ioannis Mitliagkas , and Daniel M Roy . 2020 . In search of robust measures of generalization . Advances in Neural Information Processing Systems , Vol. 33 (2020). Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Caballero, Linbo Wang, Ioannis Mitliagkas, and Daniel M Roy. 2020. In search of robust measures of generalization. Advances in Neural Information Processing Systems, Vol. 33 (2020)."},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1045"},{"key":"e_1_3_2_2_11_1","volume-title":"Sharpness-aware Minimization for Efficiently Improving Generalization. In International Conference on Learning Representations.","author":"Foret Pierre","year":"2020","unstructured":"Pierre Foret , Ariel Kleiner , Hossein Mobahi , and Behnam Neyshabur . 2020 . Sharpness-aware Minimization for Efficiently Improving Generalization. In International Conference on Learning Representations. Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. 2020. Sharpness-aware Minimization for Efficiently Improving Generalization. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_13_1","volume-title":"Conference on Neural Information Processing Systems. 8803--8812","author":"Garipov Timur","year":"2018","unstructured":"Timur Garipov , Pavel Izmailov , Dmitrii Podoprikhin , Dmitry Vetrov , and Andrew Gordon Wilson . 2018 . Loss surfaces, mode connectivity, and fast ensembling of DNNs . In Conference on Neural Information Processing Systems. 8803--8812 . Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, and Andrew Gordon Wilson. 2018. Loss surfaces, mode connectivity, and fast ensembling of DNNs. In Conference on Neural Information Processing Systems. 8803--8812."},{"key":"e_1_3_2_2_14_1","volume-title":"Conference on Uncertainty in Artificial Intelligence. 876--885","author":"Izmailov P","year":"2018","unstructured":"P Izmailov , AG Wilson , D Podoprikhin , D Vetrov , and T Garipov . 2018 . Averaging weights leads to wider optima and better generalization . In Conference on Uncertainty in Artificial Intelligence. 876--885 . P Izmailov, AG Wilson, D Podoprikhin, D Vetrov, and T Garipov. 2018. Averaging weights leads to wider optima and better generalization. In Conference on Uncertainty in Artificial Intelligence. 876--885."},{"key":"e_1_3_2_2_15_1","volume-title":"Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur.","author":"Jiang Yiding","year":"2020","unstructured":"Yiding Jiang , Pierre Foret , Scott Yak , Daniel M Roy , Hossein Mobahi , Gintare Karolina Dziugaite , Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur. 2020 . Neurips 2020 competition: Predicting generalization in deep learning. arXiv preprint arXiv:2012.07976 (2020). Yiding Jiang, Pierre Foret, Scott Yak, Daniel M Roy, Hossein Mobahi, Gintare Karolina Dziugaite, Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur. 2020. Neurips 2020 competition: Predicting generalization in deep learning. arXiv preprint arXiv:2012.07976 (2020)."},{"key":"e_1_3_2_2_16_1","volume-title":"Predicting the Generalization Gap in Deep Networks with Margin Distributions. In International Conference on Learning Representations.","author":"Jiang Yiding","year":"2018","unstructured":"Yiding Jiang , Dilip Krishnan , Hossein Mobahi , and Samy Bengio . 2018 . Predicting the Generalization Gap in Deep Networks with Margin Distributions. In International Conference on Learning Representations. Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. 2018. Predicting the Generalization Gap in Deep Networks with Margin Distributions. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_17_1","volume-title":"International Conference on Learning Representations.","author":"Jiang Yiding","year":"2019","unstructured":"Yiding Jiang , Behnam Neyshabur , Hossein Mobahi , Dilip Krishnan , and Samy Bengio . 2019 . Fantastic Generalization Measures and Where to Find Them . In International Conference on Learning Representations. Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. 2019. Fantastic Generalization Measures and Where to Find Them. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_18_1","volume-title":"Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171--4186","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171--4186 . Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171--4186."},{"key":"e_1_3_2_2_19_1","volume-title":"International Conference on Learning Representations.","author":"Lan Zhenzhong","year":"2019","unstructured":"Zhenzhong Lan , Mingda Chen , Sebastian Goodman , Kevin Gimpel , Piyush Sharma , and Radu Soricut . 2019 . ALBERT: A lite BERT for self-supervised learning of language representations . In International Conference on Learning Representations. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A lite BERT for self-supervised learning of language representations. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference. 2479--2490","author":"Le Hang","year":"2020","unstructured":"Hang Le , Lo\u00efc Vial , Jibril Frej , Vincent Segonne , Maximin Coavoux , Benjamin Lecouteux , Alexandre Allauzen , Benoit Crabb\u00e9 , Laurent Besacier , and Didier Schwab . 2020 . FlauBERT: Unsupervised Language Model Pre-training for French . In Proceedings of the 12th Language Resources and Evaluation Conference. 2479--2490 . Hang Le, Lo\u00efc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoit Crabb\u00e9, Laurent Besacier, and Didier Schwab. 2020. FlauBERT: Unsupervised Language Model Pre-training for French. In Proceedings of the 12th Language Resources and Evaluation Conference. 2479--2490."},{"key":"e_1_3_2_2_21_1","volume-title":"Roberta: A robustly optimized BERT pretraining approach. Technical Report Preprint: arXiv:1907.11692.","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019 . Roberta: A robustly optimized BERT pretraining approach. Technical Report Preprint: arXiv:1907.11692. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. Technical Report Preprint: arXiv:1907.11692."},{"key":"e_1_3_2_2_23_1","volume-title":"International Conference on Machine Learning. 4284--4293","author":"Martin Charles H","year":"2019","unstructured":"Charles H Martin and Michael W Mahoney . 2019 . Traditional and heavy tailed self regularization in neural network models . In International Conference on Machine Learning. 4284--4293 . Charles H Martin and Michael W Mahoney. 2019. Traditional and heavy tailed self regularization in neural network models. In International Conference on Machine Learning. 4284--4293."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611976236.57"},{"key":"e_1_3_2_2_25_1","first-page":"1","article-title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","volume":"22","author":"Martin Charles H","year":"2021","unstructured":"Charles H Martin and Michael W Mahoney . 2021 a. Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning . Journal of Machine Learning Research , Vol. 22 , 165 (2021), 1 -- 73 . Charles H Martin and Michael W Mahoney. 2021a. Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning. Journal of Machine Learning Research, Vol. 22, 165 (2021), 1--73.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-24025-8"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/307400.307435"},{"key":"e_1_3_2_2_30_1","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Nagarajan Vaishnavh","year":"2019","unstructured":"Vaishnavh Nagarajan and J Zico Kolter . 2019 . Uniform convergence may be unable to explain generalization in deep learning . Advances in Neural Information Processing Systems , Vol. 32 (2019). Vaishnavh Nagarajan and J Zico Kolter. 2019. Uniform convergence may be unable to explain generalization in deep learning. Advances in Neural Information Processing Systems, Vol. 32 (2019)."},{"key":"e_1_3_2_2_31_1","volume-title":"Deep Double Descent: Where Bigger Models and More Data Hurt. In International Conference on Learning Representations.","author":"Nakkiran Preetum","year":"2019","unstructured":"Preetum Nakkiran , Gal Kaplun , Yamini Bansal , Tristan Yang , Boaz Barak , and Ilya Sutskever . 2019 . Deep Double Descent: Where Bigger Models and More Data Hurt. In International Conference on Learning Representations. Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. 2019. Deep Double Descent: Where Bigger Models and More Data Hurt. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_32_1","first-page":"5947","article-title":"Exploring Generalization in Deep Learning","volume":"30","author":"Neyshabur Behnam","year":"2017","unstructured":"Behnam Neyshabur , Srinadh Bhojanapalli , David Mcallester , and Nati Srebro . 2017 . Exploring Generalization in Deep Learning . Advances in Neural Information Processing Systems , Vol. 30 (2017), 5947 -- 5956 . Behnam Neyshabur, Srinadh Bhojanapalli, David Mcallester, and Nati Srebro. 2017. Exploring Generalization in Deep Learning. Advances in Neural Information Processing Systems, Vol. 30 (2017), 5947--5956.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_33_1","volume-title":"International Conference on Learning Representations.","author":"Neyshabur Behnam","year":"2018","unstructured":"Behnam Neyshabur , Srinadh Bhojanapalli , and Nathan Srebro . 2018 . A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks . In International Conference on Learning Representations. Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. 2018. A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_34_1","volume-title":"Conference on Learning Theory. PMLR, 1376--1401","author":"Neyshabur Behnam","year":"2015","unstructured":"Behnam Neyshabur , Ryota Tomioka , and Nathan Srebro . 2015 . Norm-based capacity control in neural networks . In Conference on Learning Theory. PMLR, 1376--1401 . Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. 2015. Norm-based capacity control in neural networks. In Conference on Learning Theory. PMLR, 1376--1401."},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-6301"},{"key":"e_1_3_2_2_36_1","volume-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311--318","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni , Salim Roukos , Todd Ward , and Wei-Jing Zhu . 2002 . BLEU: a method for automatic evaluation of machine translation . In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311--318 . Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311--318."},{"key":"e_1_3_2_2_38_1","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever etal 2019. Language models are unsupervised multitask learners. OpenAI blog Vol. 1 8 (2019) 9.  Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI blog Vol. 1 8 (2019) 9."},{"key":"e_1_3_2_2_39_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , Peter J Liu , 2020 . Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Learning Research , Vol. 21 , 140 (2020), 1 -- 67 . Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, Vol. 21, 140 (2020), 1--67.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_2_43_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1101"},{"key":"e_1_3_2_2_45_1","volume-title":"Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.","author":"Wolf Thomas","year":"2020","unstructured":"Thomas Wolf , Lysandre Debut , Victor Sanh , Julien Chaumond , Clement Delangue , Anthony Moi , Pierric Cistac , Tim Rault , R\u00e9mi Louf , Morgan Funtowicz , Joe Davison , Sam Shleifer , Patrick von Platen , Clara Ma , Yacine Jernite , Julien Plu , Canwen Xu , Teven Le Scao , Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 . Transformers : State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics , 38--45. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R\u00e9mi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics, 38--45."},{"key":"e_1_3_2_2_46_1","volume-title":"Thirty-Fifth Conference on Neural Information Processing Systems.","author":"Yang Yaoqing","year":"2021","unstructured":"Yaoqing Yang , Liam Hodgkinson , Ryan Theisen , Joe Zou , Joseph E Gonzalez , Kannan Ramchandran , and Michael W Mahoney . 2021 . Taxonomizing local versus global structure in neural network loss landscapes . In Thirty-Fifth Conference on Neural Information Processing Systems. Yaoqing Yang, Liam Hodgkinson, Ryan Theisen, Joe Zou, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. 2021. Taxonomizing local versus global structure in neural network loss landscapes. In Thirty-Fifth Conference on Neural Information Processing Systems."},{"key":"e_1_3_2_2_47_1","volume-title":"Advances in Neural Information Processing Systems","volume":"33","author":"Yang Yaoqing","year":"2020","unstructured":"Yaoqing Yang , Rajiv Khanna , Yaodong Yu , Amir Gholami , Kurt Keutzer , Joseph E Gonzalez , Kannan Ramchandran , and Michael W Mahoney . 2020 . Boundary thickness and robustness in learning models . Advances in Neural Information Processing Systems , Vol. 33 (2020). Yaoqing Yang, Rajiv Khanna, Yaodong Yu, Amir Gholami, Kurt Keutzer, Joseph E Gonzalez, Kannan Ramchandran, and Michael W Mahoney. 2020. Boundary thickness and robustness in learning models. Advances in Neural Information Processing Systems, Vol. 33 (2020)."},{"key":"e_1_3_2_2_49_1","first-page":"27263","article-title":"Bartscore: Evaluating generated text as text generation","volume":"34","author":"Yuan Weizhe","year":"2021","unstructured":"Weizhe Yuan , Graham Neubig , and Pengfei Liu . 2021 . Bartscore: Evaluating generated text as text generation . Advances in Neural Information Processing Systems , Vol. 34 (2021), 27263 -- 27277 . Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. Advances in Neural Information Processing Systems, Vol. 34 (2021), 27263--27277.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_50_1","volume-title":"BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations.","author":"Zhang Tianyi","year":"2019","unstructured":"Tianyi Zhang , Varsha Kishore , Felix Wu , Kilian Q Weinberger , and Yoav Artzi . 2019 . BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_51_1","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 270--278","author":"Zhang Yizhe","year":"2020","unstructured":"Yizhe Zhang , Siqi Sun , Michel Galley , Yen-Chun Chen , Chris Brockett , Xiang Gao , Jianfeng Gao , Jingjing Liu , and William B Dolan . 2020 . DIALOGPT: Large-Scale Generative Pre-training for Conversational Response Generation . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 270--278 . Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B Dolan. 2020. DIALOGPT: Large-Scale Generative Pre-training for Conversational Response Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 270--278."}],"event":{"name":"KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Long Beach CA USA","acronym":"KDD '23","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599518","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580305.3599518","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580305.3599518","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:52Z","timestamp":1750178272000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599518"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,4]]},"references-count":42,"alternative-id":["10.1145\/3580305.3599518","10.1145\/3580305"],"URL":"https:\/\/doi.org\/10.1145\/3580305.3599518","relation":{},"subject":[],"published":{"date-parts":[[2023,8,4]]},"assertion":[{"value":"2023-08-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}