{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T02:11:05Z","timestamp":1778033465681,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":76,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,6,3]],"date-time":"2024-06-03T00:00:00Z","timestamp":1717372800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,6,3]]},"DOI":"10.1145\/3630106.3658966","type":"proceedings-article","created":{"date-parts":[[2024,6,5]],"date-time":"2024-06-05T13:14:21Z","timestamp":1717593261000},"page":"1199-1210","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7133-695X","authenticated-orcid":false,"given":"Robert","family":"Wolfe","sequence":"first","affiliation":[{"name":"University of Washington, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1911-2374","authenticated-orcid":false,"given":"Isaac","family":"Slaughter","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5280-9456","authenticated-orcid":false,"given":"Bin","family":"Han","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7980-9555","authenticated-orcid":false,"given":"Bingbing","family":"Wen","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0136-6465","authenticated-orcid":false,"given":"Yiwei","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6952-4361","authenticated-orcid":false,"given":"Lucas","family":"Rosenblatt","sequence":"additional","affiliation":[{"name":"New York University, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5453-4994","authenticated-orcid":false,"given":"Bernease","family":"Herman","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2564-0373","authenticated-orcid":false,"given":"Eva","family":"Brown","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5313-0076","authenticated-orcid":false,"given":"Zening","family":"Qu","sequence":"additional","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6008-3763","authenticated-orcid":false,"given":"Nic","family":"Weber","sequence":"additional","affiliation":[{"name":"University of Washington, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8588-8472","authenticated-orcid":false,"given":"Bill","family":"Howe","sequence":"additional","affiliation":[{"name":"University of Washington, United States"}]}],"member":"320","published-online":{"date-parts":[[2024,6,5]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978318"},{"key":"e_1_3_2_1_2_1","unstructured":"Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra Cojocaru M\u00e9rouane Debbah \u00c9tienne Goffinet Daniel Hesslow Julien Launay Quentin Malartic 2023. The Falcon Series of Open Language Models. arXiv preprint arXiv:2311.16867 (2023) ."},{"key":"e_1_3_2_1_3_1","unstructured":"Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen Nova DasSarma Dawn Drain Stanislav Fort Deep Ganguli Tom Henighan 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022) ."},{"key":"e_1_3_2_1_4_1","unstructured":"Jeff Beckman. 2023. OpenAI Statistics 2023: Growth Users and More. https:\/\/techreport.com\/statistics\/openai-statistics\/. [Accessed 19-01-2024]."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.eacl-main.168"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_3_2_1_7_1","unstructured":"Lukas Biewald. 2020. Experiment Tracking with Weights and Biases. https:\/\/www.wandb.com\/ Software available from wandb.com."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533083"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-4470-8_2"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Rishi Bommasani Kevin Klyman Shayne Longpre Sayash Kapoor Nestor Maslej Betty Xiong Daniel Zhang and Percy Liang. 2023. The foundation model transparency index. arXiv preprint arXiv:2310.12941 (2023) .","DOI":"10.1609\/aies.v7i1.31628"},{"key":"e_1_3_2_1_11_1","volume-title":"Language models are few-shot learners. Advances in neural information processing systems 33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared\u00a0D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877\u20131901."},{"key":"e_1_3_2_1_12_1","unstructured":"Yaroslav Bulatov. 2018. Fitting larger networks into memory.https:\/\/medium.com\/tensorflow\/fitting-larger-networks-into-memory-583e3c758ff9. [Accessed 19-01-2024]."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.261"},{"key":"e_1_3_2_1_14_1","unstructured":"Jerry Chee Yaohui Cai Volodymyr Kuleshov and Christopher De\u00a0Sa. 2023. Quip: 2-bit quantization of large language models with guarantees. arXiv preprint arXiv:2307.13304 (2023) ."},{"key":"e_1_3_2_1_15_1","unstructured":"Tianqi Chen Bing Xu Chiyuan Zhang and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016) ."},{"key":"e_1_3_2_1_16_1","unstructured":"Pierre Colombo Nathan Noiry Ekhine Irurozki and Stephan Clemencon. 2022. What are the best systems? New perspectives on NLP Benchmarking. arXiv preprint arXiv:2202.03799 (2022) ."},{"key":"e_1_3_2_1_17_1","unstructured":"Andrew\u00a0M Dai and Quoc\u00a0V Le. 2015. Semi-supervised sequence learning. Advances in neural information processing systems 28 (2015) ."},{"key":"e_1_3_2_1_18_1","unstructured":"Hai Dang Lukas Mecke Florian Lehmann Sven Goller and Daniel Buschek. 2022. How to prompt? Opportunities and challenges of zero-and few-shot learning for human-AI interaction in creative applications of generative models. arXiv preprint arXiv:2209.01390 (2022) ."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_20_1","unstructured":"Tim Dettmers Mike Lewis Younes Belkada and Luke Zettlemoyer. 2022. Llm. int8 (): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339 (2022) ."},{"key":"e_1_3_2_1_21_1","volume-title":"Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314","author":"Dettmers Tim","year":"2023","unstructured":"Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314, (2023),."},{"key":"e_1_3_2_1_22_1","volume-title":"International Conference on Machine Learning. PMLR,,, 7750\u20137774","author":"Dettmers Tim","year":"2023","unstructured":"Tim Dettmers and Luke Zettlemoyer. 2023. The case for 4-bit precision: k-bit inference scaling laws. In International Conference on Machine Learning. PMLR,,, 7750\u20137774."},{"key":"e_1_3_2_1_23_1","volume-title":"Climate-fever: A dataset for verification of real-world climate claims. arXiv preprint arXiv:2012.00614","author":"Diggelmann Thomas","year":"2020","unstructured":"Thomas Diggelmann, Jordan Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita, and Markus Leippold. 2020. Climate-fever: A dataset for verification of real-world climate claims. arXiv preprint arXiv:2012.00614, (2020),."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_2_1_25_1","unstructured":"Aos Fatos. 2024. Aos Fatos. https:\/\/www.aosfatos.org\/. [Accessed 22-01-2024]."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367564"},{"key":"e_1_3_2_1_27_1","volume-title":"Deep learning with label differential privacy. Advances in neural information processing systems 34","author":"Ghazi Badih","year":"2021","unstructured":"Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, and Chiyuan Zhang. 2021. Deep learning with label differential privacy. Advances in neural information processing systems 34 (2021), 27131\u201327145."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2305016120"},{"key":"e_1_3_2_1_29_1","volume-title":"A survey on differentially private machine learning","author":"Gong Maoguo","year":"2020","unstructured":"Maoguo Gong, Yu Xie, Ke Pan, Kaiyuan Feng, and Alex\u00a0Kai Qin. 2020. A survey on differentially private machine learning. IEEE computational intelligence magazine 15, 2 (2020), 49\u201364."},{"key":"e_1_3_2_1_30_1","unstructured":"Bin Han Haotian Zhu Sitong Zhou Sofia Ahmed M Rahman Fei Xia and Kevin Lybarger. 2023. Huskyscribe at mediqa-sum 2023: Summarizing clinical dialogues with transformers. In. CLEF ."},{"key":"e_1_3_2_1_31_1","unstructured":"Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Dawn Song and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. arxiv:2009.03300\u00a0[cs.CY]"},{"key":"e_1_3_2_1_32_1","volume-title":"Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685","author":"Hu J","year":"2021","unstructured":"Edward\u00a0J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, (2021),."},{"key":"e_1_3_2_1_33_1","volume-title":"Florian Bressand, Gianna Lengyel","author":"Jiang Q","year":"2023","unstructured":"Albert\u00a0Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra\u00a0Singh Chaplot, Diego de\u00a0las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, 2023. Mistral 7B. arXiv preprint arXiv:2310.06825, (2023),."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3390\/app11146421"},{"key":"e_1_3_2_1_35_1","volume-title":"Information Fusion, (2023","author":"Koco\u0144 Jan","year":"2023","unstructured":"Jan Koco\u0144, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szyd\u0142o, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, 2023. ChatGPT: Jack of all trades, master of none. Information Fusion, (2023), 101861."},{"key":"e_1_3_2_1_36_1","volume-title":"International Conference on Machine Learning. PMLR,,, 5637\u20135664","author":"Koh Pang\u00a0Wei","year":"2021","unstructured":"Pang\u00a0Wei Koh, Shiori Sagawa, Henrik Marklund, Sang\u00a0Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard\u00a0Lanas Phillips, Irena Gao, 2021. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning. PMLR,,, 5637\u20135664."},{"key":"e_1_3_2_1_37_1","unstructured":"Percy Liang Rishi Bommasani Tony Lee Dimitris Tsipras Dilara Soylu Michihiro Yasunaga Yian Zhang Deepak Narayanan Yuhuai Wu Ananya Kumar Benjamin Newman Binhang Yuan Bobby Yan Ce Zhang Christian Cosgrove Christopher\u00a0D. Manning Christopher R\u00e9 Diana Acosta-Navas Drew\u00a0A. Hudson Eric Zelikman Esin Durmus Faisal Ladhak Frieda Rong Hongyu Ren Huaxiu Yao Jue Wang Keshav Santhanam Laurel Orr Lucia Zheng Mert Yuksekgonul Mirac Suzgun Nathan Kim Neel Guha Niladri Chatterji Omar Khattab Peter Henderson Qian Huang Ryan Chi Sang\u00a0Michael Xie Shibani Santurkar Surya Ganguli Tatsunori Hashimoto Thomas Icard Tianyi Zhang Vishrav Chaudhary William Wang Xuechen Li Yifan Mai Yuhui Zhang and Yuta Koreeda. 2023. Holistic Evaluation of Language Models. arxiv:2211.09110\u00a0[cs.CL]"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3571884.3604316"},{"key":"e_1_3_2_1_39_1","unstructured":"Lefteris Loukas Ilias Stogiannidis Prodromos Malakasiotis and Stavros Vassos. 2023. Breaking the Bank with ChatGPT: Few-Shot Text Classification for Finance. In Proceedings of the Fifth Workshop on Financial Technology and Natural Language Processing and the Second Multimodal AI For Financial Forecasting Chung-Chi Chen Hiroya Takamura Puneet Mathur Remit Sawhney Hen-Hsen Huang and Hsin-Hsi Chen (Eds.). - Macao 74\u201380. https:\/\/aclanthology.org\/2023.finnlp-1.7"},{"key":"e_1_3_2_1_40_1","unstructured":"Meedan. 2024. Meedan. https:\/\/meedan.com\/. [Accessed 22-01-2024]."},{"key":"e_1_3_2_1_42_1","volume-title":"OpenAI Blog, (Nov","author":"AI.","year":"2022","unstructured":"OpenAI. 2022. Introducing ChatGPT. OpenAI Blog, (Nov 2022),."},{"key":"e_1_3_2_1_43_1","unstructured":"OpenAI. 2024. Models. https:\/\/platform.openai.com\/docs\/models\/. [Accessed 19-01-2024]."},{"key":"e_1_3_2_1_44_1","unstructured":"OpenAI. 2024. Pricing. https:\/\/openai.com\/pricing. [Accessed 19-01-2024]."},{"key":"e_1_3_2_1_45_1","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730\u201327744.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1038\/s43588-023-00585-1"},{"key":"e_1_3_2_1_47_1","volume-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics.,, 311\u2013318","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics.,, 311\u2013318."},{"key":"e_1_3_2_1_48_1","unstructured":"Dylan Patel and Gerald Wong. 2023. GPT-4 Architecture Infrastructure Training Dataset Costs Vision MoE. https:\/\/www.semianalysis.com\/p\/gpt-4-architecture-infrastructure. [Accessed 19-01-2024]."},{"key":"e_1_3_2_1_49_1","unstructured":"Guilherme Penedo Quentin Malartic Daniel Hesslow Ruxandra Cojocaru Alessandro Cappelli Hamza Alobeidli Baptiste Pannier Ebtesam Almazrouei and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data and web data only. arXiv preprint arXiv:2306.01116 (2023) ."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539597.3575792"},{"key":"e_1_3_2_1_51_1","volume-title":"The Verge, (Nov","author":"Porter Jon","year":"2023","unstructured":"Jon Porter. 2023. ChatGPT continues to be one of the fastest-growing services ever. The Verge, (Nov 2023),."},{"key":"e_1_3_2_1_52_1","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans Ilya Sutskever 2018. Improving language understanding by generative pre-training. (2018) ."},{"key":"e_1_3_2_1_53_1","volume-title":"Language models are unsupervised multitask learners. OpenAI blog 1, 8","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9."},{"key":"e_1_3_2_1_54_1","unstructured":"Rafael Rafailov Archit Sharma Eric Mitchell Stefano Ermon Christopher\u00a0D Manning and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290 (2023) ."},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iotcps.2023.04.003"},{"key":"e_1_3_2_1_56_1","unstructured":"Anna Rogers Niranjan Balasubramanian Leon Derczynski Jesse Dodge Alexander Koller Sasha Luccioni Maarten Sap Roy Schwartz Noah\u00a0A. Smith and Emma Strubell. 2023. Closed AI Models Make Bad Baselines. https:\/\/hackingsemantics.xyz\/2023\/closed-baselines\/"},{"key":"e_1_3_2_1_57_1","unstructured":"Victor Sanh Albert Webson Colin Raffel Stephen\u00a0H. Bach Lintang Sutawika Zaid Alyafeai Antoine Chaffin Arnaud Stiegler Teven Le\u00a0Scao Arun Raja Manan Dey M\u00a0Saiful Bari Canwen Xu Urmish Thakker Shanya Sharma Eliza Szczechla Taewoon Kim Gunjan Chhablani Nihal\u00a0V. Nayak Debajyoti Datta Jonathan Chang Mike Tian-Jian Jiang Han Wang Matteo Manica Sheng Shen Zheng-Xin Yong Harshit Pandey Michael McKenna Rachel Bawden Thomas Wang Trishala Neeraj Jos Rozen Abheesht Sharma Andrea Santilli Thibault Fevry Jason\u00a0Alan Fries Ryan Teehan Tali Bers Stella Biderman Leo Gao Thomas Wolf and Alexander\u00a0M. Rush. 2022. Multitask Prompted Training Enables Zero-Shot Task Generalization. In International Conference on Learning Representations. . https:\/\/research.ibm.com\/publications\/multitask-prompt-tuning-enables-zero-shot-task-generalization"},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.575"},{"key":"e_1_3_2_1_59_1","volume-title":"Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, (2023),."},{"key":"e_1_3_2_1_60_1","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023) ."},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"crossref","unstructured":"Tao Tu Shekoofeh Azizi Danny Driess Mike Schaekermann Mohamed Amin Pi-Chuan Chang Andrew Carroll Chuck Lau Ryutaro Tanno Ira Ktena 2023. Towards generalist biomedical AI. arXiv preprint arXiv:2307.14334 (2023) .","DOI":"10.1056\/AIoa2300138"},{"key":"e_1_3_2_1_62_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017) ."},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41433-023-02595-9"},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3324926"},{"key":"e_1_3_2_1_65_1","unstructured":"Zengzhi Wang Qiming Xie Zixiang Ding Yi Feng and Rui Xia. 2023. Is ChatGPT a good sentiment analyzer? A preliminary study. arXiv preprint arXiv:2304.04339 (2023) ."},{"key":"e_1_3_2_1_66_1","volume-title":"Climatebert: A pretrained language model for climate-related text. arXiv preprint arXiv:2110.12010","author":"Webersinke Nicolas","year":"2021","unstructured":"Nicolas Webersinke, Mathias Kraus, Julia\u00a0Anna Bingler, and Markus Leippold. 2021. Climatebert: A pretrained language model for climate-related text. arXiv preprint arXiv:2110.12010, (2021),."},{"key":"e_1_3_2_1_67_1","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc\u00a0V Le, Denny Zhou, 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824\u201324837.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_68_1","volume-title":"Characterizing LLM Abstention Behavior in Science QA with Context Perturbations. arXiv preprint arXiv:2404.12452","author":"Wen Bingbing","year":"2024","unstructured":"Bingbing Wen, Bill Howe, and Lucy\u00a0Lu Wang. 2024. Characterizing LLM Abstention Behavior in Science QA with Context Perturbations. arXiv preprint arXiv:2404.12452 (2024)."},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"crossref","unstructured":"Thomas Wolf Lysandre Debut Victor Sanh Julien Chaumond Clement Delangue Anthony Moi Pierric Cistac Tim Rault R\u00e9mi Louf Morgan Funtowicz 2019. Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019) .","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_1_70_1","unstructured":"Lukas Wutschitz Huseyin\u00a0A. Inan and Andre Manoel. 2022. dp-transformers: Training transformer models with differential privacy. https:\/\/www.microsoft.com\/en-us\/research\/project\/dp-transformers."},{"key":"e_1_3_2_1_71_1","doi-asserted-by":"crossref","unstructured":"Wen-wai Yim A\u00a0Ben Abacha N Snider G Adams and Meliha Yetisgen. 2023. Overview of the mediqa-sum task at imageclef 2023: Summarization and classification of doctor-patient conversations. In CLEF. .","DOI":"10.18653\/v1\/2023.clinicalnlp-1.52"},{"key":"e_1_3_2_1_72_1","volume-title":"Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109.12298","author":"Yousefpour Ashkan","year":"2021","unstructured":"Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, 2021. Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109.12298, (2021),."},{"key":"e_1_3_2_1_73_1","unstructured":"Da Yu Saurabh Naik Arturs Backurs Sivakanth Gopi Huseyin\u00a0A Inan Gautam Kamath Janardhan Kulkarni Yin\u00a0Tat Lee Andre Manoel Lukas Wutschitz 2021. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500 (2021) ."},{"key":"e_1_3_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"e_1_3_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-016-0200-9"},{"key":"e_1_3_2_1_76_1","volume-title":"BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations.,,.","author":"Zhang Tianyi","year":"2019","unstructured":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian\u00a0Q Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations.,,."},{"key":"e_1_3_2_1_77_1","unstructured":"Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu Yonghao Zhuang Zi Lin Zhuohan Li Dacheng Li Eric\u00a0P. Xing Hao Zhang Joseph\u00a0E. Gonzalez and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arxiv:2306.05685\u00a0[cs.CL]"}],"event":{"name":"FAccT '24: The 2024 ACM Conference on Fairness, Accountability, and Transparency","location":"Rio de Janeiro Brazil","acronym":"FAccT '24"},"container-title":["The 2024 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630106.3658966","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3630106.3658966","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:58Z","timestamp":1750287058000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630106.3658966"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,3]]},"references-count":76,"alternative-id":["10.1145\/3630106.3658966","10.1145\/3630106"],"URL":"https:\/\/doi.org\/10.1145\/3630106.3658966","relation":{},"subject":[],"published":{"date-parts":[[2024,6,3]]},"assertion":[{"value":"2024-06-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}