{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T08:02:02Z","timestamp":1776931322396,"version":"3.51.2"},"publisher-location":"New York, NY, USA","reference-count":27,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,11,16]]},"DOI":"10.1145\/3731599.3767410","type":"proceedings-article","created":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T16:18:44Z","timestamp":1762532324000},"page":"545-552","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Automated MCQA Benchmarking at Scale: Evaluating Reasoning Traces as Retrieval Sources for Domain Adaptation of Small Language Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5299-1983","authenticated-orcid":false,"given":"Ozan","family":"Gokdemir","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Chicago, Chicago, IL, USA and Data Science and Learning Division, Argonne National Laboratory (ANL), Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0535-8362","authenticated-orcid":false,"given":"Neil","family":"Getty","sequence":"additional","affiliation":[{"name":"Data Science and Learning Division, Argonne National Laboratory (ANL), Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1464-729X","authenticated-orcid":false,"given":"Robert","family":"Underwood","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Division, Argonne National Laboratory (ANL), Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0437-8655","authenticated-orcid":false,"given":"Sandeep","family":"Madireddy","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Division, Argonne National Laboratory (ANL), Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7890-3934","authenticated-orcid":false,"given":"Franck","family":"Cappello","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Division, Argonne National Laboratory (ANL), Lemont, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1622-5488","authenticated-orcid":false,"given":"Arvind","family":"Ramanathan","sequence":"additional","affiliation":[{"name":"Data Science and Learning Division, Argonne National Laboratory (ANL), Lemont, IL, USA and Department of Computer Science, University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2129-5269","authenticated-orcid":false,"given":"Ian T.","family":"Foster","sequence":"additional","affiliation":[{"name":"Data Science and Learning Division, Argonne National Laboratory (ANL), Lemont, IL, USA and Department of Computer Science, University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4268-4020","authenticated-orcid":false,"given":"Rick L.","family":"Stevens","sequence":"additional","affiliation":[{"name":"Data Science and Learning Division, Argonne National Laboratory (ANL), Lemont, IL, USA and Department of Computer Science, University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,11,15]]},"reference":[{"key":"e_1_3_3_2_2_2","unstructured":"Allen Institute for AI. 2015. Semantic Scholar. https:\/\/www.semanticscholar.org\/. Accessed: 2025-08-22."},{"key":"e_1_3_3_2_3_2","volume-title":"RADIATION AND CANCER BIOLOGY STUDY GUIDE","author":"Oncology American Society for Radiation","year":"2023","unstructured":"American Society for Radiation Oncology. 2023. RADIATION AND CANCER BIOLOGY STUDY GUIDE. https:\/\/www.astro.org\/ASTRO\/media\/ASTRO\/AffiliatePages\/arro\/PDFs\/RadBio_StudyGuide_23.pdf"},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3307681.3325400"},{"key":"e_1_3_3_2_5_2","unstructured":"Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang Xiaodong Deng Yang Fan Wenbin Ge Yu Han Fei Huang Binyuan Hui Luo Ji Mei Li Junyang Lin Runji Lin Dayiheng Liu Gao Liu Chengqiang Lu Keming Lu Jianxin Ma Rui Men Xingzhang Ren Xuancheng Ren Chuanqi Tan Sinan Tan Jianhong Tu Peng Wang Shijie Wang Wei Wang Shengguang Wu Benfeng Xu Jin Xu An Yang Hao Yang Jian Yang Shusheng Yang Yang Yao Bowen Yu Hongyi Yuan Zheng Yuan Jianwei Zhang Xingxuan Zhang Yichang Zhang Zhenru Zhang Chang Zhou Jingren Zhou Xiaohuan Zhou and Tianhang Zhu. 2023. Qwen Technical Report. arxiv:https:\/\/arXiv.org\/abs\/2309.16609\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2309.16609"},{"key":"e_1_3_3_2_6_2","doi-asserted-by":"publisher","unstructured":"J. Beattie S. Neufeld D.X. Yang C. Chukwuma N.B. Desai M. Dohopolski and S.B. Jiang. 2024. Using Large Language Models to Create Patient Centered Consent Forms. International Journal of Radiation Oncology*Biology*Physics 120 2 (Oct. 2024) e612. 10.1016\/j.ijrobp.2024.07.1346","DOI":"10.1016\/j.ijrobp.2024.07.1346"},{"key":"e_1_3_3_2_7_2","volume-title":"North American Chapter of the Association for Computational Linguistics (NAACL), Workshop","author":"Clark Peter","year":"2018","unstructured":"Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and \u00d8yvind Tafjord. 2018. Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. In North American Chapter of the Association for Computational Linguistics (NAACL), Workshop. Dataset: ARC (AI2 Reasoning Challenge)."},{"key":"e_1_3_3_2_8_2","volume-title":"Science and Engineering Indicators 2024: The State of U.S. Science and Engineering","author":"Deitz Steven","year":"2024","unstructured":"Steven Deitz and Christina Freyman. 2024. Science and Engineering Indicators 2024: The State of U.S. Science and Engineering. Technical Report NSB-2024-3. National Science Foundation, Alexandria, VA. https:\/\/ncses.nsf.gov\/pubs\/nsb20243"},{"key":"e_1_3_3_2_9_2","unstructured":"Peng Ding. 2024. argo-proxy. https:\/\/github.com\/Oaklight\/argo-proxy. Accessed: 2025-08-23."},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Matthijs Douze Alexandr Guzhva Chengqi Deng Jeff Johnson Gergely Szilvasy Pierre-Emmanuel Mazar\u00e9 Maria Lomeli Lucas Hosseini and Herv\u00e9 J\u00e9gou. 2024. The Faiss library. arxiv:https:\/\/arXiv.org\/abs\/2401.08281\u00a0[cs.LG]","DOI":"10.1109\/TBDATA.2025.3618474"},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3732775.3733586"},{"key":"e_1_3_3_2_12_2","unstructured":"Aaron Grattafiori Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Alex Vaughan Amy Yang Angela Fan Anirudh Goyal Anthony Hartshorn Aobo Yang Archi Mitra Archie Sravankumar Artem Korenev Arthur Hinsvark Arun Rao Aston Zhang Aurelien Rodriguez Austen Gregerson Ava Spataru Baptiste Roziere Bethany Biron Binh Tang Bobbie Chern Charlotte Caucheteux Chaya Nayak Chloe Bi Chris Marra Chris McConnell Christian Keller Christophe Touret Chunyang Wu Corinne Wong Cristian\u00a0Canton Ferrer Cyrus Nikolaidis Damien Allonsius Daniel Song Danielle Pintz Danny Livshits Danny Wyatt David Esiobu Dhruv Choudhary Dhruv Mahajan Diego Garcia-Olano Diego Perino Dieuwke Hupkes Egor Lakomkin Ehab AlBadawy Elina Lobanova Emily Dinan Eric\u00a0Michael Smith Filip Radenovic Francisco Guzm\u00e1n Frank Zhang Gabriel Synnaeve Gabrielle Lee Georgia\u00a0Lewis Anderson Govind Thattai Graeme Nail Gregoire Mialon Guan Pang Guillem Cucurell Hailey Nguyen Hannah Korevaar Hu Xu Hugo Touvron Iliyan Zarov Imanol\u00a0Arrieta Ibarra Isabel Kloumann Ishan Misra Ivan Evtimov Jack Zhang Jade Copet Jaewon Lee Jan Geffert Jana Vranes Jason Park Jay Mahadeokar Jeet Shah Jelmer van\u00a0der Linde Jennifer Billock Jenny Hong Jenya Lee Jeremy Fu Jianfeng Chi Jianyu Huang Jiawen Liu Jie Wang Jiecao Yu Joanna Bitton Joe Spisak Jongsoo Park Joseph Rocca Joshua Johnstun Joshua Saxe Junteng Jia Kalyan\u00a0Vasuden Alwala Karthik Prasad Kartikeya Upasani Kate Plawiak Ke Li Kenneth Heafield Kevin Stone Khalid El-Arini Krithika Iyer Kshitiz Malik Kuenley Chiu Kunal Bhalla Kushal Lakhotia Lauren Rantala-Yeary Laurens van\u00a0der Maaten Lawrence Chen Liang Tan Liz Jenkins Louis Martin Lovish Madaan Lubo Malo Lukas Blecher Lukas Landzaat Luke de Oliveira Madeline Muzzi Mahesh Pasupuleti Mannat Singh Manohar Paluri Marcin Kardas Maria Tsimpoukelli Mathew Oldham Mathieu Rita Maya Pavlova Melanie Kambadur Mike Lewis Min Si Mitesh\u00a0Kumar Singh Mona Hassan Naman Goyal Narjes Torabi Nikolay Bashlykov Nikolay Bogoychev Niladri Chatterji Ning Zhang Olivier Duchenne Onur \u00c7elebi Patrick Alrassy Pengchuan Zhang Pengwei Li Petar Vasic Peter Weng Prajjwal Bhargava Pratik Dubal Praveen Krishnan Punit\u00a0Singh Koura Puxin Xu Qing He Qingxiao Dong Ragavan Srinivasan Raj Ganapathy Ramon Calderer Ricardo\u00a0Silveira Cabral Robert Stojnic Roberta Raileanu Rohan Maheswari Rohit Girdhar Rohit Patel Romain Sauvestre Ronnie Polidoro Roshan Sumbaly Ross Taylor Ruan Silva Rui Hou Rui Wang Saghar Hosseini Sahana Chennabasappa Sanjay Singh Sean Bell Seohyun\u00a0Sonia Kim Sergey Edunov Shaoliang Nie Sharan Narang Sharath Raparthy Sheng Shen Shengye Wan Shruti Bhosale Shun Zhang Simon Vandenhende Soumya Batra Spencer Whitman Sten Sootla Stephane Collot Suchin Gururangan Sydney Borodinsky Tamar Herman Tara Fowler Tarek Sheasha Thomas Georgiou Thomas Scialom Tobias Speckbacher Todor Mihaylov Tong Xiao Ujjwal Karn Vedanuj Goswami Vibhor Gupta Vignesh Ramanathan Viktor Kerkez Vincent Gonguet Virginie Do Vish Vogeti V\u00edtor Albiero Vladan Petrovic Weiwei Chu Wenhan Xiong Wenyin Fu Whitney Meers Xavier Martinet Xiaodong Wang Xiaofang Wang Xiaoqing\u00a0Ellen Tan Xide Xia Xinfeng Xie Xuchao Jia Xuewei Wang Yaelle Goldschlag Yashesh Gaur Yasmine Babaei Yi Wen Yiwen Song Yuchen Zhang Yue Li Yuning Mao Zacharie\u00a0Delpierre Coudert Zheng Yan Zhengxing Chen Zoe Papakipos Aaditya Singh Aayushi Srivastava Abha Jain Adam Kelsey Adam Shajnfeld Adithya Gangidi Adolfo Victoria Ahuva Goldstand Ajay Menon Ajay Sharma Alex Boesenberg Alexei Baevski Allie Feinstein Amanda Kallet Amit Sangani Amos Teo Anam Yunus Andrei Lupu Andres Alvarado Andrew Caples Andrew Gu Andrew Ho Andrew Poulton Andrew Ryan Ankit Ramchandani Annie Dong Annie Franco Anuj Goyal Aparajita Saraf Arkabandhu Chowdhury Ashley Gabriel Ashwin Bharambe Assaf Eisenman Azadeh Yazdan Beau James Ben Maurer Benjamin Leonhardi Bernie Huang Beth Loyd Beto\u00a0De Paola Bhargavi Paranjape Bing Liu Bo Wu Boyu Ni Braden Hancock Bram Wasti Brandon Spence Brani Stojkovic Brian Gamido Britt Montalvo Carl Parker Carly Burton Catalina Mejia Ce Liu Changhan Wang Changkyu Kim Chao Zhou Chester Hu Ching-Hsiang Chu Chris Cai Chris Tindal Christoph Feichtenhofer Cynthia Gao Damon Civin Dana Beaty Daniel Kreymer Daniel Li David Adkins David Xu Davide Testuggine Delia David Devi Parikh Diana Liskovich Didem Foss Dingkang Wang Duc Le Dustin Holland Edward Dowling Eissa Jamil Elaine Montgomery Eleonora Presani Emily Hahn Emily Wood Eric-Tuan Le Erik Brinkman Esteban Arcaute Evan Dunbar Evan Smothers Fei Sun Felix Kreuk Feng Tian Filippos Kokkinos Firat Ozgenel Francesco Caggioni Frank Kanayet Frank Seide Gabriela\u00a0Medina Florez Gabriella Schwarz Gada Badeer Georgia Swee Gil Halpern Grant Herman Grigory Sizov Guangyi Zhang Guna Lakshminarayanan Hakan Inan Hamid Shojanazeri Han Zou Hannah Wang Hanwen Zha Haroun Habeeb Harrison Rudolph Helen Suk Henry Aspegren Hunter Goldman Hongyuan Zhan Ibrahim Damlaj Igor Molybog Igor Tufanov Ilias Leontiadis Irina-Elena Veliche Itai Gat Jake Weissman James Geboski James Kohli Janice Lam Japhet Asher Jean-Baptiste Gaya Jeff Marcus Jeff Tang Jennifer Chan Jenny Zhen Jeremy Reizenstein Jeremy Teboul Jessica Zhong Jian Jin Jingyi Yang Joe Cummings Jon Carvill Jon Shepard Jonathan McPhie Jonathan Torres Josh Ginsburg Junjie Wang Kai Wu Kam\u00a0Hou U Karan Saxena Kartikay Khandelwal Katayoun Zand Kathy Matosich Kaushik Veeraraghavan Kelly Michelena Keqian Li Kiran Jagadeesh Kun Huang Kunal Chawla Kyle Huang Lailin Chen Lakshya Garg Lavender A Leandro Silva Lee Bell Lei Zhang Liangpeng Guo Licheng Yu Liron Moshkovich Luca Wehrstedt Madian Khabsa Manav Avalani Manish Bhatt Martynas Mankus Matan Hasson Matthew Lennie Matthias Reso Maxim Groshev Maxim Naumov Maya Lathi Meghan Keneally Miao Liu Michael\u00a0L. Seltzer Michal Valko Michelle Restrepo Mihir Patel Mik Vyatskov Mikayel Samvelyan Mike Clark Mike Macey Mike Wang Miquel\u00a0Jubert Hermoso Mo Metanat Mohammad Rastegari Munish Bansal Nandhini Santhanam Natascha Parks Natasha White Navyata Bawa Nayan Singhal Nick Egebo Nicolas Usunier Nikhil Mehta Nikolay\u00a0Pavlovich Laptev Ning Dong Norman Cheng Oleg Chernoguz Olivia Hart Omkar Salpekar Ozlem Kalinli Parkin Kent Parth Parekh Paul Saab Pavan Balaji Pedro Rittner Philip Bontrager Pierre Roux Piotr Dollar Polina Zvyagina Prashant Ratanchandani Pritish Yuvraj Qian Liang Rachad Alao Rachel Rodriguez Rafi Ayub Raghotham Murthy Raghu Nayani Rahul Mitra Rangaprabhu Parthasarathy Raymond Li Rebekkah Hogan Robin Battey Rocky Wang Russ Howes Ruty Rinott Sachin Mehta Sachin Siby Sai\u00a0Jayesh Bondu Samyak Datta Sara Chugh Sara Hunt Sargun Dhillon Sasha Sidorov Satadru Pan Saurabh Mahajan Saurabh Verma Seiji Yamamoto Sharadh Ramaswamy Shaun Lindsay Shaun Lindsay Sheng Feng Shenghao Lin Shengxin\u00a0Cindy Zha Shishir Patil Shiva Shankar Shuqiang Zhang Shuqiang Zhang Sinong Wang Sneha Agarwal Soji Sajuyigbe Soumith Chintala Stephanie Max Stephen Chen Steve Kehoe Steve Satterfield Sudarshan Govindaprasad Sumit Gupta Summer Deng Sungmin Cho Sunny Virk Suraj Subramanian Sy Choudhury Sydney Goldman Tal Remez Tamar Glaser Tamara Best Thilo Koehler Thomas Robinson Tianhe Li Tianjun Zhang Tim Matthews Timothy Chou Tzook Shaked Varun Vontimitta Victoria Ajayi Victoria Montanez Vijai Mohan Vinay\u00a0Satish Kumar Vishal Mangla Vlad Ionescu Vlad Poenaru Vlad\u00a0Tiberiu Mihailescu Vladimir Ivanov Wei Li Wenchen Wang Wenwen Jiang Wes Bouaziz Will Constable Xiaocheng Tang Xiaojian Wu Xiaolan Wang Xilun Wu Xinbo Gao Yaniv Kleinman Yanjun Chen Ye Hu Ye Jia Ye Qi Yenda Li Yilin Zhang Ying Zhang Yossi Adi Youngjin Nam Yu Wang Yu Zhao Yuchen Hao Yundi Qian Yunlu Li Yuzi He Zach Rait Zachary DeVito Zef Rosnbrick Zhaoduo Wen Zhenyu Yang Zhiwei Zhao and Zhiyu Ma. 2024. The Llama 3 Herd of Models. arxiv:https:\/\/arXiv.org\/abs\/2407.21783\u00a0[cs.AI] https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_3_2_13_2","doi-asserted-by":"publisher","unstructured":"Yu Gu Robert Tinn Hao Cheng Michael Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao and Hoifung Poon. 2021. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. ACM Transactions on Computing for Healthcare 3 1 (Oct. 2021) 1\u201323. 10.1145\/3458754","DOI":"10.1145\/3458754"},{"key":"e_1_3_3_2_14_2","unstructured":"Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Dawn Song and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2009.03300 (2021). Dataset: MMLU."},{"key":"e_1_3_3_2_15_2","unstructured":"HuggingFaceTB. 2025. SmolLM3-3B. https:\/\/huggingface.co\/HuggingFaceTB\/SmolLM3-3B. Accessed: 2025-08-23."},{"key":"e_1_3_3_2_16_2","unstructured":"Albert\u00a0Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra\u00a0Singh Chaplot Diego de\u00a0las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier L\u00e9lio\u00a0Renard Lavaud Marie-Anne Lachaux Pierre Stock Teven\u00a0Le Scao Thibaut Lavril Thomas Wang Timoth\u00e9e Lacroix and William\u00a0El Sayed. 2023. Mistral 7B. arxiv:https:\/\/arXiv.org\/abs\/2310.06825\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2310.06825"},{"key":"e_1_3_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1259"},{"key":"e_1_3_3_2_18_2","unstructured":"Jakub L\u00e1la Odhran O\u2019Donoghue Aleksandar Shtedritski Sam Cox Samuel\u00a0G Rodriques and Andrew\u00a0D White. 2023. PaperQA: Retrieval-Augmented Generative Agent for Scientific Research. (2023)."},{"key":"e_1_3_3_2_19_2","doi-asserted-by":"publisher","unstructured":"Esther Landhuis. 2016. Scientific literature: Information overload. Nature 535 (07 2016) 457\u2013458. 10.1038\/nj7612-457a","DOI":"10.1038\/nj7612-457a"},{"key":"e_1_3_3_2_20_2","unstructured":"OpenAI. 2025. Introducing GPT-5. https:\/\/openai.com\/index\/introducing-gpt-5\/"},{"key":"e_1_3_3_2_21_2","unstructured":"Aurora Pi-Groeneveld Luca\u00a0De Martini Jesse Dodge Oyvind Tafjord Ben Hutchinson Matt Gardner Noah Smith Luke Zettlemoyer and Nathan Schneider. 2024. OLMo: Accelerating the Science of Language Models. (2024). arxiv:https:\/\/arXiv.org\/abs\/2402.00838\u00a0[cs.CL]"},{"key":"e_1_3_3_2_22_2","unstructured":"David Rein Betty\u00a0Li Hou Asa\u00a0Cooper Stickland Jackson Petty Richard\u00a0Yuanzhe Pang Julien Dirani Julian Michael and Samuel\u00a0R. Bowman. 2023. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. arxiv:https:\/\/arXiv.org\/abs\/2311.12022\u00a0[cs.AI] https:\/\/arxiv.org\/abs\/2311.12022"},{"key":"e_1_3_3_2_23_2","unstructured":"Carlo Siebenschuh Kyle Hippe Ozan Gokdemir Alexander Brace Arham Khan Khalid Hossain Yadu Babuji Nicholas Chia Venkatram Vishwanath Rick Stevens Arvind Ramanathan Ian Foster and Robert Underwood. 2025. AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine. arxiv:https:\/\/arXiv.org\/abs\/2505.01435\u00a0[cs.IR] https:\/\/arxiv.org\/abs\/2505.01435"},{"key":"e_1_3_3_2_24_2","unstructured":"Gemma Team Aishwarya Kamath Johan Ferret Shreya Pathak Nino Vieillard Ramona Merhej Sarah Perrin Tatiana Matejovicova Alexandre Ram\u00e9 Morgane Rivi\u00e8re Louis Rouillard Thomas Mesnard Geoffrey Cideron Jean bastien Grill Sabela Ramos Edouard Yvinec Michelle Casbon Etienne Pot Ivo Penchev Ga\u00ebl Liu Francesco Visin Kathleen Kenealy Lucas Beyer Xiaohai Zhai Anton Tsitsulin Robert Busa-Fekete Alex Feng Noveen Sachdeva Benjamin Coleman Yi Gao Basil Mustafa Iain Barr Emilio Parisotto David Tian Matan Eyal Colin Cherry Jan-Thorsten Peter Danila Sinopalnikov Surya Bhupatiraju Rishabh Agarwal Mehran Kazemi Dan Malkin Ravin Kumar David Vilar Idan Brusilovsky Jiaming Luo Andreas Steiner Abe Friesen Abhanshu Sharma Abheesht Sharma Adi\u00a0Mayrav Gilady Adrian Goedeckemeyer Alaa Saade Alex Feng Alexander Kolesnikov Alexei Bendebury Alvin Abdagic Amit Vadi Andr\u00e1s Gy\u00f6rgy Andr\u00e9\u00a0Susano Pinto Anil Das Ankur Bapna Antoine Miech Antoine Yang Antonia Paterson Ashish Shenoy Ayan Chakrabarti Bilal Piot Bo Wu Bobak Shahriari Bryce Petrini Charlie Chen Charline\u00a0Le Lan Christopher\u00a0A. Choquette-Choo CJ Carey Cormac Brick Daniel Deutsch Danielle Eisenbud Dee Cattle Derek Cheng Dimitris Paparas Divyashree\u00a0Shivakumar Sreepathihalli Doug Reid Dustin Tran Dustin Zelle Eric Noland Erwin Huizenga Eugene Kharitonov Frederick Liu Gagik Amirkhanyan Glenn Cameron Hadi Hashemi Hanna Klimczak-Pluci\u0144ska Harman Singh Harsh Mehta Harshal\u00a0Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini Jean Pouget-Abadie Jetha Chan Joe Stanton John Wieting Jonathan Lai Jordi Orbay Joseph Fernandez Josh Newlan Ju yeong Ji Jyotinder Singh Kat Black Kathy Yu Kevin Hui Kiran Vodrahalli Klaus Greff Linhai Qiu Marcella Valentine Marina Coelho Marvin Ritter Matt Hoffman Matthew Watson Mayank Chaturvedi Michael Moynihan Min Ma Nabila Babar Natasha Noy Nathan Byrd Nick Roy Nikola Momchev Nilay Chauhan Noveen Sachdeva Oskar Bunyan Pankil Botarda Paul Caron Paul\u00a0Kishan Rubenstein Phil Culliton Philipp Schmid Pier\u00a0Giuseppe Sessa Pingmei Xu Piotr Stanczyk Pouya Tafti Rakesh Shivanna Renjie Wu Renke Pan Reza Rokni Rob Willoughby Rohith Vallu Ryan Mullins Sammy Jerome Sara Smoot Sertan Girgin Shariq Iqbal Shashir Reddy Shruti Sheth Siim P\u00f5der Sijal Bhatnagar Sindhu\u00a0Raghuram Panyam Sivan Eiger Susan Zhang Tianqi Liu Trevor Yacovone Tyler Liechty Uday Kalra Utku Evci Vedant Misra Vincent Roseberry Vlad Feinberg Vlad Kolesnikov Woohyun Han Woosuk Kwon Xi Chen Yinlam Chow Yuvein Zhu Zichuan Wei Zoltan Egyed Victor Cotruta Minh Giang Phoebe Kirk Anand Rao Kat Black Nabila Babar Jessica Lo Erica Moreira Luiz\u00a0Gustavo Martins Omar Sanseviero Lucas Gonzalez Zach Gleicher Tris Warkentin Vahab Mirrokni Evan Senter Eli Collins Joelle Barral Zoubin Ghahramani Raia Hadsell Yossi Matias D. Sculley Slav Petrov Noah Fiedel Noam Shazeer Oriol Vinyals Jeff Dean Demis Hassabis Koray Kavukcuoglu Clement Farabet Elena Buchatskaya Jean-Baptiste Alayrac Rohan Anil Dmitry Lepikhin Sebastian Borgeaud Olivier Bachem Armand Joulin Alek Andreev Cassidy Hardin Robert Dadashi and L\u00e9onard Hussenot. 2025. Gemma 3 Technical Report. arxiv:https:\/\/arXiv.org\/abs\/2503.19786\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2503.19786"},{"key":"e_1_3_3_2_25_2","doi-asserted-by":"publisher","unstructured":"Yuan-Sen Ting Tuan\u00a0Dung Nguyen Tirthankar Ghosal Rui Pan Hardik Arora Zechang Sun Tijmen de Haan Nesar Ramachandra Azton Wells Sandeep Madireddy and Alberto Accomazzi. 2024. AstroMLab 1: Who Wins Astronomy Jeopardy!?10.48550\/ARXIV.2407.11194","DOI":"10.48550\/ARXIV.2407.11194"},{"key":"e_1_3_3_2_26_2","volume-title":"Extended Semantic Web Conference","author":"Wang Shansong","year":"2024","unstructured":"Shansong Wang, Mingzhe Hu, Qiang Li, Mojtaba Safari, and Xiaofeng Yang. 2024. Large Language Models for Scientific Question Answering: An Extensive Analysis of the SciQA Benchmark. In Extended Semantic Web Conference."},{"key":"e_1_3_3_2_27_2","unstructured":"Johannes Welbl Pontus Stenetorp and Sebastian Riedel. 2017. Crowdsourcing Multiple Choice Science Questions. Transactions of the Association for Computational Linguistics 5 (2017) 301\u2013312. Dataset: SciQ."},{"key":"e_1_3_3_2_28_2","unstructured":"Peiyuan Zhang Guangtao Zeng Tianduo Wang and Wei Lu. 2024. TinyLlama: An Open-Source Small Language Model. arxiv:https:\/\/arXiv.org\/abs\/2401.02385\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2401.02385"}],"event":{"name":"SC Workshops '25: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis","location":"St Louis MO USA","acronym":"SC Workshops '25","sponsor":["SIGHPC ACM Special Interest Group on High Performance Computing, Special Interest Group on High Performance Computing"]},"container-title":["Proceedings of the SC '25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3731599.3767410","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T19:32:22Z","timestamp":1767987142000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731599.3767410"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,15]]},"references-count":27,"alternative-id":["10.1145\/3731599.3767410","10.1145\/3731599"],"URL":"https:\/\/doi.org\/10.1145\/3731599.3767410","relation":{},"subject":[],"published":{"date-parts":[[2025,11,15]]},"assertion":[{"value":"2025-11-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}