{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T09:03:32Z","timestamp":1772183012770,"version":"3.50.1"},"reference-count":31,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T00:00:00Z","timestamp":1772150400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T00:00:00Z","timestamp":1772150400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100000266","name":"EPSRC","doi-asserted-by":"crossref","award":["EP\/T022108\/1"],"award-info":[{"award-number":["EP\/T022108\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Science and Technology Facilities Council (STFC) Consolidated","award":["ST\/X00063X\/1"],"award-info":[{"award-number":["ST\/X00063X\/1"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2026,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>We present specialized large language models (LLMs) for theoretical high-energy physics, obtained as 20 fine-tuned variants of the 8\u2009billion parameter Llama-3.1 model. Each variant was trained on arXiv abstracts (through August 2024) from different combinations of hep-th, hep-ph and gr-qc. For a comparative study, we also trained models on datasets that contained abstracts from disparate fields such as the q-bio and cs categories. All models were fine-tuned using two distinct low-rank adaptation fine-tuning approaches and varying dataset sizes, and outperformed the base model on hep-th abstract completion tasks. We compare performance against leading commercial LLMs (ChatGPT, Claude, Gemini, DeepSeek) and derive insights for further developing specialized language models for high-energy theoretical physics.<\/jats:p>","DOI":"10.1088\/2632-2153\/ae47bb","type":"journal-article","created":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T22:55:51Z","timestamp":1771455351000},"page":"025012","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["FeynTune: large language models for high-energy theory"],"prefix":"10.1088","volume":"7","author":[{"given":"Paul","family":"Richmond","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6760-5942","authenticated-orcid":true,"given":"Constantinos","family":"Papageorgakis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vasilis","family":"Niarchos","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Borun","family":"Chowdhury","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Prarit","family":"Agarwal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2026,2,27]]},"reference":[{"key":"mlstae47bbbib1","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2022","type":"preprint"},{"key":"mlstae47bbbib2","article-title":"GPT-4 technical report","author":"OpenAI","year":"2024","type":"preprint"},{"key":"mlstae47bbbib3","article-title":"LLaMA: open and efficient foundation language models","author":"Touvron","year":"2023","type":"preprint"},{"key":"mlstae47bbbib4","author":"Touvron","year":"2023","type":"preprint"},{"key":"mlstae47bbbib5","article-title":"The llama 3 herd of models","author":"Dubey","year":"2024","type":"preprint"},{"key":"mlstae47bbbib6","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.emnlp-main.498","type":"preprint","article-title":"A comprehensive survey of scientific large language models and their applications in scientific discovery","author":"Zhang","year":"2024"},{"key":"mlstae47bbbib7","article-title":"Early science acceleration experiments with GPT-5","author":"Bubeck","year":"2025","type":"preprint"},{"key":"mlstae47bbbib8","article-title":"Building astroBERT, a language model for astronomy and astrophysics","author":"Grezes","year":"2021","type":"preprint"},{"key":"mlstae47bbbib9","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.wiesp-1.7","type":"preprint","article-title":"AstroLLaMA: towards specialized foundation models in astronomy","author":"Nguyen","year":"2023"},{"key":"mlstae47bbbib10","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3847\/2515-5172\/ad1abe","type":"journal-article","article-title":"AstroLLaMA-chat: scaling astrollama with conversational and diverse datasets","volume":"8","author":"UniverseTBD Collaboration","year":"2024","journal-title":"Res. Notes AAS"},{"key":"mlstae47bbbib11","doi-asserted-by":"crossref","DOI":"10.2139\/ssrn.4922372","type":"preprint","article-title":"AstroMLab 1: who wins astronomy jeopardy!?","author":"Ting","year":"2024"},{"key":"mlstae47bbbib12","doi-asserted-by":"crossref","DOI":"10.3847\/2515-5172\/ad1abe","type":"preprint","article-title":"Astrollama-chat: scaling astrollama with conversational and diverse datasets","author":"Perkowski","year":"2024"},{"key":"mlstae47bbbib13","doi-asserted-by":"publisher","DOI":"10.1016\/j.ascom.2025.100934","type":"journal-article","article-title":"Cosmosage: a natural-language assistant for cosmology","volume":"51","author":"de Haan","year":"2025","journal-title":"Astron. Comput."},{"key":"mlstae47bbbib14","doi-asserted-by":"crossref","DOI":"10.1063\/5.0238090","type":"preprint","article-title":"Physbert: a text embedding model for physics scientific literature","author":"Hellert","year":"2024"},{"key":"mlstae47bbbib15","article-title":"Can LLMs learn from a single example?","author":"Howard","year":"2023","type":"other"},{"key":"mlstae47bbbib16","article-title":"Solving quantitative reasoning problems with language models","author":"Lewkowycz","year":"2022","type":"preprint"},{"key":"mlstae47bbbib17","first-page":"pp 7750","type":"conference-proceedings","article-title":"The case for 4-bit precision: k-bit inference scaling laws","author":"Dettmers","year":"2023"},{"key":"mlstae47bbbib18","article-title":"QLoRa: efficient finetuning of quantized LLMs","author":"Dettmers","year":"2023","type":"preprint"},{"key":"mlstae47bbbib19","article-title":"LoRA: low-rank adaptation of large language models","author":"Hu","year":"2022","type":"conference-proceedings"},{"key":"mlstae47bbbib20","article-title":"FlashAttention-2: faster attention with better parallelism and work partitioning","author":"Dao","year":"2024","type":"conference-proceedings"},{"key":"mlstae47bbbib21","author":"Huyen","year":"2024","type":"book"},{"key":"mlstae47bbbib22","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/P19-1285","type":"preprint","article-title":"Transformer-xl: attentive language models beyond a fixed-length context","author":"Dai","year":"2019"},{"key":"mlstae47bbbib23","article-title":"SemScore: automated evaluation of instruction-tuned LLMs based on semantic textual similarity","author":"Aynetdinov","year":"2024","type":"preprint"},{"key":"mlstae47bbbib24","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","type":"journal-article","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J. Chiropractic Med."},{"key":"mlstae47bbbib25","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/adfcb0","type":"journal-article","article-title":"Theoretical physics benchmark (TPBench)\u2014a dataset and study of AI reasoning capabilities in theoretical physics","volume":"6","author":"Chung","year":"2025","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstae47bbbib26","type":"dataset","article-title":"FeynTune data","author":"Richmond","year":"2025","unstructured":"Richmond P Papageorgakis C Niarchos V Chowdhury B Agarwal P 2025 FeynTune data Hugging Face (available at: https:\/\/huggingface.co\/LLMsForHepth )"},{"key":"mlstae47bbbib27","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1214\/aoms\/1177730491","type":"journal-article","article-title":"On a test of whether one of two random variables is stochastically larger than the other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Stat."},{"key":"mlstae47bbbib28","doi-asserted-by":"publisher","first-page":"11.IT.3.1","DOI":"10.2466\/11.IT.3.1","type":"journal-article","article-title":"The simple difference formula: an approach to teaching nonparametric correlation","volume":"3","author":"Kerby","year":"2014","journal-title":"Compr. Psychol."},{"key":"mlstae47bbbib29","author":"Cohen","year":"1988","edition":"2nd edn","type":"book"},{"key":"mlstae47bbbib30","doi-asserted-by":"publisher","first-page":"72","DOI":"10.2307\/1412159","type":"journal-article","article-title":"The proof and measurement of association between two things","volume":"15","author":"Spearman","year":"1904","journal-title":"Am. J. Psychol."},{"key":"mlstae47bbbib31","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","type":"journal-article","article-title":"Intraclass correlations: uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol. Bull."}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T08:19:31Z","timestamp":1772180371000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae47bb"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,27]]},"references-count":31,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2,27]]},"published-print":{"date-parts":[[2026,4,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ae47bb","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,27]]},"assertion":[{"value":"FeynTune: large language models for high-energy theory","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2026 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2025-10-18","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-02-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-02-27","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}