{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T06:16:38Z","timestamp":1783577798630,"version":"3.55.0"},"reference-count":36,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"},{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/questions\/alternative-author-rights-policies\/"},{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/publishingsupport.iopscience.iop.org\/questions\/alternative-author-rights-policies\/"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2026,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    The rapid emergence of agentic AI presents new opportunities and challenges for accelerating scientific discovery through tool-augmented reasoning, autonomous workflows, and reproducible results. To explore these capabilities in a hands-on, community-driven setting, we hosted the Agentic AI for Science Hackathon 2025, which attracted 352 registered participants. The event was designed to benchmark, test, and extend agentic AI workflows across scientific tasks while remaining model-agnostic and lowering the barrier to entry for participants from diverse backgrounds. Participants engaged with a unified API ecosystem centered on the fully open-access AtomGPT.org API, implementing tool calling, asynchronous agents, and multi-model reasoning to solve problems spanning materials database retrieval, literature search, mathematical reasoning, and scientific workflow automation. The hackathon prompt emphasized reproducibility, transparent tool usage, and explicit agent-tool interaction, encouraging participants to move beyond single-prompt question answering toward structured, auditable pipelines. Outcomes included working agentic prototypes, identification of failure modes in contemporary chatbots, and actionable insights into best practices for agent design in scientific contexts. This paper documents the hackathon design, task structure, and key lessons learned, highlighting how hackathon-based evaluations can complement formal benchmarks and inform the development of robust agentic AI systems for science. Notebooks from participants who consented to public sharing are provided in the supplementary information. The GitHub repository for the hackathon is available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/atomgptlab\/aai4science\">https:\/\/github.com\/atomgptlab\/aai4science<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1088\/2632-2153\/ae7f6a","type":"journal-article","created":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T22:53:33Z","timestamp":1781823213000},"page":"040501","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Lessons learned from the 2025 agentic AI for science hackathon"],"prefix":"10.1088","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4280-9144","authenticated-orcid":true,"given":"Jaehyung","family":"Lee","sequence":"first","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-3269-1005","authenticated-orcid":true,"given":"Harichandana","family":"Neralla","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charles","family":"Rhys Campbell","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"},{"name":"West Virginia University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-3289-8747","authenticated-orcid":true,"given":"Kent","family":"Zhang","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5214-183X","authenticated-orcid":true,"given":"Akshaya","family":"Ajith","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Ely","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-6592-2275","authenticated-orcid":true,"given":"Hua","family":"Jiang","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9150-1805","authenticated-orcid":true,"given":"Zhijian","family":"Zhou","sequence":"additional","affiliation":[{"name":"University of Pennsylvania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6864-220X","authenticated-orcid":true,"given":"Nikhil","family":"Ramlukan","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaolong","family":"Li","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0042-7958","authenticated-orcid":true,"given":"Huat","family":"Thart Chiang","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arijit","family":"Mahesh Kulkarni","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9638-3100","authenticated-orcid":true,"given":"Amruthesh","family":"Thirumalaiswamy","sequence":"additional","affiliation":[{"name":"Arizona State University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8442-6388","authenticated-orcid":true,"given":"Sampanna","family":"Pahi","sequence":"additional","affiliation":[{"name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7705-984X","authenticated-orcid":true,"given":"Md","family":"Habibur Rahman","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4065-8443","authenticated-orcid":true,"given":"Raghunandan","family":"Pratoori","sequence":"additional","affiliation":[{"name":"Iowa State University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erwei","family":"Huang","sequence":"additional","affiliation":[{"name":"Schr\u00f6dinger Inc"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asif","family":"Iqbal Bhatti","sequence":"additional","affiliation":[{"name":"Vrije University Brussels"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4037-2207","authenticated-orcid":true,"given":"Prem","family":"Prabhakaran","sequence":"additional","affiliation":[{"name":"Hannam University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8435-5551","authenticated-orcid":true,"given":"Manish","family":"Kumar","sequence":"additional","affiliation":[{"name":"Washington University in St. Louis"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5968-0571","authenticated-orcid":true,"given":"Aldo H","family":"Romero","sequence":"additional","affiliation":[{"name":"West Virginia University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Sun","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6067-6051","authenticated-orcid":true,"given":"Christopher","family":"Stiles","sequence":"additional","affiliation":[{"name":"Johns Hopkins Applied Physics Laboratory"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chinmay","family":"Maheshwari","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7645-009X","authenticated-orcid":true,"given":"Peirong","family":"Liu","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4038-8620","authenticated-orcid":true,"given":"Laixi","family":"Shi","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9737-8074","authenticated-orcid":false,"given":"Kamal","family":"Choudhary","sequence":"additional","affiliation":[{"name":"Johns Hopkins University"},{"name":"Johns Hopkins University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"key":"mlstae7f6abib1","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1038\/s41586-023-06792-0","type":"journal-article","article-title":"Autonomous chemical research with large language models","volume":"624","author":"Boiko","year":"2023","journal-title":"Nature"},{"key":"mlstae7f6abib2","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1038\/s42256-024-00832-8","type":"journal-article","article-title":"Augmenting large language models with chemistry tools","volume":"6","author":"Bran","year":"2024","journal-title":"Nat. Mach. Intell."},{"key":"mlstae7f6abib3","article-title":"LLMatDesign: autonomous materials discovery with large language models","author":"Jia","year":"2024","type":"preprint"},{"key":"mlstae7f6abib4","doi-asserted-by":"publisher","first-page":"962","DOI":"10.1038\/s43588-025-00849-y","type":"journal-article","article-title":"SciToolAgent: a knowledge-graph-driven scientific agent for multitool integration","volume":"5","author":"Ding","year":"2025","journal-title":"Nat. Comput. Sci."},{"key":"mlstae7f6abib5","article-title":"MatExpert decomposing materials discovery by mimicking human experts","author":"Ding","year":"2024","type":"preprint"},{"key":"mlstae7f6abib6","article-title":"MatSciBench: benchmarking the reasoning ability of large language models in materials science","author":"Zhang","year":"2025","type":"preprint"},{"key":"mlstae7f6abib7","article-title":"DREAMS: density functional theory based research engine for agentic materials simulation","author":"Wang","year":"2025","type":"preprint"},{"key":"mlstae7f6abib8","article-title":"MDCrow: automating molecular dynamics workflows with large language models","author":"Campbell","year":"2025","type":"preprint"},{"key":"mlstae7f6abib9","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.findings-emnlp.192","type":"preprint","article-title":"HoneyComb: a flexible LLM-based agent system for materials science","author":"Zhang","year":"2024"},{"key":"mlstae7f6abib10","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2414074122","type":"journal-article","article-title":"Automating alloy design and discovery with physics-aware multimodal multiagent AI","volume":"122","author":"Ghafarollahi","year":"2025","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"mlstae7f6abib11","doi-asserted-by":"publisher","DOI":"10.1016\/j.matt.2025.102263","type":"journal-article","article-title":"El Agente: an autonomous agent for quantum chemistry","volume":"8","author":"Zou","year":"2025","journal-title":"Matter"},{"key":"mlstae7f6abib12","article-title":"AGAPI-Agents: an open-access agentic AI platform for accelerated materials design on AtomGPT.org","author":"Lee","year":"2025","type":"preprint"},{"key":"mlstae7f6abib13","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1007\/s40192-025-00410-9","type":"journal-article","article-title":"ChatGPT material explorer: design and implementation of a custom GPT assistant for materials science applications","volume":"14","author":"Choudhary","year":"2025","journal-title":"Integr. Mater. Manuf. Innov."},{"key":"mlstae7f6abib14","doi-asserted-by":"publisher","first-page":"6909","DOI":"10.1021\/acs.jpclett.4c01126","type":"journal-article","article-title":"AtomGPT: atomistic generative pretrained transformer for forward and inverse materials design","volume":"15","author":"Choudhary","year":"2024","journal-title":"J. Phys. Chem. Lett."},{"key":"mlstae7f6abib15","doi-asserted-by":"publisher","first-page":"2110","DOI":"10.1021\/acs.jpclett.4c03137","type":"journal-article","article-title":"DiffractGPT: atomic structure determination from x-ray diffraction patterns using a generative pretrained transformer","volume":"16","author":"Choudhary","year":"2025","journal-title":"J. Phys. Chem. Lett."},{"key":"mlstae7f6abib16","doi-asserted-by":"publisher","first-page":"7028","DOI":"10.1021\/acs.jpclett.5c01257","type":"journal-article","article-title":"MicroscopyGPT: generating atomic-structure captions from microscopy images of 2D materials with vision-language transformers","volume":"16","author":"Choudhary","year":"2025","journal-title":"J. Phys. Chem. Lett."},{"key":"mlstae7f6abib17","article-title":"Survey on evaluation of LLM-based agents","author":"Yehudai","year":"2025","type":"preprint"},{"key":"mlstae7f6abib18","article-title":"Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey","author":"Zhu","year":"2025","type":"preprint"},{"key":"mlstae7f6abib19","article-title":"AgentBench: Evaluating LLMs as Agents","author":"Liu","year":"2023","type":"preprint"},{"key":"mlstae7f6abib20","article-title":"Introducing the model context protocol(Anthropic news)","author":"Anthropic","year":"2024","type":"other"},{"key":"mlstae7f6abib21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2506.08423","type":"journal-article","article-title":"Mic-hackathon 2024: Hackathon on machine learning for electron and scanning probe microscopy","volume":"6","author":"Pratiush","year":"2025","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstae7f6abib22","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1039\/D3DD00113J","type":"journal-article","article-title":"14 examples of how LLMs can transform materials science and chemistry: a reflection on a large language model hackathon","volume":"2","author":"Jablonka","year":"2023","journal-title":"Digit. Discovery"},{"key":"mlstae7f6abib23","doi-asserted-by":"crossref","DOI":"10.1080\/10447318.2022.2075601","type":"preprint","article-title":"The developers\u2019 design thinking toolbox in hackathons: a study on the recurring design methods in software development marathons","author":"Gama","year":"2023"},{"key":"mlstae7f6abib24","article-title":"Short datathon for the interdisciplinary development of data analysis and visualization skills","author":"Salinas","year":"2019","type":"preprint"},{"key":"mlstae7f6abib25","doi-asserted-by":"publisher","DOI":"10.1080\/2331186X.2024.2392420","type":"journal-article","article-title":"Evaluation of the impact of hackathons in education","volume":"11","author":"Oyetade","year":"2024","journal-title":"Cogent Educ."},{"key":"mlstae7f6abib26","first-page":"206:1","type":"journal-article","article-title":"On Hackathons: a multidisciplinary literature review","volume":"54","author":"Chau","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"mlstae7f6abib27","doi-asserted-by":"publisher","first-page":"488","DOI":"10.3390\/info16060488","type":"journal-article","article-title":"Driving international collaboration beyond boundaries through Hackathons: a comparative analysis of four Hackathon setups","volume":"16","author":"Barana","year":"2025","journal-title":"Information"},{"key":"mlstae7f6abib28","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1101\/gr.228460.117","type":"journal-article","article-title":"Hackathons as a means of accelerating scientific discoveries and knowledge transfer","volume":"28","author":"Ghouila","year":"2018","journal-title":"Genome Res."},{"key":"mlstae7f6abib29","article-title":"The MICCAI Hackathon on reproducibility, diversity, and selection of papers at the MICCAI conference","author":"Balsiger","year":"2021","type":"preprint"},{"key":"mlstae7f6abib30","doi-asserted-by":"publisher","first-page":"e6","DOI":"10.1017\/eds.2024.35","type":"journal-article","article-title":"Improving the reproducibility in geoscientific papers: lessons learned from a hackathon in climate science","volume":"4","author":"Coca-Castro","year":"2024","journal-title":"Environ. Data Sci."},{"key":"mlstae7f6abib31","article-title":"Reflections from the 2024 large language model (LLM) hackathon for applications in materials science and chemistry","author":"Zimmermann","year":"2024","type":"preprint"},{"key":"mlstae7f6abib32","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1038\/s41524-020-00440-1","type":"journal-article","article-title":"The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design","volume":"6","author":"Choudhary","year":"2020","journal-title":"npj Comput. Mater."},{"key":"mlstae7f6abib33","doi-asserted-by":"publisher","DOI":"10.1016\/j.commatsci.2025.114063","type":"journal-article","article-title":"The JARVIS infrastructure is all you need for materials design","volume":"259","author":"Choudhary","year":"2025","journal-title":"Comput. Mater. Sci."},{"key":"mlstae7f6abib34","doi-asserted-by":"publisher","DOI":"10.1016\/j.mtphys.2024.101560","type":"journal-article","article-title":"Improving machine-learning models in materials science through large datasets","volume":"48","author":"Schmidt","year":"2024","journal-title":"Mater. Today Phys."},{"key":"mlstae7f6abib35","article-title":"The future of Hackathon research and practice","author":"Falk","year":"2022","type":"preprint"},{"key":"mlstae7f6abib36","article-title":"AAI4Science (Agentic AI for Science Hackathon)","author":"AtomGPTLab","year":"2025","type":"web-resource"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T05:27:03Z","timestamp":1783574823000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ae7f6a"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,9]]},"references-count":36,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,7,9]]},"published-print":{"date-parts":[[2026,8,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ae7f6a","relation":{"has-preprint":[{"id-type":"doi","id":"10.26434\/chemrxiv.15001345\/v1","asserted-by":"object"}]},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,9]]},"assertion":[{"value":"Lessons learned from the 2025 agentic AI for science hackathon","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2026 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2026-03-04","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-06-18","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2026-07-09","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}