{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T12:46:53Z","timestamp":1782910013401,"version":"3.54.5"},"reference-count":13,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T00:00:00Z","timestamp":1774828800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100013362","name":"Swiss Cancer Research","doi-asserted-by":"publisher","award":["KFS-6477-08-2025-R"],"award-info":[{"award-number":["KFS-6477-08-2025-R"]}],"id":[{"id":"10.13039\/501100013362","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>To quantify run-to-run reproducibility of Gemini 3 Flash Preview and GPT-5.2 for trial-success classification across temperature and reasoning\/thinking settings and determine whether single-run reporting suffices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We utilized 250 trial abstracts labeled based on primary endpoint success. We evaluated Gemini across thinking levels (minimal, low, medium, high) and temperatures 0.0-2.0 and GPT-5.2 across reasoning-effort levels (none to x-high) with an additional temperature sweep when reasoning was disabled. Each setting was run 3 times.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Reproducibility was high for Gemini (\u03ba\u2009=\u20090.942-1.000; invalid outputs 0%-1.5%) and GPT-5.2 (\u03ba\u2009=\u20090.984-0.995; no invalid outputs). F1 remained stable (mean\/majority vote 0.955-0.971), with marginal gains from majority voting.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>For binary biomedical classification with tightly constrained outputs, both models were reproducible across decoding and reasoning settings, suggesting single runs are often sufficient, with minimal replication as a practical stability check.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocag039","type":"journal-article","created":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T18:17:10Z","timestamp":1774894630000},"page":"1179-1184","source":"Crossref","is-referenced-by-count":3,"title":["Is one run enough? Reproducibility of flagship large language models across temperature and reasoning settings in biomedical text processing"],"prefix":"10.1093","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1040-4888","authenticated-orcid":false,"given":"Paul","family":"Windisch","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, Cantonal Hospital Winterthur , Winterthur, 8401,","place":["Switzerland"]},{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carole","family":"Koechli","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Dennst\u00e4dt","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel M","family":"Aebersold","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel R","family":"Zwahlen","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Cantonal Hospital Winterthur , Winterthur, 8401,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"F\u00f6rster","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Cantonal Hospital Winterthur , Winterthur, 8401,","place":["Switzerland"]},{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christina","family":"Schr\u00f6der","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Cantonal Hospital Winterthur , Winterthur, 8401,","place":["Switzerland"]},{"name":"Department of Radiation Oncology, Inselspital, Bern University Hospital, University of Bern , Bern, 3012,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,3,30]]},"reference":[{"key":"2026052218573557500_ocag039-B1","doi-asserted-by":"crossref","first-page":"100887","DOI":"10.1016\/j.patter.2023.100887","article-title":"Enhancing phenotype recognition in clinical notes using large language models: PhenoBCBERT and PhenoGPT","volume":"5","author":"Yang","year":"2024","journal-title":"Patterns (N Y)"},{"key":"2026052218573557500_ocag039-B2","doi-asserted-by":"crossref","first-page":"9074","DOI":"10.1038\/s41467-024-53081-z","article-title":"Matching patients to clinical trials with large language models","volume":"15","author":"Jin","year":"2024","journal-title":"Nat Commun."},{"key":"2026052218573557500_ocag039-B3","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1002\/jrsm.1715","article-title":"Can large language models replace humans in systematic reviews? Evaluating GPT-4\u2019s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages","volume":"15","author":"Khraisha","year":"2024","journal-title":"Res Synth Methods."},{"key":"2026052218573557500_ocag039-B4","doi-asserted-by":"crossref","first-page":"e57978","DOI":"10.2196\/57978","article-title":"The evaluation of generative AI should include repetition to assess stability","volume":"12","author":"Zhu","year":"2024","journal-title":"JMIR MHealth UHealth."},{"key":"2026052218573557500_ocag039-B5","doi-asserted-by":"crossref","first-page":"e53559","DOI":"10.2196\/53559","article-title":"The temperature feature of ChatGPT: modifying creativity for clinical research","volume":"11","author":"Davis","year":"2024","journal-title":"JMIR Hum Factors."},{"key":"2026052218573557500_ocag039-B6","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1161\/CIRCRESAHA.114.303819","article-title":"Reproducibility in science: improving the standard for basic and preclinical research","volume":"116","author":"Begley","year":"2015","journal-title":"Circ Res."},{"key":"2026052218573557500_ocag039-B7","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1186\/s12909-026-08656-3","article-title":"Consistency over accuracy: run-to-run stability of contemporary large language models on Turkish curriculum-aligned theoretical anatomy multiple-choice questions","volume":"26","author":"G\u00fcrses","year":"2026","journal-title":"BMC Med Educ."},{"key":"2026052218573557500_ocag039-B8","first-page":"1","article-title":"Variation in large language model recommendations in challenging inpatient management scenarios","author":"Landon","year":"2025","journal-title":"J Gen Intern Med."},{"key":"2026052218573557500_ocag039-B9","doi-asserted-by":"crossref","first-page":"222","DOI":"10.2519\/jospt.2024.12151","article-title":"Performance of ChatGPT compared to clinical practice guidelines in making informed decisions for lumbosacral radicular pain: a cross-sectional study","volume":"54","author":"Gianola","year":"2024","journal-title":"J Orthop Sports Phys Ther."},{"key":"2026052218573557500_ocag039-B10","doi-asserted-by":"crossref","first-page":"e101139","DOI":"10.1136\/bmjhci-2024-101139","article-title":"Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation","volume":"32","author":"Ntinopoulos","year":"2025","journal-title":"BMJ Health Care Inform"},{"key":"2026052218573557500_ocag039-B11","first-page":"e59857","article-title":"Accuracy and repeatability of ChatGPT based on a set of multiple-choice questions on objective tests of hearing","volume":"16","author":"Kochanek","year":"2024","journal-title":"Cureus"},{"key":"2026052218573557500_ocag039-B12","first-page":"657","article-title":"ChatGPT\u2019s response consistency: a study on repeated queries of medical examination questions","volume":"14","author":"Funk","year":"2024","journal-title":"Eur J Investig Health Psychol Educ."},{"key":"2026052218573557500_ocag039-B13","doi-asserted-by":"publisher","DOI":"10.1097\/JNC.0000000000000545","article-title":"Assessing variability in ChatGPT responses: a case study on simulating online user inputs","author":"Hswen","year":"2025","journal-title":"J Assoc Nurses AIDS Care"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocag039\/67659220\/ocag039.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/6\/1179\/67659220\/ocag039.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/6\/1179\/67659220\/ocag039.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T22:57:44Z","timestamp":1779490664000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/33\/6\/1179\/8559659"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,30]]},"references-count":13,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,3,30]]},"published-print":{"date-parts":[[2026,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocag039","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,6]]},"published":{"date-parts":[[2026,3,30]]}}}