{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T09:03:05Z","timestamp":1775725385423,"version":"3.50.1"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T00:00:00Z","timestamp":1774483200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"},{"start":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T00:00:00Z","timestamp":1775692800000},"content-version":"vor","delay-in-days":14,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"DOI":"10.1038\/s41598-026-42829-w","type":"journal-article","created":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T12:37:09Z","timestamp":1774528629000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Zero-shot performance of selected large language and multimodal models on the 2023 Brazilian Portuguese medical residency exam"],"prefix":"10.1038","volume":"16","author":[{"given":"C\u00e9sar Augusto Madid","family":"Truyts","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Amanda Gomes","family":"Rabelo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriel Mesquita de","family":"Souza","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel Scaldaferri","family":"Lages","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adriano Jos\u00e9","family":"Pereira","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Uri Adrian Prync","family":"Flato","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eduardo Pontes dos","family":"Reis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joaquim Edson","family":"Vieira","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paulo Sergio Panse","family":"Silveira","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edson Amaro","family":"Junior","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,3,26]]},"reference":[{"key":"42829_CR1","doi-asserted-by":"publisher","unstructured":"Sarkar, C. et al. Artificial intelligence and machine learning technology driven modern drug discovery and development. Int. J. Mol. Sci. 24. https:\/\/doi.org\/10.3390\/ijms24032026 (2023).","DOI":"10.3390\/ijms24032026"},{"key":"42829_CR2","doi-asserted-by":"publisher","unstructured":"Shah, N.H., Entwistle, D., & Pfeffer, M.A. Creation and adoption of large language models in medicine. JAMA. 330. https:\/\/doi.org\/10.1001\/jama.2023.14217 (2023).","DOI":"10.1001\/jama.2023.14217"},{"key":"42829_CR3","doi-asserted-by":"publisher","unstructured":"Haug, C. J. & Drazen, J. M. Artificial intelligence and machine learning in clinical medicine. 2023. N. Engl. J. Med.388. https:\/\/doi.org\/10.1056\/nejmra2302038 (2023).","DOI":"10.1056\/nejmra2302038"},{"key":"42829_CR4","doi-asserted-by":"publisher","unstructured":"Marafino, B. J. et al. Validation of prediction models for critical care outcomes using natural language processing of electronic health record data. JAMA Netw. Open. 1. https:\/\/doi.org\/10.1001\/jamanetworkopen.2018.5097 (2018).","DOI":"10.1001\/jamanetworkopen.2018.5097"},{"key":"42829_CR5","doi-asserted-by":"publisher","unstructured":"Moskovitch, R., Polubriaginof, F., Weiss, A., Ryan, P. & Tatonetti, N. Procedure prediction from symbolic Electronic Health Records via time intervals analytics. J Biomed Inform. 75. https:\/\/doi.org\/10.1016\/j.jbi.2017.07.018 (2017).","DOI":"10.1016\/j.jbi.2017.07.018"},{"key":"42829_CR6","doi-asserted-by":"publisher","unstructured":"Harutyunyan, H., Khachatrian, H., Kale, D. C., Steeg, G. V. & Galstyan, A. Multitask learning and benchmarking with clinical time series data. Sci Data. 6. https:\/\/doi.org\/10.1038\/s41597-019-0103-9 (2019).","DOI":"10.1038\/s41597-019-0103-9"},{"key":"42829_CR7","doi-asserted-by":"publisher","unstructured":"Bommasani, R., Liang, P. & Lee, T. Holistic Evaluation of Language Models. Ann. N Y Acad. Sci. 1525. https:\/\/doi.org\/10.1111\/nyas.15007 (2023).","DOI":"10.1111\/nyas.15007"},{"key":"42829_CR8","doi-asserted-by":"publisher","unstructured":"Singhal, K. et al. Large language models encode clinical knowledge. Nature. 620. https:\/\/doi.org\/10.1038\/s41586-023-06291-2 (2023).","DOI":"10.1038\/s41586-023-06291-2"},{"key":"42829_CR9","unstructured":"Srivastava, A., Rastogi, A., Rao, A., Shoeb, A.A.M., Abid, A., Fisch, A., et al.: Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. Available from: (2023). https:\/\/arxiv.org\/abs\/2206.04615."},{"key":"42829_CR10","unstructured":"Nori, H., Lee, Y.T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., et al. Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine; 2023. Available from: https:\/\/arxiv.org\/abs\/2311.16452."},{"key":"42829_CR11","first-page":"e240476","volume-title":"RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering","author":"S Tayebi Arasteh","year":"2025","unstructured":"Tayebi Arasteh, S. et al. RadioRAG: Online Retrieval-augmented Generation for Radiology Question Answering e240476 (Artificial Intelligence, Radiology, 2025)."},{"key":"42829_CR12","doi-asserted-by":"crossref","unstructured":"Zakka, C., Shad, R., Chaurasia, A., Dalal, A.R., Kim, J.L., Moor, M., et al. Almanac\u2013retrieval-augmented language models for clinical medicine. NEJM AI. 1(2), AIoa2300068 (2024).","DOI":"10.1056\/AIoa2300068"},{"key":"42829_CR13","unstructured":"Wind, S., Sopa, J., Truhn, D., Lotfinia, M., Nguyen, T.T., Bressem, K., et al. Agentic large language models improve retrieval-based radiology question answering. arXiv preprint arXiv:2508.00743. (2025)."},{"key":"42829_CR14","doi-asserted-by":"crossref","unstructured":"Plaat, A., van Duijn, M., van Stein, N., Preuss, M., van der Putten, P., Batenburg, K.J. Agentic large language models: a survey. arXiv preprint arXiv:2503.23037. (2025).","DOI":"10.1613\/jair.1.18675"},{"key":"42829_CR15","unstructured":"Ranathunga, S., de Silva, N.: Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World (2022). Available from: https:\/\/arxiv.org\/abs\/2210.08523."},{"key":"42829_CR16","unstructured":"Wikipedia. Portuguese-speaking world \u2014 Wikipedia, The Free Encyclopedia; 2024. Available from: https:\/\/en.wikipedia.org\/wiki\/Portuguese-speaking_world."},{"key":"42829_CR17","unstructured":"Wikipedia. List of languages by number of native speakers; 2025. Accessed: 2025\u201301-11. Available from: https:\/\/en.wikipedia.org\/wiki\/List_of_languages_by_number_of_native_speakers."},{"key":"42829_CR18","unstructured":"W3Techs. Usage Statistics of Content Languages for Websites; 2025. Accessed: 2025\u201301-11. Available from: https:\/\/w3techs.com\/technologies\/overview\/content_language."},{"key":"42829_CR19","doi-asserted-by":"publisher","unstructured":"Lorenzoni, G., Gregori, D., Bressan, S., Ocagli, H., Azzolina, D., Dalt, L.D., et al. Use of a large language model to identify and classify injuries with free-text emergency department data. JAMA Netw Open. 5(7), e2413208\u20138. https:\/\/doi.org\/10.1001\/jamanetworkopen.2024.13208 (2024).","DOI":"10.1001\/jamanetworkopen.2024.13208"},{"key":"42829_CR20","doi-asserted-by":"publisher","unstructured":"Liu, M. et al. Performance of ChatGPT across different versions in medical licensing examinations worldwide: systematic review and meta-analysis. J. Med. Internet Res. 26, e60807. https:\/\/doi.org\/10.2196\/60807 (2024).","DOI":"10.2196\/60807"},{"key":"42829_CR21","doi-asserted-by":"publisher","unstructured":"Frei, J. & Kramer, F. Annotated dataset creation through large language models for non-english medical NLP. J. Biomed. Inform. 145. https:\/\/doi.org\/10.1016\/j.jbi.2023.104478 (2023).","DOI":"10.1016\/j.jbi.2023.104478"},{"key":"42829_CR22","unstructured":"Garcia, G.L., Paiola, P.H., Morelli, L.H., Candido, G., J\u00fanior, A.C., Jodas, D.S., et al. Introducing Bode: A Fine-Tuned Large Language Model for Portuguese Prompt-Based Task. Available from: https:\/\/arxiv.org\/abs\/2401.02909 (2024)."},{"key":"42829_CR23","unstructured":"Almeida, T.S., Abonizio, H., Nogueira, R., Pires, R. Sabi\u00e1-2: A New Generation of Portuguese Large Language Models (2024). Available from: https:\/\/arxiv.org\/abs\/2403.09887."},{"key":"42829_CR24","doi-asserted-by":"publisher","unstructured":"Guillen-Grima, F. et al. Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine. Clin. Pract.13, 35. https:\/\/doi.org\/10.3390\/clinpract13060130 (2023).","DOI":"10.3390\/clinpract13060130"},{"key":"42829_CR25","unstructured":"Mart\u00edn-Sanjos\u00e9, J.F., Juan, M.C., Viv\u00f3, R., Abad, F. The effects of images on multiple-choice questions in computer-based formative assessment. Digit. Educ. Rev. (2015)."},{"key":"42829_CR26","first-page":"14","volume-title":"Owen C","author":"MG Sagoo","year":"2021","unstructured":"Sagoo, M. G., Vorstenbosch, M. A. T. M., Bazira, P. J., Ellis, H. & Kambouri, M. Owen C 14 (The Effect of Images on Medical Students\u2019 Performance. Anat Sci Educ, Online Assessment of Applied Anatomy Knowledge, 2021)."},{"key":"42829_CR27","doi-asserted-by":"publisher","unstructured":"Wang, Z., Ardasheva, Y., Carbonneau, K. & Liu, Q. Testing the seductive details effect: Does the format or the amount of seductive details matter? Appl Cogn Psychol.35, 25. https:\/\/doi.org\/10.1002\/acp.3801 (2021).","DOI":"10.1002\/acp.3801"},{"key":"42829_CR28","doi-asserted-by":"publisher","unstructured":"Pouw, W., Rop, G., de Koning, B. & Paas, F. The Cognitive Basis for the Split-Attention Effect. J Exp Psychol Gen. 148. https:\/\/doi.org\/10.1037\/xge0000578 (2019).","DOI":"10.1037\/xge0000578"},{"key":"42829_CR29","doi-asserted-by":"publisher","unstructured":"Crisp, V. & Sweiry, E. Can a picture ruin a thousand words? The effects of visual resources in exam questions. Educ. Res. 48. https:\/\/doi.org\/10.1080\/00131880600732249 (2006).","DOI":"10.1080\/00131880600732249"},{"key":"42829_CR30","doi-asserted-by":"publisher","unstructured":"Kung, T. H. et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PloS Digit Health. 2. https:\/\/doi.org\/10.1371\/journal.pdig.0000198 (2023).","DOI":"10.1371\/journal.pdig.0000198"},{"key":"42829_CR31","doi-asserted-by":"publisher","unstructured":"Chan, S. et al. Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol. Ther. 10. https:\/\/doi.org\/10.1007\/s13555-020-00372-0 (2020).","DOI":"10.1007\/s13555-020-00372-0"},{"key":"42829_CR32","doi-asserted-by":"publisher","unstructured":"Du, A. X., Emam, S. & Gniadecki, R. Review of machine learning in predicting dermatological outcomes. Front Med. 7. https:\/\/doi.org\/10.3389\/fmed.2020.00266 (2020).","DOI":"10.3389\/fmed.2020.00266"},{"key":"42829_CR33","doi-asserted-by":"publisher","unstructured":"Hogarty, D. T. et al. Artificial intelligence in dermatology-where we are and the way to the future: A review. Am. J. Clin. Dermatol. 21. https:\/\/doi.org\/10.1007\/s40257-019-00462-6 (2020).","DOI":"10.1007\/s40257-019-00462-6"},{"key":"42829_CR34","doi-asserted-by":"crossref","unstructured":"Panagoulias, D.P., Tsoureli-Nikita, E., Virvou, M., Tsihrintzis, G.A. Dermacen Analytica: A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatology (2024). arxiv: https:\/\/arxiv.org\/abs\/2403.14243.","DOI":"10.1016\/j.ijmedinf.2025.105898"},{"key":"42829_CR35","doi-asserted-by":"publisher","unstructured":"Kelly, B.S., Judge, C., Bollard, S.M., Clifford, S.M., Healy, G.M., Aziz, A., et al. Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE). Eur. Radiol. 32. https:\/\/doi.org\/10.1007\/s00330-022-08784-6 (2022).","DOI":"10.1007\/s00330-022-08784-6"},{"key":"42829_CR36","doi-asserted-by":"publisher","unstructured":"Katal, S., York, B., Gholamrezanezhad, A. A. I. & in radiology: From promise to practice - A guide to effective integration. Eur. J. Radiol. 12, 181. https:\/\/doi.org\/10.1016\/j.ejrad.2024.111798 (2024).","DOI":"10.1016\/j.ejrad.2024.111798"},{"key":"42829_CR37","unstructured":"Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., et al. Hallucination of Multimodal Large Language Models: A Survey (2024). Available from: arxiv: https:\/\/arxiv.org\/abs\/2404.18930."},{"key":"42829_CR38","doi-asserted-by":"publisher","unstructured":"Li, A. et al. Mitigating Hallucinations in Large Language Models: A Comparative Study of RAG-enhanced vs. Human-Generated Medical Templates. medRxiv. (2024). Available from: https:\/\/doi.org\/10.1101\/2024.09.27.24314506.","DOI":"10.1101\/2024.09.27.24314506"},{"key":"42829_CR39","unstructured":"Gu, Z., Yin, C., Liu, F., Zhang, P. MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context (2024). Available from: arxiv: https:\/\/arxiv.org\/abs\/2407.02730."},{"key":"42829_CR40","doi-asserted-by":"crossref","unstructured":"Ngo, A. et al. ChatGPT 3.5 fails to write appropriate multiple choice practice exam questions. Acad Pathol. 11, (2024).","DOI":"10.1016\/j.acpath.2023.100099"},{"key":"42829_CR41","doi-asserted-by":"publisher","unstructured":"Cecil, J., Lermer, E., Hudecek, M. F. C., Sauer, J. & Gaube, S. Explainability does not mitigate the negative impact of incorrect AI advice in a personnel selection task. Sci Rep. 14, 9736. https:\/\/doi.org\/10.1038\/s41598-024-60220-5 (2024).","DOI":"10.1038\/s41598-024-60220-5"},{"key":"42829_CR42","doi-asserted-by":"publisher","unstructured":"Osama, M., Afridi, S. & Maaz, M. ChatGPT: Transcending Language Limitations in Scientific Research Using Artificial Intelligence. J. Coll. Phys. Surg. Pak.33. https:\/\/doi.org\/10.29271\/jcpsp.2023.10.1198 (2023).","DOI":"10.29271\/jcpsp.2023.10.1198"},{"key":"42829_CR43","unstructured":"Bedi, S., Cui, H., Fuentes, M., Unell, A., Wornow, M., Banda, J.M., et al. MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks (2025). Available from: arxiv: https:\/\/arxiv.org\/abs\/2505.23802."},{"key":"42829_CR44","unstructured":"FUVEST. Resid\u00eancia M\u00e9dica 2024 - FUVEST divulga quest\u00f5es de prova objetiva de concurso para resid\u00eancia m\u00e9dica da FMUSP - Fuvest; 2024. Available from: https:\/\/www.fuvest.br\/residencia-medica-2024-fuvest-divulga-questoes-de-prova-objetiva-de-concurso-para-residencia-medica-da-fmusp\/."},{"key":"42829_CR45","doi-asserted-by":"publisher","unstructured":"Ali, R., Tang, O.Y., Connolly, I.D., Fridley, J.S., Shin, J.H., Sullivan, P.L.Z., et al. Performance of ChatGPT, GPT-4, and Google Bard on a Neurosurgery Oral Boards Preparation Question Bank. Neurosurgery. 93(5). https:\/\/doi.org\/10.1227\/neu.0000000000002551 (2023).","DOI":"10.1227\/neu.0000000000002551"},{"key":"42829_CR46","doi-asserted-by":"publisher","unstructured":"Garabet, R., Mackey, B. P., Cross, J. & Weingarten, M. ChatGPT-4 Performance on USMLE Step 1 Style Questions and Its Implications for Medical Education: A Comparative Study Across Systems and Disciplines. Med. Sci. Educ. 34. https:\/\/doi.org\/10.1007\/s40670-023-01956-z (2024).","DOI":"10.1007\/s40670-023-01956-z"},{"key":"42829_CR47","doi-asserted-by":"publisher","unstructured":"Mihalache, A., Huang, R. S., Popovic, M. M. & Muni, R. H. ChatGPT-4: An assessment of an upgraded artificial intelligence chatbot in the United States Medical Licensing Examination. Med. Teacher.46. https:\/\/doi.org\/10.1080\/0142159X.2023.2249588 (2024).","DOI":"10.1080\/0142159X.2023.2249588"},{"key":"42829_CR48","unstructured":"Lin, H. hausekeep: A Collection of Utility Functions for Data Science and Statistics - Compute standard errors (within-subjects);. Available from: https:\/\/hauselin.github.io\/hausekeep\/reference\/seWithin.html."},{"key":"42829_CR49","unstructured":"Lenth, R.V. emmeans: Estimated marginal means, aka least-squares means. R package (version 1.7.1). R Foundation for Statistical Computing. 34,(2021)."},{"key":"42829_CR50","unstructured":"Team, R.C. R: A Language and Environment for Statistical Computing (2024). Available from: https:\/\/www.R-project.org\/."},{"issue":"50","key":"42829_CR51","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1002\/bimj.200810425","volume":"6","author":"T Hothorn","year":"2008","unstructured":"Hothorn, T., Bretz, F. & Westfall, P. Simultaneous inference in general parametric models. Biometr. J. 6(50), 346\u2013363. https:\/\/doi.org\/10.1002\/bimj.200810425 (2008).","journal-title":"Biometr. J."},{"key":"42829_CR52","doi-asserted-by":"publisher","unstructured":"Silveira, P. S. P. & Siqueira, J. O. Better to be in agreement than in bad company: A critical analysis of many kappa-like tests. Behav Res Methods.. https:\/\/doi.org\/10.3758\/s13428-022-01950-0 (2022).","DOI":"10.3758\/s13428-022-01950-0"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-42829-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-42829-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-42829-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T08:11:48Z","timestamp":1775722308000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-42829-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,26]]},"references-count":52,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["42829"],"URL":"https:\/\/doi.org\/10.1038\/s41598-026-42829-w","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,26]]},"assertion":[{"value":"4 August 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"11756"}}