{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T12:04:13Z","timestamp":1772021053732,"version":"3.50.1"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686080","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T00:00:00Z","timestamp":1754524800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,8,7]]},"abstract":"<jats:p>In this study, we examined how well the open-source foundational large language models (LLMs) can extract symptoms and signs (S&amp;S), along with their corresponding ICD-10 codes, from clinical notes found in the public MTSamples dataset. The dataset comprising notes of patients with genitourinary conditions was manually annotated to compare the S&amp;S extraction results with outputs generated by LLMs. We assessed three versions of the Llama model\u2014Llama 3.1-13B, Llama 3.3-70B, and Me-Llama-13B\u2014focusing on their consistency, runtime, and performance. Each model was tested on two tasks: (1) S&amp;S extraction and (2) ICD-10 code generation. Our findings indicate that Llama 3.3-70B performed the best overall. With fast runtime and high consistency, it achieved an average recall of 0.87 and an average precision of 0.71 for S&amp;S extraction, as well as an average recall of 0.71 and an average precision of 0.54 for ICD-10 code generation.<\/jats:p>","DOI":"10.3233\/shti250923","type":"book-chapter","created":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T11:35:17Z","timestamp":1754566517000},"source":"Crossref","is-referenced-by-count":1,"title":["Performance of Open-Source Large Language Models to Extract Symptoms from Clinical Notes"],"prefix":"10.3233","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4910-4497","authenticated-orcid":false,"given":"Yunbing","family":"Bai","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah"}]},{"given":"Wanting","family":"Cui","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah"}]},{"given":"Joseph","family":"Finkelstein","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah"}]}],"member":"7437","container-title":["Studies in Health Technology and Informatics","MEDINFO 2025 \u2014 Healthcare Smart \u00d7 Medicine Deep"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/SHTI250923","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,7]],"date-time":"2025-08-07T11:35:18Z","timestamp":1754566518000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI250923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,7]]},"ISBN":["9781643686080"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/shti250923","relation":{},"ISSN":["0926-9630","1879-8365"],"issn-type":[{"value":"0926-9630","type":"print"},{"value":"1879-8365","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,7]]}}}