{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T22:02:48Z","timestamp":1790114568852,"version":"4.0.1"},"reference-count":39,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Res Protoc"],"abstract":"\n Background<\/jats:title>\n Electronic medical charts within Workplace Safety and Insurance Board (WSIB) specialty programs contain rich but unstructured clinical and sociodemographic data essential for injury classification, treatment planning, and compensation decisions. Manual chart review is time-consuming, inconsistent, and susceptible to reviewer bias. Large language models (LLMs) extract complex information from unstructured clinical text, yet their application to workplace injury rehabilitation remains unexplored.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study will evaluate the accuracy, fairness, and methodological implications of using LLMs to extract clinical and rehabilitative information from WSIB medical charts at Trillium Health Partners (THP), Canada. We will assess model performance, examine algorithmic bias across demographic subgroups, and explore ethical implications for clinical decision-making and workplace compensation.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We will conduct a retrospective review of 50 medical charts from the WSIB Back and Neck specialty program at THP, spanning January 2018 to December 2024. General-purpose models (Qwen3-VL-8B and InternVL3.5-8B) and domain-specific biomedical models (MedGemma-27B and LLaMA-3-Meditron-8B) will be evaluated off the shelf using zero-shot and few-shot prompting; the biomedical models will also be fine-tuned. Charts will be partitioned at the chart level into development, training, and held-out test sets, preventing leakage across purposes. Ground truth will be established by independent human annotation of all 50 charts, with interannotator agreement quantified using Cohen \u03ba. The primary outcome is the macroaveraged F1-score across categorical variables on the held-out test set; secondary outcomes are the per variable F1-score, mean absolute error and root mean squared error for continuous variables, and span-level F1-score. Progression to the larger study requires a macroaveraged F1-score of at least 0.80, a pragmatic feasibility criterion; fairness and continuous-variable results are supporting outcomes. Algorithmic fairness will be examined using demographic parity and equalized odds across subgroups. A secure hybrid architecture will keep all identifiable personal health information within the THP infrastructure, with cloud compute restricted to transient processing.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n This study was funded by the Data Sciences Institute at the University of Toronto in April 2025; the award supported protocol development and ended on April 30, 2026. As of August 12, 2026, neither had a research ethics application been submitted nor had any chart been accessed. Applications to the THP and University of Toronto research ethics boards are anticipated in late 2026, and no data will be retrieved or processed before approval from both boards. Contingent on approval and further funding, data collection is anticipated through 2027, analysis in late 2027 to early 2028, and results submitted for publication in 2028.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This protocol describes the first systematic evaluation of LLM-based data extraction applied to workplace injury medical charts. Findings will inform best practices for the responsible, reproducible, and equitable integration of these tools into occupational health and workers\u2019 compensation decision-making.<\/jats:p>\n <\/jats:sec>\n \n International Registered Report Identifier (IRRID)<\/jats:title>\n PRR1-10.2196\/99807<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/99807","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T21:15:08Z","timestamp":1790111708000},"page":"e99807","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Models for Clinical Data Extraction in Workplace Injury Rehabilitation: Protocol for a Retrospective Pilot Study of Accuracy, Fairness, and Methodological Considerations in Workers\u2019 Compensation Medical Chart Review"],"prefix":"10.2196","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-4748-0071","authenticated-orcid":false,"given":"Armaan Rehman","family":"Shah","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0485-4654","authenticated-orcid":false,"given":"Barbara","family":"Gorczyca Abel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4695-3072","authenticated-orcid":false,"given":"Majid","family":"Komeili","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8131-1190","authenticated-orcid":false,"given":"Basem","family":"Gohar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5586-4282","authenticated-orcid":false,"given":"Behdin","family":"Nowrouzi-Kia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1007\/s10926-016-9690-x"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1007\/s10926-005-8038-8"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare11101403"},{"key":"ref4","unstructured":"WSIB specialty programsWorkplace Safety and Insurance Board20242026-03-15https:\/\/www.wsib.ca"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/dyw248"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.3352\/jeehp.2013.10.12"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btz682"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-1909"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-021-00455-y"},{"key":"ref12","unstructured":"BaiSCaiYChenRChenKChenXChengZDengLDingWGaoCGeCGeWGuoZHuangQHuangJHuangFHuiBJiangSLiZLiMQwen3-VL technical reportarXivPreprint posted online on November 26, 2025https:\/\/arxiv.org\/abs\/2511.21631"},{"key":"ref13","unstructured":"WangWGaoZGuLPuHCuiLWeiXLiuZJingLYeSShaoJWangZChenZZhangHYangGWangHInternVL3.5: advancing open-source multimodal models in versatility, reasoning, and efficiencyarXivPreprint posted online on August 25, 2025https:\/\/arxiv.org\/abs\/2508.18265"},{"key":"ref14","unstructured":"NoriHKingNMcKinneySMCarignanDHorvitzECapabilities of GPT-4 on medical challenge problemsarXivPreprint posted online on March 20, 2023https:\/\/arxiv.org\/abs\/2303.13375"},{"key":"ref15","unstructured":"SellergrenAKazemzadehSJaroensriTKiralyATraverseMKohlbergerTXuSJamilFHughesCLauCChenJMahvarFYatzivLChenTSterlingBBabySAMedGemma technical reportarXivPreprint posted online on July 7, 2025https:\/\/arxiv.org\/abs\/2507.05201"},{"key":"ref16","unstructured":"SallinenASolergibertA-JZhangMBoy\u00e9GDupont-RocMTheimer-LienhardXBoissonEBernathBHadhriHTranARabbaniTBrokowskiTMeditron Medical Doctor Working GroupRudnerTGJHartleyM-ALlama-3-Meditron: an open-weight suite of medical LLMs based on Llama-3.12025Workshop on Large Language Models and Generative AI for Health at AAAI 2025March 4, 2025Philadelphia, PA"},{"key":"ref17","unstructured":"ChenZCanoAHRomanouABonnetAMatobaKSalviFPagliardiniMFangSK\u00f6pfAMohtashamiASallinenASakhaeiradASwamyVKrawczukIBayazitDMarmetAMontariolSHartleyM-AJaggiMBosselutAMEDITRON-70B: scaling medical pretraining for large language modelsarXivPreprint posted online on November 27, 2023https:\/\/arxiv.org\/abs\/2311.16079"},{"key":"ref18","unstructured":"TouvronHLavrilTIzacardGMartinetXLachauxM-ALacroixTRozi\u00e8reBGoyalNHambroEAzharFRodriguezAJoulinAGraveELampleGLLaMA: open and efficient foundation language modelsarXivPreprint posted online on February 27, 2023https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2024-067223"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1126\/science.aax2342"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.7326\/m18-1990"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaf233"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.2196\/65984"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01233-2"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01429-0"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000800"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1007\/s10926-023-10127-1"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-biodatasci-092820-114757"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.11613\/bm.2012.031"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2023.100804"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.adk3452"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03425-5"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1186\/s12939-025-02419-0"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1016\/s2589-7500(23)00225-x"},{"key":"ref36","year":"2022","journal-title":"Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans, TCPS 2"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2019180052"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06160-y"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1080\/08989621.2025.2544331"}],"container-title":["JMIR Research Protocols"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.researchprotocols.org\/2026\/1\/e99807","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T21:15:11Z","timestamp":1790111711000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchprotocols.org\/2026\/1\/e99807"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":39,"URL":"https:\/\/doi.org\/10.2196\/99807","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.99807","asserted-by":"object"}]},"ISSN":["1929-0748"],"issn-type":[{"value":"1929-0748","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]}}}