{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T15:17:43Z","timestamp":1790090263016,"version":"4.0.1"},"reference-count":44,"publisher":"BMJ","issue":"2","license":[{"start":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T00:00:00Z","timestamp":1770768000000},"content-version":"unspecified","delay-in-days":10,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001350","name":"Ministry of Health, Singapore","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001350","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100030677","name":"National Health Innovation Centre","doi-asserted-by":"crossref","award":["I2Adopt Grant Number NHIC-I2A-2305009"],"award-info":[{"award-number":["I2Adopt Grant Number NHIC-I2A-2305009"]}],"id":[{"id":"10.13039\/100030677","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2026,2,2]]},"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To describe the structured process of threshold optimisation for a commercially available multiclass chest X-ray (CXR) deep learning model, to evaluate its diagnostic performance across different operating thresholds, and to estimate its potential operational impact within an artificial intelligence (AI)-enabled triage workflow in a primary care setting.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design<\/jats:title>\n                    <jats:p>Retrospective diagnostic performance evaluation with threshold-based analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Setting<\/jats:title>\n                    <jats:p>Primary care radiography services in Singapore, using data derived from two primary care clinics and a tertiary hospital.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Participants<\/jats:title>\n                    <jats:p>A total of 816 adult frontal chest radiographs were included (multiethnic Asian, 464 males, 352 females; mean age 60.8 years). Images were selected to represent the spectrum of findings often encountered in primary care. Exclusion criteria included paediatric studies, lateral or oblique radiographs, and findings not supported by the AI model (eg, bony abnormalities and medical devices).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Primary and secondary outcome measures<\/jats:title>\n                    <jats:p>Primary outcome measures were sensitivity, specificity, and negative and positive predictive value (NPV and PPV). Secondary outcomes included estimated potential operational improvement, which is calculated by dividing the number of true negatives by the total number of CXRs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>At the default threshold of 0.15, the AI model achieved a sensitivity of 87.3% (95% CI 83.9% to 90.4%) and an NPV of 87.0% (95% CI 83.6% to 90.2%). Lowering the threshold to 0.10 increased sensitivity to 93.2% (95% CI 90.7% to 95.5%) and NPV to 91.3% (95% CI 88.2% to 94.3%), with specificity of 71.7% (95% CI 67.3% to 76.1%). These trade-offs were considered acceptable for a safety-focused co-triage workflow prioritising minimisation of false negatives.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Threshold optimisation is critical for adapting AI models to context-specific clinical workflows. Our study shows that adjusting the operating threshold enabled prioritisation of sensitivity and NPV, supporting safe AI-assisted triage in primary care. This is a deeply collaborative process that must involve radiology and clinical teams: selecting appropriate thresholds aligned with clinical objectives for safe and effective implementation. Future work will assess real-world operational impact and user acceptance following prospective deployment.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjopen-2025-111127","type":"journal-article","created":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T14:40:40Z","timestamp":1770907240000},"page":"e111127","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":4,"title":["How threshold customisation affects the performance of a multiclass X-ray AI model for primary care triage: a retrospective study"],"prefix":"10.1136","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4164-1643","authenticated-orcid":false,"given":"Jordan Zheng Ting","family":"Sim","sequence":"first","affiliation":[{"name":"Diagnostic Radiology, Tan Tock Seng Hospital, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia","family":"Lin","sequence":"additional","affiliation":[{"name":"Clinical Research and Innovation Office, Tan Tock Seng Hospital, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi Wei","family":"Fong","sequence":"additional","affiliation":[{"name":"Geylang Polyclinic, National Healthcare Group Polyclinics, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alvin Yong Quan","family":"Soon","sequence":"additional","affiliation":[{"name":"Diagnostic Radiology, Tan Tock Seng Hospital, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lay Wai","family":"Khin","sequence":"additional","affiliation":[{"name":"Clinical Research and Innovation Office, Tan Tock Seng Hospital, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sreejith","family":"Balakrishnan","sequence":"additional","affiliation":[{"name":"Resaro, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Lin","sequence":"additional","affiliation":[{"name":"Resaro, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9220-1763","authenticated-orcid":false,"given":"Sabrina","family":"Wong","sequence":"additional","affiliation":[{"name":"Clinical Research Unit, National Healthcare Group Polyclinics, Singapore, Singapore"},{"name":"Lee Kong Chian School of Medicine, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cher Heng","family":"Tan","sequence":"additional","affiliation":[{"name":"Diagnostic Radiology, Tan Tock Seng Hospital, Singapore, Singapore"},{"name":"Lee Kong Chian School of Medicine, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2026,2,12]]},"reference":[{"key":"2026021206403000000_16.2.e111127.1","doi-asserted-by":"crossref","first-page":"W13","DOI":"10.1097\/RTI.0000000000000200","article-title":"ACR Appropriateness Criteria\u00ae Routine Chest Radiography","volume":"31","author":"McComb","year":"2016","journal-title":"J Thorac Imaging"},{"key":"2026021206403000000_16.2.e111127.2","unstructured":"Health M of. Agency for Care Effectiveness (ACE) . When not to order chest x-ray. ace clinical guidance (acg) [internet]. singapore. 2021. Available: http:\/\/go.gov.sg\/acg-when-not-to-order-chest-x-ray"},{"key":"2026021206403000000_16.2.e111127.3","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2332031796"},{"key":"2026021206403000000_16.2.e111127.4","unstructured":"Izamin IM , Rizal AM . Chest x-ray as an essential part of routine medical examination: is it necessary?"},{"key":"2026021206403000000_16.2.e111127.5","unstructured":"Diagnostics NHG . Imaging, 2024. Available: https:\/\/www.nhgd.com.sg\/ourservices [Accessed 19 Nov 2024]."},{"key":"2026021206403000000_16.2.e111127.6","unstructured":"Ministry of Health S . Healthcare services act, healthcare services (clinical laboratory service and radiological service) regulations 2021 [internet]. singapore statutes online, s 842\/2023 singapore statutes. 2023. Available: https:\/\/sso.agc.gov.sg\/SL\/HSA2020-S1036-2021?DocDate=20231214&WholeDoc=1#pr1"},{"key":"2026021206403000000_16.2.e111127.7","article-title":"Accuracy of an artificial intelligence-enabled diagnostic assistance device in recognizing normal chest radiographs: a service evaluation","volume":"6","author":"Kumar","year":"2024","journal-title":"BJR Open"},{"key":"2026021206403000000_16.2.e111127.8","doi-asserted-by":"crossref","DOI":"10.1148\/radiol.231236","article-title":"Commercially Available Chest Radiograph AI Tools for Detecting Airspace Disease, Pneumothorax, and Pleural Effusion","volume":"308","author":"Lind Plesner","year":"2023","journal-title":"Radiology"},{"key":"2026021206403000000_16.2.e111127.9","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-019-51503-3","article-title":"Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems","volume":"9","author":"Qin","year":"2019","journal-title":"Sci Rep"},{"key":"2026021206403000000_16.2.e111127.10","doi-asserted-by":"crossref","DOI":"10.1183\/13993003.02159-2016","article-title":"An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients","volume":"49","author":"Rahman","year":"2017","journal-title":"Eur Respir J"},{"key":"2026021206403000000_16.2.e111127.11","unstructured":"Oakden-Rayner L , Dunnmon J , Carneiro G , et al . Hidden stratification causes clinically meaningful failures in machine learning for medical imaging. ACM CHIL \u201920; Toronto Ontario Canada, April 2, 2020 doi:10.1145\/3368555.3384468 Available: https:\/\/dl.acm.org\/doi\/proceedings\/10.1145\/3368555"},{"key":"2026021206403000000_16.2.e111127.12","doi-asserted-by":"crossref","first-page":"105618","DOI":"10.1016\/j.compbiomed.2022.105618","article-title":"Multilevel threshold image segmentation for COVID-19 chest radiography: A framework using horizontal and vertical multiverse optimization","volume":"146","author":"Su","year":"2022","journal-title":"Comput Biol Med"},{"key":"2026021206403000000_16.2.e111127.13","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-020-0273-z","article-title":"Automated abnormality classification of chest radiographs using deep convolutional neural networks","volume":"3","author":"Tang","year":"2020","journal-title":"NPJ Digit Med"},{"key":"2026021206403000000_16.2.e111127.14","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1007\/s11760-022-02183-6","article-title":"A Novel Threshold-Based Segmentation Method for Quantification of COVID-19 Lung Abnormalities","volume":"17","author":"Khan","year":"2023","journal-title":"SIViP"},{"key":"2026021206403000000_16.2.e111127.15","doi-asserted-by":"crossref","first-page":"111783","DOI":"10.1016\/j.ejrad.2024.111783","article-title":"Real-World evaluation of an AI triaging system for chest X-rays: A prospective clinical study","volume":"181","author":"Sridharan","year":"2024","journal-title":"Eur J Radiol"},{"key":"2026021206403000000_16.2.e111127.16","doi-asserted-by":"crossref","first-page":"7255","DOI":"10.1007\/s00330-024-10794-5","article-title":"Performance of AI to exclude normal chest radiographs to reduce radiologists\u2019 workload","volume":"34","author":"Schalekamp","year":"2024","journal-title":"Eur Radiol"},{"key":"2026021206403000000_16.2.e111127.17","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000813","article-title":"Comparison of different Lunit INSIGHT CXR software versions when reading chest radiographs for tuberculosis","volume":"4","author":"Codlin","year":"2025","journal-title":"PLOS Digit Health"},{"key":"2026021206403000000_16.2.e111127.18","doi-asserted-by":"crossref","DOI":"10.1183\/13993003.03061-2020","article-title":"Development and validation of a deep learning algorithm detecting 10 common abnormalities on chest radiographs","volume":"57","author":"Nam","year":"2021","journal-title":"Eur Respir J"},{"key":"2026021206403000000_16.2.e111127.19","unstructured":"Lunit . Lunit AI Receives HSA Approval, Now Commercially Available in Singapore, 2021. Available: https:\/\/www.lunit.io\/en\/company\/news\/lunit-ai-receives-hsa-approval-now-commercially-available-in-singapore [Accessed 29 May 2025]."},{"key":"2026021206403000000_16.2.e111127.20","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-023-10124-1"},{"key":"2026021206403000000_16.2.e111127.21","doi-asserted-by":"crossref","first-page":"3837","DOI":"10.1007\/s00330-020-07480-7","article-title":"Smart chest X-ray worklist prioritization using artificial intelligence: a clinical workflow simulation","volume":"31","author":"Baltruschat","year":"2021","journal-title":"Eur Radiol"},{"key":"2026021206403000000_16.2.e111127.22","doi-asserted-by":"publisher","DOI":"10.1183\/09031936.00111012"},{"key":"2026021206403000000_16.2.e111127.23","unstructured":"Resaro . Resaro. Available: https:\/\/resaro.ai\/ [Accessed 6 Dec 2024]."},{"key":"2026021206403000000_16.2.e111127.24","unstructured":"Ministry of health s . Artificial Intelligence in Healthcare Guidelines (AIHGIe), 2021. Available: https:\/\/isomer-user-content.by.gov.sg\/3\/9c0db09d-104c-48af-87c9-17e01695c67c\/1-0-artificial-in-healthcare-guidelines-(aihgle)_publishedoct21.pdf"},{"key":"2026021206403000000_16.2.e111127.25","unstructured":"International Organization for Standardization (ISO), The International Electrotechnical Commission (IEC) . ISO\/IEC TR 24029-1:2021 - Robustness of neural networks \u2014 Part 1, Available: https:\/\/www.iso.org\/standard\/77609.html [Accessed 4 Dec 2024]."},{"key":"2026021206403000000_16.2.e111127.26","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1214\/aos\/1176324703","article-title":"Convergence of the Iterative Proportional Fitting Procedure","volume":"23","author":"Ruschendorf","year":"1995","journal-title":"Ann Statist"},{"key":"2026021206403000000_16.2.e111127.27","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2024-089047","article-title":"Cohort profile: AI-driven national Platform for CCTA for clinicaL and industriaL applicatiOns (APOLLO)","volume":"14","author":"Baskaran","year":"2024","journal-title":"BMJ Open"},{"key":"2026021206403000000_16.2.e111127.28","unstructured":"Leonie K , Jonas S , Markus A . Risk thresholds for frontier ai. arXiv [Preprint] 2024."},{"key":"2026021206403000000_16.2.e111127.29","unstructured":"OpenAI . Preparedness framework (beta). 2023. Available: https:\/\/cdn.openai.com\/openai-preparedness-framework-beta.pdf [Accessed 17 Dec 2024]."},{"key":"2026021206403000000_16.2.e111127.30","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-024-82775-z","article-title":"Optimizing adult-oriented artificial intelligence for pediatric chest radiographs by adjusting operating points","volume":"14","author":"Shin","year":"2024","journal-title":"Sci Rep"},{"key":"2026021206403000000_16.2.e111127.31","doi-asserted-by":"publisher","DOI":"10.1016\/j.wneu.2021.12.004"},{"key":"2026021206403000000_16.2.e111127.32","doi-asserted-by":"publisher","DOI":"10.1016\/j.gie.2020.06.059"},{"key":"2026021206403000000_16.2.e111127.33","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1148\/radiol.2018180921","article-title":"Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks","volume":"291","author":"Annarumma","year":"2019","journal-title":"Radiology"},{"key":"2026021206403000000_16.2.e111127.34","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.neucom.2022.02.040","article-title":"Multi-modal trained artificial intelligence solution to triage chest X-ray for COVID-19 using pristine ground-truth, versus radiologists","volume":"485","author":"Tan","year":"2022","journal-title":"Neurocomputing (Amst)"},{"key":"2026021206403000000_16.2.e111127.35","doi-asserted-by":"crossref","unstructured":"Sim J , Ting Y-H , Tang Y , et al . n.d. Diagnostic Performance of a Deep Learning Model Deployed at a National COVID-19 Screening Facility for Detection of Pneumonia on Frontal Chest Radiographs. Healthcare (Basel) 10:175. doi:10.3390\/healthcare10010175","DOI":"10.3390\/healthcare10010175"},{"key":"2026021206403000000_16.2.e111127.36","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1016\/j.amepre.2024.02.006","article-title":"Accuracy of Online Artificial Intelligence Models in Primary Care Settings","volume":"66","author":"Kassab","year":"2024","journal-title":"Am J Prev Med"},{"key":"2026021206403000000_16.2.e111127.37","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1001\/jamainternmed.2023.7965","article-title":"Using Artificial Intelligence to Improve Primary Care for Patients and Clinicians","volume":"184","author":"Sarkar","year":"2024","journal-title":"JAMA Intern Med"},{"key":"2026021206403000000_16.2.e111127.38","unstructured":"Ministry of health s . In: White Paper on Healthier SG. Singapore, 2022."},{"key":"2026021206403000000_16.2.e111127.39","unstructured":"Barnfield A , Stegeman I . Health Promotion and Primary Prevention in 21 European Countries A Comparative Overview of Key Policies, Approaches. Examples of Good Practice, and Gaps and Needs,"},{"key":"2026021206403000000_16.2.e111127.40","doi-asserted-by":"crossref","DOI":"10.3389\/fmedt.2023.1281500","article-title":"Harnessing artificial intelligence in radiology to augment population health","volume":"5","author":"Sim","year":"2023","journal-title":"Front Med Technol"},{"key":"2026021206403000000_16.2.e111127.41","doi-asserted-by":"crossref","first-page":"109993","DOI":"10.1016\/j.clinimag.2023.109993","article-title":"Improving radiology workflow using ChatGPT and artificial intelligence","volume":"103","author":"Mese","year":"2023","journal-title":"Clin Imaging"},{"key":"2026021206403000000_16.2.e111127.42","first-page":"1122","article-title":"A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia","volume":"6","author":"Xu","year":"2020","journal-title":"Engineering (Beijing)"},{"key":"2026021206403000000_16.2.e111127.43","unstructured":"Feng S , Liming X , Fei S , et al . Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classification. Arxiv [Preprint] 2020. Available: https:\/\/arxiv.org\/abs\/2003.09860"},{"key":"2026021206403000000_16.2.e111127.44","unstructured":"Zhang J , Xie Y , Li Y , et al . COVID-19 Screening on Chest X-ray Images Using Deep Learning based Anomaly Detection. IEEE Trans Med Imaging 2020."}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2025-111127","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T14:40:51Z","timestamp":1770907251000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2025-111127"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2]]},"references-count":44,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2,12]]},"published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1136\/bmjopen-2025-111127"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2025-111127","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2]]}}}