{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T04:54:58Z","timestamp":1793422498021,"version":"4.5.0"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T00:00:00Z","timestamp":1734912000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T00:00:00Z","timestamp":1734912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Health~Holland, the Netherlands","award":["LSHM20103"],"award-info":[{"award-number":["LSHM20103"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["World J Emerg Surg"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Acute abdominal pain (AAP) constitutes 5\u201310% of all emergency department (ED) visits, with appendicitis being a prevalent AAP etiology often necessitating surgical intervention. The variability in AAP symptoms and causes, combined with the challenge of identifying appendicitis, complicate timely intervention. To estimate the risk of appendicitis, scoring systems such as the Alvarado score have been developed. However, diagnostic errors and delays remain common. Although various machine learning (ML) models have been proposed to enhance appendicitis detection, none have been seamlessly integrated into the ED workflows for AAP or are specifically designed to diagnose appendicitis as early as possible within the clinical decision-making process. To mimic daily clinical practice, this proof-of-concept study aims to develop ML models that support decision-making using comprehensive clinical data up to key decision points in the ED workflow to detect appendicitis in patients presenting with AAP.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Data from the Dutch triage system at the ED, vital signs, complete medical history and physical examination findings and routine laboratory test results were retrospectively extracted from 350 AAP patients presenting to the ED of a Dutch teaching hospital from 2016 to 2023. Two eXtreme Gradient Boosting ML models were developed to differentiate cases with appendicitis from other AAP causes: one model used all data up to and including physical examination, and the other was extended with routine laboratory test results. The performance of both models was evaluated on a validation set (\n                      <jats:italic>n<\/jats:italic>\n                      \u2009=\u200968) and compared to the Alvarado scoring system as well as three ED physicians in a reader study.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The ML models achieved AUROCs of 0.919 without laboratory test results and 0.923 with the addition of laboratory test results. The Alvarado scoring system attained an AUROC of 0.824. ED physicians achieved AUROCs of 0.894, 0.826, and 0.791 without laboratory test results, increasing to AUROCs of 0.923, 0.892, and 0.859 with laboratory test results.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Both ML models demonstrated comparable high accuracy in predicting appendicitis in patients with AAP, outperforming the Alvarado scoring system. The ML models matched or surpassed ED physician performance in detecting appendicitis, with the largest potential performance gain observed in absence of laboratory test results. Integration could assist ED physicians in early and accurate diagnosis of appendicitis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Graphical abstract<\/jats:title>\n                  <\/jats:sec>","DOI":"10.1186\/s13017-024-00570-7","type":"journal-article","created":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T00:11:54Z","timestamp":1734912714000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Machine-learning based prediction of appendicitis for patients presenting with acute abdominal pain at the emergency department"],"prefix":"10.1186","volume":"19","author":[{"given":"Anoeska","family":"Schipper","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Belgers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rory","family":"O\u2019Connor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kim Ellis","family":"Jie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robin","family":"Dooijes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joeran Sander","family":"Bosma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steef","family":"Kurstjens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ron","family":"Kusters","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bram","family":"van Ginneken","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthieu","family":"Rutten","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,12,23]]},"reference":[{"key":"570_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ienj.2018.03.006","volume":"40","author":"MC van der Linden","year":"2018","unstructured":"van der Linden MC, van Loon M, Feenstra NSF, van der Linden N. Assessing bottlenecks in Emergency Department flow of patients with abdominal pain. Int Emerg Nurs. 2018;40:1\u20135.","journal-title":"Int Emerg Nurs"},{"key":"570_CR2","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1016\/j.ajem.2010.01.045","volume":"29","author":"RS Hastings","year":"2011","unstructured":"Hastings RS, Powers RD. Abdominal pain in the ED: A 35-year retrospective. Am J Emerg Med. 2011;29:711\u201316.","journal-title":"Am J Emerg Med"},{"key":"570_CR3","doi-asserted-by":"publisher","first-page":"362","DOI":"10.21037\/atm.2016.09.10","volume":"4","author":"G Cervellini","year":"2016","unstructured":"Cervellini G, Mora R, Ticinesi A, Meschi T, Comelli I, Catena F, Lippi G. Epidemiology and outcomes of acute abdominal pain in a large urban Emergency Department: Retrospective analysis of 5,340 cases. Ann Transl Med. 2016;4:362\u201340.","journal-title":"Ann Transl Med"},{"key":"570_CR4","doi-asserted-by":"publisher","first-page":"1278","DOI":"10.1016\/S0140-6736(15)00275-5","volume":"386","author":"A Bhangu","year":"2015","unstructured":"Bhangu A, S\u00f8reide K, di Saverio S, Assarsson JH, Drake FT. Acute appendicitis: Modern understanding of pathogenesis, diagnosis, and management. Lancet. 2015;386:1278\u201387.","journal-title":"Lancet"},{"key":"570_CR5","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1016\/j.yasu.2013.03.003","volume":"47","author":"FT Drake","year":"2013","unstructured":"Drake FT, Flum DR. Improvement in the diagnosis of appendicitis. Adv Surg. 2013;47:299\u2013328.","journal-title":"Adv Surg"},{"key":"570_CR6","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1111\/j.1553-2712.2000.tb00470.x","volume":"7","author":"L Graff","year":"2000","unstructured":"Graff L, Russell J, Seashore J, Tate J, Elwell A, Prete M, et al. False-negative and false-positive errors in abdominal pain evaluation: Failure to diagnose acute appendicitis and unnecessary surgery. Acad Emerg Med. 2000;7:1244\u201355.","journal-title":"Acad Emerg Med"},{"key":"570_CR7","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.ijsu.2020.04.041","volume":"79","author":"K Gelpke","year":"2020","unstructured":"Gelpke K, Hamminga JTH, van Bastelaar JJ, de Vos B, Bodegom ME, Heineman E, et al. Reducing the negative appendectomy rate with the laparoscopic appendicitis score; a multicenter prospective cohort and validation study. Int J Surg. 2020;79:257\u201364.","journal-title":"Int J Surg"},{"key":"570_CR8","doi-asserted-by":"publisher","first-page":"e200612","DOI":"10.1001\/jamanetworkopen.2020.0612","volume":"3","author":"P Mahajan","year":"2020","unstructured":"Mahajan P, Basu T, Pai CW, Singh H, Petersen NM, Bellolio F, et al. Factors Associated With Potentially Missed Diagnosis of Appendicitis in the Emergency Department. JAMA Netw Open. 2020;3:e200612.","journal-title":"JAMA Netw Open"},{"key":"570_CR9","doi-asserted-by":"publisher","first-page":"592","DOI":"10.1111\/acem.12144","volume":"20","author":"JA Naiditch","year":"2013","unstructured":"Naiditch JA, Lautz TB, Daley S, Pierce MC, Reynolds M. The implications of missed opportunities to diagnose appendicitis in children. Acad Emerg Med. 2013;20:592\u20136.","journal-title":"Acad Emerg Med"},{"key":"570_CR10","doi-asserted-by":"publisher","first-page":"1843","DOI":"10.1007\/s00268-008-9649-y","volume":"32","author":"M Andersson","year":"2008","unstructured":"Andersson M, Andersson RE. The Appendicitis Inflammatory Response Score: a tool for the diagnosis of acute appendicitis that outperforms the Alvarado score. World J Surg. 2008;32:1843\u20139.","journal-title":"World J Surg"},{"key":"570_CR11","doi-asserted-by":"publisher","first-page":"877","DOI":"10.1053\/jpsu.2002.32893","volume":"37","author":"M Samuel","year":"2002","unstructured":"Samuel M. Pediatric appendicitis score. J Pediatr Surg. 2002;37:877\u201381.","journal-title":"J Pediatr Surg"},{"key":"570_CR12","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1016\/S0196-0644(86)80993-3","volume":"15","author":"A Alvarado","year":"1986","unstructured":"Alvarado A. A practical score for the early diagnosis of acute appendicitis. Ann Emerg Med. 1986;15:557\u201364.","journal-title":"Ann Emerg Med"},{"key":"570_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13017-020-00306-3","volume":"15","author":"S di Saverio","year":"2020","unstructured":"di Saverio S, Podda M, de Simone B, Ceresoli M, Augustin G, Gori A, et al. Diagnosis and treatment of acute appendicitis: 2020 update of the WSES Jerusalem guidelines. World J Emerg Surg. 2020;15:1\u201342.","journal-title":"World J Emerg Surg"},{"key":"570_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1741-7015-9-139","volume":"9","author":"R Ohle","year":"2011","unstructured":"Ohle R, O\u2019Reilly F, O\u2019Brien KK, Fahey T, Dimitrov BD. The Alvarado score for predicting acute appendicitis: A systematic review. BMC Med. 2011;9:1\u201313.","journal-title":"BMC Med"},{"key":"570_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13017-023-00527-2","volume":"18","author":"M Issaiy","year":"2023","unstructured":"Issaiy M, Zarei D, Saghazadeh A. Artificial Intelligence and Acute Appendicitis: A Systematic Review of Diagnostic and Prognostic Models. World J Emerg Surg. 2023;18:1\u201331.","journal-title":"World J Emerg Surg"},{"key":"570_CR16","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.artmed.2013.03.007","volume":"58","author":"YH Lee","year":"2013","unstructured":"Lee YH, Hu PJ, Cheng TH, Huang TC, Chuang WY. A preclustering-based ensemble learning technique for acute appendicitis diagnoses. Artif Intell Med. 2013;58:115\u201324.","journal-title":"Artif Intell Med"},{"key":"570_CR17","first-page":"785","volume":"56","author":"L Shahmoradi","year":"2019","unstructured":"Shahmoradi L, Safdari R, Mir Hosseini M, Arji G, Jannt B, Abdar M. Predicting risk of acute appendicitis: a comparison of artificial neural network and logistic regression models. Acta Med Iran. 2019;56:785.","journal-title":"Acta Med Iran"},{"key":"570_CR18","first-page":"14","volume":"8","author":"R Goswami","year":"2020","unstructured":"Goswami R, Kour H, Manhas J, Sharma V. Comparison and Analysis of Machine Learning Techniques for the Prediction of Acute Appendicitis. J Appl Inf Sci. 2020;8:14\u201321.","journal-title":"J Appl Inf Sci"},{"key":"570_CR19","doi-asserted-by":"publisher","first-page":"12722","DOI":"10.1038\/s41598-024-63513-x","volume":"14","author":"I Males","year":"2024","unstructured":"Males I, Boban Z, Kumric M, Vrdoljak J, Berkovic K, Pogorelic Z, Bozic J. Applying an explainable machine learning model might reduce the number of negative appendectomies in pediatric patients with a high probability of acute appendicitis. Sci Rep. 2024;14:12722.","journal-title":"Sci Rep"},{"key":"570_CR20","first-page":"559","volume":"23","author":"SY Park","year":"2015","unstructured":"Park SY, Kim SM. Acute appendicitis diagnosis using artificial neural networks. Technol Health. 2015;23:559\u201365.","journal-title":"Technol Health"},{"key":"570_CR21","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1186\/1472-6947-12-17","volume":"12","author":"CS Son","year":"2012","unstructured":"Son CS, Jang BK, Seo ST, Kim MS, Kim YN. A hybrid decision support model to discover informative knowledge in diagnosing acute appendicitis. BMC Med Inf Decis Mak. 2012;12:17.","journal-title":"BMC Med Inf Decis Mak"},{"key":"570_CR22","first-page":"3896263","volume":"2020","author":"Y Zhao","year":"2020","unstructured":"Zhao Y, Yang L, Sun C, Li Y, He Y, Zhang L, et al. Discovery of urinary proteomic signature for differential diagnosis of acute appendicitis. Biomed Res Int. 2020;2020:3896263.","journal-title":"Biomed Res Int"},{"issue":"1","key":"570_CR23","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1016\/j.surg.2010.03.023","volume":"149","author":"CH Hsieh","year":"2011","unstructured":"Hsieh CH, Lu RH, Lee NH, Chiu WT, Hsu MH, Li YC. Novel solutions for an old disease: diagnosis of acute appendicitis with random forest, support vector machines, and artificial neural networks. Surgery. 2011;149(1):87\u201393.","journal-title":"Surgery"},{"key":"570_CR24","unstructured":"Nakayama H. 2019. Doccano (Version 1.4) [Software]. https:\/\/doccano.github.io"},{"key":"570_CR25","unstructured":"Xgboost developers. 2022. DMLC XGBoost (Version 2.0.3.) [Software]. https:\/\/xgboost.readthedocs.io"},{"key":"570_CR26","unstructured":"Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018). 2018. arXiv:1706.09516."},{"key":"570_CR27","unstructured":"McGinnis W. 2022. Category Encoders (Version 2.6.3.) [Software]. https:\/\/contrib.scikit-learn.org\/category_encoders\/catboost.html"},{"key":"570_CR28","unstructured":"Takuya A, Shotaro S, Toshihiko Y, Takeru O, Masanori K. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework (Version 3.6.1). https:\/\/optuna.org"},{"key":"570_CR29","unstructured":"Lundberg S. 2018. SHAP (Version 0.41.0) [Software]. https:\/\/shap.readthedocs.io"},{"key":"570_CR30","volume-title":"Principles of biostatistics","author":"M Pagano","year":"2000","unstructured":"Pagano M, Gauvreau K. Principles of biostatistics. 2nd ed. Brooks\/Cole: Cengage Learning; 2000.","edition":"2"},{"issue":"6","key":"570_CR31","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1007\/s00383-020-04655-7","volume":"36","author":"E Aydin","year":"2020","unstructured":"Aydin E, T\u00fcrkmen \u0130U, Namli G, \u00d6zt\u00fcrk \u00c7, Esen AB, Eray YN, et al. A novel and simple machine learning algorithm for preoperative diagnosis of acute appendicitis in children. Pediatr Surg Int. 2020;36(6):735\u201342.","journal-title":"Pediatr Surg Int"},{"key":"570_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/s11739-024-03738-w","author":"G Roshanaei","year":"2024","unstructured":"Roshanaei G, Salimi R, Mahjub H, Faradmal J, Yamini A, Tarokhian A. Accurate diagnosis of acute appendicitis in the emergency department: an artificial intelligence-based approach. Intern Emerg Med. 2024. https:\/\/doi.org\/10.1007\/s11739-024-03738-w.","journal-title":"Intern Emerg Med"},{"key":"570_CR33","doi-asserted-by":"publisher","first-page":"401","DOI":"10.1016\/S1726-4901(10)70087-3","volume":"73","author":"HW Ting","year":"2010","unstructured":"Ting HW, Wu JT, Chan CL, Lin SL, Chen MH. Decision model for acute appendicitis treatment with decision tree technology-a modification of the Alvarado scoring system. J Chin Med Assoc. 2010;73:401\u20136.","journal-title":"J Chin Med Assoc"},{"key":"570_CR34","doi-asserted-by":"publisher","first-page":"1245","DOI":"10.1016\/j.ajem.2011.06.019","volume":"30","author":"\u00d6 Yolda\u015f","year":"2012","unstructured":"Yolda\u015f \u00d6, Tez M, Karaca T. Artificial neural networks in the diagnosis of acute appendicitis. Am J Emerg Med. 2012;30:1245\u20137.","journal-title":"Am J Emerg Med"},{"key":"570_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.healthpol.2018.11.001","volume":"123","author":"N Baier","year":"2019","unstructured":"Baier N, Geissler A, Bech M, Bernstein D, Cowling TE, Jackson T, et al. Emergency and urgent care systems in Australia, Denmark, England, France, Germany and the Netherlands \u2013 Analyzing organization, payment and reforms. Health Pol. 2019;123:1\u201310.","journal-title":"Health Pol"},{"key":"570_CR36","doi-asserted-by":"crossref","unstructured":"Van der Meer JWM, Postma CT, Blockmans DE, de Rooij SEJA. Anamnese en lichamelijk onderzoek. Bohn Stafleu van Loghum; 2019.","DOI":"10.1007\/978-90-368-2072-1"},{"key":"570_CR37","doi-asserted-by":"crossref","unstructured":"de Hond AAH, Leeuwenberg AM, Hooft L, Kant IMJ, Nijman SWJ, van Os HJA et al. (2022). Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit Med. 2022;5:1\u201313.","DOI":"10.1038\/s41746-021-00549-7"},{"key":"570_CR38","doi-asserted-by":"publisher","unstructured":"la Roi-Teeuw HM, van Royen FS, de Hond A, Zahra A, de Vries S, Bartels R et al. (2024). Don\u2019t be misled: Three misconceptions about external validation of clinical prediction models. J Clin Epidemiol. 2024. https:\/\/doi.org\/10.1016\/j.jclinepi.2024.111387","DOI":"10.1016\/j.jclinepi.2024.111387"}],"container-title":["World Journal of Emergency Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13017-024-00570-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13017-024-00570-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13017-024-00570-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,23]],"date-time":"2024-12-23T13:03:55Z","timestamp":1734959035000},"score":1,"resource":{"primary":{"URL":"https:\/\/wjes.biomedcentral.com\/articles\/10.1186\/s13017-024-00570-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,23]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["570"],"URL":"https:\/\/doi.org\/10.1186\/s13017-024-00570-7","relation":{},"ISSN":["1749-7922"],"issn-type":[{"value":"1749-7922","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,23]]},"assertion":[{"value":"20 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was conducted according to the declaration of Helsinki, Guidelines for Good Clinical Practice. The execution of this retrospective observational study of patient records was approved by the local review board of the Jeroen Bosch Hospital. The execution of this retrospective observational study of patient records was judged by the Medical Research Ethics Committee Brabant (METC Brabant), which waived this study to be subject to the regulations of the WMO (Dutch Medical Research Involving Human Subjects Act), including a waiver of informed written consent and a consent for publication (METC nr. NW2024-05).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval & informed consent"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"40"}}