{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T16:06:17Z","timestamp":1791561977495,"version":"4.3.4"},"reference-count":33,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Artificial intelligence (AI) is increasingly used in radiology, but its development in pediatric imaging remains limited, particularly for emergent conditions. Ileocolic intussusception is an important cause of acute abdominal pain in infants and toddlers and requires timely diagnosis to prevent complications such as bowel ischemia or perforation. While ultrasonography is the diagnostic standard due to its high sensitivity and specificity, its accessibility may be limited, especially outside tertiary centers. Abdominal radiographs (AXRs), despite their limited sensitivity, are often the first-line imaging modality in clinical practice. In this context, AI could support early screening and triage by analyzing AXRs and identifying patients who require further ultrasonography evaluation.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to upgrade and externally validate an AI model for screening ileocolic intussusception using pediatric AXRs with multicenter data and to assess the diagnostic performance of the model in comparison with radiologists of varying experience levels with and without AI assistance.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This retrospective study included pediatric patients (\u22645 years) who underwent both AXRs and ultrasonography for suspected intussusception. Based on the preliminary study from hospital A, the AI model was retrained using data from hospital B and validated with external datasets from hospitals C and D. Diagnostic performance of the upgraded AI model was evaluated using sensitivity, specificity, and the area under the receiver operating characteristic curve (AUC). A reader study was conducted with 3 radiologists, including 2 trainees and 1 pediatric radiologist, to evaluate diagnostic performance with and without AI assistance.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Based on the previously developed AI model trained on 746 patients from hospital A, an additional 431 patients from hospital B (including 143 intussusception cases) were used for further training to develop an upgraded AI model. External validation was conducted using data from hospital C (n=68; 19 intussusception cases) and hospital D (n=90; 30 intussusception cases). The upgraded AI model achieved a sensitivity of 81.7% (95% CI 68.6%\u201090%) and a specificity of 81.7% (95% CI 73.3%\u201087.8%), with an AUC of 86.2% (95% CI 79.2%\u201092.1%) in the external validation set. Without AI assistance, radiologists showed lower performance (overall AUC 64%; sensitivity 49.7%; specificity 77.1%). With AI assistance, radiologists\u2019 specificity improved to 93% (difference +15.9%;\n P<\/jats:italic>\n <.001), and AUC increased to 79.2% (difference +15.2%;\n P<\/jats:italic>\n =.05). The least experienced reader showed the largest improvement in specificity (+37.6%;\n P<\/jats:italic>\n <.001) and AUC (+14.7%;\n P<\/jats:italic>\n =.08).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The upgraded AI model improved diagnostic performance for screening ileocolic intussusception on pediatric AXRs. It effectively enhanced the specificity and overall accuracy of radiologists, particularly those with less experience in pediatric radiology. A user-friendly software platform was introduced to support broader clinical validation and underscores the potential of AI as a screening and triage tool in pediatric emergency settings.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/72097","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T07:32:22Z","timestamp":1752478342000},"page":"e72097-e72097","source":"Crossref","is-referenced-by-count":8,"title":["External Validation of an Upgraded AI Model for Screening Ileocolic Intussusception Using Pediatric Abdominal Radiographs: Multicenter Retrospective Study"],"prefix":"10.2196","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1789-8270","authenticated-orcid":false,"given":"Jeong Hoon","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4276-8803","authenticated-orcid":false,"given":"Pyeong Hwa","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6192-8852","authenticated-orcid":false,"given":"Nak-Hoon","family":"Son","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5687-7237","authenticated-orcid":false,"given":"Kyunghwa","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5308-9825","authenticated-orcid":false,"given":"Yeseul","family":"Kang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1377-9924","authenticated-orcid":false,"given":"Sejin","family":"Jeong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3368-5013","authenticated-orcid":false,"given":"Eun-Kyung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0581-8656","authenticated-orcid":false,"given":"Haesung","family":"Yoon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6928-4967","authenticated-orcid":false,"given":"Sergios","family":"Gatidis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1999-6595","authenticated-orcid":false,"given":"Shreyas","family":"Vasanawala","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6491-5734","authenticated-orcid":false,"given":"Hee Mang","family":"Yoon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7462-2609","authenticated-orcid":false,"given":"Hyun Joo","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,7,8]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1007\/s00247-009-1178-9","article-title":"Intussusception in children: evidence-based diagnosis and treatment","volume":"39 Suppl 2","author":"Applegate","journal-title":"Pediatr Radiol"},{"issue":"5","key":"R2","doi-asserted-by":"publisher","first-page":"584","DOI":"10.1097\/PEC.0b013e31828e62d6","article-title":"Comparison of clinicians to radiologists in assessment of abdominal radiographs for suspected intussusception","volume":"29","author":"Aronson","journal-title":"Pediatr Emerg Care"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"57","DOI":"10.14366\/usg.23153","article-title":"Feasibility of a deep learning artificial intelligence model for the diagnosis of pediatric ileocolic intussusception with grayscale ultrasonography","volume":"43","author":"Kim","journal-title":"Ultrasonography"},{"issue":"4","key":"R4","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1007\/s00247-018-4277-7","article-title":"Machine learning concepts, concerns and opportunities for a pediatric radiologist","volume":"49","author":"Moore","journal-title":"Pediatr Radiol"},{"issue":"11","key":"R5","doi-asserted-by":"publisher","first-page":"2139","DOI":"10.1007\/s00247-021-05057-0","article-title":"Current and emerging artificial intelligence applications for pediatric abdominal imaging","volume":"52","author":"Dillman","journal-title":"Pediatr Radiol"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-55536-6","article-title":"Performance of deep learning-based algorithm for detection of ileocolic intussusception on abdominal radiographs of young children","volume":"9","author":"Kim","journal-title":"Sci Rep"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1155\/2022\/9285238","article-title":"Performance of deep learning-based algorithm for detection of pediatric intussusception on abdominal ultrasound images","volume":"2022","author":"Li","journal-title":"Gastroenterol Res Pract"},{"issue":"4","key":"R8","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2023.106456","article-title":"Development and evaluation of an artificial intelligence system for children intussusception diagnosis using ultrasound images","volume":"26","author":"Chen","journal-title":"iScience"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-74653-1","article-title":"Deep learning algorithms for detecting and visualising intussusception on plain abdominal radiography in children: a retrospective multicenter study","volume":"10","author":"Kwon","journal-title":"Sci Rep"},{"key":"R10","unstructured":"Mingxing T Quoc L . EfficientNet: rethinking model scaling for convolutional neural networks. Presented at: Proceedings of the 36th International Conference on Machine Learning (ICML); Jun 10-15, 2019; Long Beach, CA, United States."},{"key":"R11","unstructured":"Lee JH Shin HJ . Pediatric Ileocolic Intussusception Screening from Abdominal X-rays. URL: http:\/\/intu.cdss.co.kr [Accessed 19-06-2025]"},{"issue":"11","key":"R12","doi-asserted-by":"publisher","first-page":"1743","DOI":"10.3348\/kjr.2021.0544","article-title":"Use of artificial intelligence-based software as medical devices for chest radiography: a position paper from the Korean Society of Thoracic Radiology","volume":"22","author":"Hwang","journal-title":"Korean J Radiol"},{"issue":"12","key":"R13","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.47172","article-title":"Evaluation of an artificial intelligence model for detection of pneumothorax and tension pneumothorax in chest radiographs","volume":"5","author":"Hillis","journal-title":"JAMA Netw Open"},{"issue":"168","key":"R14","doi-asserted-by":"publisher","first-page":"37286217","DOI":"10.1183\/16000617.0259-2022","article-title":"Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis","volume":"32","author":"Sugibayashi","journal-title":"Eur Respir Rev"},{"issue":"11","key":"R15","doi-asserted-by":"publisher","first-page":"8263","DOI":"10.1007\/s00330-023-09747-1","article-title":"Can incorrect artificial intelligence (AI) results impact radiologists, and if so, what can we do about it? A multi-reader pilot study of lung cancer detection with chest radiography","volume":"33","author":"Bernstein","journal-title":"Eur Radiol"},{"issue":"12","key":"R16","doi-asserted-by":"publisher","first-page":"9664","DOI":"10.1007\/s00330-021-08074-7","article-title":"AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset","volume":"31","author":"Yoo","journal-title":"Eur Radiol"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1148\/radiol.2020203538","article-title":"Deep learning demonstrates potential for lung cancer detection in chest radiography","volume":"297","author":"Armato","journal-title":"Radiology"},{"issue":"5","key":"R18","doi-asserted-by":"publisher","first-page":"792","DOI":"10.3348\/kjr.2020.0941","article-title":"Automated bone age assessment using artificial intelligence: the future of bone age assessment","volume":"22","author":"Lee","journal-title":"Korean J Radiol"},{"issue":"11","key":"R19","doi-asserted-by":"publisher","first-page":"1151","DOI":"10.3348\/kjr.2023.0092","article-title":"Bone age assessment using artificial intelligence in Korean pediatric population: a comparison of deep-learning models trained with healthy chronological and Greulich-Pyle ages as labels","volume":"24","author":"Kim","journal-title":"Korean J Radiol"},{"issue":"7","key":"R20","doi-asserted-by":"publisher","first-page":"683","DOI":"10.3349\/ymj.2022.63.7.683","article-title":"Re-assessment of applicability of Greulich and Pyle-based bone age to Korean children using manual and deep learning-based automated method","volume":"63","author":"Hwang","journal-title":"Yonsei Med J"},{"issue":"11","key":"R21","doi-asserted-by":"publisher","first-page":"679","DOI":"10.3349\/ymj.2023.0244","article-title":"Bone age estimation and prediction of final adult height using deep learning","volume":"64","author":"Suh","journal-title":"Yonsei Med J"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1148\/radiol.210937","article-title":"Improving radiographic fracture recognition performance and efficiency using artificial intelligence","volume":"302","author":"Guermazi","journal-title":"Radiology"},{"issue":"12","key":"R23","doi-asserted-by":"publisher","first-page":"594","DOI":"10.1016\/j.diii.2022.06.004","article-title":"Added value of an artificial intelligence solution for fracture detection in the radiologist\u2019s daily trauma emergencies workflow","volume":"103","author":"Canoni-Meynet","journal-title":"Diagn Interv Imaging"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1038\/s41746-023-00829-4","article-title":"The impact of artificial intelligence on the reading times of radiologists for chest radiographs","volume":"6","author":"Shin","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0282123","article-title":"Hospital-wide survey of clinical experience with artificial intelligence applied to daily chest radiographs","volume":"18","author":"Shin","journal-title":"PLoS ONE"},{"issue":"2","key":"R26","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1148\/radiol.2021202818","article-title":"Added value of deep learning-based detection system for multiple major findings on chest radiographs: a randomized crossover study","volume":"299","author":"Sung","journal-title":"Radiology"},{"issue":"2","key":"R27","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1007\/s00247-021-05072-1","article-title":"Artificial intelligence development in pediatric body magnetic resonance imaging: best ideas to adapt from adults","volume":"52","author":"Moore","journal-title":"Pediatr Radiol"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-14519-w","article-title":"Diagnostic performance of artificial intelligence approved for adults for the interpretation of pediatric chest radiographs","volume":"12","author":"Shin","journal-title":"Sci Rep"},{"issue":"1121","key":"R29","doi-asserted-by":"publisher","DOI":"10.1259\/bjr.20201263","article-title":"Automated detection of pneumonia cases using deep transfer learning with paediatric chest X-ray images","volume":"94","author":"Salehi","journal-title":"Br J Radiol"},{"issue":"1","key":"R30","doi-asserted-by":"publisher","first-page":"34376402","DOI":"10.1136\/bmjresp-2021-001045","article-title":"Chest radiograph-based artificial intelligence predictive model for mortality in community-acquired pneumonia","volume":"8","author":"Quah","journal-title":"BMJ Open Respir Res"},{"issue":"11","key":"R31","doi-asserted-by":"publisher","first-page":"2065","DOI":"10.1007\/s00247-021-05086-9","article-title":"The current and future roles of artificial intelligence in pediatric radiology","volume":"52","author":"Otjen","journal-title":"Pediatr Radiol"},{"issue":"21","key":"R32","doi-asserted-by":"publisher","first-page":"1981","DOI":"10.1056\/NEJMra2301725","article-title":"The current and future state of AI interpretation of medical images","volume":"388","author":"Rajpurkar","journal-title":"N Engl J Med"},{"issue":"2","key":"R33","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1053\/j.ro.2023.01.005","article-title":"Artificial intelligence in radiology: overview of application types, design, and challenges","volume":"58","author":"Moassefi","journal-title":"Semin Roentgenol"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2025\/1\/e72097","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:02:13Z","timestamp":1782842533000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2025\/1\/e72097"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,8]]},"references-count":33,"URL":"https:\/\/doi.org\/10.2196\/72097","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,8]]},"article-number":"v27i5e72097"}}