{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T17:03:15Z","timestamp":1789578195309,"version":"build-2803163510"},"reference-count":22,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T00:00:00Z","timestamp":1789516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Clinical prediction models may support staged triage in acute pelvic pain, but simplified integer scores can lose information and destabilise point assignments. We evaluated a tiered clinical\u2013ultrasound prediction pathway and the stability of its simplified score within an ultrasound-documented acute-pelvic-pain cohort. Methods: This single-centre retrospective cohort included 4800 encounters in 4320 women. Encounters from 2013 to 2019 formed the development cohort (n = 3360), and patient-separated encounters from 2020 to 2022 formed the temporal-evaluation cohort (n = 1440). M1 contained clinical and laboratory predictors without imaging variables, M2 added ultrasound descriptors, and M3 added cross-sectional-imaging descriptors. Multiple imputation, ridge logistic regression, patient-level cluster bootstrap, calibration, decision-curve analysis, reader reproducibility and score-stability analyses were used. Results: Urgent pathology occurred in 20.0% of development and 18.0% of temporal encounters. Temporal AUC was 0.841 for M1 and 0.845 for M2 (difference 0.004; 95% CI 0.001\u20130.009). Because inclusion required documented ultrasound, this difference estimates incremental information from the selected ultrasound descriptors within an already ultrasound-documented pathway and does not estimate the pathway-level benefit of performing ultrasound. The locked integer rule classified 2.9% as low risk, with 99.6% sensitivity and 3.5% specificity. Categorisation accounted for 0.057 of the total 0.060 AUC loss, and 798 distinct score cards arose in 800 bootstrap resamples. Conclusions: Continuous M2 retained good temporal discrimination, but the small M1-M2 difference does not establish whether ultrasound improves pathway-level triage or can safely alter subsequent management. The integer score had low rule-out yield and highly variable point assignments; independent evaluation should prioritise local calibration, clinical utility and score-card stability before implementation.<\/jats:p>","DOI":"10.3390\/diagnostics16183003","type":"journal-article","created":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T16:07:29Z","timestamp":1789574849000},"page":"3003","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Development and Temporal Evaluation of a Tiered Clinical\u2013Ultrasound Prediction Model and Simplified Integer Score for Time-Sensitive Pathology in Women with Acute Pelvic Pain: A Retrospective Cohort Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2769-886X","authenticated-orcid":false,"given":"Marijana","family":"Basta Nikoli\u0107","sequence":"first","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21137 Novi Sad, Serbia"},{"name":"Center for Radiology, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8005-046X","authenticated-orcid":false,"given":"Dragan","family":"Vasin","sequence":"additional","affiliation":[{"name":"Emergency Radiology Department, Center for Radiology, University Clinical Center of Serbia, 11000 Belgrade, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jelena","family":"Pilipovi\u0107 Grubor","sequence":"additional","affiliation":[{"name":"Center for Radiology, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5222-7096","authenticated-orcid":false,"given":"Mirela","family":"Jukovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21137 Novi Sad, Serbia"},{"name":"Center for Radiology, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dijana","family":"Ni\u0107iforovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21137 Novi Sad, Serbia"},{"name":"Center for Radiology, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3602-3522","authenticated-orcid":false,"given":"Dragan","family":"Nikoli\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21137 Novi Sad, Serbia"},{"name":"Clinic for Vascular and Endovascular Surgery, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanja","family":"Stojanovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, University of Novi Sad, 21137 Novi Sad, Serbia"},{"name":"Center for Radiology, University Clinical Center of Vojvodina, 21000 Novi Sad, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"20210281","DOI":"10.1259\/bjr.20210281","article-title":"Imaging of acute pelvic pain","volume":"94","author":"Spasic","year":"2021","journal-title":"Br. J. Radiol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Expert Panel on GYN and OB Imaging, Brook, O.R., Dadour, J.R., Robbins, J.B., Wasnik, A.P., Akin, E.A., Borloz, M.P., Dawkins, A.A., Feldman, M.K., and Jones, L.P. (2024). ACR Appropriateness Criteria\u00ae Acute Pelvic Pain in the Reproductive Age Group: 2023 Update. J. Am. Coll. Radiol., 21, S3\u2013S20.","DOI":"10.1016\/j.jacr.2024.02.014"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"573","DOI":"10.7326\/0003-4819-147-8-200710160-00010","article-title":"STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement","volume":"147","author":"Altman","year":"2007","journal-title":"Ann. Intern Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.jclinepi.2015.04.005","article-title":"Prediction models need appropriate internal, internal-external, and external validation","volume":"69","author":"Steyerberg","year":"2016","journal-title":"J. Clin. Epidemiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e074819","DOI":"10.1136\/bmj-2023-074819","article-title":"Evaluation of clinical prediction models (part 1): From development to external validation","volume":"384","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e074820","DOI":"10.1136\/bmj-2023-074820","article-title":"Evaluation of clinical prediction models (part 2): How to undertake an external validation study","volume":"384","author":"Riley","year":"2024","journal-title":"BMJ"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1002\/sim.2331","article-title":"Dichotomizing continuous predictors in multiple regression: A bad idea","volume":"25","author":"Royston","year":"2006","journal-title":"Stat. Med."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sinkovec, H., Heinze, G., Blagus, R., and Geroldinger, A. (2021). To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. BMC Med. Res. Methodol., 21.","DOI":"10.1186\/s12874-021-01374-y"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"m441","DOI":"10.1136\/bmj.m441","article-title":"Calculating the sample size required for developing a clinical prediction model","volume":"368","author":"Riley","year":"2020","journal-title":"BMJ"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1186\/s41512-020-00082-3","article-title":"Impact of sample size on the stability of risk scores from clinical prediction models: A case study in cardiovascular disease","volume":"4","author":"Pate","year":"2020","journal-title":"Diagn. Progn. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e2200302","DOI":"10.1002\/bimj.202200302","article-title":"Stability of clinical prediction models developed using statistical or machine learning methods","volume":"65","author":"Riley","year":"2023","journal-title":"Biom. J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2455","DOI":"10.1177\/0962280218784726","article-title":"Sample size for binary logistic prediction models: Beyond events per variable criteria","volume":"28","author":"Moons","year":"2019","journal-title":"Stat. Methods Med. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1002\/sim.4067","article-title":"Multiple imputation using chained equations: Issues and guidance for practice","volume":"30","author":"White","year":"2011","journal-title":"Stat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3591","DOI":"10.1002\/sim.8682","article-title":"Handling missing predictor values when validating and applying a prediction model to new patients","volume":"39","author":"Hoogland","year":"2020","journal-title":"Stat. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2252","DOI":"10.1002\/sim.7654","article-title":"Bootstrap inference when using multiple imputation","volume":"37","author":"Schomaker","year":"2018","journal-title":"Stat. Med."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Steyerberg, E.W. (2019). Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating, Springer. [2nd ed.].","DOI":"10.1007\/978-3-030-16399-0"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1002\/sim.1742","article-title":"Presentation of multivariate data for clinical use: The Framingham Study risk score functions","volume":"23","author":"Sullivan","year":"2004","journal-title":"Stat. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: A novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med. Decis. Mak."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1186\/s41512-019-0064-7","article-title":"A simple, step-by-step guide to interpreting decision curve analysis","volume":"3","author":"Vickers","year":"2019","journal-title":"Diagn. Progn. Res."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/18\/3003\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T16:18:33Z","timestamp":1789575513000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/18\/3003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,16]]},"references-count":22,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["diagnostics16183003"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16183003","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,16]]}}}