{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T05:21:16Z","timestamp":1788153676237,"version":"build-2803163510"},"reference-count":36,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2026,8,30]],"date-time":"2026-08-30T00:00:00Z","timestamp":1788048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Parotid ultrasound requires integration of sonographic morphology with clinical context under uncertainty, but conventional agreement and performance metrics provide limited insight into examiner-dependent classification patterns. Building on previously derived examiner-specific surrogate models, we applied a behavioral decision-science framework to examine observable classification patterns and case-level ambiguity. Methods: In this exploratory retrospective single-center secondary analysis, six examiners rated predefined ultrasound image sets from 149 histopathologically verified lesions. Surrogate-tree structure, feature\u2013reference-class associations, signal detection metrics, and case-level ambiguity were analyzed. Internal robustness was assessed using cross-validation and sensitivity analyses. Results: Pruned surrogate trees identified a context-based classification pattern, a morphology-based classification pattern, and a hierarchical morphology-context pattern. Tumor history and lesion boundary were the only retained upper-level classification cues and showed the strongest overall categorical and sonographic associations with the histopathological reference class, respectively. Across 30 cross-validation trees, boundary was the root split in 22 and tumor history in eight; no other root feature occurred. Examiners differed in diagnostic discrimination and estimated decision criterion, with Examiner 3 showing the highest discrimination. Boundary remained the sonographic feature most strongly associated with the histopathological reference class across sensitivity analyses. Descriptor disagreement was associated with decision disagreement and error burden, and four cases showed unanimous but incorrect classifications. Conclusions: In this exploratory retrospective single-center cohort of histopathologically verified lesions, examiner-dependent classification was characterized by observable, model-derived cue structure, feature-use\/reference-association alignment, estimated decision-criterion patterns, and case-level ambiguity. These cohort-specific findings do not directly measure examiner cognition and should not be generalized to unselected parotid-ultrasound populations; however, the complementary analyses provide an empirically grounded framework for characterizing observable examiner-dependent classification and identifying testable targets for prospective studies of examiner feedback, descriptor standardization, and decision support.<\/jats:p>","DOI":"10.3390\/diagnostics16172793","type":"journal-article","created":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T04:24:03Z","timestamp":1788150243000},"page":"2793","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Observable Classification Patterns and Diagnostic Uncertainty in Parotid Ultrasound: A Multimethod Secondary Analysis"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9057-7684","authenticated-orcid":false,"given":"Lukas","family":"Pillong","sequence":"first","affiliation":[{"name":"Department of Otorhinolaryngology, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]},{"given":"Oliver","family":"Walzer","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Marlene","family":"Peters","sequence":"additional","affiliation":[{"name":"Medical Faculty, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"investigation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Ida","family":"Ohnesorg","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine I\u2014Hematology, Oncology, Clinical Immunology and Rheumatology, Saarland University Medical Center, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"investigation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1122-1557","authenticated-orcid":false,"given":"Jan","family":"Palm","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Saarland University Medical Center, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"resources","vocabulary":"credit"}]},{"given":"Victoria","family":"Bozzato","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"resources","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Malvina","family":"Garner","sequence":"additional","affiliation":[{"name":"Department of Neuroradiology, Saarland University Medical Center, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"investigation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Bernhard","family":"Schick","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9648-8834","authenticated-orcid":false,"given":"Alessandro","family":"Bozzato","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"validation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"given":"Thomas","family":"J\u00e4ger","sequence":"additional","affiliation":[{"name":"Medical Faculty, Saarland University, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"software","vocabulary":"credit"}]},{"given":"Adrian","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Microsystems Technology, University of Applied Sciences Kaiserslautern, Amerikastra\u00dfe 1, 66482 Zweibr\u00fccken, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"software","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"}]},{"given":"Susan","family":"Pulham","sequence":"additional","affiliation":[{"name":"Faculty of Business Administration and Economics, Saarland University of Applied Sciences (Hochschule f\u00fcr Technik und Wirtschaft, htw saar), Waldhausweg 14, 66123 Saarbr\u00fccken, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5466-6579","authenticated-orcid":false,"given":"Manfred","family":"Voges","sequence":"additional","affiliation":[{"name":"Department of Radiology, Saarland University Medical Center, Kirrberger Stra\u00dfe 100, 66421 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1148\/rg.263055024","article-title":"US of the Major Salivary Glands: Anatomy and Spatial Relationships, Pathologic Conditions, and Pitfalls1","volume":"26","author":"Bialek","year":"2006","journal-title":"RadioGraphics"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.21037\/gs-20-530","article-title":"Narrative Review of Multiparametric Ultrasound in Parotid Gland Evaluation","volume":"9","author":"Martino","year":"2020","journal-title":"Gland Surg."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3300","DOI":"10.1016\/j.ejrad.2012.01.004","article-title":"Multimodal Ultrasonographic Characterisation of Parotid Gland Lesions\u2014A Pilot Study","volume":"81","author":"Knopf","year":"2012","journal-title":"Eur. J. Radiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1055\/s-0031-1299130","article-title":"Evaluation of Parotid Gland Lesions with Standard Ultrasound, Color Duplex Sonography, Sonoelastography, and Acoustic Radiation Force Impulse Imaging\u2014A Pilot Study","volume":"33","author":"Mansour","year":"2012","journal-title":"Ultraschall Med. Eur. J. Ultrasound"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1055\/s-0035-1553267","article-title":"Multimodal Ultrasonographic Pathway of Parotid Gland Lesions","volume":"38","author":"Mansour","year":"2015","journal-title":"Ultraschall Med. Eur. J. Ultrasound"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"20210484","DOI":"10.1259\/dmfr.20210484","article-title":"Diagnostic Grading of Parotid Lesions by Conventional Ultrasound: A Pilot Study","volume":"51","author":"Wang","year":"2022","journal-title":"Dentomaxillofacial Radiol."},{"key":"ref_7","first-page":"194","article-title":"Ultrasonography Techniques in the Preoperative Diagnosis of Parotid Gland Tumors\u2014An Updated Review of the Literature","volume":"23","author":"Stoia","year":"2020","journal-title":"Med. Ultrason."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20220972","DOI":"10.1259\/bjr.20220972","article-title":"Interobserver Variability Studies in Diagnostic Imaging: A Methodological Systematic Review","volume":"96","author":"Quinn","year":"2023","journal-title":"Br. J. Radiol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Pillong, L., Ohnesorg, I., Brust, L.A., Palm, J., Schulze-Berge, J., Bozzato, V., Voges, M., M\u00fcller, A., Garner, M., and Bozzato, A. (2026). Differential Diagnosis of Parotid Tumors on Ultrasound: Interobserver Variability and Examiner-Specific Decision Rules\u2014A Machine Learning Approach. Diagnostics, 16.","DOI":"10.3390\/diagnostics16060880"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2533","DOI":"10.1016\/j.ejrad.2015.09.001","article-title":"Pilot Study of Ultrasound Parotid Imaging Reporting and Data System (PIRADS): Inter-Observer Agreement","volume":"84","author":"Razek","year":"2015","journal-title":"Eur. J. Radiol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1016\/j.diii.2020.09.001","article-title":"Interobserver Agreement Issues in Radiology","volume":"101","author":"Benchoufi","year":"2020","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/S1078-8174(98)80030-8","article-title":"Evaluation of Diagnostic Performance in Radiography","volume":"4","author":"Manning","year":"1998","journal-title":"Radiography"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1007\/s11606-017-4164-1","article-title":"Defining and Measuring Diagnostic Uncertainty in Medicine: A Systematic Review","volume":"33","author":"Bhise","year":"2018","journal-title":"J. Gen. Intern. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1053\/j.semnuclmed.2011.07.001","article-title":"New Developments in Observer Performance Methodology in Medical Imaging","volume":"41","author":"Chakraborty","year":"2011","journal-title":"Semin. Nucl. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1037\/h0047470","article-title":"Representative Design and Probabilistic Theory in a Functional Psychology","volume":"62","author":"Brunswik","year":"1955","journal-title":"Psychol. Rev."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1177\/0272989X8800800404","article-title":"Use of Linear Models to Analyze Physicians\u2019 Decisions","volume":"8","author":"Wigton","year":"1988","journal-title":"Med. Decis. Mak."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Scarffe, A., Coates, A., Brand, K., and Michalowski, W. (2024). Decision Threshold Models in Medical Decision Making: A Scoping Literature Review. BMC Med. Inform. Decis. Mak., 24.","DOI":"10.1186\/s12911-024-02681-2"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1146\/annurev-psych-120709-145346","article-title":"Heuristic Decision Making","volume":"62","author":"Gigerenzer","year":"2011","journal-title":"Annu. Rev. Psychol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1111\/jep.13998","article-title":"Dual Process Models of Clinical Reasoning: The Central Role of Knowledge in Diagnostic Expertise","volume":"30","author":"Norman","year":"2024","journal-title":"J. Eval. Clin. Pract."},{"key":"ref_20","first-page":"539","article-title":"Cognitive Biases and Heuristics in Medical Decision Making","volume":"35","author":"Krieger","year":"2014","journal-title":"Med. Decis. Mak."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1126\/science.3287615","article-title":"Measuring the Accuracy of Diagnostic Systems","volume":"240","author":"Swets","year":"1988","journal-title":"Science"},{"key":"ref_22","unstructured":"Swets, J.A. (1996). Signal Detection Theory and ROC Analysis in Psychology and Diagnostics, Psychology Press."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/S0001-2998(78)80014-2","article-title":"Basic Principles of ROC Analysis","volume":"8","author":"Metz","year":"1978","journal-title":"Semin. Nucl. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1097\/00004424-198609000-00009","article-title":"ROC Methodology in Radiologic Imaging","volume":"21","author":"Metz","year":"1986","journal-title":"Investig. Radiol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1056\/NEJM198005153022003","article-title":"The Threshold Approach to Clinical Decision Making","volume":"302","author":"Pauker","year":"1980","journal-title":"N. Engl. J. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1146\/annurev-orgpsych-012420-090506","article-title":"Smart Heuristics for Individuals, Teams, and Organizations","volume":"9","author":"Gigerenzer","year":"2022","journal-title":"Annu. Rev. Organ. Psychol. Organ. Behav."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/s10459-009-9179-x","article-title":"Dual Processing and Diagnostic Errors","volume":"14","author":"Norman","year":"2009","journal-title":"Adv. Health Sci. Educ."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1017\/S0140525X00003447","article-title":"Simple Heuristics That Make Us Smart","volume":"23","author":"Todd","year":"2000","journal-title":"Behav. Brain Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.suronc.2017.11.007","article-title":"Metastases to the Parotid Gland\u2014A Review of the Clinicopathological Evolution, Molecular Mechanisms and Management","volume":"27","author":"Badlani","year":"2018","journal-title":"Surg. Oncol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1007\/s12070-020-01981-z","article-title":"Parotid Gland Tumors: A Multicenter Analysis of 1020 Cases. Increasing Incidence of Warthin\u2019s Tumor","volume":"74","author":"Saravakos","year":"2022","journal-title":"Indian J. Otolaryngol. Head Neck Surg."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pecorari, G., Pizzo, C., Briguglio, M., Cravero, E., and Riva, G. (2023). Primary and Secondary Tumors of the Parotid Gland: Clinical Features and Prognosis. Cancers, 15.","DOI":"10.3390\/cancers15041293"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1055\/s-2008-1056246","article-title":"Die Erkrankungen Der Parotis Im Hochaufl\u00f6senden Real-Time-Schnittbild","volume":"137","author":"Pirschel","year":"1982","journal-title":"Front. R\u00f6ntgenstr."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"454","DOI":"10.1055\/s-0042-109171","article-title":"Parotid Gland Lesions: Multiparametric Ultrasound and MRI Features","volume":"37","author":"Cantisani","year":"2016","journal-title":"Ultraschall Med. Eur. J. Ultrasound."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chiari, F., Dalmazzini, C., Fermi, M., Presutti, L., Caporale, C.D., and Guarino, P. (2026). Basal Cell Adenocarcinoma of the Parotid Gland: Prognostic Factors and Survival Outcomes: A Systematic Review and Pooled Patient-Level Analysis. Eur. Arch. Oto-Rhino-Laryngol., 283.","DOI":"10.1007\/s00405-026-10313-w"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1037\/0033-2909.130.6.959","article-title":"The Role of Representative Design in an Ecological Approach to Cognition","volume":"130","author":"Dhami","year":"2004","journal-title":"Psychol. Bull."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1037\/0033-2909.134.3.404","article-title":"Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies","volume":"134","author":"Karelaia","year":"2008","journal-title":"Psychol. Bull."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/17\/2793\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T04:54:45Z","timestamp":1788152085000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/17\/2793"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,30]]},"references-count":36,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["diagnostics16172793"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16172793","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,30]]}}}