{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T09:58:09Z","timestamp":1785491889564,"version":"3.56.0"},"reference-count":34,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T00:00:00Z","timestamp":1785456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["JP23K16235"],"award-info":[{"award-number":["JP23K16235"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Clin. Pract.","Clinics and Practice"],"abstract":"<jats:p>Background\/Objectives: The mandibular cortical index (MCI) is a valuable screening tool for osteoporosis on dental panoramic radiographs, but its assessment is subject to inter-examiner variability. This study evaluated the reproducibility and inter-rater agreement of MCI classification by a closed-source generative AI tool (NotebookLM, Google) compared with eight dentists of varying clinical experience. Methods: One hundred panoramic radiographs were classified according to the three-category MCI in two sessions held at least two weeks apart. Intra-examiner reliability, inter-examiner agreement, and agreement with a reference radiologist were assessed using linearly weighted kappa coefficients. The study was designed as a descriptive reliability study rather than a formal equivalence trial. Results: The intra-examiner reliability of the AI was exceptionally high (\u03ba = 0.987). However, agreement between the AI and the dentists remained at \u201cslight agreement\u201d or lower (\u03ba &lt; 0.2) for every pairing, with 95% confidence intervals that included zero; no formal global hypothesis test was performed, and these individual interval estimates should not be interpreted as proof of the absence of agreement beyond chance. A \u201ctwo-level discrepancy,\u201d in which the AI interchanged Class 1 (normal) and Class 3 (severe), occurred in 10\u201318% of cases. The dentists showed a possible learning effect, with inter-examiner agreement improving between sessions. Conclusions: Despite the high reproducibility of the NotebookLM configuration evaluated in this study, agreement with the dentists remained at \u201cslight\u201d or lower (\u03ba &lt; 0.2) in MCI classification. As classifications were not validated against bone mineral density or an adjudicated reference standard, these findings concern reproducibility and agreement rather than diagnostic or screening performance and characterize a single LLM-based platform rather than generative AI in general.<\/jats:p>","DOI":"10.3390\/clinpract16080142","type":"journal-article","created":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T08:20:18Z","timestamp":1785486018000},"page":"142","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Reproducibility of Mandibular Cortical Index Classification Among Dental Examiners and a Single Generative AI Platform: An Observer Agreement Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6130-1259","authenticated-orcid":false,"given":"Keisuke","family":"Seki","sequence":"first","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minori","family":"Kashima","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taiki","family":"Akiyama","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Atsushi","family":"Kobayashi","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5545-1771","authenticated-orcid":false,"given":"Ko","family":"Dezawa","sequence":"additional","affiliation":[{"name":"Department of Oral and Maxillofacial Radiology, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9432-7478","authenticated-orcid":false,"given":"Yoshimasa","family":"Takeuchi","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mika","family":"Furuchi","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8139-9354","authenticated-orcid":false,"given":"Atsushi","family":"Kamimoto","sequence":"additional","affiliation":[{"name":"Department of Comprehensive Dentistry and Clinical Education, Nihon University School of Dentistry, Tokyo 101-8310, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s11282-021-00518-6","article-title":"Clinical guidelines for the application of panoramic radiographs in screening for osteoporosis","volume":"37","author":"Taguchi","year":"2021","journal-title":"Oral Radiol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Seki, K., Nagasaki, M., Yoshino, T., Yano, M., Kawamoto, A., and Shimizu, O. (2024). Radiographical diagnostic evaluation of mandibular cortical index classification and mandibular cortical width in female patients prescribed antiosteoporosis medication. Diagnostics, 14.","DOI":"10.3390\/diagnostics14101009"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1111\/j.1600-0722.1994.tb01156.x","article-title":"Pantomography in assessment of the osteoporosis risk group","volume":"102","author":"Klemetti","year":"1994","journal-title":"Eur. J. Oral Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101754","DOI":"10.1016\/j.berh.2022.101754","article-title":"Osteoporosis in 2022: Care gaps to screening and personalised medicine","volume":"36","author":"Curtis","year":"2022","journal-title":"Best Pract. Res. Clin. Rheumatol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1080\/03007995.2022.2141483","article-title":"Closing the gap in osteoporosis management: The critical role of primary care in bone health","volume":"39","author":"Singer","year":"2023","journal-title":"Curr. Med. Res. Opin."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/sj\/dmfr\/4600427","article-title":"The reproducibility of the mandibular cortical index","volume":"28","author":"Jowitt","year":"1999","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1016\/j.prosdent.2021.02.010","article-title":"Artificial intelligence applications in restorative dentistry: A systematic review","volume":"128","author":"Vyas","year":"2022","journal-title":"J. Prosthet. Dent."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.prosdent.2021.05.008","article-title":"Artificial intelligence applications in implant dentistry: A systematic review","volume":"129","author":"Vyas","year":"2023","journal-title":"J. Prosthet. Dent."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1093\/dmfr\/twaf060","article-title":"The performance of large language models in dentomaxillofacial radiology: A systematic review","volume":"54","author":"Liu","year":"2025","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nguyen, V.A., Vuong, T.Q.T., and Nguyen, V.H. (2025). Benchmarking large-language-model vision capabilities in oral and maxillofacial anatomy: A cross-sectional study. PLoS ONE, 20.","DOI":"10.1371\/journal.pone.0335775"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"World Medical Association (2013). World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. JAMA, 310, 2191\u20132194.","DOI":"10.1001\/jama.2013.281053"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1136\/bmj.39335.541782.AD","article-title":"Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies","volume":"335","author":"Altman","year":"2007","journal-title":"BMJ"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.jclinepi.2010.03.002","article-title":"Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed","volume":"64","author":"Kottner","year":"2011","journal-title":"J. Clin. Epidemiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1259\/dmfr\/56808511","article-title":"Mandibular cortical index evaluation in masked and unmasked panoramic radiographs","volume":"38","author":"Yasar","year":"2009","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1037\/h0026256","article-title":"Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit","volume":"70","author":"Cohen","year":"1968","journal-title":"Psychol. Bull."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1038\/bmt.2012.244","article-title":"Investigation of the freely available easy-to-use software \u2018EZR\u2019 for medical statistics","volume":"48","author":"Kanda","year":"2013","journal-title":"Bone Marrow Transplant."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/0895-4356(90)90158-L","article-title":"High agreement but low kappa: I. The problems of two paradoxes","volume":"43","author":"Feinstein","year":"1990","journal-title":"J. Clin. Epidemiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1348\/000711006X126600","article-title":"Computing inter-rater reliability and its variance in the presence of high agreement","volume":"61","author":"Gwet","year":"2008","journal-title":"Br. J. Math. Stat. Psychol."},{"key":"ref_20","unstructured":"Gwet, K.L. (2014). Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters, Advanced Analytics, LLC. [4th ed.]."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"20220108","DOI":"10.1259\/dmfr.20220108","article-title":"Comparison of five convolutional neural networks for predicting osteoporosis based on mandibular cortical index on panoramic radiographs","volume":"51","author":"Tassoker","year":"2022","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"20220135","DOI":"10.1259\/dmfr.20220135","article-title":"Osteoporosis screening support system from panoramic radiographs using deep learning by convolutional neural network","volume":"51","author":"Nakamoto","year":"2022","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_23","first-page":"100932","article-title":"Large language models in medical image analysis: A systematic survey and future directions","volume":"48","author":"Urooj","year":"2025","journal-title":"Biomed. J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e232756","DOI":"10.1148\/radiol.232756","article-title":"Chatbots and large language models in radiology: A practical primer for clinical and research applications","volume":"310","author":"Bhayana","year":"2024","journal-title":"Radiology"},{"key":"ref_25","unstructured":"Personal Information Protection Commission, and Ministry of Health, Labour and Welfare (2026, April 26). Guidance on Appropriate Handling of Personal Information by Medical and Long-Term Care Service Providers. Available online: https:\/\/www.ppc.go.jp\/personalinfo\/legal\/iryoukaigo_guidance\/."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"20552076251343959","DOI":"10.1177\/20552076251343959","article-title":"Data privacy in healthcare: Global challenges and solutions","volume":"11","author":"Conduah","year":"2025","journal-title":"Digit. Health"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Meszaros, J., Minari, J., and Huys, I. (2022). The future regulation of artificial intelligence systems in healthcare services and medical research in the European Union. Front. Genet., 13.","DOI":"10.3389\/fgene.2022.927721"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1093\/jamia\/ocaf005","article-title":"Development of secure infrastructure for advancing generative artificial intelligence research in healthcare at an academic medical center","volume":"32","author":"Ng","year":"2025","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"15006","DOI":"10.1038\/s41598-021-94093-9","article-title":"Clinically applicable artificial intelligence system for dental diagnosis with CBCT","volume":"11","author":"Ezhov","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ding, H., Wu, J., Zhao, W., Matinlinna, J.P., Burrow, M.F., and Tsoi, J.K.H. (2023). Artificial intelligence in dentistry\u2014A review. Front. Dent. Med., 4.","DOI":"10.3389\/fdmed.2023.1085251"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e52346","DOI":"10.2196\/52346","article-title":"Artificial intelligence in dental education: Opportunities and challenges of large language models and multimodal foundation models","volume":"10","author":"Claman","year":"2024","journal-title":"JMIR Med. Educ."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1111\/eje.13074","article-title":"Integrating generative AI in dental education: A scoping review of current practices and recommendations","volume":"29","author":"Uribe","year":"2025","journal-title":"Eur. J. Dent. Educ."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"105650","DOI":"10.1016\/j.jdent.2025.105650","article-title":"Artificial intelligence for osteoporosis detection on panoramic radiography: A systematic review and meta-analysis","volume":"156","author":"Ghasemi","year":"2025","journal-title":"J. Dent."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00198-024-07229-8","article-title":"Diagnostic accuracy of artificial intelligence models in detecting osteoporosis using dental images: A systematic review and meta-analysis","volume":"36","author":"Khadivi","year":"2025","journal-title":"Osteoporos. Int."}],"container-title":["Clinics and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2039-7283\/16\/8\/142\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T09:07:32Z","timestamp":1785488852000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2039-7283\/16\/8\/142"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,31]]},"references-count":34,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["clinpract16080142"],"URL":"https:\/\/doi.org\/10.3390\/clinpract16080142","relation":{},"ISSN":["2039-7283"],"issn-type":[{"value":"2039-7283","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,31]]}}}