{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,29]],"date-time":"2026-10-29T22:00:30Z","timestamp":1793311230892,"version":"4.5.0"},"reference-count":41,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:00:00Z","timestamp":1642723200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ho Chi Minh University of Pharmacy and Medicine"},{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["110-2628-B-038-014 and DP2-110-21121-01-A-04 and DP2-109-21121-01-A-04"],"award-info":[{"award-number":["110-2628-B-038-014 and DP2-110-21121-01-A-04 and DP2-109-21121-01-A-04"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010002","name":"Ministry of Education","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Background<\/jats:title>\n<jats:p>A brief gonadotropin-releasing hormone analogues (GnRHa) stimulation test which solely focused on LH 30-minute post-stimulation was considered to identify girls with central precocious puberty (CPP). However, it was tested using traditional statistical methods. With advanced computer science, we aimed to develop a machine learning-based diagnostic model that processed baseline CPP-related variables and a brief GnRHa stimulation test for CPP diagnosis.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Methods<\/jats:title>\n<jats:p>We recruited girls suspected of precocious puberty and underwent a GnRHa stimulation test at Children Hospital 2, Vietnam, and Cathay General Hospital, Taiwan. Clinical data, bone age measurement, and 30-min post-stimulation blood test were used to build up the predictive model. The candidate model was developed by different machine learning algorithms that were mainly evaluated by sensitivity, specificity, the area under the receiver operator characteristic curve (AUC), and F1-score in internal and external validation data to classify girls as CPP and non-CPP at different time-points (0-min, 30-min, 60-min, and 120-min post-stimulation).<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Results<\/jats:title>\n<jats:p>Among the 614 girls diagnosed with PP, 524 (85.3%) had CPP. The random forest algorithm yielded the highest value of F1-score (0.976), specificity (0.893), positive predicted value (0.987), and relatively high value of AUC (0.972) that contributed to high probability to identify CPP. The performance metrics of the 30-min post-stimulation diagnostic model including sensitivity and specificity surpassed those of the 0-minute model (0-min) and were equivalent to those of the model obtained 60-min and 120-min post-stimulation. Hence, our machine learning-based model helps shorten the stimulation test to 30 minutes after GnRHa injection, in general, it requires 120 minutes for a completed GnRHa stimulation test.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec004\">\n<jats:title>Conclusions<\/jats:title>\n<jats:p>We developed a diagnostic model based on clinical features and a single sample 30-minute post-stimulation to identify CPP in girls that can reduce distress for children caused by multiple blood samplings.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pone.0261965","type":"journal-article","created":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T19:37:51Z","timestamp":1642793871000},"page":"e0261965","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":21,"title":["Development and Validation of Clinical Diagnostic Model for Girls with Central Precocious Puberty: Machine-learning Approaches"],"prefix":"10.1371","volume":"17","author":[{"given":"Quynh Thi Vu","family":"Huynh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4896-7926","authenticated-orcid":true,"given":"Nguyen Quoc Khanh","family":"Le","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2914-5513","authenticated-orcid":true,"given":"Shih-Yi","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ban Tran","family":"Ho","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tru Huy","family":"Vu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong Thi Minh","family":"Pham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"An Le","family":"Pham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia-Woei","family":"Hou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5655-7292","authenticated-orcid":true,"given":"Ngan Thi Kim","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0682-3716","authenticated-orcid":true,"given":"Yang Ching","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2022,1,21]]},"reference":[{"issue":"10","key":"pone.0261965.ref001","doi-asserted-by":"crossref","first-page":"e2015665","DOI":"10.1001\/jamanetworkopen.2020.15665","article-title":"Trends in the Incidence of central precocious puberty and normal variant puberty among children in Denmark, 1998 to 2017","volume":"3","author":"EV Br\u00e4uner","year":"2020","journal-title":"AMA Netw Open"},{"issue":"3","key":"pone.0261965.ref002","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/S2213-8587(15)00380-0","article-title":"Causes, diagnosis, and treatment of central precocious puberty","volume":"4","author":"AC Latronico","year":"2016","journal-title":"Lancet Diabetes Endocrinol"},{"issue":"22","key":"pone.0261965.ref003","doi-asserted-by":"crossref","first-page":"2366","DOI":"10.1056\/NEJMcp0800459","article-title":"Precocious puberty","volume":"358","author":"J-C Carel","year":"2008","journal-title":"N Engl J Med"},{"key":"pone.0261965.ref004","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/B978-012440905-7\/50266-8","article-title":"Endocrinology and growth in children and adolescents","author":"M GAYA ARANOFF","year":"2004","journal-title":"Principles of Gender-Specific Medicine: Gender in the Genomic Era"},{"issue":"1","key":"pone.0261965.ref005","first-page":"1","article-title":"Puberty timing associated with diabetes, cardiovascular disease and also diverse health outcomes in men and women: the UK Biobank study","volume":"5","author":"FR Day","year":"2015","journal-title":"Scientific reports"},{"issue":"4","key":"pone.0261965.ref006","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1007\/s10802-013-9798-9","article-title":"Early pubertal timing as a vulnerability to depression symptoms: Differential effects of race and sex","volume":"42","author":"EJ Hamlat","year":"2014","journal-title":"J Abnorm Child Psychol"},{"issue":"1","key":"pone.0261965.ref007","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1093\/aje\/kwu113","article-title":"Age at menarche and risks of all-cause and cardiovascular death: a systematic review and meta-analysis","volume":"180","author":"D Charalampopoulos","year":"2014","journal-title":"Am J Epidemiol"},{"issue":"6","key":"pone.0261965.ref008","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1038\/ng.3841","article-title":"Genomic analyses identify hundreds of variants associated with age at menarche and support a role for puberty timing in cancer risk","volume":"49","author":"FR Day","year":"2017","journal-title":"Nat Genet"},{"issue":"11","key":"pone.0261965.ref009","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1007\/s12098-015-1761-z","article-title":"A gonadotropin-releasing hormone (GnRH) stimulation test before and after GnRH analogue treatment for central precocious puberty: has the gnrh test been adequately simplified?","volume":"82","author":"MS Kim","year":"2015","journal-title":"Indian J Pediatr"},{"issue":"1","key":"pone.0261965.ref010","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/S0022-3476(95)70254-7","article-title":"Normal ranges for immunochemiluminometric gonadotropin assays","volume":"127","author":"EK Neely","year":"1995","journal-title":"J Pediatr"},{"issue":"10","key":"pone.0261965.ref011","first-page":"3539","article-title":"Diagnostic value of fluorometric assays in the evaluation of precocious puberty","volume":"84","author":"V Brito","year":"1999","journal-title":"J Clin Endocrinol Metab"},{"issue":"6","key":"pone.0261965.ref012","doi-asserted-by":"crossref","first-page":"e1059","DOI":"10.1542\/peds.2008-1180","article-title":"Adequacy of a single unstimulated luteinizing hormone level to diagnose central precocious puberty in girls","volume":"123","author":"CP Houk","year":"2009","journal-title":"Pediatrics"},{"issue":"4","key":"pone.0261965.ref013","doi-asserted-by":"crossref","first-page":"196","DOI":"10.6065\/apem.2013.18.4.196","article-title":"Basal luteinizing hormone and follicular stimulating hormone: is it sufficient for the diagnosis of precocious puberty in girls?","volume":"18","author":"DS Lee","year":"2013","journal-title":"Ann Pediatr Endocrinol Metab"},{"issue":"S1","key":"pone.0261965.ref014","doi-asserted-by":"crossref","first-page":"P72","DOI":"10.1186\/1687-9856-2013-S1-P72","article-title":"Basal luteinizing hormone and follicular stimulating hormone: is it sufficient for diagnosis of precocious puberty in Korean girls?","volume":"2013","author":"SH Lee","year":"2013","journal-title":"Int J Pediatr Endocrinol"},{"issue":"166","key":"pone.0261965.ref015","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1530\/EJE-11-0720","article-title":"The utility of basal serum LH in prediction of central precocious puberty in girls","volume":"166","author":"Y Pasternak","year":"2012","journal-title":"Eur J Endocrinol"},{"issue":"6","key":"pone.0261965.ref016","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1111\/j.1365-2265.1995.tb02692.x","article-title":"A simplified gonadotrophin\u2010releasing hormone test for precocious puberty","volume":"42","author":"A Cavallo","year":"1995","journal-title":"Clin Endocrinol (Oxf)"},{"issue":"12","key":"pone.0261965.ref017","first-page":"4536","article-title":"A single sample subcutaneous luteinizing hormone (LH)-releasing hormone (LHRH) stimulation test for monitoring LH suppression in children with central precocious puberty receiving LHRH agonists","volume":"84","author":"ML Lawson","year":"1999","journal-title":"J Clin Endocrinol Metab"},{"issue":"1","key":"pone.0261965.ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1687-9856-2012-23","article-title":"A single sample GnRHa stimulation test in the diagnosis of precocious puberty","volume":"2012","author":"P Yazdani","year":"2012","journal-title":"Int J Pediatr Endocrinol"},{"issue":"1","key":"pone.0261965.ref019","doi-asserted-by":"crossref","first-page":"e11728","DOI":"10.2196\/11728","article-title":"Development of Prediction Models Using Machine Learning Algorithms for Girls with Suspected Central Precocious Puberty: Retrospective Study","volume":"7","author":"L Pan","year":"2019","journal-title":"JMIR Med Inform"},{"key":"pone.0261965.ref020","article-title":"Machine learning identifies girls with central precocious puberty based on multisource data","author":"L Pan","year":"2020","journal-title":"JAMIA Open"},{"issue":"4","key":"pone.0261965.ref021","doi-asserted-by":"crossref","first-page":"e752","DOI":"10.1542\/peds.2008-1783","article-title":"Consensus statement on the use of gonadotropin-releasing hormone analogs in children","volume":"123","author":"J-C Carel","year":"2009","journal-title":"Pediatrics"},{"issue":"235","key":"pone.0261965.ref022","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1136\/adc.44.235.291","article-title":"Variations in pattern of pubertal changes in girls","volume":"44","author":"WA Marshall","year":"1969","journal-title":"Arch Dis Child"},{"key":"pone.0261965.ref023","volume-title":"WHO AnthroPlus for personal computers manual: software for assessing growth of the world\u2019s children and adolescents","author":"M Bl\u00f6ssner","year":"2009"},{"issue":"4","key":"pone.0261965.ref024","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1038\/sj.ijo.0803300","article-title":"Body mass index in children and adolescents: considerations for population-based applications","volume":"30","author":"A Must","year":"2006","journal-title":"International journal of obesity"},{"key":"pone.0261965.ref025","doi-asserted-by":"crossref","DOI":"10.1097\/00000441-195909000-00030","volume-title":"Radiographic atlas of skeletal development of the hand and wrist","author":"WW Greulich","year":"1959"},{"issue":"4","key":"pone.0261965.ref026","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TSMC.1985.6313426","article-title":"A fuzzy K-nearest neighbor algorithm","volume":"SMC-15","author":"JM Keller","year":"1985","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"key":"pone.0261965.ref027","unstructured":"Rish I, editor An empirical study of the naive Bayes classifier. IJCAI 2001 workshop on empirical methods in artificial intelligence; 2001."},{"issue":"5","key":"pone.0261965.ref028","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1016\/S1532-0464(03)00034-0","article-title":"Logistic regression and artificial neural network classification models: a methodology review","volume":"35","author":"S Dreiseitl","year":"2002","journal-title":"J Biomed Inform"},{"key":"pone.0261965.ref029","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C, editors. \" Why should i trust you?\" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016.","DOI":"10.1145\/2939672.2939778"},{"key":"pone.0261965.ref030","doi-asserted-by":"crossref","unstructured":"Liu N, Kumara S, Reich E, editors. Explainable data-driven modeling of patient satisfaction survey data. 2017 IEEE International Conference on Big Data (Big Data); 2017: IEEE.","DOI":"10.1109\/BigData.2017.8258391"},{"issue":"1","key":"pone.0261965.ref031","doi-asserted-by":"crossref","first-page":"12","DOI":"10.4274\/jcrpe.v3i1.03","article-title":"GnRH stimulation test in precocious puberty: single sample is adequate for diagnosis and dose adjustment","volume":"3","author":"N Kandemir","year":"2011","journal-title":"J Clin Res Pediatr Endocrinol"},{"key":"pone.0261965.ref032","article-title":"Diagnostic evaluation of central precocious puberty in girls","author":"S-N Yeh","year":"2020","journal-title":"Pediatr Neonatol"},{"issue":"2","key":"pone.0261965.ref033","first-page":"109","article-title":"Development and verification of a diagnostic prediction model for girls with central precocious puberty","volume":"16","author":"W Wenyong","year":"2021","journal-title":"CJEBM"},{"key":"pone.0261965.ref034","first-page":"67","volume-title":"Sensitivity, specificity, accuracy, associated confidence interval and ROC analysis with practical SAS implementations","author":"W Zhu","year":"2010"},{"issue":"3\u20134","key":"pone.0261965.ref035","first-page":"215","article-title":"In children with premature adrenarche, bone age advancement by 2 or more years is common and generally benign","volume":"26","author":"DJ DeSalvo","year":"2013","journal-title":"J Pediatr Endocrinol Metab"},{"issue":"1","key":"pone.0261965.ref036","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.jpedp.2017.08.020","article-title":"Advanced bone age as an indicator facilitates the diagnosis of precocious puberty","volume":"94","author":"YQ Xu","year":"2018","journal-title":"Jornal de Pediatria (Vers\u00e3o em Portugu\u00eas)"},{"issue":"1","key":"pone.0261965.ref037","doi-asserted-by":"crossref","first-page":"20","DOI":"10.5152\/tpa.2015.2281","article-title":"Clinical and laboratory findings in the differential diagnosis of central precocious puberty and premature thelarche","volume":"50","author":"G \u00c7atl\u0131","year":"2015","journal-title":"Turk Pediatri Ars"},{"issue":"2","key":"pone.0261965.ref038","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1210\/jcem.84.2.5533","article-title":"Precocious puberty: who has it? Who should be treated?","volume":"84","author":"K. Oerter Klein","year":"1999","journal-title":"J Clin Endocrinol Metab"},{"issue":"11","key":"pone.0261965.ref039","doi-asserted-by":"crossref","first-page":"e0141844","DOI":"10.1371\/journal.pone.0141844","article-title":"A significant increase in the incidence of central precocious puberty among Korean girls from 2004 to 2010","volume":"10","author":"SH Kim","year":"2015","journal-title":"PLoS One"},{"issue":"11","key":"pone.0261965.ref040","doi-asserted-by":"crossref","first-page":"E537","DOI":"10.1503\/cmaj.121286","article-title":"Variation of a test\u2019s sensitivity and specificity with disease prevalence","volume":"185","author":"MM Leeflang","year":"2013","journal-title":"Cmaj"},{"issue":"4","key":"pone.0261965.ref041","doi-asserted-by":"crossref","first-page":"e195881","DOI":"10.1001\/jamapediatrics.2019.5881","article-title":"Worldwide secular trends in age at pubertal onset assessed by breast development among girls: a systematic review and meta-analysis","volume":"174","author":"C Eckert-Lind","year":"2020","journal-title":"JAMA Pediatr"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0261965","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T19:39:13Z","timestamp":1642793953000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0261965"}},"subtitle":[],"editor":[{"given":"Alok","family":"Deoraj","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,1,21]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1,21]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0261965","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,21]]}}}