{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:46:09Z","timestamp":1782845169437,"version":"3.54.5"},"reference-count":57,"publisher":"JMIR Publications Inc.","issue":"9","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>In the case of Korean institutions and enterprises that collect nonstandardized and nonunified formats of electronic medical examination results from multiple medical institutions, a group of experienced nurses who can understand the results and related contexts initially classified the reports manually. The classification guidelines were established by years of workers\u2019 clinical experiences and there were attempts to automate the classification work. However, there have been problems in which rule-based algorithms or human labor\u2013intensive efforts can be time-consuming or limited owing to high potential errors. We investigated natural language processing (NLP) architectures and proposed ensemble models to create automated classifiers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to develop practical deep learning models with electronic medical records from 284 health care institutions and open-source corpus data sets for automatically classifying 3 thyroid conditions: healthy, caution required, and critical. The primary goal is to increase the overall accuracy of the classification, yet there are practical and industrial needs to correctly predict healthy (negative) thyroid condition data, which are mostly medical examination results, and minimize false-negative rates under the prediction of healthy thyroid conditions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The data sets included thyroid and comprehensive medical examination reports. The textual data are not only documented in fully complete sentences but also written in lists of words or phrases. Therefore, we propose static and contextualized ensemble NLP network (SCENT) systems to successfully reflect static and contextual information and handle incomplete sentences. We prepared each convolution neural network (CNN)-, long short-term memory (LSTM)-, and efficiently learning an encoder that classifies token replacements accurately (ELECTRA)-based ensemble model by training or fine-tuning them multiple times. Through comprehensive experiments, we propose 2 versions of ensemble models, SCENT-v1 and SCENT-v2, with the single-architecture\u2013based CNN, LSTM, and ELECTRA ensemble models for the best classification performance and practical use, respectively. SCENT-v1 is an ensemble of CNN and ELECTRA ensemble models, and SCENT-v2 is a hierarchical ensemble of CNN, LSTM, and ELECTRA ensemble models. SCENT-v2 first classifies the 3 labels using an ELECTRA ensemble model and then reclassifies them using an ensemble model of CNN and LSTM if the ELECTRA ensemble model predicted them as \u201chealthy\u201d labels.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>SCENT-v1 outperformed all the suggested models, with the highest F1 score (92.56%). SCENT-v2 had the second-highest recall value (94.44%) and the fewest misclassifications for caution-required thyroid condition while maintaining 0 classification error for the critical thyroid condition under the prediction of the healthy thyroid condition.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The proposed SCENT demonstrates good classification performance despite the unique characteristics of the Korean language and problems of data lack and imbalance, especially for the extremely low amount of critical condition data. The result of SCENT-v1 indicates that different perspectives of static and contextual input token representations can enhance classification performance. SCENT-v2 has a strong impact on the prediction of healthy thyroid conditions.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/30223","type":"journal-article","created":{"date-parts":[[2021,8,2]],"date-time":"2021-08-02T19:36:21Z","timestamp":1627932981000},"page":"e30223","source":"Crossref","is-referenced-by-count":13,"title":["Automatic Classification of Thyroid Findings Using Static and Contextualized Ensemble Natural Language Processing Systems: Development Study"],"prefix":"10.2196","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8398-234X","authenticated-orcid":false,"given":"Dongyup","family":"Shin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1522-4539","authenticated-orcid":false,"given":"Hye Jin","family":"Kam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7722-7906","authenticated-orcid":false,"given":"Min-Seok","family":"Jeon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5853-7589","authenticated-orcid":false,"given":"Ha Young","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2021,9,21]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.2008.0816"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1093\/heapol\/czn037"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.4093\/dmj.2014.38.5.395"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1016\/j.imr.2013.06.004"},{"key":"ref5","unstructured":"Korea Ministry of Employment and LaborKorea Occupational Safety and Health Act of Korea2018-08-01http:\/\/www.moleg.go.kr\/english"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-002411"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000607"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2012-000820"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz200"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacr.2019.12.026"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1200\/cci.18.00080"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-019-0937-2"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-017-0468-7"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-019-0935-4"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2020.104140"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocy173"},{"key":"ref17","author":"LeCun, Y","year":"2016","journal-title":"Deep Learning"},{"key":"ref18","author":"Mikolov, T","year":"2013","journal-title":"arXiv"},{"key":"ref19","author":"Devlin, J","year":"2018","journal-title":"arXiv"},{"key":"ref20","author":"Clark, K","year":"2020","journal-title":"arXiv"},{"key":"ref21","unstructured":"ParkJKoELECTRA: Pretrained ELECTRA Model for KoreanGitHub repository20202020-11-18https:\/\/github.com\/monologg\/KoELECTRA"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"ref24","unstructured":"American Thyroid AssociationThyroid Function Tests20192020-12-28https:\/\/www.thyroid.org\/wp-content\/uploads\/patients\/brochures\/FunctionTests_brochure"},{"key":"ref25","unstructured":"National Cancer InstituteThyroid Cancer Treatment (Adult)20202020-12-28https:\/\/www.cancer.gov\/types\/thyroid\/patient\/thyroid-treatment-pdq"},{"key":"ref26","unstructured":"TsarfatyRSeddahDK\u00fcblerSNivreJParsing Morphologically Rich Languages: Introduction to the Special Issue2013032021-09-08https:\/\/aclanthology.org\/J13-1003.pdf"},{"key":"ref27","unstructured":"SeddahDK\u00fcblerSTsarfatyRIntroducing the spmrl 2014 shared task on parsing morphologically-rich languages20142021-09-08https:\/\/aclanthology.org\/W14-6111.pdf"},{"key":"ref28","unstructured":"LeeYWYooYHMeCab-Ko-Dic, A Piece of Silver Coin Project2020-09-15http:\/\/eunjeon.blogspot.com\/"},{"key":"ref29","unstructured":"KudoTYamamotoKMatsumotoYApplying Conditional Random Fields to Japanese Morphological Analysis2021-09-08https:\/\/aclanthology.org\/W04-3230.pdf"},{"key":"ref30","unstructured":"ParkEChoSKoNLPy: Korean natural language processing in PythonProceedings of the 26th Annual Conference on Human & Cognitive Language Technology201426th Annual Conference on Human & Cognitive Language TechnologyOctober 2014Chuncheon, Korea133136"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2012.6289079"},{"key":"ref32","author":"Sennrich, R","year":"2015","journal-title":"arXiv"},{"key":"ref33","author":"Wu, Y","year":"2016","journal-title":"arXiv"},{"key":"ref34","unstructured":"VaswaniAShazeerNParmarNUszkoreitJJonesLGomezANKaiser\u0141PolosukhinIAttention is all you need201731st Conference on Neural Information Processing Systems (NIPS 2017)December 4-9, 2017Long Beach, CA59986008"},{"key":"ref35","unstructured":"IoffeSSzegedyCBatch normalization: Accelerating deep network training by reducing internal covariate shiftProceedings of the 32nd International Conference on Machine Learning2015060132nd International Conference on Machine Learning2015Lille, France448456"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1109\/cvpr.2016.90"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/n16-1174"},{"key":"ref38","unstructured":"Wikipedia:Database Download2020-06-25https:\/\/dumps.wikimedia.org\/kowiki\/"},{"key":"ref39","author":"Song, X","year":"2020","journal-title":"arXiv"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2016.2577031"},{"key":"ref41","doi-asserted-by":"crossref","unstructured":"RedmonJDivvalaSGirshickRFarhadiAYou only look once: Unified, real-time object detection20162016 IEEE Conference on Computer Vision and Pattern RecognitionJune 27-30, 2016Las Vegas, NVNew York, NYIEEE","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref42","author":"Tan, M","year":"2019","journal-title":"arXiv"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1109\/mci.2018.2840738"},{"key":"ref44","unstructured":"NairVHintonGRectified linear units improve restricted boltzmann machinesProceedings of the 27th International Conference on Machine Learning2010010127th International Conference on Machine LearningJune 21-24, 2010Haifa, IsraelMadison, WIOmnipress"},{"issue":"1","key":"ref45","first-page":"1929","volume":"15","author":"Srivastava, N","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"ref46","author":"Van Laarhoven, T","year":"2017","journal-title":"arXiv"},{"key":"ref47","author":"Kingma, D","year":"2014","journal-title":"arXiv"},{"key":"ref48","author":"Abadi, M","year":"2016","journal-title":"arXiv"},{"key":"ref49","unstructured":"PaszkeAGrossSMassaFLererABradburyJChananGKilleenTLinZGimelsheinNAntigaLDesmaisonAK\u00f6pfAYangEDeVitoZRaisonMTejaniAChilamkurthySSteinerBFangLBaiJChintalaSPytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems201933rd Conference on Neural Information Processing Systems (NeurIPS 2019)December 2, 2019Vancouver, Canada80268037"},{"key":"ref50","unstructured":"WolfTDebutLSanhVChaumondJDelangueCMoiACistacPRaultTLoufRFuntowiczMDavisonJShleiferSvon PlatenPMaCJerniteYPluJXuCLe ScaoTGuggerSDrameMLhoestQRushAMTransformers: State-of-the-art natural language processing202010Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System DemonstrationsNovember 16-20, 2020VirtualStroudsburg, PAAssociation for Computational Linguistics3845"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooz031"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms5022"},{"key":"ref53","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0701361104"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.2105\/ajph.11.10.908"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1016\/s0271-7123(80)91733-2"},{"issue":"1","key":"ref56","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1023\/A:1008286314688","volume":"8","author":"Thagard, P","year":"1998","journal-title":"Minds and Machines"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1016\/j.shpsc.2009.09.006"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2021\/9\/e30223","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:45:53Z","timestamp":1782841553000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2021\/9\/e30223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,21]]},"references-count":57,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021]]}},"URL":"https:\/\/doi.org\/10.2196\/30223","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.30223","asserted-by":"object"}]},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,21]]}}}