{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T17:03:47Z","timestamp":1778346227564,"version":"3.51.4"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T00:00:00Z","timestamp":1641945600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T00:00:00Z","timestamp":1641945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2022,12]]},"abstract":"Abstract<\/jats:title>Background<\/jats:title>Automated ICD coding on medical texts via machine learning has been a hot topic. Related studies from medical field heavily relies on conventional bag-of-words (BoW) as the feature extraction method, and do not commonly use more complicated methods, such as word2vec (W2V<\/jats:italic>) and large pretrained models likeBERT<\/jats:italic>. This study aimed at uncovering the most effective feature extraction methods for coding models by comparingBoW<\/jats:italic>,W2V<\/jats:italic>andBERT<\/jats:italic>variants.<\/jats:p><\/jats:sec>Methods<\/jats:title>We experimented with a Chinese dataset from Fuwai Hospital, which contains 6947 records and 1532 unique ICD codes, and a public Spanish dataset, which contains 1000 records and 2557 unique ICD codes. We designed coding tasks with different code frequency thresholds (denoted as$$f_s$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub><\/mml:math><\/jats:alternatives><\/jats:inline-formula>), with a lower threshold indicating a more complex task. Using traditional classifiers, we comparedBoW<\/jats:italic>,W2V<\/jats:italic>andBERT<\/jats:italic>variants on accomplishing these coding tasks.<\/jats:p><\/jats:sec>Results<\/jats:title>When$$f_s$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub><\/mml:math><\/jats:alternatives><\/jats:inline-formula>was equal to or greater than 140 for Fuwai dataset, and 60 for the Spanish dataset, theBERT<\/jats:italic>variants with the whole network fine-tuned was the best method, leading to aMicro-F<\/jats:italic>1 of 93.9% for Fuwai data when$$f_s=200$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub>=<\/mml:mo>200<\/mml:mn><\/mml:mrow><\/mml:math><\/jats:alternatives><\/jats:inline-formula>, and aMicro-F<\/jats:italic>1 of 85.41% for the Spanish dataset when$$f_s=180$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub>=<\/mml:mo>180<\/mml:mn><\/mml:mrow><\/mml:math><\/jats:alternatives><\/jats:inline-formula>. When$$f_s$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub><\/mml:math><\/jats:alternatives><\/jats:inline-formula>fell below 140 for Fuwai dataset, and 60 for the Spanish dataset,BoW<\/jats:italic>turned out to be the best, leading to aMicro-F<\/jats:italic>1 of 83% for Fuwai dataset when$$f_s=20$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub>=<\/mml:mo>20<\/mml:mn><\/mml:mrow><\/mml:math><\/jats:alternatives><\/jats:inline-formula>, and aMicro-F<\/jats:italic>1 of 39.1% for the Spanish dataset when$$f_s=20$$<\/jats:tex-math>f<\/mml:mi>s<\/mml:mi><\/mml:msub>=<\/mml:mo>20<\/mml:mn><\/mml:mrow><\/mml:math><\/jats:alternatives><\/jats:inline-formula>. Our experiments also showed that both theBERT<\/jats:italic>variants andBoW<\/jats:italic>possessed good interpretability, which is important for medical applications of coding models.<\/jats:p><\/jats:sec>Conclusions<\/jats:title>This study shed light on building promising machine learning models for automated ICD coding by revealing the most effective feature extraction methods. Concretely, our results indicated that fine-tuning the whole network of theBERT<\/jats:italic>variants was the optimal method for tasks covering only frequent codes, especially codes that represented unspecified diseases, whileBoW<\/jats:italic>was the best for tasks involving both frequent and infrequent codes. The frequency threshold where the best-performing method varied differed between different datasets due to factors like language and codeset.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12911-022-01753-5","type":"journal-article","created":{"date-parts":[[2022,1,12]],"date-time":"2022-01-12T13:03:18Z","timestamp":1641992598000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Comparison of different feature extraction methods for applicable automated ICD coding"],"prefix":"10.1186","volume":"22","author":[{"given":"Zhao","family":"Shuai","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diao","family":"Xiaolin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuan","family":"Jing","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huo","family":"Yanni","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cui","family":"Meng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wang","family":"Yuxin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhao","family":"Wei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,1,12]]},"reference":[{"key":"1753_CR1","unstructured":"Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv e-prints, 2013;1301\u20133781."},{"key":"1753_CR2","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv e-prints, 2018;1810\u201304805."},{"issue":"3","key":"1753_CR3","first-page":"0173410","volume":"12","author":"Y Chen","year":"2017","unstructured":"Chen Y, Lu H, Li L. Automatic ICD-10 coding algorithm using an improved longest common subsequence based on semantic similarity. PLoS ONE. 2017;12(3):0173410.","journal-title":"PLoS ONE"},{"key":"1753_CR4","doi-asserted-by":"crossref","unstructured":"de Lima LR, Laender AH, Ribeiro-Neto BA. A hierarchical approach to the automatic categorization of medical documents. In: International conference on information and knowledge management, 1998;132\u2013139.","DOI":"10.1145\/288627.288649"},{"key":"1753_CR5","doi-asserted-by":"crossref","unstructured":"Ferrao JC, Janela F, Oliveira MD, Martins HM. Using structured ehr data and svm to support icd-9-cm coding. In: IEEE international conference on healthcare informatics, pp. 511\u2013516. IEEE, ;2013.","DOI":"10.1109\/ICHI.2013.79"},{"issue":"11","key":"1753_CR6","doi-asserted-by":"publisher","first-page":"956","DOI":"10.1016\/j.ijmedinf.2015.08.004","volume":"84","author":"B Koopman","year":"2015","unstructured":"Koopman B, Zuccon G, Nguyen A, Bergheim A, Grayson N. Automatic ICD-10 classification of cancers from free-text death certificates. Int J Med Inform. 2015;84(11):956\u201365.","journal-title":"Int J Med Inform"},{"key":"1753_CR7","first-page":"73","volume":"252","author":"R Kaur","year":"2018","unstructured":"Kaur R, Ginige JA. Comparative analysis of algorithmic approaches for auto-coding with icd-10-am and achi. Stud Health Technol Inform. 2018;252:73\u20139.","journal-title":"Stud Health Technol Inform"},{"key":"1753_CR8","doi-asserted-by":"crossref","unstructured":"Karimi S, Dai X, Hassanzadeh H, Nguyen A. Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods. In: BioNLP, 2017;328\u2013332.","DOI":"10.18653\/v1\/W17-2342"},{"issue":"1","key":"1753_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12911-016-0269-4","volume":"16","author":"W Ning","year":"2016","unstructured":"Ning W, Yu M, Zhang R. A hierarchical method to automatically encode Chinese diagnoses through semantic similarity estimation. BMC Med Inform Decis Mak. 2016;16(1):1\u201312.","journal-title":"BMC Med Inform Decis Mak"},{"key":"1753_CR10","doi-asserted-by":"publisher","first-page":"104135","DOI":"10.1016\/j.ijmedinf.2020.104135","volume":"139","author":"A Sonabend","year":"2020","unstructured":"Sonabend A, Cai W, Ahuja Y, Ananthakrishnan A, Xia Z, Yu S, Hong C. Automated ICD coding via unsupervised knowledge integration (unite). Int J Med Inform. 2020;139:104135.","journal-title":"Int J Med Inform"},{"issue":"5","key":"1753_CR11","doi-asserted-by":"publisher","first-page":"866","DOI":"10.1093\/jamia\/ocv201","volume":"23","author":"M Subotin","year":"2016","unstructured":"Subotin M, Davis AR. A method for modeling co-occurrence propensity of clinical codes with application to icd-10-pcs auto-coding. J Am Med Inform Assoc. 2016;23(5):866\u201371.","journal-title":"J Am Med Inform Assoc"},{"key":"1753_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12911-020-1085-4","volume":"20","author":"L Zhou","year":"2020","unstructured":"Zhou L, Cheng C, Ou D, Huang H. Construction of a semi-automatic icd-10 coding system. BMC Med Inform Decis Mak. 2020;20:1\u201312.","journal-title":"BMC Med Inform Decis Mak"},{"key":"1753_CR13","first-page":"1","volume":"00","author":"M Docherty","year":"2021","unstructured":"Docherty M, Regnier SA, Capkun G, Balp M-M, Ye Q, Janssens N, Tietz A, L\u00f6ffler J, Cai J, Pedrosa MC, Schattenberg JM. Development of a novel machine learning model to predict presence of nonalcoholic steatohepatitis. J Am Med Inform Assoc. 2021;00:1\u20137.","journal-title":"J Am Med Inform Assoc"},{"issue":"e1","key":"1753_CR14","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1093\/jamia\/ocv115","volume":"23","author":"E Scheurwegs","year":"2016","unstructured":"Scheurwegs E, Luyckx K, Luyten L, Daelemans W, Van den Bulcke T. Data integration of structured and unstructured sources for assigning clinical codes to patient stays. J Am Med Inform Assoc. 2016;23(e1):11\u20139.","journal-title":"J Am Med Inform Assoc"},{"key":"1753_CR15","doi-asserted-by":"crossref","unstructured":"Cao P, Chen Y, Liu K, Zhao J, Liu S, Chong W. Hypercore: Hyperbolic and co-graph representation for automatic icd coding. In: Annual meeting of the association for computational linguistics, 2020;3105\u20133114.","DOI":"10.18653\/v1\/2020.acl-main.282"},{"key":"1753_CR16","doi-asserted-by":"crossref","unstructured":"Cao P, Yan C, Fu X, Chen Y, Liu K, Zhao J, Liu S, Chong W. Clinical-coder: Assigning interpretable icd-10 codes to chinese clinical notes. In: Annual meeting of the association for computational linguistics: system demonstrations, 2020;294\u2013301.","DOI":"10.18653\/v1\/2020.acl-demos.33"},{"key":"1753_CR17","doi-asserted-by":"crossref","unstructured":"Li F, Yu H. Icd coding from clinical text using multi-filter residual convolutional neural network. In: AAAI conference on artificial intelligence, 2020;34, 8180\u20138187.","DOI":"10.1609\/aaai.v34i05.6331"},{"key":"1753_CR18","doi-asserted-by":"crossref","unstructured":"Mullenbach J, Wiegreffe S, Duke J, Sun J, Eisenstein J. Explainable prediction of medical codes from clinical text. In: Annual conference of the North American chapter of the association for computational linguistics: human language technologies, 2018;1101\u20131111.","DOI":"10.18653\/v1\/N18-1100"},{"key":"1753_CR19","unstructured":"Shi H, Xie P, Hu Z, Zhang M, Xing EP. Towards automated icd coding using deep learning. arXiv e-prints, 2017;1711\u201304075."},{"key":"1753_CR20","doi-asserted-by":"crossref","unstructured":"Vu T, Nguyen DQ, Nguyen A. A label attention model for icd coding from clinical text. In: International joint conference on artificial intelligence, 2020;3335\u20133341.","DOI":"10.24963\/ijcai.2020\/461"},{"key":"1753_CR21","doi-asserted-by":"crossref","unstructured":"Xie P, Xing E. A neural architecture for automated icd coding. In: Annual meeting of the association for computational linguistics, 2018;1,1066\u20131076.","DOI":"10.18653\/v1\/P18-1098"},{"key":"1753_CR22","doi-asserted-by":"crossref","unstructured":"Xie X, Xiong Y, Yu PS, Zhu Y. EHR coding with multi-scale feature attention and structured knowledge graph propagation. In: ACM international conference on information and knowledge management, 2019;649\u2013658.","DOI":"10.1145\/3357384.3357897"},{"key":"1753_CR23","unstructured":"Xu K, Lam M, Pang J, Gao X, Band C, Mathur P, Papay F, Khanna AK, Cywinski JB, Maheshwari K. Multimodal machine learning for automated icd coding. In: Machine learning for healthcare conference, pp. 197\u2013215. PMLR;2019."},{"key":"1753_CR24","doi-asserted-by":"publisher","first-page":"103114","DOI":"10.1016\/j.jbi.2019.103114","volume":"91","author":"Y Yu","year":"2019","unstructured":"Yu Y, Li M, Liu L, Fei Z, Wu F-X, Wang J. Automatic icd code assignment of chinese clinical notes based on multilayer attention birnn. J Biomed Inform. 2019;91:103114.","journal-title":"J Biomed Inform"},{"key":"1753_CR25","doi-asserted-by":"crossref","unstructured":"Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng C -K, Stanley HE. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. Circulation 2000;101(23), 215\u2013220.","DOI":"10.1161\/01.CIR.101.23.e215"},{"key":"1753_CR26","doi-asserted-by":"publisher","unstructured":"Johnson A, Pollard T, Mark R. MIMIC-III clinical database (version 1.4). PhysioNet;2016. https:\/\/doi.org\/10.13026\/C2XW26.","DOI":"10.13026\/C2XW26"},{"issue":"1","key":"1753_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AE Johnson","year":"2016","unstructured":"Johnson AE, Pollard TJ, Shen L, Li-Wei HL, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG. MIMIC-III, a freely accessible critical care database. Sci data. 2016;3(1):1\u20139.","journal-title":"Sci data"},{"issue":"4","key":"1753_CR28","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234\u201340.","journal-title":"Bioinformatics"},{"key":"1753_CR29","doi-asserted-by":"crossref","unstructured":"Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q. Ernie: Enhanced language representation with informative entities. arXiv e-prints, 2019;1905\u201307129.","DOI":"10.18653\/v1\/P19-1139"},{"key":"1753_CR30","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. Roberta: A robustly optimized bert pretraining approach. arXiv e-prints, 2019;1907\u201311692."},{"key":"1753_CR31","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. ALBERT: a lite BERT for self-supervised learning of language representations."},{"key":"1753_CR32","unstructured":"Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le QV. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv e-prints, 2020;1906\u201308237."},{"key":"1753_CR33","doi-asserted-by":"crossref","unstructured":"Jiao X, Yin Y, Shang L, Jiang X, Chen X, Li L, Wang F, Liu Q. Tinybert: Distilling bert for natural language understanding. arXiv e-prints, 2020;1909\u201310351.","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"1753_CR34","unstructured":"Xu Z. Roberta-wwm-ext fine-tuning for chinese text classification. arXiv e-prints, 2021;2103\u201300492."},{"key":"1753_CR35","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. arXiv e-prints, 2017;1706\u201303762."},{"key":"1753_CR36","doi-asserted-by":"publisher","first-page":"154290","DOI":"10.1109\/ACCESS.2019.2946594","volume":"7","author":"Z Gao","year":"2019","unstructured":"Gao Z, Feng A, Song X, Wu X. Target-dependent sentiment classification with bert. IEEE Access. 2019;7:154290\u20139.","journal-title":"IEEE Access"},{"key":"1753_CR37","unstructured":"Yang W, Zhang H, Lin J. Simple applications of bert for ad hoc document retrieval. arXiv e-prints, 2019;1903\u201310972."},{"key":"1753_CR38","volume-title":"Data mining: concepts and techniques","author":"J Han","year":"2011","unstructured":"Han J, Pei J, Kamber M. Data mining: concepts and techniques. New York: Elsevier; 2011."},{"key":"1753_CR39","unstructured":"Platt JC. Sequential minimal optimization: A fast algorithm for training support vector machines. Report, Advances in Kernel Methods\u2014Support Vector Learning ;1998."},{"key":"1753_CR40","unstructured":"Miranda-Escalada A, Gonzalez-Agirre A, Armengol-Estap\u00e9 J, Krallinger M. Overview of automatic clinical coding: Annotations, guidelines, and solutions for non-english clinical cases at codiesp track of clef ehealth 2020. In: CLEF (Working Notes);2020."},{"key":"1753_CR41","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: Machine learning in Python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"1753_CR42","unstructured":"\u0158eh\u016f\u0159ek R, Sojka P. Software framework for topic modelling with large corpora. In: LREC 2010 workshop on new challenges for NLP frameworks, 2010;45\u201350."},{"key":"1753_CR43","unstructured":"Turc I, Chang M-W, Lee K, Toutanova K. Well-read students learn better: On the importance of pre-training compact models. arXiv e-prints, 2019;1908\u201308962."},{"key":"1753_CR44","doi-asserted-by":"crossref","unstructured":"Zhao Z, Chen H, Zhang J, Zhao X, Liu T, Lu W, Chen X, Deng H, Ju Q, Du X. Uer: An open-source toolkit for pre-training models. arXiv e-prints, 1909-05658;2019.","DOI":"10.18653\/v1\/D19-3041"},{"key":"1753_CR45","unstructured":"Su J. Wobert: Word-based chinese bert model - zhuiyiai. Technical report ;2020. https:\/\/github.com\/ZhuiyiTechnology\/WoBERT."},{"key":"1753_CR46","doi-asserted-by":"crossref","unstructured":"Bhargava P, Drozd A, Rogers A. Generalization in nli: Ways (not) to go beyond simple heuristics. arXiv preprint, 01518;2021.","DOI":"10.18653\/v1\/2021.insights-1.18"},{"key":"1753_CR47","unstructured":"Turc I, Chang M-W, Lee K, Toutanova K. Well-read students learn better: the impact of student initialization on knowledge distillation. arXiv preprint 13, 08962 ;2019."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-01753-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-022-01753-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-01753-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,15]],"date-time":"2023-11-15T18:59:58Z","timestamp":1700074798000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-022-01753-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,12]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["1753"],"URL":"https:\/\/doi.org\/10.1186\/s12911-022-01753-5","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,12]]},"assertion":[{"value":"1 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 January 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The Ethics Committee at Fuwai Hospital approved the current research and granted the permission of using the dataset in this study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"11"}}