{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T21:45:51Z","timestamp":1775252751002,"version":"3.50.1"},"reference-count":53,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,9,11]],"date-time":"2021-09-11T00:00:00Z","timestamp":1631318400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,11]],"date-time":"2021-09-11T00:00:00Z","timestamp":1631318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>It is essential for radiologists to communicate actionable findings to the referring clinicians reliably. Natural language processing (NLP) has been shown to help identify free-text radiology reports including actionable findings. However, the application of recent deep learning techniques to radiology reports, which can improve the detection performance, has not been thoroughly examined. Moreover, free-text that clinicians input in the ordering form (order information) has seldom been used to identify actionable reports. This study aims to evaluate the benefits of two new approaches: (1) bidirectional encoder representations from transformers (BERT), a recent deep learning architecture in NLP, and (2) using order information in addition to radiology reports.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We performed a binary classification to distinguish actionable reports (i.e., radiology reports tagged as <jats:italic>actionable<\/jats:italic> in actual radiological practice) from non-actionable ones (those without an <jats:italic>actionable<\/jats:italic> tag). 90,923 Japanese radiology reports in our\u00a0hospital were used, of which 788 (0.87%) were actionable. We evaluated four methods, statistical machine learning with logistic regression (LR) and with gradient boosting decision tree (GBDT), and deep learning with a bidirectional long short-term memory (LSTM) model and a publicly available Japanese BERT model. Each method was used with two different inputs, radiology reports alone and pairs of order information and radiology reports. Thus, eight experiments were conducted to examine the performance.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Without order information, BERT achieved the highest area under the precision-recall curve (AUPRC) of 0.5138, which showed a statistically significant improvement over LR, GBDT, and LSTM, and the highest area under the receiver operating characteristic curve (AUROC) of 0.9516. Simply coupling the order information with the radiology reports slightly increased the AUPRC of BERT but did not lead to a statistically significant improvement. This may be due to the complexity of clinical decisions made by radiologists.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>BERT was assumed to be useful to detect actionable reports. More sophisticated methods are required to use order information effectively.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01623-6","type":"journal-article","created":{"date-parts":[[2021,9,12]],"date-time":"2021-09-12T13:03:46Z","timestamp":1631451826000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers"],"prefix":"10.1186","volume":"21","author":[{"given":"Yuta","family":"Nakamura","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shouhei","family":"Hanaoka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yukihiro","family":"Nomura","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takahiro","family":"Nakao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Soichiro","family":"Miki","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takeyuki","family":"Watadani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takeharu","family":"Yoshikawa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Naoto","family":"Hayashi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Osamu","family":"Abe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,9,11]]},"reference":[{"issue":"6","key":"1623_CR1","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1016\/j.jacr.2013.12.016","volume":"11","author":"PA Larson","year":"2014","unstructured":"Larson PA, Berland LL, Griffith B, Kahn CE, Liebscher LA. Actionable findings and the role of IT support: report of the ACR Actionable Reporting Work Group. J Am Coll Radiol. 2014;11(6):552\u20138.","journal-title":"J Am Coll Radiol"},{"issue":"12","key":"1623_CR2","doi-asserted-by":"publisher","first-page":"1579","DOI":"10.1016\/j.acra.2014.07.006","volume":"21","author":"CE Sloan","year":"2014","unstructured":"Sloan CE, Chadalavada SC, Cook TS, Langlotz CP, Schnall MD, Zafar HM. Assessment of follow-up completeness and notification preferences for imaging findings of possible cancer: what happens after radiologists submit their reports? Acad Radiol. 2014;21(12):1579\u201386.","journal-title":"Acad Radiol"},{"issue":"2","key":"1623_CR3","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1007\/s10916-018-1150-z","volume":"43","author":"SJ Baccei","year":"2019","unstructured":"Baccei SJ, DiRoberto C, Greene J, Rosen MP. Improving communication of actionable findings in radiology imaging studies and procedures using an EMR-independent system. J Med Syst. 2019;43(2):30.","journal-title":"J Med Syst"},{"issue":"5","key":"1623_CR4","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1016\/j.jacr.2017.01.024","volume":"14","author":"TS Cook","year":"2017","unstructured":"Cook TS, Lalevic D, Sloan C, Chadalavada SC, Langlotz CP, Schnall MD, et al. Implementation of an automated radiology recommendation-tracking engine for abdominal imaging findings of possible cancer. J Am Coll Radiol. 2017;14(5):629\u201336.","journal-title":"J Am Coll Radiol"},{"issue":"1","key":"1623_CR5","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1148\/radiol.2531091088","volume":"253","author":"CP Langlotz","year":"2009","unstructured":"Langlotz CP. Structured radiology reporting: are we there yet? Radiology. 2009;253(1):23\u20135.","journal-title":"Radiology"},{"issue":"2","key":"1623_CR6","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1148\/radiol.16142770","volume":"279","author":"E Pons","year":"2016","unstructured":"Pons E, Braun LM, Hunink MG, Kors JA. Natural language processing in radiology: a systematic review. Radiology. 2016;279(2):329\u201343.","journal-title":"Radiology"},{"key":"1623_CR7","first-page":"1546","volume":"264","author":"X Meng","year":"2019","unstructured":"Meng X, Heinz MV, Ganoe CH, Sieberg RT, Cheung YY, Hassanpour S. Understanding urgency in radiology reporting: identifying associations between clinical findings in radiology reports and their prompt communication to referring physicians. Stud Health Technol Inform. 2019;264:1546\u20137.","journal-title":"Stud Health Technol Inform"},{"issue":"9","key":"1623_CR8","doi-asserted-by":"publisher","first-page":"1299","DOI":"10.1016\/j.jacr.2019.05.038","volume":"16","author":"ME Heilbrun","year":"2019","unstructured":"Heilbrun ME, Chapman BE, Narasimhan E, Patel N, Mowery D. Feasibility of natural language processing-assisted auditing of critical findings in chest radiology. J Am Coll Radiol. 2019;16(9):1299\u2013304.","journal-title":"J Am Coll Radiol"},{"issue":"3","key":"1623_CR9","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1016\/j.jacr.2018.10.020","volume":"16","author":"E Carrodeguas","year":"2019","unstructured":"Carrodeguas E, Lacson R, Swanson W, Khorasani R. Use of machine learning to identify follow-up recommendations in radiology reports. J Am Coll Radiol. 2019;16(3):336\u201343.","journal-title":"J Am Coll Radiol"},{"key":"1623_CR10","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.jbi.2012.12.005","volume":"46","author":"M Yetisgen-Yildiz","year":"2013","unstructured":"Yetisgen-Yildiz M, Gunn ML, Xia F, Payne TH. A text processing pipeline to extract recommendations from radiology reports. J Biomed Inform. 2013;46:354\u201362.","journal-title":"J Biomed Inform"},{"key":"1623_CR11","first-page":"1593","volume":"2011","author":"M Yetisgen-Yildiz","year":"2011","unstructured":"Yetisgen-Yildiz M, Gunn ML, Xia F, Payne TH. Automatic identification of critical follow-up recommendation sentences in radiology reports. AMIA Annu Symp Proc. 2011;2011:1593\u2013602.","journal-title":"AMIA Annu Symp Proc"},{"key":"1623_CR12","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.annemergmed.2013.02.001","volume":"62","author":"S Dutta","year":"2013","unstructured":"Dutta S, Long WJ, Brown DF, Reisner AT. Automated detection using natural language processing of radiologists recommendations for additional imaging of incidental findings. Ann Emerg Med. 2013;62:162\u20139.","journal-title":"Ann Emerg Med"},{"key":"1623_CR13","first-page":"335","volume":"2020","author":"W Lau","year":"2020","unstructured":"Lau W, Payne TH, Uzuner O, Yetisgen M. Extraction and analysis of clinically important follow-up recommendations in a large radiology dataset. AMIA Jt Summits Transl Sci Proc. 2020;2020:335\u201344.","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"1623_CR14","doi-asserted-by":"publisher","first-page":"313","DOI":"10.2214\/AJR.07.3508","volume":"191","author":"PA Dang","year":"2008","unstructured":"Dang PA, Kalra MK, Blake MA, Schultz TJ, Halpern EF, Dreyer KJ. Extraction of recommendation features in radiology with natural language processing: exploratory study. AJR Am J Roentgenol. 2008;191:313\u201320.","journal-title":"AJR Am J Roentgenol"},{"key":"1623_CR15","first-page":"540","volume":"129","author":"T Imai","year":"2007","unstructured":"Imai T, Aramaki E, Kajino M, Miyo K, Onogi Y, Ohe K. Finding malignant findings from radiological reports using medical attributes and syntactic information. Stud Health Technol Inform. 2007;129:540\u20134.","journal-title":"Stud Health Technol Inform"},{"issue":"1","key":"1623_CR16","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s10278-019-00271-7","volume":"33","author":"R Lou","year":"2020","unstructured":"Lou R, Lalevic D, Chambers C, Zafar HM, Cook TS. Automated detection of radiology reports that require follow-up imaging using natural language processing feature engineering and machine learning classification. J Digit Imaging. 2020;33(1):131\u20136.","journal-title":"J Digit Imaging"},{"key":"1623_CR17","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1097\/JTO.0b013e31825bd9f5","volume":"7","author":"KN Danforth","year":"2012","unstructured":"Danforth KN, Early MI, Ngan S, Kosco AE, Zheng C, Gould MK. Automated identification of patients with pulmonary nodules in an integrated health system using administrative health plan data, radiology reports, and natural language processing. J Thorac Oncol. 2012;7:1257\u201362.","journal-title":"J Thorac Oncol"},{"key":"1623_CR18","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1016\/j.jbi.2013.06.014","volume":"46","author":"V Garla","year":"2013","unstructured":"Garla V, Taylor C, Brandt C. Semi-supervised clinical text classification with Laplacian SVMs: an application to cancer case management. J Biomed Inform. 2013;46:869\u201375.","journal-title":"J Biomed Inform"},{"key":"1623_CR19","doi-asserted-by":"publisher","DOI":"10.13063\/2327-9214.1254","author":"F Farjah","year":"2016","unstructured":"Farjah F, Halgrim S, Buist DSM, Gould MK, Zeliadt SB, Loggers ET, et al. An automated method for identifying individuals with a lung nodule can be feasibly implemented across health systems. EGEMS. 2016. https:\/\/doi.org\/10.13063\/2327-9214.1254.","journal-title":"EGEMS"},{"key":"1623_CR20","first-page":"465","volume":"2011","author":"EF Gershanik","year":"2011","unstructured":"Gershanik EF, Lacson R, Khorasani R. Critical finding capture in the impression section of radiology reports. AMIA Annu Symp Proc. 2011;2011:465\u20139.","journal-title":"AMIA Annu Symp Proc"},{"key":"1623_CR21","first-page":"1027","volume":"216","author":"L Oliveira","year":"2015","unstructured":"Oliveira L, Tellis R, Qian Y, Trovato K, Mankovich G. Identification of incidental pulmonary nodules in free-text radiology reports: an initial investigation. Stud Health Technol Inform. 2015;216:1027.","journal-title":"Stud Health Technol Inform"},{"issue":"1","key":"1623_CR22","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1186\/1471-2105-15-266","volume":"15","author":"A-D Pham","year":"2014","unstructured":"Pham A-D, N\u00e9v\u00e9ol A, Lavergne T, Yasunaga D, Cl\u00e9ment O, Meyer G, et al. Natural language processing of radiology reports for the detection of thromboembolic diseases and clinically relevant incidental findings. BMC Bioinform. 2014;15(1):266.","journal-title":"BMC Bioinform"},{"key":"1623_CR23","first-page":"1090","volume":"245","author":"T Mabotuwana","year":"2017","unstructured":"Mabotuwana T, Hall CS, Dalal S, Tieder J, Gunn ML. Extracting follow-up recommendations and associated anatomy from radiology reports. Stud Health Technol Inform. 2017;245:1090\u20134.","journal-title":"Stud Health Technol Inform"},{"key":"1623_CR24","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1007\/s10278-016-9889-6","volume":"29","author":"C Morioka","year":"2016","unstructured":"Morioka C, Meng F, Taira R, Sayre J, Zimmerman P, Ishimitsu D, et al. Automatic classification of ultrasound screening examinations of the abdominal aorta. J Digit Imaging. 2016;29:742\u20138.","journal-title":"J Digit Imaging"},{"issue":"5","key":"1623_CR25","doi-asserted-by":"publisher","first-page":"792","DOI":"10.1136\/amiajnl-2012-000812","volume":"19","author":"Y Xu","year":"2012","unstructured":"Xu Y, Tsujii J, Chang EIC. Named entity recognition of follow-up and time information in 20 000 radiology reports. J Am Med Inform Assoc. 2012;19(5):792\u20139.","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"1623_CR26","doi-asserted-by":"publisher","first-page":"e12109","DOI":"10.2196\/12109","volume":"7","author":"S Fu","year":"2019","unstructured":"Fu S, Leung LY, Wang Y, Raulli A-O, Kallmes DF, Kinsman KA, et al. Natural language processing for the identification of silent brain infarcts from neuroimaging reports. JMIR Med Inform. 2019;7(2):e12109.","journal-title":"JMIR Med Inform"},{"key":"1623_CR27","first-page":"5998","volume":"30","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:5998\u20136008.","journal-title":"Adv Neural Inf Process Syst"},{"key":"1623_CR28","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 4171\u201386."},{"key":"1623_CR29","doi-asserted-by":"crossref","unstructured":"Lin Y, Tan YC, Frank R. Open Sesame: getting inside BERT's linguistic knowledge. In: Proceedings of the 2019 ACL workshop BlackboxNLP: analyzing and interpreting neural networks for NLP. Florence, Italy: Association for Computational Linguistics; 2019. p. 241\u201353.","DOI":"10.18653\/v1\/W19-4825"},{"issue":"Suppl 1","key":"1623_CR30","first-page":"S148","volume":"15","author":"R Kuwabara","year":"2020","unstructured":"Kuwabara R, Han C, Murao K, Satoh S. BERT-based few-shot learning for automatic anomaly classification from Japanese multi-institutional CT scan reports. Int J Comput Assist Radiol Surg. 2020;15(Suppl 1):S148\u20139.","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"1623_CR31","doi-asserted-by":"crossref","unstructured":"Peng Y, Lee S, Elton DC, Shen T, Tang Y-X, Chen Q, et al. Automatic recognition of abdominal lymph nodes from clinical text. In: Proceedings of the 3rd clinical natural language processing workshop. Association for Computational Linguistics; 2020. pp. 101\u201310.","DOI":"10.18653\/v1\/2020.clinicalnlp-1.12"},{"key":"1623_CR32","unstructured":"American College of Radiology. ACR practice parameter for communication of diagnostic imaging findings revised 2020. 2020. https:\/\/www.acr.org\/-\/media\/ACR\/Files\/Practice-Parameters\/CommunicationDiag.pdf?la=en. Accessed 10 Feb 2021."},{"key":"1623_CR33","doi-asserted-by":"crossref","unstructured":"Kudo T, Richardson J. SentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. In: Proceedings of the 2018 conference on empirical methods in natural language processing: system demonstrations. Brussels, Belgium: Association for Computational Linguistics; 2018. p. 66\u201371.","DOI":"10.18653\/v1\/D18-2012"},{"key":"1623_CR34","unstructured":"Kikuta Y. BERT pretrained model trained on Japanese Wikipedia articles. 2019. https:\/\/github.com\/yoheikikuta\/bert-japanese. Accessed 10 Feb 2021."},{"key":"1623_CR35","unstructured":"Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: 3rd International conference on learning representations, ICLR 2015. San Diego, CA, USA: 2015."},{"key":"1623_CR36","unstructured":"Kingma DP, Ba J. Adam: a method for stochastic optimization. In: 3rd International conference on learning representations, ICLR 2015. San Diego, CA, USA: 2015."},{"issue":"3","key":"1623_CR37","doi-asserted-by":"publisher","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","volume":"10","author":"T Saito","year":"2015","unstructured":"Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE. 2015;10(3):e0118432.","journal-title":"PLoS ONE"},{"key":"1623_CR38","doi-asserted-by":"crossref","unstructured":"Davis J, Goadrich M. The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd international conference on machine learning. New York, NY, USA: Association for Computing Machinery; 233\u2013240, 2006. p. 233\u201340.","DOI":"10.1145\/1143844.1143874"},{"key":"1623_CR39","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9:1735\u201380.","journal-title":"Neural Comput"},{"key":"1623_CR40","unstructured":"Dan J, James HM. Speech and language processing, 3rd edition in draft. 2020. https:\/\/web.stanford.edu\/~jurafsky\/slp3\/ed3book_dec302020.pdf. Accessed 10 Feb 2021."},{"key":"1623_CR41","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29:1189\u2013232.","journal-title":"Ann Stat"},{"key":"1623_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v033.i01","volume":"33","author":"J Friedman","year":"2010","unstructured":"Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33:1\u201322.","journal-title":"J Stat Softw"},{"key":"1623_CR43","unstructured":"Prokhorenkova L, Gusev G, Vorobev A, Dorogush AV, Gulin A. CatBoost: unbiased boosting with categorical features. In: Proceedings of the 32nd international conference on neural information processing systems. Red Hook, NY, USA: Curran Associates Inc.; 2018. p. 6639\u201349."},{"key":"1623_CR44","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1111\/opo.12131","volume":"34","author":"RA Armstrong","year":"2014","unstructured":"Armstrong RA. When to use the Bonferroni correction. Ophthalmic Physiol Opt. 2014;34:502\u20138.","journal-title":"Ophthalmic Physiol Opt"},{"key":"1623_CR45","doi-asserted-by":"publisher","first-page":"364","DOI":"10.2214\/ajr.184.2.01840364","volume":"184","author":"NA Obuchowski","year":"2005","unstructured":"Obuchowski NA. ROC analysis. AJR Am J Roentgenol. 2005;184:364\u201372.","journal-title":"AJR Am J Roentgenol"},{"key":"1623_CR46","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett. 2006;27:861\u201374.","journal-title":"Pattern Recognit Lett"},{"key":"1623_CR47","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","volume":"106","author":"M Buda","year":"2018","unstructured":"Buda M, Maki A, Mazurowski MA. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 2018;106:249\u201359.","journal-title":"Neural Netw"},{"issue":"3","key":"1623_CR48","first-page":"176","volume":"7","author":"A Ali","year":"2015","unstructured":"Ali A, Shamsuddin SM, Ralescu AL. Classification with class imbalance problem: a review. Int J Adv Soft Comput Appl. 2015;7(3):176\u2013204.","journal-title":"Int J Adv Soft Comput Appl"},{"key":"1623_CR49","doi-asserted-by":"crossref","unstructured":"Wei J, Zou K. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics; 2019. p. 6382\u20138.","DOI":"10.18653\/v1\/D19-1670"},{"key":"1623_CR50","unstructured":"Madabushi HT, Kochkina E, Castelle M. Cost-sensitive BERT for generalisable sentence classification with imbalanced data. In: Proceedings of the second workshop on natural language processing for internet freedom: censorship, disinformation, and propaganda. Hong Kong, China: Association for Computational Linguistics; 2019. p. 125\u201334."},{"key":"1623_CR51","doi-asserted-by":"crossref","unstructured":"Li X, Sun X, Meng Y, Liang J, Wu F, Li J. Dice loss for data-imbalanced NLP tasks. In: Proceedings of the 58th annual meeting of the association for computational linguistics. 2020. p. 465\u201376.","DOI":"10.18653\/v1\/2020.acl-main.45"},{"issue":"3","key":"1623_CR52","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1148\/radiol.2019182826","volume":"291","author":"LR Cochon","year":"2019","unstructured":"Cochon LR, Kapoor N, Carrodeguas E, Ip IK, Lacson R, Boland G, et al. Variation in follow-up imaging recommendations in radiology reports: patient, modality, and radiologist predictors. Radiology. 2019;291(3):700\u20137.","journal-title":"Radiology"},{"key":"1623_CR53","doi-asserted-by":"publisher","unstructured":"Kawazoe Y, Shibata D, Shinohara E, Aramaki E, Ohe K. A clinical specific BERT developed with huge size of Japanese clinical narrative. medRxiv. 2020. https:\/\/doi.org\/10.1101\/2020.07.07.20148585.","DOI":"10.1101\/2020.07.07.20148585"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01623-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01623-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01623-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,12]],"date-time":"2021-09-12T18:03:39Z","timestamp":1631469819000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01623-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,11]]},"references-count":53,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1623"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01623-6","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,11]]},"assertion":[{"value":"17 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 September 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was approved by the institutional review board at The University of Tokyo Hospital (No.:2561-(18), approval date: 25 May 2009, last renewal date: 22 January 2020). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and\/or national research committee and with the 1975 Declaration of Helsinki, as revised in 2008(5). The institutional review board above stated that formal consent was not required for this study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The radiology reports and order information involved in this study are not publicly available because publishing the dataset is not approved by the institutional review board of the University of Tokyo Hospital. For more information, please contact the corresponding authors.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Availability of data and materials"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"262"}}