{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T16:23:05Z","timestamp":1781886185200,"version":"3.54.5"},"reference-count":14,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T00:00:00Z","timestamp":1736294400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T00:00:00Z","timestamp":1736294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100018736","name":"Kaken Pharmaceutical","doi-asserted-by":"publisher","award":["21H03840"],"award-info":[{"award-number":["21H03840"]}],"id":[{"id":"10.13039\/100018736","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004206","name":"Osaka University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004206","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Purpose<\/jats:title>\n            <jats:p>Systems equipped with natural language (NLP) processing can reduce missed radiological findings by physicians, but the annotation costs are burden in the development. This study aimed to compare the effects of active learning (AL) algorithms in NLP for estimating the significance of head computed tomography (CT) reports using bidirectional encoder representations from transformers (BERT).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>A total of 3728 head CT reports annotated with five categories of importance were used and UTH-BERT was adopted as the pre-trained BERT model. We assumed that 64% (2385 reports) of the data were initially in the unlabeled data pool (UDP), while the labeled data set (LD) used to train the model was empty. Twenty-five reports were repeatedly selected from the UDP and added to the LD, based on seven metrices: random sampling (RS: control), four uncertainty sampling (US) methods (least confidence (LC), margin sampling (MS), ratio of confidence (RC), and entropy sampling (ES)), and two distance-based sampling (DS) methods (cosine distance (CD) and Euclidian distance (ED)). The transition of accuracy of the model was evaluated using the test dataset.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>The accuracy of the models with US was significantly higher than RS when reports in LD were\u2009&lt;\u20091800, whereas DS methods were significantly lower than RS. Among the US methods, MS and RC were even better than the others. With the US methods, the required labeled data decreased by 15.4\u201340.5%, and most efficient in RC. In addition, in the US methods, data for minor categories tended to be added to LD earlier than RS and DS.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>In the classification task for the importance of head CT reports, US methods, especially RC and MS can lead to the effective fine-tuning of BERT models and reduce the imbalance of categories. AL can contribute to other studies on larger datasets by providing effective annotation.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s11548-024-03316-7","type":"journal-article","created":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T03:44:08Z","timestamp":1736307848000},"page":"687-701","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Comparison of active learning algorithms in classifying head computed tomography reports using bidirectional encoder representations from transformers"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1224-6601","authenticated-orcid":false,"given":"Tomohiro","family":"Wataya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Azusa","family":"Miura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Takahisa","family":"Sakisuka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3423-7658","authenticated-orcid":false,"given":"Masahiro","family":"Fujiwara","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6148-0672","authenticated-orcid":false,"given":"Hisashi","family":"Tanaka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Hiraoka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1206-2036","authenticated-orcid":false,"given":"Junya","family":"Sato","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miyuki","family":"Tomiyama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7006-3408","authenticated-orcid":false,"given":"Daiki","family":"Nishigaki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5825-3348","authenticated-orcid":false,"given":"Kosuke","family":"Kita","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1572-5771","authenticated-orcid":false,"given":"Yuki","family":"Suzuki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5293-4110","authenticated-orcid":false,"given":"Shoji","family":"Kido","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9918-7327","authenticated-orcid":false,"given":"Noriyuki","family":"Tomiyama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,8]]},"reference":[{"key":"3316_CR1","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1007\/s11604-024-01549-9","volume":"42","author":"T Wataya","year":"2024","unstructured":"Wataya T, Miura A, Sakisuka T et al (2024) Comparison of natural language processing algorithms in assessing the importance of head computed tomography reports written in Japanese. Jpn J Radiol 42:697\u2013708. https:\/\/doi.org\/10.1007\/s11604-024-01549-9","journal-title":"Jpn J Radiol"},{"key":"3316_CR2","volume-title":"AL literature survey. Computer sciences technical report 1648","author":"B Settles","year":"2009","unstructured":"Settles B (2009) AL literature survey. Computer sciences technical report 1648. University of Wisconsin, Madison"},{"issue":"9","key":"3316_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3472291","volume":"54","author":"P Ren","year":"2022","unstructured":"Ren P, Xiao Y, Chang X, Huang PY, Li Z, Gupta BB et al (2022) A survey of deep active learning. ACM Comput Surv 54(9):1\u201340","journal-title":"ACM Comput Surv"},{"key":"3316_CR4","doi-asserted-by":"crossref","unstructured":"Lewis DD, Gale WA (1994) A sequential algorithm for training text classifiers. In: Croft BW, van Rijsbergen CJ (eds.) SIGIR \u201994. Springer, London","DOI":"10.1007\/978-1-4471-2099-5_1"},{"key":"3316_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32606-7_10","author":"S Mabu","year":"2020","unstructured":"Mabu S, Kido S, Hirano Y, Kuremoto T (2020) Opacity labeling of diffuse lung diseases in CT images using unsupervised and semi-supervised learning. Deep Learn Healthc Paradig Appl. https:\/\/doi.org\/10.1007\/978-3-030-32606-7_10","journal-title":"Deep Learn Healthc Paradig Appl"},{"key":"3316_CR6","doi-asserted-by":"crossref","unstructured":"Zhang Z, Strubell E, Hovy E (2022) A survey of active learning for natural language processing. In: proceedings of the 2022 conference on empirical methods in natural language processing [Internet]. Stroudsburg, PA, USA: Association for Computational Linguistics. pp. 6166\u20136190. Available from: https:\/\/aclanthology.org\/2022.emnlp-main.414","DOI":"10.18653\/v1\/2022.emnlp-main.414"},{"key":"3316_CR7","doi-asserted-by":"publisher","unstructured":"Zhang, Y, Matthew L, Byron W (2017) Active discriminative text representation learning. In: proceedings of the AAAI conference on artificial intelligence. vol. 31. No. 1. https:\/\/doi.org\/10.1609\/aaai.v31i1.10962","DOI":"10.1609\/aaai.v31i1.10962"},{"key":"3316_CR8","doi-asserted-by":"publisher","unstructured":"Schr\u00f6der C, Niekler A, Potthast M (2022) Revisiting uncertainty-based query strategies for active learning with transformers. Findings of the association for computational linguistics: ACL 2022. Association for Computational Linguistics. Stroudsburg, PA, USA. pp. 2194\u2013203. https:\/\/doi.org\/10.18653\/v1\/2022.findings-acl.172","DOI":"10.18653\/v1\/2022.findings-acl.172"},{"key":"3316_CR9","doi-asserted-by":"publisher","unstructured":"LE Dor, A Halfon, A Gera, E Shnarch, L Dankin, L Choshen, M Danilevsky, R Aharonov, Y Katz, N Slonim (2020) Active learning for BERT: an empirical study. In: proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), Association for Computational Linguistics. pp. 7949\u20137962, Online. https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.638","DOI":"10.18653\/v1\/2020.emnlp-main.638"},{"key":"3316_CR10","unstructured":"Devlin, Jacob, et al. (2019) \u201cBERT: pre-training of deep bidirectional transformers for language understanding\u201d. In: proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, vol. 1 (Long and Short Papers)"},{"key":"3316_CR11","doi-asserted-by":"publisher","first-page":"642347","DOI":"10.3389\/fpsyg.2021.642347","volume":"12","author":"U Ahmed","year":"2021","unstructured":"Ahmed U, Mukhiya SK, Srivastava G, Lamo Y, Lin JC (2021) Attention-based deep entropy active learning using lexical algorithm for mental health treatment. Front Psychol 12:642347. https:\/\/doi.org\/10.3389\/fpsyg.2021.642347","journal-title":"Front Psychol"},{"issue":"1","key":"3316_CR12","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1186\/s12859-021-04047-1","volume":"22","author":"K De Angeli","year":"2021","unstructured":"De Angeli K, Gao S, Alawad M, Yoon HJ, Schaefferkoetter N, Wu XC, Durbin EB, Doherty J, Stroup A, Coyle L, Penberthy L, Tourassi G (2021) Deep active learning for classifying cancer pathology reports. BMC Bioinform 22(1):113. https:\/\/doi.org\/10.1186\/s12859-021-04047-1","journal-title":"BMC Bioinform"},{"issue":"11","key":"3316_CR13","doi-asserted-by":"publisher","first-page":"e0259763","DOI":"10.1371\/journal.pone.0259763","volume":"16","author":"Y Kawazoe","year":"2021","unstructured":"Kawazoe Y, Shibata D, Shinohara E, Aramaki E, Ohe K (2021) A clinical specific BERT developed using a huge Japanese clinical text corpus. PLoS One 16(11):e0259763. https:\/\/doi.org\/10.1371\/journal.pone.0259763","journal-title":"PLoS One"},{"key":"3316_CR14","doi-asserted-by":"publisher","unstructured":"Seung HS, Opper M, Sompolinsky H (1992) Query by committee. In: proceedings of the fifth annual workshop on computational learning theory. COLT \u201992. Association for Computing Machinery, New York, NY, USA. pp. 287\u201394. https:\/\/doi.org\/10.1145\/130385.130417.","DOI":"10.1145\/130385.130417"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03316-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03316-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03316-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,27]],"date-time":"2025-04-27T10:02:54Z","timestamp":1745748174000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03316-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,8]]},"references-count":14,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["3316"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03316-7","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,8]]},"assertion":[{"value":"27 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Yu Hiraoka is the director of GramEye Inc. and owns stock of the company. The other authors declare no conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study was approved by Ethics Review Board of Osaka University Hospital (3\/22\/2022 #21498).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Statement that all human and animal studies have been approved and performed in accordance with ethical standards.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Human or animal participants"}},{"value":"The need for informed consent was waived because of the retrospective nature of this study.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}