{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T22:00:55Z","timestamp":1768341655869,"version":"3.49.0"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,18]],"date-time":"2021-12-18T00:00:00Z","timestamp":1639785600000},"content-version":"vor","delay-in-days":17,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Crowd Sourcing Labels from Electronic Medical Records to Enable Biomedical Research Award","award":["1 UH2 CA203708-01"],"award-info":[{"award-number":["1 UH2 CA203708-01"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Information retrieval (IR) help clinicians answer questions posed to large collections of electronic medical records (EMRs), such as how best to identify a patient\u2019s cancer stage. One of the more promising approaches to IR for EMRs is to expand a keyword query with similar terms (e.g., augmenting<jats:italic>cancer<\/jats:italic>with<jats:italic>mets<\/jats:italic>). However, there is a large range of clinical chart review tasks, such that fixed sets of similar terms is insufficient. Current language models, such as Bidirectional Encoder Representations from Transformers (BERT) embeddings, do not capture the full non-textual context of a task. In this study, we present new methods that provide similar terms dynamically by adjusting with the context of the chart review task.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We introduce a vector space for medical-context in which each word is represented by a vector that captures the word\u2019s usage in different medical contexts (e.g., how frequently<jats:italic>cancer<\/jats:italic>is used when ordering a prescription versus describing family history) beyond the context learned from the surrounding text. These vectors are transformed into a vector space for customizing the set of similar terms selected for different chart review tasks. We evaluate the vector space model with multiple chart review tasks, in which supervised machine learning models learn to predict the preferred terms of clinically knowledgeable reviewers. To quantify the usefulness of the predicted similar terms to a baseline of standard word2vec embeddings, we measure (1) the prediction performance of the medical-context vector space model using the area under the receiver operating characteristic curve (AUROC) and (2) the labeling effort required to train the models.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The vector space outperformed the baseline word2vec embeddings in all three chart review tasks with an average AUROC of 0.80 versus 0.66, respectively. Additionally, the medical-context vector space significantly reduced the number of labels required to learn and predict the preferred similar terms of reviewers. Specifically, the labeling effort was reduced to 10% of the entire dataset in all three tasks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>The set of preferred similar terms that are relevant to a chart review task can be learned by leveraging the medical context of the task.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12911-021-01724-2","type":"journal-article","created":{"date-parts":[[2021,12,18]],"date-time":"2021-12-18T12:02:37Z","timestamp":1639828957000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Leveraging medical context to recommend semantically similar terms for chart reviews"],"prefix":"10.1186","volume":"21","author":[{"given":"Cheng","family":"Ye","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bradley A.","family":"Malin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Fabbri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,12,18]]},"reference":[{"issue":"6","key":"1724_CR1","doi-asserted-by":"publisher","first-page":"e20162966","DOI":"10.1542\/peds.2016-2966","volume":"140","author":"LS Beers","year":"2017","unstructured":"Beers LS, Godoy L, John T, Long M, Biel MG, Anthony B, et al. Mental health screening quality improvement learning collaborative in pediatric primary care. Pediatrics. 2017;140(6):e20162966.","journal-title":"Pediatrics"},{"key":"1724_CR2","doi-asserted-by":"crossref","unstructured":"Rasmussen L V. The electronic health record for translational research. J Cardiovasc Transl Res. 2014 [cited 2017 Jun 23];7(6):607\u201314.","DOI":"10.1007\/s12265-014-9579-z"},{"key":"1724_CR3","doi-asserted-by":"publisher","unstructured":"Chen L, Guo U, Illipparambil LC, Netherton MD, Sheshadri B, Karu E, et al. Racing against the clock: internal medicine residents\u2019 time spent on electronic health records. J Grad Med Educ. 2016 [cited 2017 Apr 19];8(1):39\u201344. https:\/\/doi.org\/10.4300\/JGME-D-15-00240.1.","DOI":"10.4300\/JGME-D-15-00240.1"},{"issue":"2","key":"1724_CR4","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1136\/jamia.2010.008441","volume":"18","author":"G Hripcsak","year":"2011","unstructured":"Hripcsak G, Vawdrey DK, Fred MR, Bostwick SB. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inf Assoc. 2011;18(2):112\u20137.","journal-title":"J Am Med Inf Assoc"},{"key":"1724_CR5","doi-asserted-by":"crossref","unstructured":"Ooi J, Ma X, Qin H, Liew SC. A survey of query expansion, query suggestion and query refinement techniques. In: 2015 4th international conference on software engineering and computer systems ICSECS 2015 Virtuous Software Solutions Big Data. 2015; p. 112\u20137.","DOI":"10.1109\/ICSECS.2015.7333094"},{"key":"1724_CR6","doi-asserted-by":"crossref","unstructured":"Goodwin T, Harabagiu SM. UTD at TREC 2014: query expansion for clinical decision support. 23rd text retrieval conference (TREC 2014) Proceedings. 2014;1.","DOI":"10.6028\/NIST.SP.500-308.clinical-UTDHLTRI"},{"key":"1724_CR7","unstructured":"Pal D, Mitra M, Bhattacharya S. Exploring query categorisation for query expansion: a study. 2015;1\u201334. arXiv:1509.05567."},{"issue":"September","key":"1724_CR8","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jbi.2018.09.008","volume":"87","author":"Y Wang","year":"2018","unstructured":"Wang Y, Liu S, Afzal N, Rastegar-Mojarad M, Wang L, Shen F, et al. A comparison of word embeddings for the biomedical natural language processing. J Biomed Inform. 2018;87(September):12\u201320. https:\/\/doi.org\/10.1016\/j.jbi.2018.09.008.","journal-title":"J Biomed Inform"},{"key":"1724_CR9","doi-asserted-by":"publisher","unstructured":"Biron P, Metzger MH, Pezet C, Sebban C, Barthuet E, Durand T. An information retrieval system for computerized patient records in the context of a daily hospital practice: the example of the L\u00e9on B\u00e9rard Cancer Center (France). Appl Clin Inform. 2014 [cited 2016 Oct 24];5(1):191\u2013205. https:\/\/doi.org\/10.4338\/ACI-2013-08-CR-0065.","DOI":"10.4338\/ACI-2013-08-CR-0065"},{"key":"1724_CR10","doi-asserted-by":"crossref","unstructured":"Natarajan K, Stein D, Jain S, Elhadad N. An analysis of clinical queries in an electronic health record search utility. Int J Med Inform. 2010 [cited 2016 Oct 24];79(7):515\u201322.","DOI":"10.1016\/j.ijmedinf.2010.03.004"},{"key":"1724_CR11","doi-asserted-by":"publisher","unstructured":"Tawfik AA, Kochendorfer KM, Saparova D, Al Ghenaimi S, Moore JL. \u201cI don\u2019t have time to dig back through this\u201d: the role of semantic search in supporting physician information seeking in an electronic health record. Perform Improv Q. 2014 [cited 2016 Oct 24];26(4):75\u201391. https:\/\/doi.org\/10.1002\/piq.21158.","DOI":"10.1002\/piq.21158"},{"key":"1724_CR12","doi-asserted-by":"crossref","unstructured":"Zalis M, Harris M. Advanced search of the electronic medical record: Augmenting safety and efficiency in radiology. J Am Coll Radiol. 2010 [cited 2016 Oct 24];7: 625\u201333.","DOI":"10.1016\/j.jacr.2010.03.011"},{"key":"1724_CR13","unstructured":"Gregg W, Jirjis J, Lorenzi NM, Giuse D. StarTracker: an integrated, web-based clinical search engine. AMIA Annu Symp Proc. 2003 [cited 2016 Oct 24];2003(1):855."},{"key":"1724_CR14","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1016\/j.jbi.2015.05.003","volume":"55","author":"DA Hanauer","year":"2015","unstructured":"Hanauer DA, Mei Q, Law J, Khanna R, Zheng K. Supporting information retrieval from electronic health records: a report of University of Michigan\u2019s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE). J Biomed Inform. 2015;55:290\u2013300. https:\/\/doi.org\/10.1016\/j.jbi.2015.05.003.","journal-title":"J Biomed Inform"},{"key":"1724_CR15","unstructured":"Mikolov T, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. Adv Neural Inf Process Syst. 2013;3111\u20139."},{"key":"1724_CR16","unstructured":"Mikolov T, Corrado G, Chen K, Dean J. Efficient estimation of word representations in vector space. In: Proceedings of International Conference on Learning Representations (ICLR 2013). 2013; p. 1\u201312."},{"key":"1724_CR17","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. 2018. arXiv:1810.04805."},{"issue":"9","key":"1724_CR18","first-page":"1","volume":"21","author":"F Li","year":"2019","unstructured":"Li F, Jin Y, Liu W, Rawat BPS, Cai P, Yu H. Fine-tuning bidirectional encoder representations from transformers (BERT)\u2013based models on large-scale electronic health record notes: an empirical study. J Med Internet Res. 2019;21(9):1\u201313.","journal-title":"J Med Internet Res"},{"key":"1724_CR19","doi-asserted-by":"crossref","unstructured":"Ye C, Fabbri D. Extracting similar terms from multiple EMR-based semantic embeddings to support chart reviews. J Biomed Inform. 2018;83(April).","DOI":"10.1016\/j.jbi.2018.05.014"},{"issue":"4","key":"1724_CR20","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2020;36(4):1234\u201340. https:\/\/doi.org\/10.1093\/bioinformatics\/btz682.","journal-title":"Bioinformatics"},{"key":"1724_CR21","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.jbi.2014.02.003","volume":"52","author":"MJ Mitchell","year":"2014","unstructured":"Mitchell MJ, King MR. Secondary use of clinical data: the vanderbilt approach. J Biomed Inform. 2014;52:28\u201335.","journal-title":"J Biomed Inform"},{"issue":"6","key":"1724_CR22","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1197\/jamia.M3037","volume":"16","author":"JC Denny","year":"2009","unstructured":"Denny JC, Spickard A, Johnson KB, Peterson NB, Peterson JF, Miller RA. Evaluation of a method to identify and categorize section headers in clinical documents. J Am Med Informatics Assoc. 2009;16(6):806\u201315.","journal-title":"J Am Med Informatics Assoc"},{"key":"1724_CR23","first-page":"273","volume":"2017","author":"C Ye","year":"2018","unstructured":"Ye C, Coco J, Epishova A, Hajaj C, Bogardus H, Novak L, et al. A crowdsourcing framework for medical data sets. AMIA Summits Transl Sci Proc. 2018;2017:273\u201380.","journal-title":"AMIA Summits Transl Sci Proc"},{"key":"1724_CR24","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.jbi.2014.04.013","volume":"51","author":"D Martinez","year":"2014","unstructured":"Martinez D, Otegi A, Soroa A, Agirre E. Improving search over Electronic Health Records using UMLS-based query expansion through random walks. J Biomed Inform. 2014;51:100\u20136. https:\/\/doi.org\/10.1016\/j.jbi.2014.04.013.","journal-title":"J Biomed Inform"},{"issue":"2","key":"1724_CR25","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1016\/j.jalz.2013.02.009","volume":"10","author":"A Malhotra","year":"2014","unstructured":"Malhotra A, Younesi E, G\u00fcndel M, M\u00fcller B, Heneka MT, Hofmann-Apitius M. ADO: a disease ontology representing the domain knowledge specific to Alzheimer\u2019s disease. Alzheimer\u2019s Dement. 2014;10(2):238\u201346.","journal-title":"Alzheimer\u2019s Dement"},{"key":"1724_CR26","first-page":"211","volume":"4","author":"C Perlich","year":"2003","unstructured":"Perlich C, Provost F, Simonoff JS. Tree induction vs. logistic regression: a learning-curve analysis. J Mach Learn Res. 2003;4:211\u201355.","journal-title":"J Mach Learn Res"},{"key":"1724_CR27","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM; 2016. p. 1135\u201344. arXiv:1602.04938.","DOI":"10.1145\/2939672.2939778"},{"key":"1724_CR28","doi-asserted-by":"crossref","unstructured":"Temple MW, Lehmann CU, Fabbri D. Predicting discharge dates From the NICU using progress note data. Pediatrics. 2015 [cited 2017 Aug 16];136(2):e395\u2013405.","DOI":"10.1542\/peds.2015-0456"},{"issue":"7","key":"1724_CR29","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1093\/jamia\/ocy013","volume":"25","author":"Y Wang","year":"2018","unstructured":"Wang Y, Zheng K, Xu H, Mei Q. Interactive medical word sense disambiguation through informed learning. J Am Med Informatics Assoc. 2018;25(7):800\u20138. https:\/\/doi.org\/10.1093\/jamia\/ocy013.","journal-title":"J Am Med Informatics Assoc"},{"key":"1724_CR30","doi-asserted-by":"crossref","unstructured":"Self JZ, Vinayagam RK, Fry JT, North C. Bridging the gap between user intention and model parameters for human-in-the-loop data analytics. In: HILDA@ SIGMOD. 2016. p. 1\u20136.","DOI":"10.1145\/2939502.2939505"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01724-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01724-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01724-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,14]],"date-time":"2024-09-14T18:18:52Z","timestamp":1726337932000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01724-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1724"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01724-2","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"19 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 December 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was approved by the Vanderbilt University Institutional Review Board (#160995 and #140917).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"353"}}