{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T21:46:10Z","timestamp":1767649570658,"version":"3.37.3"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2019,3,23]],"date-time":"2019-03-23T00:00:00Z","timestamp":1553299200000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1054199","1633370"],"award-info":[{"award-number":["1054199","1633370"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"crossref","award":["2R01LM010681-05"],"award-info":[{"award-number":["2R01LM010681-05"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Medical word sense disambiguation (WSD) is challenging and often requires significant training with data labeled by domain experts. This work aims to develop an interactive learning algorithm that makes efficient use of expert\u2019s domain knowledge in building high-quality medical WSD models with minimal human effort.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We developed an interactive learning algorithm with expert labeling instances and features. An expert can provide supervision in 3 ways: labeling instances, specifying indicative words of a sense, and highlighting supporting evidence in a labeled instance. The algorithm learns from these labels and iteratively selects the most informative instances to ask for future labels. Our evaluation used 3 WSD corpora: 198 ambiguous terms from Medical Subject Headings (MSH) as MEDLINE indexing terms, 74 ambiguous abbreviations in clinical notes from the University of Minnesota (UMN), and 24 ambiguous abbreviations in clinical notes from Vanderbilt University Hospital (VUH). For each ambiguous term and each learning algorithm, a learning curve that plots the accuracy on the test set against the number of labeled instances was generated. The area under the learning curve was used as the primary evaluation metric.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our interactive learning algorithm significantly outperformed active learning, the previous fastest learning algorithm for medical WSD. Compared to active learning, it achieved 90% accuracy for the MSH corpus with 42% less labeling effort, 35% less labeling effort for the UMN corpus, and 16% less labeling effort for the VUH corpus.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>High-quality WSD models can be efficiently trained with minimal supervision by inviting experts to label informative instances and provide domain knowledge through labeling\/highlighting contextual features.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocy013","type":"journal-article","created":{"date-parts":[[2018,2,9]],"date-time":"2018-02-09T20:12:32Z","timestamp":1518207152000},"page":"800-808","source":"Crossref","is-referenced-by-count":16,"title":["Interactive medical word sense disambiguation through informed learning"],"prefix":"10.1093","volume":"25","author":[{"given":"Yue","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, The University of Michigan, Ann Arbor, MI, 48109, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Informatics, The University of California, Irvine, CA, 92697, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hua","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, 77030, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiaozhu","family":"Mei","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, The University of Michigan, Ann Arbor, MI, 48109, USA"},{"name":"School of Information, The University of Michigan, Ann Arbor, MI, 48109, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2018,3,23]]},"reference":[{"issue":"1","key":"2020110620013553900_ocy013-B1","first-page":"2","article-title":"Introduction to the special issue on word sense disambiguation: the state of the art","volume":"24","author":"Ide","year":"1998","journal-title":"Comput Linguist"},{"issue":"5","key":"2020110620013553900_ocy013-B2","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1089\/cmb.2005.12.554","article-title":"Word sense disambiguation in the biomedical domain: an overview","volume":"12","author":"Schuemie","year":"2005","journal-title":"J Comput Biol"},{"issue":"4","key":"2020110620013553900_ocy013-B3","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1197\/jamia.M1533","article-title":"A multi-aspect comparison study of supervised word sense disambiguation","volume":"11","author":"Liu","year":"2004","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110620013553900_ocy013-B4","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1186\/1471-2105-7-334","article-title":"Machine learning and word sense disambiguation in the biomedical domain: design and evaluation issues","volume":"7","author":"Xu","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2020110620013553900_ocy013-B5","first-page":"171","article-title":"Clinical abbreviation disambiguation using neural word embeddings","volume-title":"proceedings of the 2015 Workshop on Biomedical Natural Language Processing (BioNLP)","author":"Wu","year":", 2015"},{"issue":"4","key":"2020110620013553900_ocy013-B6","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1006\/jbin.2001.1023","article-title":"Disambiguating ambiguous biomedical terms in biomedical narrative text: an unsupervised method","volume":"34","author":"Liu","year":"2001","journal-title":"J Biomed Inform"},{"key":"2020110620013553900_ocy013-B7","first-page":"1004","article-title":"Combining corpus-derived sense profiles with estimated frequency information to disambiguate clinical abbreviations","volume":"Vol. 2012","author":"Xu","year":"2012","journal-title":"AMIA Ann Symp Proc"},{"key":"2020110620013553900_ocy013-B8","first-page":"560","article-title":"Towards comprehensive clinical abbreviation disambiguation using machine-labeled training data","volume":"Vol. 2016","author":"Finley","year":"2016","journal-title":"AMIA Ann Symp Proc"},{"issue":"6","key":"2020110620013553900_ocy013-B9","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1197\/jamia.M1101","article-title":"Automatic resolution of ambiguous terms based on machine learning and conceptual relations in the UMLS","volume":"9","author":"Liu","year":"2002","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110620013553900_ocy013-B10","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.jbi.2006.06.001","article-title":"Using MEDLINE as a knowledge source for disambiguating abbreviations and acronyms in full-text biomedical journal articles","volume":"40","author":"Yu","year":"2007","journal-title":"J Biomed Inform"},{"volume-title":"Natural Language Annotation for Machine Learning: A Guide to Corpus-building for Applications","year":"2012","author":"Pustejovsky","key":"2020110620013553900_ocy013-B11"},{"volume-title":"Active Learning Literature Survey","author":"Settles","key":"2020110620013553900_ocy013-B12"},{"issue":"5","key":"2020110620013553900_ocy013-B13","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1136\/amiajnl-2012-001244","article-title":"Applying active learning to supervised word sense disambiguation in MEDLINE","volume":"20","author":"Chen","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110620013553900_ocy013-B14","first-page":"2062","article-title":"Clinical word sense disambiguation with interactive search and classification","volume":"Vol. 2016","author":"Wang","year":"2016","journal-title":"AMIA Ann Symp Proc"},{"key":"2020110620013553900_ocy013-B15","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1145\/1390334.1390436","article-title":"Learning from labeled features using generalized expectation criteria","volume-title":"proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Druck","year":", 2008"},{"key":"2020110620013553900_ocy013-B16","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1186\/1471-2105-12-223","article-title":"Exploiting MeSH indexing in MEDLINE to generate a data set for word sense disambiguation","volume":"12","author":"Jimeno-Yepes","year":"2011","journal-title":"BMC Bioinformatics"},{"issue":"2","key":"2020110620013553900_ocy013-B17","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1136\/amiajnl-2012-001506","article-title":"A sense inventory for clinical abbreviations and acronyms created using clinical notes and medical dictionary resources","volume":"21","author":"Moon","year":"2014","journal-title":"J Am Med Inform Assoc"},{"key":"2020110620013553900_ocy013-B18","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1145\/2512089.2512096","article-title":"A prototype application for real-time recognition and disambiguation of clinical abbreviations","volume-title":"Proceedings of the 7th International Workshop on Data and Text Mining in Biomedical Informatics","author":"Wu","year":"2013"},{"key":"2020110620013553900_ocy013-B19","first-page":"412","article-title":"A comparative study on feature selection in text categorization","volume-title":"Proceedings of the 14th International Conference on Machine Learning","author":"Yang","year":", 1997"},{"key":"2020110620013553900_ocy013-B20","first-page":"19","article-title":"Results of the Active Learning Challenge","volume":"16","author":"Guyon","year":"2011","journal-title":"JMLR: Workshop Conf Proc"},{"issue":"6","key":"2020110620013553900_ocy013-B21","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual comparisons by ranking methods","volume":"1","author":"Wilcoxon","year":"1945","journal-title":"Biometrics Bull"},{"issue":"1","key":"2020110620013553900_ocy013-B22","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1197\/jamia.M3378","article-title":"MedEx: a medication information extraction system for clinical narratives","volume":"17","author":"Xu","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110620013553900_ocy013-B23","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1136\/amiajnl-2011-000203","article-title":"2010 i2b2\/VA challenge on concepts, assertions, and relations in clinical text","volume":"18","author":"Uzuner","year":"2011","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110620013553900_ocy013-B24","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1136\/amiajnl-2013-001628","article-title":"Evaluating temporal relations in clinical text: 2012 i2b2 Challenge","volume":"20","author":"Sun","year":"2013","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/7\/800\/34150646\/ocy013.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/7\/800\/34150646\/ocy013.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,14]],"date-time":"2022-08-14T04:30:57Z","timestamp":1660451457000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/25\/7\/800\/4953034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,23]]},"references-count":24,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,3,23]]},"published-print":{"date-parts":[[2018,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocy013","relation":{},"ISSN":["1527-974X"],"issn-type":[{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2018,7]]},"published":{"date-parts":[[2018,3,23]]}}}