{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T01:09:31Z","timestamp":1784077771291,"version":"3.55.0"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2019,10,9]],"date-time":"2019-10-09T00:00:00Z","timestamp":1570579200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61572139"],"award-info":[{"award-number":["61572139"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61872094"],"award-info":[{"award-number":["61872094"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Municipal Science and Technology Major Project","award":["2017SHZDZX01"],"award-info":[{"award-number":["2017SHZDZX01"]}]},{"name":"Shanghai Municipal Science and Technology Major Project","award":["B18015"],"award-info":[{"award-number":["B18015"]}]},{"name":"Shanghai Science and Technology","award":["16JC1420402"],"award-info":[{"award-number":["16JC1420402"]}]},{"name":"Shanghai Municipal Science and Technology Major Project","award":["2018SHZDZX01"],"award-info":[{"award-number":["2018SHZDZX01"]}]},{"name":"JST ACCEL","award":["JPMJAC1503"],"award-info":[{"award-number":["JPMJAC1503"]}]},{"name":"MEXT Kakenhi","award":["16H02868"],"award-info":[{"award-number":["16H02868"]}]},{"name":"MEXT Kakenhi","award":["19H04169"],"award-info":[{"award-number":["19H04169"]}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>With the rapidly growing biomedical literature, automatically indexing biomedical articles by Medical Subject Heading (MeSH), namely MeSH indexing, has become increasingly important for facilitating hypothesis generation and knowledge discovery. Over the past years, many large-scale MeSH indexing approaches have been proposed, such as Medical Text Indexer, MeSHLabeler, DeepMeSH and MeSHProbeNet. However, the performance of these methods is hampered by using limited information, i.e. only the title and abstract of biomedical articles.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We propose FullMeSH, a large-scale MeSH indexing method taking advantage of the recent increase in the availability of full text articles. Compared to DeepMeSH and other state-of-the-art methods, FullMeSH has three novelties: (i) Instead of using a full text as a whole, FullMeSH segments it into several sections with their normalized titles in order to distinguish their contributions to the overall performance. (ii) FullMeSH integrates the evidence from different sections in a \u2018learning to rank\u2019 framework by combining the sparse and deep semantic representations. (iii) FullMeSH trains an Attention-based Convolutional Neural Network for each section, which achieves better performance on infrequent MeSH headings. FullMeSH has been developed and empirically trained on the entire set of 1.4 million full-text articles in the PubMed Central Open Access subset. It achieved a Micro F-measure of 66.76% on a test set of 10\u00a0000 articles, which was 3.3% and 6.4% higher than DeepMeSH and MeSHLabeler, respectively. Furthermore, FullMeSH demonstrated an average improvement of 4.7% over DeepMeSH for indexing Check Tags, a set of most frequently indexed MeSH headings.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>The software is available upon request.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz756","type":"journal-article","created":{"date-parts":[[2019,10,3]],"date-time":"2019-10-03T19:28:53Z","timestamp":1570130933000},"page":"1533-1541","source":"Crossref","is-referenced-by-count":34,"title":["FullMeSH: improving large-scale MeSH indexing with full text"],"prefix":"10.1093","volume":"36","author":[{"given":"Suyang","family":"Dai","sequence":"first","affiliation":[{"name":"School of Computer Science and Shanghai Key Lab of Intelligent Information Processing , Fudan University, Shanghai 200433, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronghui","family":"You","sequence":"additional","affiliation":[{"name":"School of Computer Science and Shanghai Key Lab of Intelligent Information Processing , Fudan University, Shanghai 200433, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9998-916X","authenticated-orcid":false,"given":"Zhiyong","family":"Lu","sequence":"additional","affiliation":[{"name":"National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH) , Bethesda, MD, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaodi","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computing and Mathematics, Charles Sturt University , Albury, NSW 2640, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiroshi","family":"Mamitsuka","sequence":"additional","affiliation":[{"name":"Bioinformatics Center, Institute for Chemical Research, Kyoto University , Uji, Kyoto Prefecture, Japan"},{"name":"Department of Computer Science, Aalto University , Espoo, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shanfeng","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Shanghai Key Lab of Intelligent Information Processing , Fudan University, Shanghai 200433, China"},{"name":"Shanghai Institute of Artificial Intelligence Algorithms and Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University , Shanghai 200433, China"},{"name":"Ministry of Education, Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University) , China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,10,9]]},"reference":[{"key":"2023060910272297600_btz756-B1","first-page":"268","article-title":"The NLM indexing initiative\u2019s medical text indexer","volume":"107","author":"Aronson","year":"2004","journal-title":"Stud. Health Technol. Inform"},{"key":"2023060910272297600_btz756-B2","author":"Bahdanau","year":"2014"},{"key":"2023060910272297600_btz756-B3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1961189.1961199","article-title":"LIBSVM: a library for support vector machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intel. Syst. Technol"},{"key":"2023060910272297600_btz756-B4","first-page":"785","volume-title":"XGBoost: A Scalable Tree Boosting System","author":"Chen","year":"2016"},{"key":"2023060910272297600_btz756-B5","author":"Comeau","year":"2019"},{"key":"2023060910272297600_btz756-B6","first-page":"484","author":"Demner-Fushman","year":"2015"},{"key":"2023060910272297600_btz756-B7","first-page":"1871","article-title":"LIBLINEAR: a library for large linear classification","volume":"9","author":"Fan","year":"2008","journal-title":"J. Mach. Learn. Res"},{"key":"2023060910272297600_btz756-B8","first-page":"271","author":"Gay","year":"2005"},{"key":"2023060910272297600_btz756-B9","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1109\/TSMCB.2012.2227998","article-title":"Efficient semisupervised MEDLINE document clustering with MeSH-semantic and global-content constraints","volume":"43","author":"Gu","year":"2013","journal-title":"IEEE Trans. Cybernetics"},{"key":"2023060910272297600_btz756-B10","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1136\/amiajnl-2010-000055","article-title":"Recommending MeSH terms for annotating biomedical articles","volume":"18","author":"Huang","year":"2011","journal-title":"J. Am. Med. Inform. Assoc"},{"key":"2023060910272297600_btz756-B11","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s10791-007-9027-7","article-title":"An empirical study of tokenization strategies for biomedical information retrieval","volume":"10","author":"Jiang","year":"2007","journal-title":"Inform. Retrieval"},{"key":"2023060910272297600_btz756-B12","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1186\/1471-2105-14-208","article-title":"MeSH indexing based on automatically generated summaries","volume":"14","author":"Jimeno-Yepes","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023060910272297600_btz756-B13","first-page":"47","author":"Jin","year":"2018"},{"key":"2023060910272297600_btz756-B14","first-page":"1188","author":"Le","year":"2014"},{"key":"2023060910272297600_btz756-B15","first-page":"115","author":"Liu","year":"2017"},{"key":"2023060910272297600_btz756-B16","doi-asserted-by":"crossref","first-page":"i339","DOI":"10.1093\/bioinformatics\/btv237","article-title":"MeSHLabeler: improving the accuracy of large-scale MeSH indexing by integrating diverse evidence","volume":"31","author":"Liu","year":"2015","journal-title":"Bioinformatics"},{"key":"2023060910272297600_btz756-B17","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/s10791-008-9074-8","article-title":"Evaluation of query expansion using MeSH in PubMed","volume":"12","author":"Lu","year":"2009","journal-title":"Inform. Retrieval"},{"key":"2023060910272297600_btz756-B18","author":"Mao","year":"2013"},{"key":"2023060910272297600_btz756-B19","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13326-017-0123-3","article-title":"MeSH Now: automatic MeSH indexing at PubMed scale via learning to rank","volume":"8","author":"Mao","year":"2017","journal-title":"J. Biomed. Seman"},{"key":"2023060910272297600_btz756-B20","first-page":"3111","author":"Mikolov","year":"2013"},{"key":"2023060910272297600_btz756-B21","author":"Mork","year":"2013"},{"key":"2023060910272297600_btz756-B22","first-page":"1328","author":"Mork","year":"2014"},{"key":"2023060910272297600_btz756-B23","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s13326-017-0113-5","article-title":"12 years on\u2013is the NLM medical text indexer still useful and relevant?","volume":"8","author":"Mork","year":"2017","journal-title":"J. Biomed. Seman"},{"key":"2023060910272297600_btz756-B24","first-page":"1101","author":"Mullenbach","year":"2018"},{"key":"2023060910272297600_btz756-B25","doi-asserted-by":"crossref","first-page":"i70","DOI":"10.1093\/bioinformatics\/btw294","article-title":"DeepMeSH: deep semantic representation for improving large-scale MeSH indexing","volume":"32","author":"Peng","year":"2016","journal-title":"Bioinformatics"},{"key":"2023060910272297600_btz756-B26","doi-asserted-by":"crossref","first-page":"D23","DOI":"10.1093\/nar\/gky1069","article-title":"Database resources of the national center for biotechnology information","volume":"47","author":"Sayers","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023060910272297600_btz756-B27","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/s10791-008-9073-9","article-title":"Exploring criteria for successful query expansion in the genomic domain","volume":"12","author":"Stokes","year":"2009","journal-title":"Inform. Retrieval"},{"key":"2023060910272297600_btz756-B28","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1186\/s12859-015-0564-6","article-title":"An overview of the BioASQ large-scale biomedical semantic indexing and question answering competition","volume":"16","author":"Tsatsaronis","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"2023060910272297600_btz756-B29","author":"Tsoumakas","year":"2013"},{"key":"2023060910272297600_btz756-B30","doi-asserted-by":"crossref","first-page":"3794","DOI":"10.1093\/bioinformatics\/btz142","article-title":"Meshprobenet: a self-attentive probe net for mesh indexing","volume":"35","author":"Xun","year":"2019","journal-title":"Bioinformatics"},{"key":"2023060910272297600_btz756-B31","first-page":"1480","author":"Yang","year":"2016"},{"key":"2023060910272297600_btz756-B32","doi-asserted-by":"crossref","first-page":"1944","DOI":"10.1093\/bioinformatics\/btp338","article-title":"Enhancing MEDLINE document clustering by incorporating MeSH semantic similarity","volume":"25","author":"Zhu","year":"2009","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btz756\/30350367\/btz756.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/5\/1533\/50552957\/bioinformatics_36_5_1533.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/5\/1533\/50552957\/bioinformatics_36_5_1533.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,9]],"date-time":"2023-06-09T10:29:04Z","timestamp":1686306544000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/5\/1533\/5584197"}},"subtitle":[],"editor":[{"given":"Robert","family":"Murphy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2019,10,9]]},"references-count":32,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz756","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,3]]},"published":{"date-parts":[[2019,10,9]]}}}