{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T10:10:06Z","timestamp":1775124606806,"version":"3.50.1"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T00:00:00Z","timestamp":1695945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Res. Metr. Anal."],"abstract":"<jats:p>Biomedical experts are facing challenges in keeping up with the vast amount of biomedical knowledge published daily. With millions of citations added to databases like MEDLINE\/PubMed each year, efficiently accessing relevant information becomes crucial. Traditional term-based searches may lead to irrelevant or missed documents due to homonyms, synonyms, abbreviations, or term mismatch. To address this, semantic search approaches employing predefined concepts with associated synonyms and relations have been used to expand query terms and improve information retrieval. The National Library of Medicine (NLM) plays a significant role in this area, indexing citations in the MEDLINE database with topic descriptors from the Medical Subject Headings (MeSH) thesaurus, enabling advanced semantic search strategies to retrieve relevant citations, despite synonymy, and polysemy of biomedical terms. Over time, advancements in semantic indexing have been made, with Machine Learning facilitating the transition from manual to automatic semantic indexing in the biomedical literature. The paper highlights the journey of this transition, starting with manual semantic indexing and the initial efforts toward automatic indexing. The BioASQ challenge has served as a catalyst in revolutionizing the domain of semantic indexing, further pushing the boundaries of efficient knowledge retrieval in the biomedical field.<\/jats:p>","DOI":"10.3389\/frma.2023.1250930","type":"journal-article","created":{"date-parts":[[2023,9,30]],"date-time":"2023-09-30T21:36:48Z","timestamp":1696109808000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["The road from manual to automatic semantic indexing of biomedical literature: a 10 years journey"],"prefix":"10.3389","volume":"8","author":[{"given":"Anastasia","family":"Krithara","sequence":"first","affiliation":[]},{"given":"James G.","family":"Mork","sequence":"additional","affiliation":[]},{"given":"Anastasios","family":"Nentidis","sequence":"additional","affiliation":[]},{"given":"Georgios","family":"Paliouras","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,9,29]]},"reference":[{"key":"B1","unstructured":"AntoniouG.\n            van HarmelenF.\n          A Semantic Web Primer, 2nd Edn. MIT Press2008"},{"key":"B2","unstructured":"AronsonA. R.\n            BodenreiderO.\n            ChangH. F.\n            HumphreyS. M.\n            MorkJ. G.\n            NelsonS.\n          The Indexing Initiative: A Report to the Board of Scientific Counselors of the Lister Hill. Technical Report1999"},{"key":"B3","article-title":"\u201cResults of the bioasq tasks of the question answering lab at CLEF 2015,\u201d","author":"Balikas","year":"2015","journal-title":"Working Notes of CLEF 2015"},{"key":"B4","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1561\/1500000032","article-title":"Semantic search on text and knowledge bases","volume":"10","author":"Bast","year":"2016","journal-title":"Found. Trends Inform. Retrieval"},{"key":"B5","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1038\/scientificamerican0501-34","article-title":"The semantic web","volume":"284","author":"Berners-Lee","year":"2001","journal-title":"Sci. Am"},{"key":"B6","first-page":"318","article-title":"From surgeon general's bookshelf to national library of medicine: a brief history","volume":"74","author":"Blake","year":"1986","journal-title":"Bull. Med. Libr. Assoc"},{"key":"B7","doi-asserted-by":"publisher","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The unified medical language system (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"B8","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1007\/978-1-59745-440-7_20","article-title":"\u201cGoPubMed: exploring Pubmed with ontological background knowledge,\u201d","author":"Dietze","year":"2008","journal-title":"Bioinformatics for Systems Biology"},{"key":"B9","unstructured":"\u201cA survey in semantic search technologies,\u201d\n            DongH.\n            HussainF. K.\n            ChangE.\n          10.1109\/DEST.2008.4635202.2008 2nd IEEE International Conference on Digital Ecosystems and Technologies2008"},{"key":"B10","article-title":"SNOMED-CT: The advanced terminology and coding system for eHealth","author":"Donnelly","year":"2006","journal-title":"Stud. Health Technol. Inform"},{"key":"B11","doi-asserted-by":"publisher","first-page":"W745","DOI":"10.1093\/nar\/gkl037","article-title":"HubMed: a web-based biomedical literature search interface","volume":"34","author":"Eaton","year":"2006","journal-title":"Nucleic Acids Res"},{"key":"B12","unstructured":"\u201cOverview of bioasq 2021-mesinesp track. Evaluation of advance hierarchical classification techniques for scientific literature, patents and clinical trials,\u201d\n            GascoL.\n            NentidisA.\n            KritharaA.\n            Estrada-ZavalaD.\n            Toshiyuki MurasakiR.\n            Primo-Pe\u00f1aE.\n          Proceedings of the 9th BioASQ Workshop A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering2021"},{"key":"B13","unstructured":"\u201cPubMed: redesigning citation data management,\u201d\n            GollnerK.\n            CaneseK.\n          Journal Article Tag Suite Conference (JATS-Con) Proceedings 2017 [Internet2017"},{"key":"B14","doi-asserted-by":"publisher","first-page":"108","DOI":"10.3163\/1536-5050.97.2.007","article-title":"The great contribution: index medicus, index-catalogue, and indexcat","volume":"97","author":"Greenberg","year":"2009","journal-title":"J. Med. Libr. Assoc"},{"key":"B15","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1038\/019431a0","article-title":"A monthly classified record of the medical literature of the world","volume":"19","year":"1879","journal-title":"Nature"},{"key":"B16","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1007\/s10618-014-0382-x","article-title":"Evaluation measures for hierarchical classification: a unified view and novel approaches","volume":"29","author":"Kosmopoulos","year":"2015","journal-title":"Data Mining Knowl. Discov"},{"key":"B17","unstructured":"\u201cOverview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023,\u201d\n            Lima-L\u00f3pezS.\n            Farr\u00e9-MaduellE.\n            Gasc\u00f3L.\n            NentidisA.\n            KritharaA.\n            KatsimprasG.\n          Working Notes of CLEF 20232023"},{"key":"B18","unstructured":"\u201cNCBI at the 2013 BioASQ challenge task: learning to rank for automatic MeSH indexing,\u201d\n            MaoY.\n            LuZ.\n          CLEF 2013 Workshop Proceedings2013"},{"key":"B19","author":"Miles","year":"1982","journal-title":"A History of the National Library of Medicine: The Nation's Treasury of Medical Knowledge"},{"key":"B20","unstructured":"\u201cOverview of distemist at bioasq: automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources,\u201d\n            Miranda-EscaladaA.\n            GascoL.\n            Lima-L\u00f3pezS.\n            Farr\u00e9-MaduellE.\n            EstradaD.\n            NentidisA.\n          Working Notes of Conference and Labs of the Evaluation (CLEF) Forum, CEUR Workshop Proceedings2022"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1186\/s13326-017-0113-5","article-title":"12 years on-is the NLM medical text indexer still useful and relevant?","author":"Mork","year":"2017","journal-title":"J. Biomed. Semant"},{"key":"B22","unstructured":"\u201cVocabulary density method for customized indexing of MEDLINE journals,\u201d\n            MorkJ. G.\n            Demner-FushmanD.\n            SchmidtS.\n            AronsonA. R.\n          AMIA2014"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1145\/3411408.3411414","article-title":"\u201cZero-shot classification of biomedical articles with emerging mesh descriptors,\u201d","author":"Mylonas","year":"2020","journal-title":"11th Hellenic Conference on Artificial Intelligence"},{"key":"B24","first-page":"171","article-title":"\u201cOverview of bioasq tasks 10a, 10b and synergy10 in CLEF2022,\u201d","author":"Nentidis","year":"2022","journal-title":"Proceedings of the Working Notes of CLEF 2022"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2020.102282","article-title":"Beyond mesh: fine-grained semantic indexing of biomedical literature based on weak supervision","author":"Nentidis","year":"2020","journal-title":"Inform. Process. Manage"},{"key":"B26","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s00799-021-00304-z","article-title":"What is all this new MeSH about?","volume":"22","author":"Nentidis","year":"2021","journal-title":"Int. J. Digit. Librar"},{"key":"B27","unstructured":"\u201cBiomedical semantic indexing and question answering,\u201d\n            NgomoA. N.\n            PaliourasG.\n          Proceedings of the First Workshop on Bio-Medical Semantic Indexing and Question Answering, a Post-Conference Workshop of Conference and Labs of the Evaluation Forum 2013 (CLEF 2013)2013"},{"key":"B28","unstructured":"50th Anniversary Medical Subject Headings (MeSH) Event.2010"},{"key":"B29","year":"2015","journal-title":"MEDLINE Data Changes\u20132016, OLDMEDLINE MeSH Mapping"},{"key":"B30","unstructured":"Archived: Detailed Indexing Statistics: 1965-2017 (Website)"},{"key":"B31","year":"","journal-title":"Incorporating Values for Indexing Method in MEDLINE\/PubMed XML"},{"key":"B32","unstructured":"Online Indexing Training Module (Website)"},{"key":"B33","unstructured":"US National Library of Medicine Mission Statement (Website)2020"},{"key":"B34","unstructured":"MEDLINE 2022 Initiative: Transition to Automated Indexing. Technical report2021"},{"key":"B35","unstructured":"Citations Added to MEDLINE by Fiscal Year (Website)"},{"key":"B36","unstructured":"MEDLINE: Overview (Website)"},{"key":"B37","unstructured":"MEDLINE PubMed Production Statistics (Website)"},{"key":"B38","unstructured":"A Brief History of NLM(Website)"},{"key":"B39","unstructured":"Frequently Asked Question About Indexing for MEDLINE (Website)"},{"key":"B40","unstructured":"Medical Subject Headings (MeSH) Homepage (Website)"},{"key":"B41","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1093\/bib\/bbl035","article-title":"Evolving research trends in bioinformatics","volume":"8","author":"Perez-Iratxeta","year":"2007","journal-title":"Brief. Bioinform"},{"key":"B42","unstructured":"\u201cA neural text ranking approach for automatic MeSH indexing,\u201d\n            RaeA. R.\n            MorkJ. G.\n            Demner-FushmanD.\n          CLEF 2021 Workshop Proceedings - BioASQ Workshop2021"},{"key":"B43","doi-asserted-by":"publisher","first-page":"e237","DOI":"10.1093\/bioinformatics\/btl302","article-title":"EBIMed-text crunching to gather facts for proteins from Medline","volume":"23","author":"Rebholz-Schuhmann","year":"2007","journal-title":"Bioinformatics"},{"key":"B44","unstructured":"ReznickJ. S.\n            KoyleK. M.\n          US National Library of Medicine. Charleston, SC: Arcadia Publishing2017"},{"key":"B45","unstructured":"\u201cOverview of mesinesp8, a Spanish medical semantic indexing task within bioasq 2020,\u201d\n            Rodriguez-PenagosC.\n            NentidisA.\n            Gonzalez-AgirreA.\n            AsensioA.\n            Armengol-Estap\u00e9J.\n            KritharaA.\n          Proceedings of the 8th BioASQ Workshop A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering2020"},{"key":"B46","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1186\/s12859-015-0564-6","article-title":"An overview of the bioasq large-scale biomedical semantic indexing and question answering competition","volume":"16","author":"Tsatsaronis","year":"2015","journal-title":"BMC Bioinformatics"},{"key":"B47","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1007\/978-0-387-09823-4_34","article-title":"\u201cMining multi-label data,\u201d","author":"Tsoumakas","year":"2009","journal-title":"Data Mining and Knowledge Discovery Handbook"},{"key":"B48","unstructured":"\u201cLarge-scale semantic indexing of biomedical publications at bioasq,\u201d\n            TsoumakasG.\n            LaliotisM.\n            MarkantonatosN.\n            VlahavasI.\n          CLEF 2013 Workshop Proceedings2013"},{"key":"B49","unstructured":"Specimen Fasciculus of a Catalogue of the National Medical Library1876"},{"key":"B50","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1093\/bioinformatics\/btaa837","article-title":"BERTMeSH: deep contextual representation learning for large-scale high-performance MeSH indexing with full text","volume":"37","author":"You","year":"2021","journal-title":"Bioinformatics"},{"key":"B51","unstructured":"YouR.\n            ZhangZ.\n            WangZ.\n            DaiS.\n            MamitsukaH.\n            ZhuS.\n          AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification. NeurIPS 20192018"},{"key":"B52","unstructured":"\u201cUsing learning-to-rank to enhance NLM medical text indexer results,\u201d\n            ZavorinI.\n            MorkJ. G.\n            Demner-FushmanD.\n          Proceedings of the Fourth BioASQ Workshop2016"}],"container-title":["Frontiers in Research Metrics and Analytics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2023.1250930\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,30]],"date-time":"2023-09-30T21:36:55Z","timestamp":1696109815000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frma.2023.1250930\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,29]]},"references-count":52,"alternative-id":["10.3389\/frma.2023.1250930"],"URL":"https:\/\/doi.org\/10.3389\/frma.2023.1250930","relation":{},"ISSN":["2504-0537"],"issn-type":[{"value":"2504-0537","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,29]]},"article-number":"1250930"}}