{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T11:02:21Z","timestamp":1790074941475,"version":"4.0.1"},"reference-count":45,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T00:00:00Z","timestamp":1783987200000},"content-version":"vor","delay-in-days":194,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["1R01LM014292-01"],"award-info":[{"award-number":["1R01LM014292-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Major research and implementation efforts have been devoted to indexing articles according to the major topics discussed, but much less effort to indexing their publication types and study designs (collectively, PTs). In this Perspective, we discuss how indexing PTs differs from topical Medical Subject Heading (MeSH) indexing and requires a different approach. Rather than focus on the technical aspects of machine learning-based indexing models, we emphasize the goals and purposes for which biomedical articles are indexed, and the surprisingly thorny question of how indexing systems should be evaluated. Topical Medical Subject Heading (MeSH) terms are assigned to articles that cover the major topics discussed; when more than one term is applicable, only the most specific term is assigned. In contrast, PTs are assigned to articles that have a given structure or use a particular design. To meet the needs of end users, particularly groups involved in evidence syntheses, PT indexing needs to be comprehensive and employ probabilistic goodness-of-fit prediction scores. Whereas existing National Library of Medicine (NLM) hierarchies place publication types and study design-related terms on separate trees from each other, we have created a unified hierarchy that permits more appropriate retrieval via automatic expansion. Automated PT indexing systems should allow users to input article records or full-text PDFs and receive scores in real time. This will offer consistent indexing across bibliographic databases, as well as preprints and unpublished manuscripts. Automated PT indexing systems, properly designed and implemented, hold the promise of greatly improving the retrieval of biomedical articles, saving substantial effort when writing evidence syntheses and benefiting other users as well. Database URL: \u00a0https:\/\/www.nlm.nih.gov\/medline\/medline_home.html<\/jats:p>","DOI":"10.1093\/database\/baag039","type":"journal-article","created":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T12:53:29Z","timestamp":1784033609000},"source":"Crossref","is-referenced-by-count":2,"title":["Goals and strategies for the indexing of publication types and study designs"],"prefix":"10.1093","volume":"2026","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1079-3406","authenticated-orcid":false,"given":"Neil R","family":"Smalheiser","sequence":"first","affiliation":[{"name":"Department of Psychiatry, College of Medicine, University of Illinois at Chicago, 1601 W. Taylor Street MC912 , Chicago, IL 60612 ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7374-7549","authenticated-orcid":false,"given":"Joe D","family":"Menke","sequence":"additional","affiliation":[{"name":"School of Information Sciences, University of Illinois Urbana-Champaign, 501 E. Daniel St. , Champaign, IL 61820 ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6110-2997","authenticated-orcid":false,"given":"Arthur W","family":"Holt","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, College of Medicine, University of Illinois at Chicago, 1601 W. Taylor Street MC912 , Chicago, IL 60612 ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3987-9393","authenticated-orcid":false,"given":"Halil","family":"Kilicoglu","sequence":"additional","affiliation":[{"name":"School of Information Sciences, University of Illinois Urbana-Champaign, 501 E. Daniel St. , Champaign, IL 61820 ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5098-5667","authenticated-orcid":false,"given":"Jodi","family":"Schneider","sequence":"additional","affiliation":[{"name":"Information School, University of Wisconsin-Madison, 1205 University Ave , Madison, WI 53706 ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,14]]},"reference":[{"key":"2026092206054465300_bib1","doi-asserted-by":"publisher","first-page":"1533","DOI":"10.1093\/bioinformatics\/btz756","article-title":"FullMeSH: improving large-scale MeSH indexing with full text","volume":"36","author":"Dai","year":"2020","journal-title":"Bioinformatics"},{"key":"2026092206054465300_bib2","first-page":"1031","article-title":"Automatic MeSH Indexing: revisiting the subheading attachment problem","volume":"2020","author":"Rae","year":"2021","journal-title":"AMIA Annu Symp Proc"},{"key":"2026092206054465300_bib3","first-page":"727","article-title":"A high recall classifier for selecting articles for MEDLINE indexing","volume":"2019","author":"Rae","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"2026092206054465300_bib4","doi-asserted-by":"publisher","first-page":"1250930","DOI":"10.3389\/frma.2023.1250930","article-title":"The road from manual to automatic semantic indexing of biomedical literature: a 10 years journey","volume":"8","author":"Krithara","year":"2023","journal-title":"Front Res Metr Anal"},{"key":"2026092206054465300_bib5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv2506.03321","article-title":"Enhancing automatic PT tagging for MEDLINE citations using transformer-based models","author":"Cid","year":"2025","journal-title":"ArXiv"},{"key":"2026092206054465300_bib6","article-title":"Robust biomedical publication type and study design classification with knowledge-guided perturbations","author":"Ming","year":"2026","journal-title":"arXiv:2605.11502. IEEE-ICHI"},{"key":"2026092206054465300_bib7","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3163\/1536-5050.101.2.004","article-title":"Comparative effectiveness research designs: an analysis of terms and coverage in Medical Subject Headings (MeSH) and Emtree","volume":"101","author":"Bekhuis","year":"2013","journal-title":"J Med Libr Assoc"},{"key":"2026092206054465300_bib8","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.jclinepi.2016.08.008","article-title":"No suitable precise or optimized epidemiologic search filters were available for bibliographic databases","volume":"82","author":"Waffenschmidt","year":"2017","journal-title":"J Clin Epidemiol"},{"key":"2026092206054465300_bib9","doi-asserted-by":"publisher","first-page":"111201","DOI":"10.1016\/j.jclinepi.2023.10.017","article-title":"Searching a methods topic: practical challenges and implications for search design","volume":"166","author":"Hirt","year":"2024","journal-title":"J Clin Epidemiol"},{"key":"2026092206054465300_bib10","doi-asserted-by":"publisher","first-page":"e52758","DOI":"10.2196\/52758","article-title":"Human-comparable sensitivity of Large Language Models in identifying eligible studies through title and abstract screening: 3-layer strategy using GPT-3.5 and GPT-4 for systematic reviews","volume":"26","author":"Matsui","year":"2024","journal-title":"J Med Internet Res"},{"key":"2026092206054465300_bib11","doi-asserted-by":"publisher","first-page":"101287","DOI":"10.1016\/j.cola.2024.101287","article-title":"Screening articles for systematic reviews with ChatGPT","volume":"80","author":"Syriani","year":"2024","journal-title":"J Comput Lang"},{"key":"2026092206054465300_bib12","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1186\/s13643-024-02575-4","article-title":"Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain","volume":"13","author":"Dennst\u00e4dt","year":"2024","journal-title":"Syst Rev."},{"key":"2026092206054465300_bib13","doi-asserted-by":"publisher","first-page":"e2420496","DOI":"10.1001\/jamanetworkopen.2024.20496","article-title":"Performance of a Large Language Model in screening citations","volume":"7","author":"Oami","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2026092206054465300_bib14","first-page":"557","article-title":"Clinical \u2018case series\u2019: a concept analysis","volume":"12","author":"Abu-Zidan","year":"2012","journal-title":"Afr Health Sci"},{"key":"2026092206054465300_bib15","doi-asserted-by":"publisher","DOI":"10.64898\/2025.12.19.25342712","article-title":"Creating an indexing scheme for case series articles","author":"Shahidehpour","year":"2025","journal-title":"medRxiv"},{"key":"2026092206054465300_bib16","doi-asserted-by":"publisher","first-page":"1321","DOI":"10.1007\/s00381-014-2460-1","article-title":"Case series and descriptive cohort studies in neurosurgery: the confusion and solution","volume":"30","author":"Esene","year":"2014","journal-title":"Childs Nerv Syst"},{"key":"2026092206054465300_bib17","doi-asserted-by":"publisher","first-page":"1035","DOI":"10.1111\/jvim.14741","article-title":"What\u2019s in a Name? The incorrect use of case series as a study design label in studies involving dogs and cats","volume":"31","author":"Sargeant","year":"2017","journal-title":"J Vet Intern Med"},{"key":"2026092206054465300_bib18","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1093\/jamia\/ocu025","article-title":"Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine","volume":"22","author":"Cohen","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2026092206054465300_bib19","volume-title":"The Autobiography of Alice B. Toklas (Modern Classics Series): Enriched Edition","author":"Stein","year":"2024"},{"key":"2026092206054465300_bib20","doi-asserted-by":"publisher","DOI":"10.1101\/2025.03.10.25323694","article-title":"Issues regarding the indexing of adaptive clinical trial articles","author":"Smalheiser","year":"2025","journal-title":"medRxiv"},{"key":"2026092206054465300_bib21","doi-asserted-by":"publisher","first-page":"822","DOI":"10.1093\/jamia\/ocab290","article-title":"A web-based tool for automatically linking clinical trials to their publications","volume":"29","author":"Smalheiser","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"2026092206054465300_bib22","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1093\/jamiaopen\/ooaa042","article-title":"New improved aggregator: predicting which clinical trial articles derive from the same registered clinical trial","volume":"3","author":"Smalheiser","year":"2020","journal-title":"JAMIA Open"},{"key":"2026092206054465300_bib23","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1186\/s13643-017-0518-3","article-title":"A systematic review of the processes used to link clinical trial registrations to their published results. Syst Rev","author":"Bashir","year":"2017"},{"key":"2026092206054465300_bib24","doi-asserted-by":"publisher","first-page":"baag022","DOI":"10.1093\/database\/baag022","article-title":"A similarity metric, rubric, and unified hierarchy for biomedical publication types and study designs","volume":"2026","author":"Smalheiser","year":"2026","journal-title":"Database (Oxford)"},{"key":"2026092206054465300_bib25","doi-asserted-by":"publisher","first-page":"318","DOI":"10.5195\/jmla.2025.2110","article-title":"Sometimes the apple does fall far from the tree: a case study on automatic indexing precision errors in PubMed","volume":"113","author":"Wilson","year":"2025","journal-title":"J Med Libr Assoc"},{"key":"2026092206054465300_bib26","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1111\/hir.70000","article-title":"Trends in use of the new MeSH term \u2018overdiagnosis\u2019: a bibliometric review","volume":"42","author":"Gram","year":"2025","journal-title":"Health Info Libr J"},{"key":"2026092206054465300_bib27","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1097\/WNO.0000000000001895","article-title":"Inappropriate indexing of case reports to the \u2018Papilledema\u2019 subject heading: a systematic review","volume":"43","author":"Tao","year":"2023","journal-title":"J Neuroophthalmol"},{"key":"2026092206054465300_bib28","doi-asserted-by":"publisher","first-page":"125","DOI":"10.11607\/jomi.4590","article-title":"Research waste: how are dental survival articles indexed and reported?","volume":"31","author":"Layton","year":"2016","journal-title":"Int J Oral Maxillofac Implants"},{"key":"2026092206054465300_bib29","doi-asserted-by":"publisher","first-page":"39","DOI":"10.5195\/jmla.2025.1936","article-title":"Algorithmic indexing in MEDLINE frequently overlooks important concepts and may compromise literature search results","volume":"113","author":"Amar-Zifkin","year":"2025","journal-title":"J Med Libr Assoc"},{"key":"2026092206054465300_bib30","doi-asserted-by":"publisher","DOI":"10.1101\/2025.04.23.25326300","article-title":"Enhancing automated indexing of publication types and study designs in biomedical literature using full-text features","author":"Menke","year":"2025","journal-title":"medRxiv"},{"key":"2026092206054465300_bib31","doi-asserted-by":"publisher","DOI":"10.1101\/2021.07.13.21260468","article-title":"Fifty ways to tag your pubtypes: multi-tagger, a set of probabilistic publication type and study design taggers to support biomedical indexing and evidence-based medicine","author":"Cohen","year":"2021","journal-title":"medRxiv"},{"key":"2026092206054465300_bib32","first-page":"818","article-title":"Publication type tagging using Transformer models and multi-label classification","volume":"2024","author":"Menke","year":"2025","journal-title":"AMIA Annu Symp Proc"},{"key":"2026092206054465300_bib33","doi-asserted-by":"publisher","first-page":"1005","DOI":"10.1017\/rsm.2025.10031","article-title":"StudyTypeTeller\u2014large language models to automatically classify research study types for systematic reviews","volume":"16","author":"Emilova\u00a0Doneva","year":"2025","journal-title":"Res Synth Methods"},{"key":"2026092206054465300_bib34","first-page":"2901","article-title":"Obtaining well calibrated probabilities using Bayesian binning","volume":"2015","author":"Naeini","year":"2015","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"2026092206054465300_bib35","first-page":"406","article-title":"Testing a filtering strategy for systematic reviews: evaluating work savings and recall","volume":"2022","author":"Proescholdt","year":"2022","journal-title":"AMIA Jt Summits Transl Sci Proc."},{"key":"2026092206054465300_bib36","doi-asserted-by":"publisher","first-page":"ooac015","DOI":"10.1093\/jamiaopen\/ooac015","article-title":"Evaluation of publication type tagging as a strategy to screen randomized controlled trial articles in preparing systematic reviews","volume":"5","author":"Schneider","year":"2022","journal-title":"JAMIA Open"},{"key":"2026092206054465300_bib37","doi-asserted-by":"publisher","first-page":"1165","DOI":"10.1093\/jamia\/ocx053","article-title":"Identifying reports of randomized controlled trials (RCTs) via a hybrid machine learning and crowdsourcing approach","volume":"24","author":"Wallace","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2026092206054465300_bib38","doi-asserted-by":"publisher","first-page":"1544","DOI":"10.1110\/ps.062184006","article-title":"A categorization approach to automated ontological function annotation","volume":"15","author":"Verspoor","year":"2006","journal-title":"Protein Sci"},{"key":"2026092206054465300_bib39","doi-asserted-by":"crossref","DOI":"10.1002\/9781119756194","article-title":"Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy","author":"Deeks","year":"2023"},{"key":"2026092206054465300_bib40","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.1001\/jama.287.13.1728","article-title":"Basic science and translational research in JAMA","volume":"287","author":"Fontanarosa","year":"2002","journal-title":"JAMA"},{"key":"2026092206054465300_bib41","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/s13326-023-00292-w","article-title":"Automatic classification of experimental models in biomedical literature to support searching for alternative methods to animal experiments","volume":"14","author":"Neves","year":"2023","journal-title":"J Biomed Semantics"},{"key":"2026092206054465300_bib42","unstructured":"Priem J., Piwowar H., Orr R. (2022). OpenAlex: a fully-open index of scholarly works, authors, venues, institutions, and concepts. STI 2022 \u2018From Global Indicators to Local Applications\u2019 Conference Proceedings. 2022; 10.5281\/zenodo.6936227"},{"key":"2026092206054465300_bib43","doi-asserted-by":"publisher","first-page":"195","DOI":"10.5195\/jmla.2020.834","article-title":"Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial","volume":"108","author":"Clark","year":"2020","journal-title":"J Med Libr Assoc"},{"key":"2026092206054465300_bib44","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1186\/s12874-024-02186-6","article-title":"An analysis of published study designs in PubMed prisoner health abstracts from 1963 to 2023: a text mining study","volume":"24","author":"Karystianis","year":"2024","journal-title":"BMC Med Res Method"},{"key":"2026092206054465300_bib45","doi-asserted-by":"crossref","DOI":"10.1093\/database\/baag039","article-title":"Goals and strategies for the indexing of publication types and study designs","volume-title":"arXiv","author":"Smalheiser","year":"2026"}],"container-title":["Database"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baag039\/68770775\/baag039.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baag039\/68770775\/baag039.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T10:05:50Z","timestamp":1790071550000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/database\/article\/doi\/10.1093\/database\/baag039\/8733907"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":45,"URL":"https:\/\/doi.org\/10.1093\/database\/baag039","relation":{},"ISSN":["1758-0463"],"issn-type":[{"value":"1758-0463","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"baag039"}}