{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T02:51:39Z","timestamp":1786157499169,"version":"3.56.0"},"reference-count":38,"publisher":"MIT Press","issue":"1","license":[{"start":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T00:00:00Z","timestamp":1615161600000},"content-version":"vor","delay-in-days":66,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,4,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We present a large-scale comparison of five multidisciplinary bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. The comparison considers scientific documents from the period 2008\u20132017 covered by these data sources. Scopus is compared in a pairwise manner with each of the other data sources. We first analyze differences between the data sources in the coverage of documents, focusing for instance on differences over time, differences per document type, and differences per discipline. We then study differences in the completeness and accuracy of citation links. Based on our analysis, we discuss the strengths and weaknesses of the different data sources. We emphasize the importance of combining a comprehensive coverage of the scientific literature with a flexible set of filters for making selections of the literature.<\/jats:p>","DOI":"10.1162\/qss_a_00112","type":"journal-article","created":{"date-parts":[[2021,1,27]],"date-time":"2021-01-27T16:19:52Z","timestamp":1611764392000},"page":"20-41","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":714,"title":["Large-scale comparison of bibliographic data sources: Scopus, Web of\n                    Science, Dimensions, Crossref, and Microsoft Academic"],"prefix":"10.1162","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5987-2389","authenticated-orcid":true,"given":"Martijn","family":"Visser","sequence":"first","affiliation":[{"name":"Centre for Science and Technology Studies, Leiden University, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8448-4521","authenticated-orcid":true,"given":"Nees Jan","family":"van Eck","sequence":"additional","affiliation":[{"name":"Centre for Science and Technology Studies, Leiden University, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8249-1752","authenticated-orcid":true,"given":"Ludo","family":"Waltman","sequence":"additional","affiliation":[{"name":"Centre for Science and Technology Studies, Leiden University, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2021,4,8]]},"reference":[{"issue":"1","key":"2021041320052498600_bib1","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1162\/qss_a_00019","article-title":"Scopus as a curated, high-quality\n                        bibliometric data source for academic research in quantitative science\n                        studies","volume":"1","author":"Baas","year":"2020","journal-title":"Quantitative Science Studies"},{"key":"2021041320052498600_bib2","article-title":"Underreporting of matched references in\n                        Crossref metadata [Blog post, February 5]","author":"Bilder","year":"2019"},{"issue":"1","key":"2021041320052498600_bib3","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1162\/qss_a_00018","article-title":"Web of Science as a data source for\n                        research on scientific and scholarly activity","volume":"1","author":"Birkle","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"1","key":"2021041320052498600_bib4","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1007\/s11192-018-2855-y","article-title":"Field classification of publications in\n                        Dimensions: A first case study testing its reliability and\n                        validity","volume":"117","author":"Bornmann","year":"2018","journal-title":"Scientometrics"},{"key":"2021041320052498600_bib5","article-title":"How I scraped data from Google\n                        Scholar","author":"Else","year":"2018","journal-title":"Nature"},{"issue":"10","key":"2021041320052498600_bib6","doi-asserted-by":"crossref","first-page":"2070","DOI":"10.1002\/asi.21372","article-title":"Accuracy and completeness of publication\n                        and citation records in the Web of Science, PsycINFO, and Google Scholar: A\n                        case study for the computation of h indices in\n                        Psychology","volume":"61","author":"Garc\u00eda-P\u00e9rez","year":"2010","journal-title":"Journal of the American Society for\n                        Information Science and Technology"},{"issue":"1","key":"2021041320052498600_bib7","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s11192-019-03114-y","article-title":"Two new kids on the block: How do\n                        Crossref and Dimensions compare with Google Scholar, Microsoft Academic,\n                        Scopus and the Web of Science?","volume":"120","author":"Harzing","year":"2019","journal-title":"Scientometrics"},{"key":"2021041320052498600_bib8","article-title":"Crowdsourcing open citations with CROCI\n                        \u2013 An analysis of the current status of open citations, and a\n                        proposal","author":"Heibi","year":"2019","journal-title":"arXiv:1902.02534"},{"issue":"2","key":"2021041320052498600_bib9","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1007\/s11192-019-03217-6","article-title":"Software review: COCI, the\n                        OpenCitations Index of Crossref open DOI-to-DOI citations","volume":"121","author":"Heibi","year":"2019","journal-title":"Scientometrics"},{"issue":"1","key":"2021041320052498600_bib10","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1162\/qss_a_00022","article-title":"Crossref: The sustainable source of\n                        community-owned scholarly metadata","volume":"1","author":"Hendricks","year":"2020","journal-title":"Quantitative\n                        Science Studies"},{"issue":"1","key":"2021041320052498600_bib11","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1162\/qss_a_00020","article-title":"Dimensions: Bringing down barriers\n                        between scientometricians and data","volume":"1","author":"Herzog","year":"2020","journal-title":"Quantitative\n                        Science Studies"},{"issue":"1","key":"2021041320052498600_bib12","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1007\/s11192-018-2854-z","article-title":"Response to the letter \u201cField classification of\n                        publications in Dimensions: A first case study testing its reliability and\n                        validity.\u201d","volume":"117","author":"Herzog","year":"2018","journal-title":"Scientometrics"},{"key":"2021041320052498600_bib13","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1038\/520429a","article-title":"The Leiden Manifesto for research\n                        metrics","volume":"520","author":"Hicks","year":"2015","journal-title":"Nature"},{"key":"2021041320052498600_bib14","doi-asserted-by":"crossref","first-page":"23","DOI":"10.3389\/frma.2018.00023","article-title":"Dimensions: Building context for search\n                        and evaluation","volume":"3","author":"Hook","year":"2018","journal-title":"Frontiers in Research Metrics and\n                        Analytics"},{"issue":"2","key":"2021041320052498600_bib15","first-page":"445","article-title":"Comparison of bibliographic data\n                        sources: Implications for the robustness of university\n                        rankings","volume":"1","author":"Huang","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"2","key":"2021041320052498600_bib16","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1177\/0165551508098603","article-title":"Comparing bibliometric country-by-country rankings derived\n                        from the Web of Science and Scopus: The effect of poorly cited journals in\n                        oncology","volume":"35","author":"L\u00f3pez-Illescas","year":"2009","journal-title":"Journal of Information Science"},{"issue":"3","key":"2021041320052498600_bib17","doi-asserted-by":"crossref","first-page":"2175","DOI":"10.1007\/s11192-018-2820-9","article-title":"Coverage of highly-cited documents in\n                        Google Scholar, Web of Science, and Scopus: A multidisciplinary\n                        comparison","volume":"116","author":"Mart\u00edn-Mart\u00edn","year":"2018","journal-title":"Scientometrics"},{"issue":"4","key":"2021041320052498600_bib18","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1016\/j.joi.2018.09.002","article-title":"Google Scholar, Web of Science, and\n                        Scopus: A systematic comparison of citations in 252 subject\n                        categories","volume":"12","author":"Mart\u00edn-Mart\u00edn","year":"2018","journal-title":"Journal of Informetrics"},{"key":"2021041320052498600_bib19","article-title":"Google Scholar, Microsoft Academic,\n                        Scopus, Dimensions, Web of Science, and OpenCitations\u2019 COCI: A\n                        multidisciplinary comparison of coverage via citations","author":"Mart\u00edn-Mart\u00edn","year":"2020","journal-title":"arXiv:2004.14329"},{"key":"2021041320052498600_bib20","article-title":"Microsoft Academic\n                        Graph (version 2019-03-22) [Data set]","author":"Microsoft\n                        Academic","year":"2019"},{"issue":"1","key":"2021041320052498600_bib21","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s11192-015-1765-5","article-title":"The journal coverage of Web of Science\n                        and Scopus: A comparative analysis","volume":"106","author":"Mongeon","year":"2016","journal-title":"Scientometrics"},{"issue":"10","key":"2021041320052498600_bib22","doi-asserted-by":"crossref","first-page":"2550","DOI":"10.1002\/asi.23590","article-title":"Evaluation of the citation matching algorithms of CWTS and\n                        iFQ in comparison to the Web of Science","volume":"67","author":"Olensky","year":"2016","journal-title":"Journal of\n                        the Association for Information Science and Technology"},{"issue":"2","key":"2021041320052498600_bib23","doi-asserted-by":"crossref","first-page":"420","DOI":"10.3145\/epi.2018.mar.21","article-title":"Dimensions: Re-discovering the ecosystem\n                        of scientific information","volume":"27","author":"Ordu\u00f1a-Malea","year":"2018","journal-title":"El Profesional de la\n                        Informaci\u00f3n"},{"issue":"1","key":"2021041320052498600_bib24","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1162\/qss_a_00023","article-title":"OpenCitations, an infrastructure\n                        organization for open scholarship","volume":"1","author":"Peroni","year":"2020","journal-title":"Quantitative\n                        Science Studies"},{"key":"2021041320052498600_bib25","article-title":"Advancing responsible research\n                        assessment [Blog post, December 16]","author":"Plume","year":"2020"},{"key":"2021041320052498600_bib26","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1201\/9781315155890-2","article-title":"Web of Science: The first citation index for data analytics\n                        and scientometrics","volume-title":"Research analytics: Boosting university productivity and\n                        competitiveness through scientometrics","author":"Schnell","year":"2017"},{"key":"2021041320052498600_bib27","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1201\/9781315155890-3","article-title":"A brief history of Scopus: The world\u2019s largest\n                        abstract and citation database of scientific literature","volume-title":"Research analytics: Boosting university productivity and\n                        competitiveness through scientometrics","author":"Schotten","year":"2017"},{"key":"2021041320052498600_bib28","volume-title":"Scopus content coverage\n                        guide","author":"Scopus","year":"2020"},{"key":"2021041320052498600_bib29","article-title":"The journal coverage of Web of Science,\n                        Scopus and Dimensions: A comparative analysis","author":"Singh","year":"2020","journal-title":"arXiv:2011.00223"},{"key":"2021041320052498600_bib30","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1145\/2740908.2742839","article-title":"An overview of Microsoft Academic\n                        Service (MAS) and applications","volume-title":"Proceedings of\n                        the 24th International World Wide Web Conference","author":"Sinha","year":"2015"},{"issue":"3","key":"2021041320052498600_bib31","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1016\/j.joi.2015.05.002","article-title":"A systematic analysis of duplicate\n                        records in Scopus","volume":"9","author":"Valderrama-Zuri\u00e1n","year":"2015","journal-title":"Journal of Informetrics"},{"key":"2021041320052498600_bib32","first-page":"1087","article-title":"Accuracy of citation data in Web of\n                        Science and Scopus","volume-title":"Proceedings of the 16th\n                        International Conference of the International Society for Scientometrics and\n                        Informetrics","author":"Van Eck","year":"2017"},{"key":"2021041320052498600_bib33","article-title":"Crossref as a new source of citation\n                        data: A comparison with Web of Science and Scopus [Blog post, January\n                        17]","author":"Van Eck","year":"2018"},{"key":"2021041320052498600_bib34","article-title":"Large-scale comparison of bibliographic\n                        data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft\n                        Academic [Data set]","author":"Visser","year":"2020","journal-title":"Zenodo"},{"issue":"1","key":"2021041320052498600_bib35","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1162\/qss_e_00026","article-title":"Special issue on bibliographic data\n                        sources","volume":"1","author":"Waltman","year":"2020","journal-title":"Quantitative Science Studies"},{"key":"2021041320052498600_bib36","doi-asserted-by":"crossref","first-page":"45","DOI":"10.3389\/fdata.2019.00045","article-title":"A review of Microsoft Academic Services\n                        for science of science studies","volume":"2","author":"Wang","year":"2019","journal-title":"Frontiers in Big\n                        Data"},{"issue":"1","key":"2021041320052498600_bib37","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1162\/qss_a_00021","article-title":"Microsoft Academic Graph: When experts\n                        are not enough","volume":"1","author":"Wang","year":"2020","journal-title":"Quantitative Science\n                        Studies"},{"issue":"2","key":"2021041320052498600_bib38","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/j.joi.2016.02.003","article-title":"Large-scale analysis of the accuracy of\n                        the journal classification systems of Web of Science and\n                        Scopus","volume":"10","author":"Wang","year":"2016","journal-title":"Journal of Informetrics"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/1\/20\/1906541\/qss_a_00112.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/qss\/article-pdf\/2\/1\/20\/1906541\/qss_a_00112.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:28:50Z","timestamp":1618446530000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/2\/1\/20\/97574\/Large-scale-comparison-of-bibliographic-data"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,4,8]]}},"URL":"https:\/\/doi.org\/10.1162\/qss_a_00112","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021]]},"published":{"date-parts":[[2021]]}}}