{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T16:52:02Z","timestamp":1771606322832,"version":"3.50.1"},"reference-count":27,"publisher":"MIT Press","issue":"4","funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["HHSN271201800033C"],"award-info":[{"award-number":["HHSN271201800033C"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.mitpressjournals.org"],"crossmark-restriction":true},"short-container-title":["Quantitative Science Studies"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p>Recent large-scale bibliometric models have largely been based on direct citation, and several recent studies have explored augmenting direct citation with other citation-based or textual characteristics. In this study we compare clustering results from direct citation, extended direct citation, a textual relatedness measure, and several citation-text hybrid measures using a set of nine million documents. Three different accuracy measures are employed, one based on references in authoritative documents, one using textual relatedness, and the last using document pairs linked by grants. We find that a hybrid relatedness measure based equally on direct citation and PubMed-related article scores gives more accurate clusters (in the aggregate) than the other relatedness measures tested. We also show that the differences in cluster contents between the different models are even larger than the differences in accuracy, suggesting that the textual and citation logics are complementary. Finally, we show that for the hybrid measure based on direct citation and related article scores, the larger clusters are more oriented toward textual relatedness, while the smaller clusters are more oriented toward citation-based relatedness.<\/jats:p>","DOI":"10.1162\/qss_a_00085","type":"journal-article","created":{"date-parts":[[2020,8,19]],"date-time":"2020-08-19T12:35:17Z","timestamp":1597840517000},"page":"1570-1585","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":18,"title":["A comparison of large-scale science models based on textual, direct citation and hybrid relatedness"],"prefix":"10.1162","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7814-8951","authenticated-orcid":true,"given":"Kevin W.","family":"Boyack","sequence":"first","affiliation":[{"name":"SciTech Strategies, Inc., Albuquerque, NM (USA)"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9118-8955","authenticated-orcid":true,"given":"Richard","family":"Klavans","sequence":"additional","affiliation":[{"name":"SciTech Strategies, Inc., Wayne, PA (USA)"}]}],"member":"281","reference":[{"issue":"2","key":"bib1","first-page":"714","volume":"1","author":"Ahlgren P.","year":"2020","journal-title":"Quantitative Science Studies"},{"key":"bib2","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.joi.2008.11.003","volume":"3","author":"Ahlgren P.","year":"2009","journal-title":"Journal of Informetrics"},{"issue":"12","key":"bib3","doi-asserted-by":"crossref","first-page":"2389","DOI":"10.1002\/asi.21419","volume":"61","author":"Boyack K. W.","year":"2010","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"bib4","volume-title":"23rd International Conference on Science and Technology Indicators (STI 2018)","author":"Boyack K. W.","year":"2018"},{"issue":"3","key":"bib5","doi-asserted-by":"crossref","first-page":"e18029","DOI":"10.1371\/journal.pone.0018029","volume":"6","author":"Boyack K. W.","year":"2011","journal-title":"PLOS ONE"},{"key":"bib6","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s11192-011-0347-4","volume":"88","author":"Gl\u00e4nzel W.","year":"2011","journal-title":"Scientometrics"},{"key":"bib7","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1007\/s11192-017-2301-6","volume":"111","author":"Gl\u00e4nzel W.","year":"2017","journal-title":"Scientometrics"},{"key":"bib8","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1007\/s11192-017-2296-z","volume":"111","author":"Gl\u00e4ser J.","year":"2017","journal-title":"Scientometrics"},{"key":"bib9","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1016\/j.joi.2018.03.004","volume":"12","author":"Haunschild R.","year":"2018","journal-title":"Journal of Informetrics"},{"issue":"10","key":"bib10","first-page":"e03000385","volume":"17","author":"Hutchins B. I.","year":"2019","journal-title":"PLOS Biology"},{"key":"bib11","first-page":"615","volume-title":"International Conference on Multidisciplinary Information Sciences and Technologies (InSciT2006)","author":"Janssens F.","year":"2006"},{"key":"bib12","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1016\/j.ipm.2009.06.003","volume":"45","author":"Janssens F.","year":"2009","journal-title":"Information Processing & Management"},{"issue":"4","key":"bib13","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1016\/j.joi.2017.10.002","volume":"11","author":"Klavans R.","year":"2017","journal-title":"Journal of Informetrics"},{"issue":"4","key":"bib14","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1002\/asi.23734","volume":"68","author":"Klavans R.","year":"2017","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"bib15","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1186\/1471-2105-8-423","volume":"8","author":"Lin J.","year":"2007","journal-title":"BMC Bioinformatics"},{"issue":"6","key":"bib16","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1002\/asi.21312","volume":"61","author":"Liu X.","year":"2010","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"bib17","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1007\/s11192-017-2366-2","volume":"111","author":"Meyer-Br\u00f6tz F.","year":"2017","journal-title":"Scientometrics"},{"issue":"1","key":"bib18","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.joi.2017.12.006","volume":"12","author":"Sj\u00f6g\u00e5rde P.","year":"2018","journal-title":"Journal of Informetrics"},{"key":"bib19","doi-asserted-by":"publisher","DOI":"10.1162\/qss_a_00004"},{"key":"bib20","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1007\/s11192-012-0896-1","volume":"96","author":"Thijs B.","year":"2013","journal-title":"Scientometrics"},{"key":"bib21","doi-asserted-by":"crossref","first-page":"5233","DOI":"10.1038\/s41598-019-41695-z","volume":"9","author":"Traag V. A.","year":"2019","journal-title":"Scientific Reports"},{"key":"bib22","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.1007\/s11192-017-2306-1","volume":"111","author":"Velden T.","year":"2017","journal-title":"Scientometrics"},{"key":"bib23","volume-title":"Paper presented at the 16th International Conference of the International Society on Scientometrics and Informetrics","author":"Waltman L.","year":"2017"},{"issue":"2","key":"bib24","first-page":"691","volume":"1","author":"Waltman L.","year":"2020","journal-title":"Quantitative Science Studies"},{"issue":"12","key":"bib25","doi-asserted-by":"crossref","first-page":"2378","DOI":"10.1002\/asi.22748","volume":"63","author":"Waltman L.","year":"2012","journal-title":"Journal of the American Society for Information Science and Technology"},{"issue":"10","key":"bib26","doi-asserted-by":"crossref","first-page":"e0187164","DOI":"10.1371\/journal.pone.0187164","volume":"12","author":"Yu D.","year":"2017","journal-title":"PLOS ONE"},{"key":"bib27","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s11192-015-1705-4","volume":"106","author":"Zhang L.","year":"2016","journal-title":"Scientometrics"}],"container-title":["Quantitative Science Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/qss_a_00085","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,12]],"date-time":"2024-08-12T02:46:18Z","timestamp":1723430778000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/qss\/article\/1\/4\/1570-1585\/96116"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":27,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["10.1162\/qss_a_00085"],"URL":"https:\/\/doi.org\/10.1162\/qss_a_00085","relation":{},"ISSN":["2641-3337"],"issn-type":[{"value":"2641-3337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]},"assertion":[{"value":"2020-05-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-08-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-12-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}