{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T07:05:42Z","timestamp":1779260742296,"version":"3.51.4"},"reference-count":35,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["TACL"],"published-print":{"date-parts":[[2018,12]]},"abstract":"<jats:p> Citations have long been used to characterize the state of a scientific field and to identify influential works. However, writers use citations for different purposes, and this varied purpose influences uptake by future scholars. Unfortunately, our understanding of how scholars use and frame citations has been limited to small-scale manual citation analysis of individual papers. We perform the largest behavioral study of citations to date, analyzing how scientific works frame their contributions through different types of citations and how this framing affects the field as a whole. We introduce a new dataset of nearly 2,000 citations annotated for their function, and use it to develop a state-of-the-art classifier and label the papers of an entire field: Natural Language Processing. We then show how differences in framing affect scientific uptake and reveal the evolution of the publication venues and the field as a whole. We demonstrate that authors are sensitive to discourse structure and publication venue when citing, and that how a paper frames its work through citations is predictive of the citation count it will receive. Finally, we use changes in citation framing to show that the field of NLP is undergoing a significant increase in consensus. <\/jats:p>","DOI":"10.1162\/tacl_a_00028","type":"journal-article","created":{"date-parts":[[2018,12,10]],"date-time":"2018-12-10T19:32:50Z","timestamp":1544470370000},"page":"391-406","source":"Crossref","is-referenced-by-count":136,"title":["Measuring the Evolution of a Scientific Field through Citation Frames"],"prefix":"10.1162","volume":"6","author":[{"given":"David","family":"Jurgens","sequence":"first","affiliation":[{"name":"University of Michigan,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Srijan","family":"Kumar","sequence":"additional","affiliation":[{"name":"Stanford University,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raine","family":"Hoover","sequence":"additional","affiliation":[{"name":"Stanford University,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dan","family":"McFarland","sequence":"additional","affiliation":[{"name":"Stanford University,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dan","family":"Jurafsky","sequence":"additional","affiliation":[{"name":"Stanford University,"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"issue":"6","key":"p_3","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","volume":"19","author":"Akaike Hirotugu","year":"1974","journal-title":"IEEE Transactions on Automatic Control"},{"issue":"1","key":"p_7","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1002\/asi.23367","volume":"67","author":"Bertin Marc","year":"2016","journal-title":"Journal of the Association for Information Science and Technology (JASIST)"},{"issue":"1","key":"p_9","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1002\/asi.4630370106","volume":"37","author":"Brooks Terrence A.","year":"1986","journal-title":"Journal of the American Society for Information Science"},{"issue":"7","key":"p_11","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1002\/(SICI)1097-4571(2000)51:7<635::AID-ASI6>3.0.CO;2-H","volume":"51","author":"Case Donald O.","year":"2000","journal-title":"Journal of the American Society for Information Science"},{"key":"p_13","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"Chawla Nitesh V.","year":"2002","journal-title":"Journal of Artificial Intelligence Research"},{"key":"p_14","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1177\/030631277500500403","volume":"5","author":"Chubin Dale E.","year":"1975","journal-title":"Social Studies of Science"},{"issue":"2","key":"p_15","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/BF01476360","volume":"9","author":"Collins Randall","year":"1994","journal-title":"Sociological Forum"},{"issue":"3","key":"p_20","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1016\/j.joi.2013.03.003","volume":"7","author":"Ding Ying","year":"2013","journal-title":"Journal of Informetrics"},{"issue":"9","key":"p_21","doi-asserted-by":"crossref","first-page":"1820","DOI":"10.1002\/asi.23256","volume":"65","author":"Ding Ying","year":"2014","journal-title":"Journal of the Association for Information Science and Technology (JASIST)"},{"issue":"1","key":"p_23","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TBDATA.2016.2521657","volume":"2","author":"Dong Yuxiao","year":"2016","journal-title":"IEEE Transactions on Big Data (TBD)"},{"key":"p_25","doi-asserted-by":"crossref","first-page":"757","DOI":"10.15195\/v3.a32","volume":"3","author":"Evans Eliza D.","year":"2016","journal-title":"Sociological Science"},{"issue":"1","key":"p_26","first-page":"3133","volume":"15","author":"Fern\u00e1ndez-Delgado Manuel","year":"2014","journal-title":"Journal of Machine Learning Research (JMLR)"},{"issue":"3","key":"p_27","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s11192-007-1777-2","volume":"72","author":"Fowler James H.","year":"2007","journal-title":"Scientometrics"},{"issue":"2","key":"p_30","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1007\/s11192-011-0343-8","volume":"87","author":"Gazni Ali","year":"2011","journal-title":"Scientometrics"},{"issue":"4","key":"p_33","first-page":"219","volume":"15","author":"Haas Andrew","year":"1989","journal-title":"Computational Linguistics"},{"issue":"3","key":"p_35","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.pragma.2008.06.001","volume":"41","author":"Harwood Nigel","year":"2009","journal-title":"Journal of Pragmatics"},{"issue":"03","key":"p_37","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1017\/S1351324915000388","volume":"22","author":"Hern\u00e1ndez-Alvarez Myriam","year":"2016","journal-title":"Natural Language Engineering"},{"issue":"4","key":"p_38","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1016\/j.joi.2013.08.005","volume":"7","author":"Hu Zhigang","year":"2013","journal-title":"Journal of Informetrics"},{"issue":"1","key":"p_43","first-page":"1","volume":"28","author":"Leech Geoffrey","year":"1992","journal-title":"Language Research"},{"issue":"2","key":"p_45","first-page":"313","volume":"19","author":"Marcus Mitchell P.","year":"1993","journal-title":"Computational Linguistics"},{"issue":"2","key":"p_47","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/000312240406900204","volume":"69","author":"Moody James","year":"2004","journal-title":"American Sociological Review"},{"issue":"1","key":"p_48","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1177\/030631277500500106","volume":"5","author":"Moravcsik Michael J.","year":"1975","journal-title":"Social Studies of Science"},{"issue":"2","key":"p_50","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/S0889-4906(97)85388-4","volume":"16","author":"Nwogu Kevin Ngozi","year":"1997","journal-title":"English for Specific Purposes"},{"issue":"5","key":"p_51","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1007\/s11135-006-9018-6","volume":"41","author":"O\u2019brien Robert M.","year":"2007","journal-title":"Quality & Quantity"},{"issue":"5","key":"p_52","first-page":"227","volume":"29","author":"Oppenheim Charles","year":"1978","journal-title":"Journal of the American Society for Information Science"},{"key":"p_53","first-page":"2825","volume":"12","author":"Pedregosa Fabian","year":"2011","journal-title":"The Journal of Machine Learning Research (JMLR)"},{"issue":"5","key":"p_56","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/BF02147226","volume":"5","author":"Peritz Bluma C.","year":"1983","journal-title":"Scientometrics"},{"issue":"387","key":"p_62","first-page":"455","volume":"44","author":"Skelton John","year":"1994","journal-title":"British Journal of General Practice"},{"issue":"2","key":"p_63","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s11192-011-0349-2","volume":"87","author":"Small Henry","year":"2011","journal-title":"Scientometrics"},{"key":"p_65","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1177\/030631277700700111","volume":"7","author":"Spiegel-R\u00fcsing Ina","year":"1977","journal-title":"Social Studies of Science"},{"issue":"11","key":"p_67","doi-asserted-by":"crossref","first-page":"e112520","DOI":"10.1371\/journal.pone.0112520","volume":"9","author":"Stern David I.","year":"2014","journal-title":"PloS One"},{"issue":"1","key":"p_75","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/BF02458400","volume":"43","author":"Vinkler Peter","year":"1998","journal-title":"Scientometrics"},{"issue":"9","key":"p_76","doi-asserted-by":"crossref","first-page":"1929","DOI":"10.1002\/asi.23083","volume":"65","author":"Wan Xiaojun","year":"2014","journal-title":"Journal of the Association for Information Science and Technology (JASIST)"},{"issue":"6154","key":"p_77","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1126\/science.1237825","volume":"342","author":"Wang Dashun","year":"2013","journal-title":"Science"},{"issue":"2","key":"p_82","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1002\/asi.23179","volume":"66","author":"Zhu Xiaodan","year":"2015","journal-title":"Journal of the Association for Information Science and Technology"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00028","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:37:58Z","timestamp":1615585078000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43437"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12]]},"references-count":35,"alternative-id":["10.1162\/tacl_a_00028"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00028","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12]]}}}