{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T04:00:31Z","timestamp":1776916831123,"version":"3.51.2"},"reference-count":65,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2017,6,25]],"date-time":"2017-06-25T00:00:00Z","timestamp":1498348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Scholarly article impact reflects the significance of academic output recognised by academic peers, and it often plays a crucial role in assessing the scientific achievements of researchers, teams, institutions and countries. It is also used for addressing various needs in the academic and scientific arena, such as recruitment decisions, promotions, and funding allocations. This article provides a comprehensive review of recent progresses related to article impact assessment and prediction. The review starts by sharing some insight into the article impact research and outlines current research status. Some core methods and recent progress are presented to outline how article impact metrics and prediction have evolved to consider integrating multiple networks. Key techniques, including statistical analysis, machine learning, data mining and network science, are discussed. In particular, we highlight important applications of each technique in article impact research. Subsequently, we discuss the open issues and challenges of article impact research. At the same time, this review points out some important research directions, including article impact evaluation by considering Conflict of Interest, time and location information, various distributions of scholarly entities, and rising stars.<\/jats:p>","DOI":"10.3390\/info8030073","type":"journal-article","created":{"date-parts":[[2017,6,27]],"date-time":"2017-06-27T02:58:05Z","timestamp":1498532285000},"page":"73","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":54,"title":["An Overview on Evaluating and Predicting Scholarly Article Impact"],"prefix":"10.3390","volume":"8","author":[{"given":"Xiaomei","family":"Bai","sequence":"first","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"},{"name":"Computing Center, Anshan Normal University, Anshan 114007, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fuli","family":"Zhang","sequence":"additional","affiliation":[{"name":"Library, Anshan Normal University, Anshan 114007, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaolong","family":"Ning","sequence":"additional","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangjie","family":"Kong","sequence":"additional","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivan","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Information Technology and Mathematical Sciences, University of South Australia, Mawson Lakes, SA 5095, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Xia","sequence":"additional","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"105","DOI":"10.5465\/amp.2011.0088","article-title":"Scholarly impact revisited","volume":"26","author":"Aguinis","year":"2012","journal-title":"Acad. Manag. Perspect."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gargouri, Y., Hajjem, C., Larivi\u00e8re, V., Gingras, Y., Carr, L., Brody, T., and Harnad, S. (2010). Self-selected or mandated, open access increases citation impact for higher quality research. PLoS ONE, 5.","DOI":"10.1371\/journal.pone.0013636"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1126\/science.1237825","article-title":"Quantifying long-term scientific impact","volume":"342","author":"Wang","year":"2013","journal-title":"Science"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1038\/493159a","article-title":"Altmetrics: Value all research products","volume":"493","author":"Piwowar","year":"2013","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.joi.2006.06.001","article-title":"Finding scientific gems with Google\u2019s PageRank algorithm","volume":"1","author":"Chen","year":"2007","journal-title":"J. Informetr."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1126\/science.aaa8415","article-title":"Machine learning: Trends, perspectives, and prospects","volume":"349","author":"Jordan","year":"2015","journal-title":"Science"},{"key":"ref_7","unstructured":"Johnson, R.A., and Wichern, D.W. (2002). Applied Multivariate Statistical Analysis, Prentice Hall."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Di Ciaccio, A., Coli, M., and Ibanez, J.M.A. (2012). Advanced Statistical Methods for the Analysis of Large Data-Sets, Springer.","DOI":"10.1007\/978-3-642-21037-2"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bai, X., Xia, F., Lee, I., Zhang, J., and Ning, Z. (2016). Identifying anomalous citations for objective evaluation of scholarly article impact. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0162364"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2003","DOI":"10.1002\/asi.23309","article-title":"Do \u2018altmetrics\u2019 correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective","volume":"66","author":"Costas","year":"2015","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_11","unstructured":"Rojas, R. (2013). Neural Networks: A Systematic Introduction, Springer."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/5254.708428","article-title":"Support vector machines","volume":"13","author":"Hearst","year":"1998","journal-title":"IEEE Intell. Syst. Their Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/BF00116251","article-title":"Induction of decision trees","volume":"1","author":"Quinlan","year":"1986","journal-title":"Mach. Learn."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., and Lemeshow, S. (2004). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/0470011815.b2a10030"},{"key":"ref_15","unstructured":"Ho, T.K. (1995, January 14\u201316). Random decision forests. Proceedings of the Third International Conference on Document Analysis and Recognition, Montreal, QC, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). Unsupervised learning. The Elements of Statistical Learning, Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1093\/rfs\/10.2.481","article-title":"A Markov model for the term structure of credit risk spreads","volume":"10","author":"Jarrow","year":"1997","journal-title":"Rev. Financ. Stud."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, T., and He, T. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on knowledge discovery and data mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_21","first-page":"18","article-title":"Importance of data mining in higher education system","volume":"6","author":"Bhise","year":"2013","journal-title":"IOSR J. Hum. Soc. Sci."},{"key":"ref_22","first-page":"20120375","article-title":"Network science","volume":"371","year":"2013","journal-title":"Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1313","DOI":"10.1002\/asi.22680","article-title":"Scholarly network similarities: How bibliographic coupling networks, citation networks, cocitation networks, topical networks, coauthorship networks, and coword networks relate to each other","volume":"63","author":"Yan","year":"2012","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1002\/asi.22790","article-title":"Author-level Eigenfactor metrics: Evaluating the influence of authors, institutions, and countries within the social science research network community","volume":"64","author":"West","year":"2013","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"ref_25","unstructured":"Page, L., Brin, S., Motwani, R., and Winograd, T. (1999). The PageRank Citation Ranking: Bringing Order to the Web, Technical Report; Stanford InfoLab."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1145\/324133.324140","article-title":"Authoritative sources in a hyperlinked environment","volume":"46","author":"Kleinberg","year":"1999","journal-title":"J. ACM"},{"key":"ref_27","unstructured":"Zhang, C., Liu, C., Yu, L., Zhang, Z.K., and Zhou, T. (arXiv, 2016). Identifying the Academic Rising Stars, arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, J., Ning, Z., Bai, X., Wang, W., Yu, S., and Xia, F. (2016, January 19\u201323). Who are the Rising Stars in Academia?. Proceedings of the 16th ACM\/IEEE-CS on Joint Conference on Digital Libraries, Newark, NJ, USA.","DOI":"10.1145\/2910896.2925436"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sugiyama, K., and Kan, M.Y. (2011, January 13\u201317). Serendipitous recommendation for scholarly papers considering relations among researchers. Proceedings of the 11th Annual International ACM\/IEEE Joint Conference on Digital Libraries, Ottawa, ON, Canada.","DOI":"10.1145\/1998076.1998133"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7426","DOI":"10.1073\/pnas.1424329112","article-title":"Defining and identifying Sleeping Beauties in science","volume":"112","author":"Ke","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bai, X., Zhang, J., Cui, H., Ning, Z., and Xia, F. (2016, January 11\u201315). PNCOIRank: Evaluating the Impact of Scholarly Articles with Positive and Negative Citations. Proceedings of the 25th International Conference Companion on World Wide Web, Montr\u00e9al, QC, Canada.","DOI":"10.1145\/2872518.2889383"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1002\/asi.23179","article-title":"Measuring academic influence: Not all citations are equal","volume":"66","author":"Zhu","year":"2015","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_33","first-page":"743","article-title":"Constructions of success in academia: An early career perspective","volume":"42","author":"Sutherland","year":"2015","journal-title":"Stud. High. Educ."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"150266","DOI":"10.1098\/rsos.150266","article-title":"The advantage of short paper titles","volume":"2","author":"Letchford","year":"2015","journal-title":"R. Soc. Open Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1338","DOI":"10.1073\/pnas.1408800112","article-title":"Hierarchical cultural values predict success and mortality in high-stakes teams","volume":"112","author":"Anicich","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_36","first-page":"E4671","article-title":"Quantifying the impact of weak, strong, and super ties in scientific careers","volume":"112","author":"Petersen","year":"2015","journal-title":"Adv. Short Pap. Titles"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1007\/s11948-014-9598-9","article-title":"Mandatory and self-citation; types, reasons, their benefits and disadvantages","volume":"21","author":"Esfe","year":"2015","journal-title":"Sci. Eng. Ethics"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"13823","DOI":"10.1073\/pnas.1502280112","article-title":"The incidence and role of negative citations in science","volume":"112","author":"Catalini","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Priem, J. (2014). Altmetrics. Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact, MIT Press.","DOI":"10.7551\/mitpress\/9445.003.0019"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1038\/nj7463-491a","article-title":"Research impact: Altmetrics make their mark","volume":"500","author":"Kwok","year":"2013","journal-title":"Nature"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1038\/494176d","article-title":"Altmetrics: Too soon for use in assessment","volume":"494","author":"Cheung","year":"2013","journal-title":"Nature"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/meet.14504701033","article-title":"Measuring scholarly impact in heterogeneous networks","volume":"47","author":"Yan","year":"2010","journal-title":"Proc. Am. Soc. Inf. Sci. Technol."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wang, Y., Tong, Y., and Zeng, M. (2013, January 14\u201318). Ranking Scientific Articles by Exploiting Citations, Authors, Journals, and Time Information. Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, Bellevue, WA, USA.","DOI":"10.1609\/aaai.v27i1.8627"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"P06010","DOI":"10.1088\/1742-5468\/2007\/06\/P06010","article-title":"Ranking scientific publications using a model of network traffic","volume":"2007","author":"Walker","year":"2007","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Sayyadi, H., and Getoor, L. (May, January 30). FutureRank: Ranking Scientific Articles by Predicting their Future PageRank. Proceedings of the SIAM International Conference on Data Mining (SDM 2009), Sparks, NV, USA.","DOI":"10.1137\/1.9781611972795.46"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"033033","DOI":"10.1088\/1367-2630\/14\/3\/033033","article-title":"Quantifying the influence of scientists and their publications: Distinguishing between prestige and popularity","volume":"14","author":"Zhou","year":"2012","journal-title":"New J. Phys."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Wang, S., Xie, S., Zhang, X., Li, Z., Yu, P.S., and Shu, X. (2014, January 24\u201326). Future influence ranking of scientific literature. Proceedings of the 2014 SIAM International Conference on Data Mining, Philadelphia, PA, USA.","DOI":"10.1137\/1.9781611973440.86"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Liu, Z., Huang, H., Wei, X., and Mao, X. (2014, January 10\u201312). Tri-Rank: An Authority Ranking Framework in Heterogeneous Academic Networks by Mutual Reinforce. Proceedings of the IEEE 26th International Conference on Tools with Artificial Intelligence (ICTAI), Limassol, Cyprus.","DOI":"10.1109\/ICTAI.2014.80"},{"key":"ref_49","unstructured":"Shah, N., and Song, Y. (arXiv, 2015). S-index: Towards better metrics for quantifying research impact, arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1007\/s11192-009-0121-z","article-title":"Maps of science as interdisciplinary discourse: Co-citation contexts and the role of analogy","volume":"83","author":"Small","year":"2010","journal-title":"Scientometrics"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1016\/j.joi.2013.09.002","article-title":"Universality of scholarly impact metrics","volume":"7","author":"Kaur","year":"2013","journal-title":"J. Informetr."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"17268","DOI":"10.1073\/pnas.0806977105","article-title":"Universality of citation distributions: Toward an objective measure of scientific impact","volume":"105","author":"Radicchi","year":"2008","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1017\/cts.2016.8","article-title":"Feasibility of common bibliometrics in evaluating translational science","volume":"1","author":"Schneider","year":"2017","journal-title":"J. Clin. Transl. Sci."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s11192-016-1979-1","article-title":"Research assessment using early citation information","volume":"108","author":"Bruns","year":"2015","journal-title":"Scientometrics"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1016\/j.joi.2016.02.006","article-title":"A data analytic approach to quantifying scientific impact","volume":"10","author":"Cao","year":"2016","journal-title":"J. Informetr."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1007\/s11192-016-1926-1","article-title":"Successful fish go with the flow: Citation impact prediction based on centrality measures for term-document networks","volume":"107","author":"Klimek","year":"2016","journal-title":"Scientometrics"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1016\/j.joi.2015.06.005","article-title":"Predicting the long-term citation impact of recent publications","volume":"9","author":"Stegehuis","year":"2015","journal-title":"J. Informetr."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1007\/s11192-014-1279-6","article-title":"Citation impact prediction for scientific papers using stepwise regression analysis","volume":"101","author":"Yu","year":"2014","journal-title":"Scientometrics"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Sarig\u00f6l, E., Pfitzner, R., Scholtes, I., Garas, A., and Schweitzer, F. (2014). Predicting scientific success based on coauthorship networks. EPJ Data Sci., 3.","DOI":"10.1140\/epjds\/s13688-014-0009-x"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"E123","DOI":"10.2196\/jmir.2012","article-title":"Can tweets predict citations? metrics of social impact based on twitter and correlation with traditional metrics of scientific impact","volume":"13","author":"Eysenbach","year":"2011","journal-title":"J. Med. Internet Res."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Timilsina, M., Davis, B., Taylor, M., and Hayes, C. (2016, January 18\u201321). Towards predicting academic impact from mainstream news and weblogs: A heterogeneous graph based approach. Proceedings of the 2016 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), San Francisco, CA, USA.","DOI":"10.1109\/ASONAM.2016.7752425"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"269","DOI":"10.25088\/ComplexSystems.14.3.269","article-title":"Read Before You Cite!","volume":"14","author":"Simkin","year":"2003","journal-title":"Complex Syst."},{"key":"ref_63","first-page":"7","article-title":"Data Science Altmetrics","volume":"1","author":"Thelwall","year":"2016","journal-title":"J. Data Inf. Sci."},{"key":"ref_64","first-page":"3","article-title":"Altmetrics as new indicators of scientific impact","volume":"10","author":"Barbaro","year":"2014","journal-title":"J. Eur. Assoc. Health Inf. Libr."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1038\/523129a","article-title":"We need a measured approach to metrics","volume":"523","author":"Wilsdon","year":"2015","journal-title":"Nature"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/8\/3\/73\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:40:19Z","timestamp":1760208019000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/8\/3\/73"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,25]]},"references-count":65,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["info8030073"],"URL":"https:\/\/doi.org\/10.3390\/info8030073","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,25]]}}}