{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T13:54:12Z","timestamp":1767707652184,"version":"3.37.3"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,1,3]],"date-time":"2021-01-03T00:00:00Z","timestamp":1609632000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,1,3]],"date-time":"2021-01-03T00:00:00Z","timestamp":1609632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2021,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>On December 31st 2019, the World Health Organization China Country Office was informed of cases of pneumonia of unknown etiology detected in Wuhan City. The cause of the syndrome was a new type of coronavirus isolated on January 7th 2020 and named Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2). SARS-CoV-2 is the cause of the coronavirus disease 2019 (COVID-19). Since January 2020 an ever increasing number of scientific works related to the new pathogen have appeared in literature. Identifying relevant research outcomes at very early stages is challenging. In this work we use COVID-19 as a use-case for investigating: (1) which tools and frameworks are mostly used for early scholarly communication; (2) to what extent altmetrics can be used to identify potential impactful research in tight (i.e. <jats:italic>quasi-zero-day<\/jats:italic>) time-windows. A literature review with rigorous eligibility criteria is performed for gathering a sample composed of scientific papers about SARS-CoV-2\/COVID-19 appeared in literature in the tight time-window ranging from January 15th 2020 to February 24th 2020. This sample is used for building a knowledge graph that represents the knowledge about papers and indicators formally. This knowledge graph feeds a data analysis process which is applied for experimenting with altmetrics as impact indicators. We find moderate correlation among traditional citation count, citations on social media, and mentions on news and blogs. Additionally, correlation coefficients are not inflated by indicators associated with zero values, which are quite common at very early stages after an article has been published. This suggests there is a common intended meaning of the citational acts associated with aforementioned indicators. Then, we define a method, i.e. the <jats:italic>Comprehensive Impact Score<\/jats:italic> (CIS), that harmonises different indicators for providing a multi-dimensional impact indicator. CIS shows promising results as a tool for selecting relevant papers even in a tight time-window. Our results foster the development of automated frameworks aimed at helping the scientific community in identifying relevant work even in case of limited literature and observation time.<\/jats:p>","DOI":"10.1007\/s11192-020-03809-7","type":"journal-article","created":{"date-parts":[[2021,1,3]],"date-time":"2021-01-03T23:02:17Z","timestamp":1609714937000},"page":"1189-1215","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Using altmetrics for detecting impactful research in quasi-zero-day time-windows: the case of COVID-19"],"prefix":"10.1007","volume":"126","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2509-159X","authenticated-orcid":false,"given":"Erik","family":"Boetto","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3257-6552","authenticated-orcid":false,"given":"Maria Pia","family":"Fantini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5568-2684","authenticated-orcid":false,"given":"Aldo","family":"Gangemi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7331-9520","authenticated-orcid":false,"given":"Davide","family":"Golinelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6886-0160","authenticated-orcid":false,"given":"Manfredi","family":"Greco","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2928-9496","authenticated-orcid":false,"given":"Andrea Giovanni","family":"Nuzzolese","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9380-5160","authenticated-orcid":false,"given":"Valentina","family":"Presutti","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Flavia","family":"Rallo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,1,3]]},"reference":[{"key":"3809_CR1","doi-asserted-by":"publisher","unstructured":"Andersen, K. G., Rambaut, A., Lipkin, W. I., Holmes, E. C., & Garry, R. F. (2020). The proximal origin of SARS-CoV-2. Nature Medicine. https:\/\/doi.org\/10.1038\/s41591-020-0820-9.","DOI":"10.1038\/s41591-020-0820-9"},{"issue":"6","key":"3809_CR2","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1002\/bult.2012.1720380607","volume":"38","author":"J Bar-Ilan","year":"2012","unstructured":"Bar-Ilan, J. (2012). JASIST 2001\u20132010. Bulletin of the Association for Information Science and Technology, 38(6), 24\u201328. https:\/\/doi.org\/10.1002\/bult.2012.1720380607.","journal-title":"Bulletin of the Association for Information Science and Technology"},{"key":"3809_CR3","doi-asserted-by":"publisher","unstructured":"Bilge, L., Dumitras, T. (2012). Before we knew it: An empirical study of zero-day attacks in the real world. In Yu, T., Danezis, G., Gligor, V. D. (Eds.), ACM conference on computer and communications security (pp. 833\u2013844). New York: ACM. https:\/\/doi.org\/10.1145\/2382196.2382284. ISBN: 978-1-4503-1651-4.","DOI":"10.1145\/2382196.2382284"},{"issue":"3","key":"3809_CR4","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.1007\/s11192-015-1565-y","volume":"103","author":"L Bornmann","year":"2015","unstructured":"Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123\u20131144. https:\/\/doi.org\/10.1007\/s11192-015-1565-y.","journal-title":"Scientometrics"},{"issue":"5","key":"3809_CR5","doi-asserted-by":"publisher","first-page":"0197133","DOI":"10.1371\/journal.pone.0197133","volume":"13","author":"L Bornmann","year":"2018","unstructured":"Bornmann, L., & Haunschild, R. (2018). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data. PloS One, 13(5), 0197133. https:\/\/doi.org\/10.1371\/journal.pone.0197133.","journal-title":"PloS One"},{"key":"3809_CR6","doi-asserted-by":"publisher","unstructured":"Ciancarini, P., Iorio, A. D., Nuzzolese, A. G., Peroni, S., & Vitali, F. (2014). Evaluating citation functions in CiTO: Cognitive issues. In Presutti, V., d\u2019Amato, C., Gandon, F., d\u2019Aquin, M., Staab, S., & Tordai A. (Eds.), The semantic web: Trends and challenges\u201411th international conference, ESWC 2014, Anissaras, Crete, Greece, May 25\u201329, proceedings. Lecture Notes in Computer Science (Vol. 8465, pp. 580\u2013594). Springer. https:\/\/doi.org\/10.1007\/978-3-319-07443-6_39.","DOI":"10.1007\/978-3-319-07443-6_39"},{"key":"3809_CR7","unstructured":"D\u2019Arcus, B., & Giasson, F. (2008). Bibliographic ontology specification, Specification Document. http:\/\/bibliontology.com\/. Accessed 30 Nov 2020."},{"issue":"6206","key":"3809_CR8","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1126\/science.1259439","volume":"346","author":"Y Gil","year":"2014","unstructured":"Gil, Y., Greaves, M., Hendler, J., & Hirsh, H. (2014). Amplify scientific discovery with artificial intelligence. Science, 346(6206), 171\u2013172. https:\/\/doi.org\/10.1126\/science.1259439.","journal-title":"Science"},{"key":"3809_CR9","doi-asserted-by":"publisher","unstructured":"Gorbalenya, A. E. (2020). Severe acute respiratory syndrome-related coronavirus-the species and its viruses, a statement of the coronavirus study group. BioRxiv. https:\/\/doi.org\/10.1101\/2020.02.07.937862.","DOI":"10.1101\/2020.02.07.937862"},{"issue":"1","key":"3809_CR10","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1007\/s11192-016-1910-9","volume":"108","author":"S Haustein","year":"2016","unstructured":"Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413\u2013423. https:\/\/doi.org\/10.1007\/s11192-016-1910-9.","journal-title":"Scientometrics"},{"issue":"1","key":"3809_CR11","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1609\/aimag.v37i1.2642","volume":"37","author":"H Kitano","year":"2016","unstructured":"Kitano, H. (2016). Artificial intelligence to win the nobel prize and beyond: Creating the engine for scientific discovery. AI Magazine, 37(1), 39\u201349. https:\/\/doi.org\/10.1609\/aimag.v37i1.2642.","journal-title":"AI Magazine"},{"key":"3809_CR12","doi-asserted-by":"publisher","unstructured":"Kousha, K., & Thelwall, M. (2020). Covid-19 publications: Database coverage, citations, readers, tweets, news, facebook walls, reddit posts. Quantitative Science Studies. https:\/\/doi.org\/10.1162\/qss_a_00066.","DOI":"10.1162\/qss_a_00066"},{"issue":"2","key":"3809_CR13","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1007\/s11192-007-1868-8","volume":"76","author":"S Lehmann","year":"2008","unstructured":"Lehmann, S., Jackson, A. D., & Lautrup, B. (2008). A quantitative analysis of indicators of scientific performance. Scientometrics, 76(2), 369\u2013390. https:\/\/doi.org\/10.1007\/s11192-007-1868-8.","journal-title":"Scientometrics"},{"key":"3809_CR14","unstructured":"Li, X., Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Archambault, Y. G., Lariviere, V. (Eds.), The 17th international conference on science and technology indicators (pp. 541\u2013551)."},{"issue":"2","key":"3809_CR15","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1007\/s11192-011-0580-x","volume":"91","author":"X Li","year":"2012","unstructured":"Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461\u2013471. https:\/\/doi.org\/10.1007\/s11192-011-0580-x.","journal-title":"Scientometrics"},{"key":"3809_CR16","doi-asserted-by":"publisher","unstructured":"Nuzzolese, A. G., Gentile, A. L., Presutti, V., & Gangemi, A. (2016). Conference linked data: The ScholarlyData Project. In Groth, P. T., Simperl, E., Gray, A. J. G., Sabou, M., Kr\u00f6tzsch, M., L\u00e9cu\u00e9, F., & Fl\u00f6ck, F. Y. (Eds.), International semantic web conference (2). Gil Lecture Notes in Computer Science (Vol. 9982, pp. 50\u2013158). ISBN: 978-3-319-46546-3. https:\/\/doi.org\/10.1007\/978-3-319-46547-0_16","DOI":"10.1007\/978-3-319-46547-0_16"},{"key":"3809_CR17","doi-asserted-by":"publisher","unstructured":"Nuzzolese, A. G. , Presutti, V., Gangemi, A., & Ciancarini, P. (2018). Extending scholarly data with research impact indicators. In Workshop on semantics, analytics, visualisation: Enhancing scholarly dissemination (SAVE-SD). Springer. https:\/\/doi.org\/10.1007\/978-3-030-01379-0_4.","DOI":"10.1007\/978-3-030-01379-0_4"},{"issue":"2","key":"3809_CR18","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1007\/s11192-018-2988-z","volume":"118","author":"AG Nuzzolese","year":"2019","unstructured":"Nuzzolese, A. G., Ciancarini, P., Gangemi, A., Peroni, S., Poggi, F., & Presutti, V. (2019). Do altmetrics work for assessing research quality? Scientometrics, 118(2), 539\u2013562. https:\/\/doi.org\/10.1007\/s11192-018-2988-z.","journal-title":"Scientometrics"},{"issue":"1","key":"3809_CR19","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1007\/s11192-020-03397-6","volume":"123","author":"S Peroni","year":"2020","unstructured":"Peroni, S., Ciancarini, P., Gangemi, A., Nuzzolese, A. G., Poggi, F., & Presutti, V. (2020). The practice of self-citations: A longitudinal study. Scientometrics, 123(1), 253\u2013282. https:\/\/doi.org\/10.1007\/s11192-020-03397-6.","journal-title":"Scientometrics"},{"key":"3809_CR20","doi-asserted-by":"publisher","first-page":"12-1","DOI":"10.5283\/bpf.205","volume":"3","author":"I Peters","year":"2014","unstructured":"Peters, I., Jobmann, A., Hoffmann, C. P., K\u00fcnne, S., Schmitz, J., & Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: Comparison of current tools. Bibliometrie-praxis und forschung, 3, 12-1\u201312-19. https:\/\/doi.org\/10.5283\/bpf.205.","journal-title":"Bibliometrie-praxis und forschung"},{"key":"3809_CR21","doi-asserted-by":"publisher","first-page":"199","DOI":"10.7717\/peerj-cs.199","volume":"5","author":"F Poggi","year":"2019","unstructured":"Poggi, F., Ciancarini, P., Gangemi, A., Nuzzolese, A. G., Peroni, S., & Presutti, V. (2019). Predicting the results of evaluation procedures of academics. PeerJ Computer Science, 5, 199. https:\/\/doi.org\/10.7717\/peerj-cs.199.","journal-title":"PeerJ Computer Science"},{"issue":"11","key":"3809_CR22","doi-asserted-by":"publisher","first-page":"48753","DOI":"10.1371\/journal.pone.0048753","volume":"7","author":"J Priem","year":"2012","unstructured":"Priem, J., Groth, P., & Taraborelli, D. (2012). The altmetrics collection. PloS One, 7(11), 48753. https:\/\/doi.org\/10.1371\/journal.pone.0048753.","journal-title":"PloS One"},{"issue":"3","key":"3809_CR23","doi-asserted-by":"publisher","first-page":"0173152","DOI":"10.1371\/journal.pone.0173152","volume":"12","author":"J Ravenscroft","year":"2017","unstructured":"Ravenscroft, J., Liakata, M., Clare, A., & Duma, D. (2017). Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements. PloS One, 12(3), 0173152. https:\/\/doi.org\/10.1371\/journal.pone.0173152.","journal-title":"PloS One"},{"key":"3809_CR24","doi-asserted-by":"publisher","DOI":"10.1002\/9781118575574","volume-title":"Multivariate density estimation: theory, practice, and visualization","author":"DW Scott","year":"2015","unstructured":"Scott, D. W. (2015). Multivariate density estimation: theory, practice, and visualization. New York: Wiley."},{"issue":"Suppl 1","key":"3809_CR25","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1186\/2041-1480-1-s1-s6","volume":"1","author":"D Shotton","year":"2010","unstructured":"Shotton, D. (2010). CiTO, the citation typing ontology. Journal of Biomedical Semantics, 1(Suppl 1), 6. https:\/\/doi.org\/10.1186\/2041-1480-1-s1-s6.","journal-title":"Journal of Biomedical Semantics"},{"key":"3809_CR26","isbn-type":"print","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3324-9","volume-title":"Density estimation for statistics and data analysis. Monographs on Statistics & Applied Probability","author":"BW Silverman","year":"1986","unstructured":"Silverman, B. W. (1986). Density estimation for statistics and data analysis. Monographs on Statistics & Applied Probability (Vol. 26). London: Chapman and Hall. ISBN: 978-0-4122-4620-3.","ISBN":"https:\/\/id.crossref.org\/isbn\/9780412246203"},{"issue":"2","key":"3809_CR27","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1007\/s11192-013-1117-2","volume":"98","author":"P Sud","year":"2014","unstructured":"Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131\u20131143. https:\/\/doi.org\/10.1007\/s11192-013-1117-2.","journal-title":"Scientometrics"},{"issue":"3","key":"3809_CR28","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1007\/s11192-018-2715-9","volume":"115","author":"M Thelwall","year":"2018","unstructured":"Thelwall, M. (2018). Early Mendeley readers correlate with later citation counts. Scientometrics, 115(3), 1231\u20131240. https:\/\/doi.org\/10.1007\/s11192-018-2715-9.","journal-title":"Scientometrics"},{"issue":"5","key":"3809_CR29","doi-asserted-by":"publisher","first-page":"64841","DOI":"10.1371\/journal.pone.0064841","volume":"8","author":"M Thelwall","year":"2013","unstructured":"Thelwall, M., Haustein, S., Larivi\u00e8re, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PloS One, 8(5), 64841. https:\/\/doi.org\/10.1371\/journal.pone.0064841.","journal-title":"PloS One"},{"issue":"10","key":"3809_CR30","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1371\/journal.pmed.0040297","volume":"4","author":"JP Vandenbroucke","year":"2007","unstructured":"Vandenbroucke, J. P., Von Elm, E., Altman, D. G., G\u00f8tzsche, P. C., Mulrow, C. D., Pocock, S. J., et al. (2007). Strengthening the reporting of observational studies in epidemiology (strobe): Explanation and elaboration. PLoS Med, 4(10), 297. https:\/\/doi.org\/10.1371\/journal.pmed.0040297.","journal-title":"PLoS Med"},{"key":"3809_CR31","doi-asserted-by":"publisher","unstructured":"White, R. W., Kules, B., & Bederson, B. B. (2005). Exploratory search interfaces: categorization, clustering and beyond: report on the XSI 2005 workshop at the human\u2013computer interaction laboratory. In University of Maryland. SIGIR Forum (Vol. 39, No. 2, pp. 52\u201356). https:\/\/doi.org\/10.1145\/1113343.1113356.","DOI":"10.1145\/1113343.1113356"},{"key":"3809_CR32","doi-asserted-by":"publisher","unstructured":"Wouters, P., Thelwall, M., Kousha, K., Waltman, L. , de Rijcke,S., Rushforth, A., et al. (2015). The metric tide: Correlation analysis of REF2014 scores and metrics (Supplementary Report II to the Independent Review of the Role of Metrics in Research Assessment and Management). London: Higher Education Funding Council for England (HEFCE) . https:\/\/doi.org\/10.13140\/RG.2.1.3362.4162.","DOI":"10.13140\/RG.2.1.3362.4162"},{"issue":"10225","key":"3809_CR33","doi-asserted-by":"publisher","first-page":"676","DOI":"10.1016\/S0140-6736(20)30461-X","volume":"395","author":"J Zarocostas","year":"2020","unstructured":"Zarocostas, J. (2020). How to fight an infodemic. The Lancet, 395(10225), 676. https:\/\/doi.org\/10.1016\/S0140-6736(20)30461-X.","journal-title":"The Lancet"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03809-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11192-020-03809-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03809-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T18:07:16Z","timestamp":1614794836000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11192-020-03809-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,3]]},"references-count":33,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,2]]}},"alternative-id":["3809"],"URL":"https:\/\/doi.org\/10.1007\/s11192-020-03809-7","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"type":"print","value":"0138-9130"},{"type":"electronic","value":"1588-2861"}],"subject":[],"published":{"date-parts":[[2021,1,3]]},"assertion":[{"value":"17 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 November 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}