{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T06:19:25Z","timestamp":1770099565635,"version":"3.49.0"},"reference-count":39,"publisher":"Walter de Gruyter GmbH","issue":"4","license":[{"start":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T00:00:00Z","timestamp":1624492800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>Building upon pioneering work by Francis Narin and others, a new methodological approach to assessing the technological impact of scientific research is presented.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design\/methodology\/approach<\/jats:title>\n                    <jats:p>It is based on the analysis of citations made in patent families included in the PATSTAT database that is to scientific papers indexed in Scopus.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Findings<\/jats:title>\n                    <jats:p>An advanced citation matching procedure is applied to the data in order to construct two indicators of technological impact: on the citing (patent) side, the country\/region in which protection is sought and a patent family's propensity to cite scientific papers are taken into account, and on the cited (paper) side, a relative citation rate is defined for patent citations to papers that is similar to the scientific paper-to-paper citation rate in classical bibliometrics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Research limitations<\/jats:title>\n                    <jats:p>The results are limited by the available data, in our case Scopus and PATSTAT, and especially by the lack of standardization of references in patents. This required a matching procedure that is neither trivial nor exact.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Practical implications<\/jats:title>\n                    <jats:p>Results at the country\/region, document type, and publication age levels are presented. The country\/region-level results in particular reveal features that have remained hidden in analyses of straight counts. Especially notable is that the rankings of some Asian countries\/regions move upwards when the proposed normalized indicator of technological impact is applied as against the case with straight counts of patent citations to those countries\/regions\u2019 published papers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Originality\/value<\/jats:title>\n                    <jats:p>In our opinion, the level of sophistication of the indicators proposed in the current paper is unparalleled in the scientific literature, and provides a solid basis for the assessment of the technological impact of scientific research in countries\/regions and institutions.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2478\/jdis-2021-0028","type":"journal-article","created":{"date-parts":[[2021,6,24]],"date-time":"2021-06-24T20:11:25Z","timestamp":1624565485000},"page":"36-61","source":"Crossref","is-referenced-by-count":5,"title":["New Indicators of the Technological Impact of Scientific Production"],"prefix":"10.2478","volume":"6","author":[{"given":"Vicente P.","family":"Guerrero-Bote","sequence":"first","affiliation":[{"name":"SCImago Group, Dept. Information and Communication , University of Extremadura , Badajoz , Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henk F.","family":"Moed","sequence":"additional","affiliation":[{"name":"Sapienza University of Rome , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"F\u00e9lix","family":"Moya-Aneg\u00f3n","sequence":"additional","affiliation":[{"name":"SCImago Group , Madrid , Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"374","published-online":{"date-parts":[[2021,6,24]]},"reference":[{"key":"2026020217551528528_j_jdis-2021-0028_ref_001","doi-asserted-by":"crossref","unstructured":"Ahmadpoor, M., & Jones, B.F. (2017). The dual frontier: Patented inventions and prior scientific advance. Science, 357(6351), 583\u2013587.","DOI":"10.1126\/science.aam9527"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_002","doi-asserted-by":"crossref","unstructured":"Archambault, E., Campbell, D., Gingras, Y., & Larivi\u00e8re, V. (2009). \u201cComparing bibliometric statistics obtained from the Web of Science and Scopus\u201d. Journal of the American Society for Information Science and Technology, 60 (7), 1320\u20131326. https:\/\/doi.org\/10.1002\/asi.21062","DOI":"10.1002\/asi.21062"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_003","doi-asserted-by":"crossref","unstructured":"Bryan, K.A., Ozcan, Y., & Sampat, B. (2020). In-text patent citations: A user's guide. Research Policy, 49(4), 103946.","DOI":"10.1016\/j.respol.2020.103946"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_004","doi-asserted-by":"crossref","unstructured":"Callaert, J., Pellens, M., & Van Looy, B. (2014). Sources of inspiration? Making sense of scientific references in patents. Scientometrics, 98(3), 1617\u20131629.","DOI":"10.1007\/s11192-013-1073-x"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_005","doi-asserted-by":"crossref","unstructured":"Coffano, M., & Tarasconi, G. (2014). Crios\u2014Patstat database: Sources, contents and access rules. Center for Research on Innovation, Organization and Strategy, Crios Working Paper n. 1. https:\/\/ssrn.com\/abstract=2404344 (accessed: 01\/06\/2020). https:\/\/doi.org\/10.2139\/ssrn.2404344","DOI":"10.2139\/ssrn.2404344"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_006","unstructured":"Derwent. (2007). World Patents Index\u2014Derwent patentee codes, Revised edition 8. Thomson Corporation. Leuven Manual. ISBN: 0 901157 38 4. http:\/\/ips.clarivate.com\/m\/pdfs\/mgr\/patenteecodes.pdf"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_007","unstructured":"European Patent Office. (2019). Data catalog Patstat global. Versi\u00f3n 5.13. EPO Patstat customers. https:\/\/www.epo.org"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_008","doi-asserted-by":"crossref","unstructured":"Gorraiz, J., Gumpenberger, C., & Wieland, M. (2011). \u201cGalton 2011 revisited: A bibliometric journey in the footprints of a universal genius\u201d. Scientometrics, 88(2), 627\u2013652. https:\/\/doi.org\/10.1007\/s11192-011-0393-y","DOI":"10.1007\/s11192-011-0393-y"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_009","doi-asserted-by":"crossref","unstructured":"Guerrero-Bote, V.P., & Moya-Aneg\u00f3n, F. (2015). \u201cAnalysis of scientific production in food science from 2003 to 2013\u201d. Journal of food science, 80(12), R2619\u2013R2626. https:\/\/doi.org\/10.1111\/1750-3841.13108","DOI":"10.1111\/1750-3841.13108"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_010","doi-asserted-by":"crossref","unstructured":"Guerrero-Bote, V.P., S\u00e1nchez-Jim\u00e9nez, R., & Moya-Aneg\u00f3n, F. (2019). \u201cThe citation from patents to scientific output revisited: A new approach to Patstat \/ Scopus matching\u201d. El profesional de la informaci\u00f3n, 28(4), e280401. https:\/\/doi.org\/10.3145\/epi.2019.jul.01","DOI":"10.3145\/epi.2019.jul.01"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_011","unstructured":"Hane, P.J. (2004). \u201cElsevier announces Scopus service\u201d. Information today. http:\/\/newsbreaks.infotoday.com\/nbreader.asp?ArticleID=16494 (accessed: 01\/06\/2020)."},{"key":"2026020217551528528_j_jdis-2021-0028_ref_012","doi-asserted-by":"crossref","unstructured":"Hinze, S., & Schmoch, U. (2004). Opening the black box. In: Moed, H.F., Gl\u00e4nzel, W., and Schmoch, U. (2004) (eds.). Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems. Dordrecht (the Netherlands): Kluwer Academic Publishers, 215\u2013235.","DOI":"10.1007\/1-4020-2755-9_10"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_013","doi-asserted-by":"crossref","unstructured":"Jacs\u00f3, P. (2011). \u201cThe h-index, h-core citation rate and the bibliometric profile of the Scopus database\u201d. Online information review, 35(3), 492\u2013501. https:\/\/doi.org\/10.1108\/14684521111151487","DOI":"10.1108\/14684521111151487"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_014","doi-asserted-by":"crossref","unstructured":"Jefferson, O.A., Jaffe, A., Ashton, D., Warren, B., Koellhofer, D., Dulleck, U., Bilder, G., Ballagh, A., Moe, J., DiCuccio, M., Ward, K., Bilder, G., Dolby, K., & Jefferson, R.A. (2018). \u201cMapping the global influence of published research on industry and innovation\u201d. Nature biotechnology, 36(1), 31\u201339. https:\/\/doi.org\/10.1038\/nbt0818-772a","DOI":"10.1038\/nbt.4049"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_015","doi-asserted-by":"crossref","unstructured":"Kang, B., & Tarasconi, G. (2016). \u201cPatstat revisited: Suggestions for better usage\u201d. World patent information, 46, 56\u201363. https:\/\/doi.org\/10.1016\/j.wpi.2016.06.001","DOI":"10.1016\/j.wpi.2016.06.001"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_016","doi-asserted-by":"crossref","unstructured":"Leydesdorff, L., Moya Aneg\u00f3n, F., & Guerrero-Bote, V.P. (2010). \u201cJournal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI\u201d. Journal of the American Society for Information Science and Technology, 61(2), 352\u2013369. https:\/\/doi.org\/10.1002\/asi.21250","DOI":"10.1002\/asi.21250"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_017","doi-asserted-by":"crossref","unstructured":"Lissoni, F. (2012). \u201cAcademic patenting in Europe: An overview of recent research and new perspectives\u201d. World patent information, 34(3), 197\u2013205. https:\/\/doi.org\/10.1016\/j.wpi.2012.03.002","DOI":"10.1016\/j.wpi.2012.03.002"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_018","doi-asserted-by":"crossref","unstructured":"Lotti, F., & Marin, G. (2013). \u201cMatching of Patstat applications to AIDA firms: Discussion of the methodology and results\u201d. Bank of Italy occasional paper, n. 166. https:\/\/ssrn.com\/abstract=2283111 (accessed: 01\/06\/2020). https:\/\/doi.org\/10.2139\/ssrn.2283111","DOI":"10.2139\/ssrn.2283111"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_019","doi-asserted-by":"crossref","unstructured":"Lundberg, J. (2007). \u201cLifting the crown\u2014citation z-score\u201d. Journal of Informetrics, 1(2), 145\u2013154. https:\/\/doi.org\/10.1016\/j.joi.2006.09.007","DOI":"10.1016\/j.joi.2006.09.007"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_020","unstructured":"Magerman, T., Van-Looy, B., & Song, X.Y. (2006). Data production methods for harmonized patent statistics: Patentee name standardization. Technical report, K.U. Leuven. https:\/\/ec.europa.eu\/eurostat\/documents\/3888793\/5836029\/KS-AV-06-002-EN.PDF (accessed: 01\/06\/2020)."},{"key":"2026020217551528528_j_jdis-2021-0028_ref_021","doi-asserted-by":"crossref","unstructured":"Maraut, S., & Mart\u00ednez, C. (2014). \u201cIdentifying author\u2013inventors from Spain: Methods and a first insight into results\u201d. Scientometrics, 101(1), 445\u2013476. https:\/\/doi.org\/10.1007\/s11192-014-1409-1","DOI":"10.1007\/s11192-014-1409-1"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_022","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez, C. (2011). \u201cPatent families: When do different definitions really matter?\u201d, Scientometrics, v. 86, pp. 39\u201363. https:\/\/doi.org\/10.1007\/s11192-010-0251-3","DOI":"10.1007\/s11192-010-0251-3"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_023","doi-asserted-by":"crossref","unstructured":"Moya-Aneg\u00f3n, F., Chinchilla-Rodr\u00edguez, Z., Vargas-Quesada, B., Corera-\u00c1lvarez, E., Mu\u00f1oz-Fern\u00e1ndez, F.J., Gonz\u00e1lez-Molina, A., & Herrero-Solana, V. (2007). \u201cCoverage analysis of Scopus: A journal metric approach\u201d. Scientometrics, 73(1), 53\u201378. https:\/\/doi.org\/10.1007\/s11192-007-1681-4","DOI":"10.1007\/s11192-007-1681-4"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_024","doi-asserted-by":"crossref","unstructured":"Moya-Aneg\u00f3n, F., Guerrero-Bote, V.P., L\u00f3pez-Illescas, C., & Moed, H.F. (2018). \u201cStatistical relationships between corresponding authorship, international co-authorship and citation impact of national research systems\u201d. Journal of informetrics, 12(4), 1251\u20131262. https:\/\/doi.org\/10.1016\/j.joi.2018.10.004","DOI":"10.1016\/j.joi.2018.10.004"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_025","doi-asserted-by":"crossref","unstructured":"Moya-Aneg\u00f3n, F., L\u00f3pez-Illescas, C., Guerrero-Bote, V.P., & Moed, H.F. (2020). The citation impact of social sciences and humanities upon patentable technology. Scientometrics, to be published.","DOI":"10.1007\/s11192-020-03530-5"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_026","doi-asserted-by":"crossref","unstructured":"Narin, F., & Olivastro, D. (1992). Status report: Linkage between technology and science. Research Policy, 21, 237\u2013249.","DOI":"10.1016\/0048-7333(92)90018-Y"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_027","doi-asserted-by":"crossref","unstructured":"Narin, F., Hamilton, K.S., & Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy, 26, 317\u2013330.","DOI":"10.1016\/S0048-7333(97)00013-9"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_028","unstructured":"Organisation for Economic Co-operation and Development. (2009). OECD Patent Statistics Manual, OECD, Paris. https:\/\/www.oecd.org\/sti\/inno\/oecdpatentstatisticsmanual.htm"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_029","unstructured":"Pickering, B. (2004). \u201cElsevier prepares Scopus to rival ISI Web of science\u201d. Information world review, n. 8."},{"key":"2026020217551528528_j_jdis-2021-0028_ref_030","doi-asserted-by":"crossref","unstructured":"Poege, F., Harhoff, D., Gaessler, F., & Baruffaldi, S. (2019). Science quality and the value of inventions. Science advances, 5(12), easy7323.","DOI":"10.1126\/sciadv.aay7323"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_031","doi-asserted-by":"crossref","unstructured":"Price, D.J.D (1965). Networks of scientific papers. Science, 149, 510\u2013515.","DOI":"10.1126\/science.149.3683.510"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_032","doi-asserted-by":"crossref","unstructured":"Raffo, J.D., & Lhuillery, S. (2009). \u201cHow to play the \u2018Names game\u2019: Patent retrieval comparing different heuristics\u201d. Research policy, 38(10), 1617\u20131627. https:\/\/doi.org\/10.2139\/ssrn.1441172","DOI":"10.1016\/j.respol.2009.08.001"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_033","doi-asserted-by":"crossref","unstructured":"Schmoch, U., & Khan, M. (2019). Methodological challenges for creating patent indicators. In: Glanzel, W., Moed, H.F., Schmoch, U., & Thelwall, M. (eds.), Springer Handbook of Science and Technology Indicators. Cham (Switzerland): Springer Nature, 907\u2013927.","DOI":"10.1007\/978-3-030-02511-3_37"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_034","doi-asserted-by":"crossref","unstructured":"Schoen, A., Heinisch, D., & Buenstorf, G. (2014). \u201cPlaying the \u2018Name game\u2019 to identify academic patents in Germany\u201d. Scientometrics, 101(1), 527\u2013545. https:\/\/doi.org\/10.1007\/s11192-014-1400-x","DOI":"10.1007\/s11192-014-1400-x"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_035","unstructured":"SCImago Research Group. (2010). SIR Methodology: General Considerations. https:\/\/www.scimagoir.com\/methodology.php (accessed: 01\/06\/2020)."},{"key":"2026020217551528528_j_jdis-2021-0028_ref_036","unstructured":"Thoma, G., & Torrisi, S. (2007). Creating powerful indicators for innovation studies with approximate matching algorithms. A test based on Patstat and Amadeus databases (No. 211). KITeS, Centre for Knowledge, Internationalization and Technology Studies, Universit\u00e0 Bocconi, Milano, Italy. http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.573.8107&rep=rep1&type=pdf (accessed: 01\/06\/2020)."},{"key":"2026020217551528528_j_jdis-2021-0028_ref_037","doi-asserted-by":"crossref","unstructured":"Van Raan, A.F.J. (2017). Patent Citations Analysis and Its Value in Research Evaluation: A Review and a New Approach to Map Technology-relevant Research. Journal of Data and Information Science, 2, 13\u201350.","DOI":"10.1515\/jdis-2017-0002"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_038","doi-asserted-by":"crossref","unstructured":"Veugelers, R., & Wang, J. (2019). Scientific novelty and technological impact. Research Policy, 48(6), 1362\u20131372.","DOI":"10.1016\/j.respol.2019.01.019"},{"key":"2026020217551528528_j_jdis-2021-0028_ref_039","doi-asserted-by":"crossref","unstructured":"Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., & van Raan, A.F. (2011). \u201cTowards a new crown indicator: Some theoretical considerations\u201d. Journal of informetrics, 5(1), 37\u201347. https:\/\/doi.org\/10.1016\/j.joi.2010.08.001","DOI":"10.1016\/j.joi.2010.08.001"}],"container-title":["Journal of Data and Information Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2021-0028\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2021-0028\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T17:55:26Z","timestamp":1770054926000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.2478\/jdis-2021-0028\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,24]]},"references-count":39,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,6,24]]},"published-print":{"date-parts":[[2021,11,1]]}},"alternative-id":["10.2478\/jdis-2021-0028"],"URL":"https:\/\/doi.org\/10.2478\/jdis-2021-0028","relation":{},"ISSN":["2543-683X"],"issn-type":[{"value":"2543-683X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,24]]}}}