{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T17:37:45Z","timestamp":1772300265205,"version":"3.50.1"},"reference-count":56,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,8,5]],"date-time":"2020-08-05T00:00:00Z","timestamp":1596585600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,8,5]],"date-time":"2020-08-05T00:00:00Z","timestamp":1596585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2020,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We discuss the trend towards using quantitative metrics for evaluating research. We claim that, rather than promoting meaningful research, purely metric-based research evaluation schemes potentially lead to a dystopian academic reality, leaving no space for creativity and intellectual initiative. After sketching what the future could look like if quantitative metrics are allowed to proliferate, we provide a more detailed discussion on why research is so difficult to evaluate and outline approaches for avoiding such a situation. In particular, we characterize meaningful research as an essentially contested concept and argue that quantitative metrics should always be accompanied by operationalized instructions for their proper use and continuously evaluated via feedback loops. Additionally, we analyze a dataset containing information about computer science publications and their citation history and indicate how quantitative metrics could potentially be calibrated via alternative evaluation methods such as test of time awards. Finally, we argue that, instead of over-relying on indicators, research environments should primarily be based on trust and personal responsibility.<\/jats:p>","DOI":"10.1007\/s11192-020-03649-5","type":"journal-article","created":{"date-parts":[[2020,8,5]],"date-time":"2020-08-05T13:02:32Z","timestamp":1596632552000},"page":"153-169","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["What is meaningful research and how should we measure it?"],"prefix":"10.1007","volume":"125","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9666-1932","authenticated-orcid":false,"given":"Sven","family":"Helmer","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8651-750X","authenticated-orcid":false,"given":"David B.","family":"Blumenthal","sequence":"additional","affiliation":[]},{"given":"Kathrin","family":"Paschen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,8,5]]},"reference":[{"key":"3649_CR1","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1093\/reseval\/rvv003","volume":"24","author":"K Aagaard","year":"2015","unstructured":"Aagaard, K., Bloch, C., & Schneider, J. W. (2015). Impacts of performance-based research funding systems: The case of the Norwegian publication indicator. Research Evaluation, 24, 106\u2013117.","journal-title":"Research Evaluation"},{"issue":"1","key":"3649_CR2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/2158244019829575","volume":"9","author":"DW Aksnes","year":"2019","unstructured":"Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019). Citations, citation indicators, and research quality: An overview of basic concepts and theories. SAGE Open, 9(1), 1\u201317.","journal-title":"SAGE Open"},{"issue":"6","key":"3649_CR3","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1016\/j.respol.2009.02.001","volume":"38","author":"DW Aksnes","year":"2009","unstructured":"Aksnes, D. W., & Rip, A. (2009). Researchers perceptions of citations. Research Policy, 38(6), 895\u2013905.","journal-title":"Research Policy"},{"key":"3649_CR4","unstructured":"Armstrong, J. (2012). A question universities need to answer: Why do we research?https:\/\/theconversation.com\/a-question-universities-need-to-answer-why-do-we-research-6230."},{"key":"3649_CR5","volume-title":"Measuring and managing performance in organizations","author":"RD Austin","year":"1996","unstructured":"Austin, R. D. (1996). Measuring and managing performance in organizations. New York: Dorset House Publishing."},{"issue":"8","key":"3649_CR6","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1080\/03075070903348404","volume":"35","author":"P Bazeley","year":"2010","unstructured":"Bazeley, P. (2010). Conceptualising research performance. Studies in Higher Education, 35(8), 889\u2013903.","journal-title":"Studies in Higher Education"},{"issue":"7415","key":"3649_CR7","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1038\/489179a","volume":"489","author":"J Beall","year":"2012","unstructured":"Beall, J. (2012). Predatory publishers are corrupting open access. Nature News, 489(7415), 179.","journal-title":"Nature News"},{"issue":"4","key":"3649_CR8","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1017\/S1049096515000827","volume":"48","author":"M Breuning","year":"2015","unstructured":"Breuning, M., Backstrom, J., Brannon, J., Gross, B. I., & Widmeier, M. (2015). Reviewer fatigue? Why scholars decline to review their peers\u2019 work. Political Science and Politics, 48(4), 595\u2013600.","journal-title":"Political Science and Politics"},{"key":"3649_CR9","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1007\/s11192-020-03348-1","volume":"123","author":"ES Brezis","year":"2020","unstructured":"Brezis, E. S., & Birukou, A. (2020). Arbitrariness in the peer review process. Scientometrics, 123, 393\u2013411.","journal-title":"Scientometrics"},{"issue":"4487","key":"3649_CR10","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1126\/science.7008199","volume":"211","author":"WJ Broad","year":"1981","unstructured":"Broad, W. J. (1981). The publishing game: Getting more for less. Science, 211(4487), 1137\u20131139.","journal-title":"Science"},{"issue":"3","key":"3649_CR11","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1080\/13569310600923782","volume":"11","author":"D Collier","year":"2006","unstructured":"Collier, D., Daniel Hidalgo, F., & Olivia Maciuceanu, A. (2006). Essentially contested concepts: Debates and applications. Journal of Political Ideologies, 11(3), 211\u2013246.","journal-title":"Journal of Political Ideologies"},{"key":"3649_CR12","unstructured":"Criley, M.\u00a0E. (2007). Contested concepts and competing conceptions. Ph.D. thesis, University of Pittsburgh."},{"key":"3649_CR13","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1038\/nj7647-747a","volume":"543","author":"A Dance","year":"2017","unstructured":"Dance, A. (2017). Flexible working: Solo scientist. Nature, 543, 747\u2013749.","journal-title":"Nature"},{"key":"3649_CR14","unstructured":"DORA. (2012). San Francisco declaration on research assessment. https:\/\/sfdora.org\/."},{"key":"3649_CR15","first-page":"27","volume":"18","author":"RM Dworkin","year":"1972","unstructured":"Dworkin, R. M. (1972). The jurisprudence of Richard Nixon. The New York Review of Books, 18, 27\u201335.","journal-title":"The New York Review of Books"},{"key":"3649_CR16","volume-title":"Taking rights seriously: New impression with a reply to critics","author":"RM Dworkin","year":"1978","unstructured":"Dworkin, R. M. (1978). Taking rights seriously: New impression with a reply to critics. Oxford: Duckworth."},{"issue":"5","key":"3649_CR17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/scipol\/scy007","volume":"45","author":"F Ferretti","year":"2018","unstructured":"Ferretti, F., Pereira, \u00c2. G., V\u00e9rtesy, D., & Hardeman, S. (2018). Research excellence indicators: Time to reimagine the \u2018making of\u2019? Science and Public Policy, 45(5), 1\u201311.","journal-title":"Science and Public Policy"},{"key":"3649_CR18","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s41073-017-0027-x","volume":"2","author":"CW Fox","year":"2017","unstructured":"Fox, C. W., Albert, A. Y. K., & Vines, T. H. (2017). Recruitment of reviewers is becoming harder at some journals: A test of the influence of reviewer fatigue at six journals in ecology and evolution. Research Integrity and Peer Review, 2, 3.","journal-title":"Research Integrity and Peer Review"},{"key":"3649_CR19","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1093\/aristotelian\/56.1.167","volume":"56","author":"WB Gallie","year":"1955","unstructured":"Gallie, W. B. (1955). Essentially contested concepts. Proceedings of the Aristotelian Society, 56, 167\u2013198.","journal-title":"Proceedings of the Aristotelian Society"},{"key":"3649_CR20","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1038\/nj7396-137a","volume":"485","author":"V Gewin","year":"2012","unstructured":"Gewin, V. (2012). Research: Uncovering misconduct. Nature, 485, 137\u2013139.","journal-title":"Nature"},{"key":"3649_CR21","series-title":"Wenner\u2013Gren international series","first-page":"29","volume-title":"Bibliometrics use and abuse in the review of research performance","author":"J Grimson","year":"2014","unstructured":"Grimson, J. (2014). Measuring research impact: Not everything that can be counted counts, and not everything that counts can be counted. In W. Blockmans, L. Engwall, & D. Weaire (Eds.), Bibliometrics use and abuse in the review of research performance (Vol. 87, pp. 29\u201341)., Wenner\u2013Gren international series London: Portland Press."},{"issue":"3","key":"3649_CR22","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1093\/reseval\/rvx018","volume":"26","author":"B Hammarfelt","year":"2017","unstructured":"Hammarfelt, B., & Rushforth, A. D. (2017). Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation. Research Evaluation, 26(3), 169\u2013180.","journal-title":"Research Evaluation"},{"issue":"2","key":"3649_CR23","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1177\/1356389011400891","volume":"17","author":"T Hellstr\u00f6m","year":"2011","unstructured":"Hellstr\u00f6m, T. (2011). Homing in on excellence: Dimensions of appraisal in center of excellence program evaluations. Evaluation, 17(2), 117\u2013131.","journal-title":"Evaluation"},{"key":"3649_CR24","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1038\/520429a","volume":"520","author":"D Hicks","year":"2015","unstructured":"Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden manifesto for research metrics. Nature, 520, 429\u2013431.","journal-title":"Nature"},{"issue":"5","key":"3649_CR25","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1093\/reseval\/rvt008","volume":"22","author":"SE Hug","year":"2013","unstructured":"Hug, S. E., Ochsner, M., & Daniel, H.-D. (2013). Criteria for assessing research quality in the humanities: A Delphi study among scholars of English literature, German literature and art history. Research Evaluation, 22(5), 369\u2013383.","journal-title":"Research Evaluation"},{"issue":"6162","key":"3649_CR26","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1126\/science.342.6162.1035","volume":"342","author":"M Hvistendahl","year":"2013","unstructured":"Hvistendahl, M. (2013). China\u2019s publication bazaar. Science, 342(6162), 1035\u20131039.","journal-title":"Science"},{"key":"3649_CR27","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1038\/d41586-018-06185-8","volume":"561","author":"JPA Ioannidis","year":"2018","unstructured":"Ioannidis, J. P. A., Klavans, R., & Boyack, K. W. (2018). Thousands of scientists publish a paper every five days. Nature, 561, 167\u2013169.","journal-title":"Nature"},{"key":"3649_CR28","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1080\/13600800903191930","volume":"31","author":"SD Jarwal","year":"2009","unstructured":"Jarwal, S. D., Brion, A. M., & King, M. L. (2009). Measuring research quality using the journal impact factor, citations and \u2019ranked journals\u2019: Blunt instruments or inspired metrics? Journal of Higher Education Policy and Management, 31, 289\u2013300.","journal-title":"Journal of Higher Education Policy and Management"},{"key":"3649_CR29","doi-asserted-by":"crossref","first-page":"1441","DOI":"10.1016\/j.cell.2019.11.029","volume":"179","author":"M Krummel","year":"2019","unstructured":"Krummel, M., Blish, C., Kuhns, M., Cadwell, K., Oberst, A., Goldrath, A., et al. (2019). Universal principled review: A community-driven method to improve peer review. Cell, 179, 1441\u20131445.","journal-title":"Cell"},{"issue":"3","key":"3649_CR30","first-page":"429","volume":"55","author":"M Kucher","year":"1998","unstructured":"Kucher, M., & G\u00f6tte, L. (1998). Trust me\u2014An empirical analysis of taxpayer honesty. Finanzarchiv, 55(3), 429\u2013444.","journal-title":"Finanzarchiv"},{"issue":"2","key":"3649_CR31","doi-asserted-by":"crossref","first-page":"1493","DOI":"10.14778\/1687553.1687577","volume":"2","author":"M Ley","year":"2009","unstructured":"Ley, M. (2009). DBLP: Some lessons learned. Proceedings of the VLDB Endowment, 2(2), 1493\u20131500.","journal-title":"Proceedings of the VLDB Endowment"},{"key":"3649_CR32","doi-asserted-by":"crossref","first-page":"2129","DOI":"10.1007\/s11192-016-2150-8","volume":"109","author":"L Leydesdorff","year":"2016","unstructured":"Leydesdorff, L., Wouters, P., & Bornmann, L. (2016). Professional and citizen bibliometrics: Complementarities and ambivalences in the development and use of indicators\u2013a state-of-the-art report. Scientometrics, 109, 2129\u20132150.","journal-title":"Scientometrics"},{"key":"3649_CR33","volume-title":"Trust and power","author":"N Luhmann","year":"2017","unstructured":"Luhmann, N. (2017). Trust and power. Cambridge: Polity."},{"issue":"3","key":"3649_CR34","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.respol.2015.11.009","volume":"45","author":"P M\u00e5rtensson","year":"2016","unstructured":"M\u00e5rtensson, P., Fors, U., Wallin, S.-B., Zander, U., & Nilsson, G. H. (2016). Evaluating research: A multidisciplinary approach to assessing research practice and quality. Research Policy, 45(3), 593\u2013603.","journal-title":"Research Policy"},{"key":"3649_CR35","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s11192-013-1015-7","volume":"98","author":"C Michels","year":"2014","unstructured":"Michels, C., & Schmoch, U. (2014). Impact of bibliometric studies on the publication behaviour of authors. Scientometrics, 98, 369\u2013385.","journal-title":"Scientometrics"},{"key":"3649_CR36","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1038\/478026a","volume":"478","author":"RV Noorden","year":"2011","unstructured":"Noorden, R. V. (2011). Science publishing: The trouble with retractions. Nature, 478, 26\u201328.","journal-title":"Nature"},{"key":"3649_CR37","doi-asserted-by":"crossref","first-page":"23","DOI":"10.21832\/9781783099245-007","volume-title":"Global academic publishing: Policies, perspectives and pedagogies","author":"LP Nygaard","year":"2017","unstructured":"Nygaard, L. P., & Bellanova, R. (2017). Lost in quantification: Scholars and the politics of bibliometrics. In M. J. Curry & T. Lillis (Eds.), Global academic publishing: Policies, perspectives and pedagogies (pp. 23\u201336). Bristol: Multilingual Matters."},{"key":"3649_CR38","volume-title":"Weapons of math destruction: How big data increases inequality and threatens democracy","author":"C O\u2019Neil","year":"2016","unstructured":"O\u2019Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Crown."},{"issue":"11","key":"3649_CR39","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1145\/1297797.1297815","volume":"50","author":"DL Parnas","year":"2007","unstructured":"Parnas, D. L. (2007). Stop the numbers game. Communications of the ACM, 50(11), 19\u201321.","journal-title":"Communications of the ACM"},{"key":"3649_CR40","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1002\/asi.22746","volume":"64","author":"FN Piro","year":"2013","unstructured":"Piro, F. N., Aksnes, D. W., & R\u00f8rstad, K. (2013). A macro analysis of productivity differences across fields: Challenges in the measurement of scientific publishing. Journal of the American Society for Information Science and Technology, 64, 307\u2013320.","journal-title":"Journal of the American Society for Information Science and Technology"},{"key":"3649_CR41","volume-title":"Managerialism and the public services: Cuts or cultural change in the 1990s?","author":"C Pollitt","year":"1993","unstructured":"Pollitt, C. (1993). Managerialism and the public services: Cuts or cultural change in the 1990s?. Oxford: Blackwell."},{"issue":"1","key":"3649_CR42","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1146\/annurev-orgpsych-012218-015009","volume":"6","author":"ED Pulakos","year":"2019","unstructured":"Pulakos, E. D., Mueller-Hanson, R., & Arad, S. (2019). The evolution of performance management: Searching for value. Annual Review of Organizational Psychology and Organizational Behavior, 6(1), 249\u2013271.","journal-title":"Annual Review of Organizational Psychology and Organizational Behavior"},{"key":"3649_CR43","unstructured":"Rafols, I., Ciarli, T., van Zwanenberg, P., & Stirling, A. (2012). Towards indicators for \u2018opening up\u2019 science and technology policy. In The internet, policy and politics conference 2012. Oxford, UK."},{"key":"3649_CR44","unstructured":"Royal Society. (2017). Research culture embedding inclusive excellence. https:\/\/royalsociety.org\/-\/media\/policy\/Publications\/2018\/research-culture-workshop-report.pdf."},{"key":"3649_CR45","first-page":"57","volume-title":"Coercive accountability: The rise of audit culture in higher education","author":"C Shore","year":"2000","unstructured":"Shore, C., & Wright, S. (2000). Coercive accountability: The rise of audit culture in higher education (pp. 57\u201389). London: Routledge."},{"key":"3649_CR46","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1080\/19409052.2014.905968","volume":"6","author":"G Singh","year":"2014","unstructured":"Singh, G. (2014). Recognition and the image of mastery as themes in black mirror (channel 4, 2011-present): An eco-jungian approach to \u2019always on\u2019 culture. International Journal of Jungian Studies, 6, 120\u2013132.","journal-title":"International Journal of Jungian Studies"},{"issue":"7","key":"3649_CR47","first-page":"e68397:1","volume":"8","author":"R\u00a0G Steen","year":"2013","unstructured":"Steen, R\u00a0. G., Casadevall, A., & Fang, F\u00a0. C. (2013). Why has the number of scientific retractions increased? PLoS ONE, 8(7), e68397:1\u20139.","journal-title":"PLoS ONE"},{"key":"3649_CR48","unstructured":"Stilgoe, J. (2014). Against excellence. https:\/\/www.theguardian.com\/science\/political-science\/2014\/dec\/19\/against-excellence."},{"key":"3649_CR49","unstructured":"Swain, H. (2011). What are universities for?https:\/\/www.theguardian.com\/education\/2011\/oct\/10\/higher-education-purpose."},{"key":"3649_CR50","doi-asserted-by":"crossref","unstructured":"Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008). ArnetMiner: Extraction and mining of academic social networks. In Proceedings of the 14th ACM international conferecne on knowledge discovery and data mining (SIGKDD\u201908), Las Vegas, Nevada, (pp. 990\u2013998).","DOI":"10.1145\/1401890.1402008"},{"issue":"19","key":"3649_CR51","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1093\/femsle\/fny204","volume":"365","author":"JP Tennant","year":"2018","unstructured":"Tennant, J. P. (2018). The state of the art in peer review. FEMS Microbiology Letters, 365(19), 204.","journal-title":"FEMS Microbiology Letters"},{"key":"3649_CR52","doi-asserted-by":"crossref","unstructured":"Troullinou, P., d\u2019Aquin, M., & Tiddi, I. (2018). Re-coding black mirror chairs\u2019 welcome & organization. Companion of the the web conference WWW\u201918 (pp. 1527\u20131528). France: Lyon.","DOI":"10.1145\/3184558.3192319"},{"issue":"1","key":"3649_CR53","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1162\/qss_a_00011","volume":"1","author":"Q Wang","year":"2020","unstructured":"Wang, Q., & Schneider, J. W. (2020). Consistency and validity of interdisciplinary measures. Quantitative Science Studies, 1(1), 239\u2013263.","journal-title":"Quantitative Science Studies"},{"issue":"1","key":"3649_CR54","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s11192-005-0007-7","volume":"62","author":"P Weingart","year":"2005","unstructured":"Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117\u2013131.","journal-title":"Scientometrics"},{"key":"3649_CR55","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198767268.001.0001","volume-title":"A university education","author":"D Willetts","year":"2017","unstructured":"Willetts, D. (2017). A university education. New York: Oxford University Press."},{"key":"3649_CR56","doi-asserted-by":"crossref","unstructured":"Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Stephen\u00a0Curry, S.\u00a0H., Jones, R., Kain, R., Kerridge, S., Thelwall, M., Jane\u00a0Tinkler, I.\u00a0V., Wouters, P., Hill, J. & Johnson, B. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. HEFCE.","DOI":"10.4135\/9781473978782"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03649-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-020-03649-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03649-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,4]],"date-time":"2021-08-04T23:27:59Z","timestamp":1628119679000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-020-03649-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,5]]},"references-count":56,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,10]]}},"alternative-id":["3649"],"URL":"https:\/\/doi.org\/10.1007\/s11192-020-03649-5","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"value":"0138-9130","type":"print"},{"value":"1588-2861","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,5]]},"assertion":[{"value":"14 November 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 August 2020","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}