{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T05:18:49Z","timestamp":1776316729066,"version":"3.50.1"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T00:00:00Z","timestamp":1733097600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T00:00:00Z","timestamp":1733097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EPJ Data Sci."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Open science practices have been widely discussed and have been implemented with varying success in different disciplines. We argue that computational-x disciplines such as computational social science, are also susceptible to the symptoms of the crises, but in terms of reproducibility. We expand the binary definition of reproducibility into a tier system which allows increasing levels of reproducibility based on external verifiability to counteract the practice of open-washing. We provide solutions for barriers in Computational Social Science that hinder researchers from obtaining the highest level of reproducibility, including the use of alternate data sources and considering reproducibility proactively.<\/jats:p>","DOI":"10.1140\/epjds\/s13688-024-00514-w","type":"journal-article","created":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T13:27:59Z","timestamp":1733146079000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Computational reproducibility in computational social science"],"prefix":"10.1140","volume":"13","author":[{"given":"David","family":"Schoch","sequence":"first","affiliation":[]},{"given":"Chung-hong","family":"Chan","sequence":"additional","affiliation":[]},{"given":"Claudia","family":"Wagner","sequence":"additional","affiliation":[]},{"given":"Arnim","family":"Bleier","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,12,2]]},"reference":[{"key":"514_CR1","doi-asserted-by":"publisher","unstructured":"Elmer T (2023) Computational social science is growing up: why puberty consists of embracing measurement validation, theory development, and open science practices. EPJ Data Sci 12(1). https:\/\/doi.org\/10.1140\/epjds\/s13688-023-00434-1","DOI":"10.1140\/epjds\/s13688-023-00434-1"},{"key":"514_CR2","unstructured":"Barba LA (2018) Terminologies for reproducible research. arXiv preprint. arXiv:1802.03311"},{"key":"514_CR3","doi-asserted-by":"publisher","unstructured":"The Turing Way Community (2022) the Turing Way: a handbook for reproducible, ethical and collaborative research. https:\/\/doi.org\/10.5281\/zenodo.3233853","DOI":"10.5281\/zenodo.3233853"},{"key":"514_CR4","doi-asserted-by":"publisher","unstructured":"Assenmacher D, Sen I, Fr\u00f6hling L, Wagner C (2023) The End of the Rehydration Era - the Problem of Sharing Harmful Twitter Research Data. ICWSM, https:\/\/doi.org\/10.36190\/2023.56","DOI":"10.36190\/2023.56"},{"key":"514_CR5","doi-asserted-by":"publisher","DOI":"10.4324\/9780429276859","volume-title":"The golden age of data: media analytics in study & practice","author":"D Grady","year":"2019","unstructured":"Grady D (2019) The golden age of data: media analytics in study & practice. Routledge, London"},{"issue":"4","key":"514_CR6","doi-asserted-by":"publisher","first-page":"665","DOI":"10.1080\/10584609.2018.1477506","volume":"35","author":"D Freelon","year":"2018","unstructured":"Freelon D (2018) Computational research in the post-api age. Polit Commun 35(4):665\u2013668. https:\/\/doi.org\/10.1080\/10584609.2018.1477506","journal-title":"Polit Commun"},{"issue":"1","key":"514_CR7","doi-asserted-by":"publisher","DOI":"10.1177\/2056305121988929","volume":"7","author":"R Tromble","year":"2021","unstructured":"Tromble R (2021) Where have all the data gone? A critical reflection on academic digital research in the Post-API age. Soc Media Soc 7(1):2056305121988929. https:\/\/doi.org\/10.1177\/2056305121988929","journal-title":"Soc Media Soc"},{"key":"514_CR8","first-page":"400","volume-title":"Proceedings of the international AAAI conference on web and social media","author":"F Morstatter","year":"2013","unstructured":"Morstatter F, Pfeffer J, Liu H, Carley K (2013) Is the sample good enough? Comparing data from Twitter\u2019s streaming api with Twitter\u2019s firehose. In: Proceedings of the international AAAI conference on web and social media, vol\u00a07, pp\u00a0400\u2013408"},{"key":"514_CR9","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/ps32z","volume-title":"Social media APIs: a quiet threat to the advancement of science","author":"BI Davidson","year":"2023","unstructured":"Davidson BI, Wischerath D, Racek D, Parry DA, Godwin E, Hinds J, Linden D, Roscoe JF, Ayravainen LEM (2023) Social media APIs: a quiet threat to the advancement of science. PsyArXiv. psyarxiv.com\/ps32z. https:\/\/doi.org\/10.31234\/osf.io\/ps32z"},{"key":"514_CR10","doi-asserted-by":"crossref","unstructured":"Yang K-C, Ferrara E, Menczer F (2022) Botometer 101: Social bot practicum for computational social scientists. J\u00a0Comput Soc Sci: 1\u201318","DOI":"10.1007\/s42001-022-00177-5"},{"issue":"10","key":"514_CR11","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0241045","volume":"15","author":"A Rauchfleisch","year":"2020","unstructured":"Rauchfleisch A, Kaiser J (2020) The false positive problem of automatic bot detection in social science research. PLoS ONE 15(10):0241045. https:\/\/doi.org\/10.1371\/journal.pone.0241045","journal-title":"PLoS ONE"},{"key":"514_CR12","doi-asserted-by":"crossref","unstructured":"Chen L, Zaharia M, Zou J (2023) How is ChatGPT\u2019s behavior changing over time? arXiv preprint. arXiv:2307.09009","DOI":"10.1162\/99608f92.5317da47"},{"key":"514_CR13","doi-asserted-by":"publisher","unstructured":"Ohme J, Araujo T, Boeschoten L, Freelon D, Ram N, Reeves BB, Robinson TN (2023) Digital trace data collection for social media effects research: APIs, data donation, and (screen) tracking. Commun Methods Meas: 1\u201318. https:\/\/doi.org\/10.1080\/19312458.2023.2181319","DOI":"10.1080\/19312458.2023.2181319"},{"key":"514_CR14","unstructured":"Chamberlain S, Wolen A, Salmon M, Possenriede D (2023) Vcr: Record \u2018HTTP\u2019 Calls to Disk. R package version 1.2.2. https:\/\/CRAN.R-project.org\/package=vcr"},{"issue":"1","key":"514_CR15","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.cub.2013.11.014","volume":"24","author":"TH Vines","year":"2014","unstructured":"Vines TH, Albert AYK, Andrew RL, D\u00e9barre F, Bock DG, Franklin MT, Gilbert KJ, Moore J-S, Renaut S, Rennison DJ (2014) The availability of research data declines rapidly with article age. Curr Biol 24(1):94\u201397. https:\/\/doi.org\/10.1016\/j.cub.2013.11.014","journal-title":"Curr Biol"},{"key":"514_CR16","doi-asserted-by":"publisher","unstructured":"Tedersoo L, K\u00fcngas R, Oras E, K\u00f6ster K, Eenmaa H, Leijen \u00c4, Pedaste M, Raju M, Astapova A, Lukner H, Kogermann K, Sepp T (2021) Data sharing practices and data availability upon request differ across scientific disciplines. Sci Data 8(1). https:\/\/doi.org\/10.1038\/s41597-021-00981-0","DOI":"10.1038\/s41597-021-00981-0"},{"issue":"4","key":"514_CR17","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.1096\/fj.12-218164","volume":"27","author":"TH Vines","year":"2013","unstructured":"Vines TH, Andrew RL, Bock DG, Franklin MT, Gilbert KJ, Kane NC, Moore J-S, Moyers BT, Renaut S, Rennison DJ, Veen T, Yeaman S (2013) Mandated data archiving greatly improves access to research data. FASEB J 27(4):1304\u20131308. https:\/\/doi.org\/10.1096\/fj.12-218164 arXiv:1301.3744 [physics, q-bio]","journal-title":"FASEB J"},{"issue":"8","key":"514_CR18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0272845","volume":"17","author":"LM Federer","year":"2022","unstructured":"Federer LM (2022) Long-term availability of data associated with articles in PLOS ONE. PLoS ONE 17(8):0272845. https:\/\/doi.org\/10.1371\/journal.pone.0272845","journal-title":"PLoS ONE"},{"key":"514_CR19","unstructured":"Taori R, Gulrajani I, Zhang T, Dubois Y, Li X, Guestrin C, Liang P, Hashimoto TB (2023) Stanford Alpaca: an Instruction-following LLaMA model. GitHub"},{"key":"514_CR20","doi-asserted-by":"publisher","DOI":"10.1145\/3571884.3604316","volume-title":"Proceedings of the 5th international conference on conversational user interfaces","author":"A Liesenfeld","year":"2023","unstructured":"Liesenfeld A, Lopez A, Dingemanse M (2023) Opening up ChatGPT: tracking openness, transparency, and accountability in instruction-tuned text generators. In: Proceedings of the 5th international conference on conversational user interfaces. https:\/\/doi.org\/10.1145\/3571884.3604316"},{"key":"514_CR21","volume-title":"Who is doing computational social science? Trends in big data research","author":"K Metzler","year":"2016","unstructured":"Metzler K, Kim DA, Allum N, Denman A (2016) Who is doing computational social science? Trends in big data research. https:\/\/repository.essex.ac.uk\/17679\/1\/compsocsci.pdf"},{"key":"514_CR22","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.13933","volume":"10","author":"L Cadwallader","year":"2022","unstructured":"Cadwallader L, Hrynaszkiewicz I (2022) A survey of researchers\u2019 code sharing and code reuse practices, and assessment of interactive notebook prototypes. PeerJ 10:13933. https:\/\/doi.org\/10.7717\/peerj.13933","journal-title":"PeerJ"},{"key":"514_CR23","unstructured":"Sutherland ME (2018) Computational social science heralds the age of interdisciplinary science. https:\/\/socialsciences.nature.com\/posts\/54262-computational-social-science-heralds-the-age-of-interdisciplinary-science\/. [Online; accessed 05-May-2023]"},{"key":"514_CR24","doi-asserted-by":"publisher","DOI":"10.1109\/secse.2009.5069155","volume-title":"2009 ICSE workshop on software engineering for computational science and engineering","author":"JE Hannay","year":"2009","unstructured":"Hannay JE, MacLeod C, Singer J, Langtangen HP, Pfahl D, Wilson G (2009) How do scientists develop and use scientific software? In: 2009 ICSE workshop on software engineering for computational science and engineering. https:\/\/doi.org\/10.1109\/secse.2009.5069155"},{"issue":"1","key":"514_CR25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1001745","volume":"12","author":"G Wilson","year":"2014","unstructured":"Wilson G, Aruliah DA, Brown CT, Hong NPC, Davis M, Guy RT, Haddock SHD, Huff KD, Mitchell IM, Plumbley MD, Waugh B, White EP, Wilson P (2014) Best practices for scientific computing. PLoS Biol 12(1):1001745. https:\/\/doi.org\/10.1371\/journal.pbio.1001745","journal-title":"PLoS Biol"},{"issue":"6","key":"514_CR26","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0286761","volume":"18","author":"C-h Chan","year":"2023","unstructured":"Chan C-h, Schoch D (2023) rang: reconstructing reproducible r computational environments. PLoS ONE 18(6):0286761. https:\/\/doi.org\/10.1371\/journal.pone.0286761","journal-title":"PLoS ONE"},{"issue":"30","key":"514_CR27","doi-asserted-by":"publisher","first-page":"774","DOI":"10.21105\/joss.00774","volume":"3","author":"K Benoit","year":"2018","unstructured":"Benoit K, Watanabe K, Wang H, Nulty P, Obeng A, M\u00fcller S, Matsuo A (2018) quanteda: an R package for the quantitative analysis of textual data. J Open Sour Softw 3(30):774. https:\/\/doi.org\/10.21105\/joss.00774","journal-title":"J Open Sour Softw"},{"key":"514_CR28","doi-asserted-by":"publisher","unstructured":"Trisovic A, Lau MK, Pasquier T, Crosas M (2022) A large-scale study on research code quality and execution. Sci Data 9(1). https:\/\/doi.org\/10.1038\/s41597-022-01143-6","DOI":"10.1038\/s41597-022-01143-6"},{"key":"514_CR29","doi-asserted-by":"crossref","unstructured":"Wilson G (2006) Software carpentry: Getting scientists to write better code by making them more productive. Comput Sci Eng Summarizes the what and why of Version 3 of the course","DOI":"10.1109\/MCSE.2006.122"},{"issue":"2","key":"514_CR30","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1093\/comjnl\/27.2.97","volume":"27","author":"DE Knuth","year":"1984","unstructured":"Knuth DE (1984) Literate programming. Comput J 27(2):97\u2013111. https:\/\/doi.org\/10.1093\/comjnl\/27.2.97https:\/\/academic.oup.com\/comjnl\/article-pdf\/27\/2\/97\/981657\/270097.pdf","journal-title":"Comput J"},{"key":"514_CR31","doi-asserted-by":"publisher","unstructured":"Schulte E, Davison D, Dye T, Dominik C (2012) A multi-language computing environment for literate programming and reproducible research. J Stat Softw 46(3). https:\/\/doi.org\/10.18637\/jss.v046.i03","DOI":"10.18637\/jss.v046.i03"},{"key":"514_CR32","doi-asserted-by":"publisher","unstructured":"Weingart SB, Burton M, Lavin MJ, Otis J (2020) Digits: Two Reports on New Units of Scholarly Publication. J Electron Publ 22(1). https:\/\/doi.org\/10.3998\/3336451.0022.105","DOI":"10.3998\/3336451.0022.105"},{"key":"514_CR33","first-page":"113","volume-title":"Proceedings of the 17th Python in science conference","author":"B Ragan-Kelley","year":"2018","unstructured":"Ragan-Kelley B, Willing C, Akici F, Lippa D, Niederhut D, Pacer M (2018) Binder 2.0-reproducible, interactive, sharable environments for science at scale. In: Akici F, Lippa D, Niederhut D, Pacer M (eds) Proceedings of the 17th Python in science conference, pp\u00a0113\u2013120"},{"key":"514_CR34","unstructured":"Arenas D, Atkins J, Austin C, Beavan D, Egea AC, Carlysle-Davies S, Carter I, Clarke R, Cunningham J, Doel T, et\u00a0al (2019) Design choices for productive, secure, data-intensive research at scale in the cloud. arXiv preprint. arXiv:1908.08737"},{"key":"514_CR35","doi-asserted-by":"crossref","unstructured":"Recker A, M\u00fcller S, Trixa J, Schumann N (2015) Paving the way for data-centric, open science: an example from the social sciences. J Librariansh Sch Commun 3(2)","DOI":"10.7710\/2162-3309.1227"}],"container-title":["EPJ Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1140\/epjds\/s13688-024-00514-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1140\/epjds\/s13688-024-00514-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1140\/epjds\/s13688-024-00514-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,2]],"date-time":"2024-12-02T14:05:19Z","timestamp":1733148319000},"score":1,"resource":{"primary":{"URL":"https:\/\/epjdatascience.springeropen.com\/articles\/10.1140\/epjds\/s13688-024-00514-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,2]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["514"],"URL":"https:\/\/doi.org\/10.1140\/epjds\/s13688-024-00514-w","relation":{},"ISSN":["2193-1127"],"issn-type":[{"value":"2193-1127","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,2]]},"assertion":[{"value":"4 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 November 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"75"}}