{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T13:30:43Z","timestamp":1756992643849,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":60,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,5,2]],"date-time":"2019-05-02T00:00:00Z","timestamp":1556755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,5,2]]},"DOI":"10.1145\/3290605.3300685","type":"proceedings-article","created":{"date-parts":[[2019,4,29]],"date-time":"2019-04-29T17:36:58Z","timestamp":1556559418000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Designing for Reproducibility"],"prefix":"10.1145","author":[{"given":"Sebastian S.","family":"Feger","sequence":"first","affiliation":[{"name":"Ludwig Maximilian University of Munich & CERN, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"S\u00fcnje","family":"Dallmeier-Tiessen","sequence":"additional","affiliation":[{"name":"CERN, Geneva, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Albrecht","family":"Schmidt","sequence":"additional","affiliation":[{"name":"Ludwig Maximilian University of Munich, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pawe\u0142 W.","family":"Wo\u017aniak","sequence":"additional","affiliation":[{"name":"Utrecht University, Utrecht, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,5,2]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Artifact Review and Badging. Website. (April","author":"ACM.","year":"2018","unstructured":"ACM. 2018. Artifact Review and Badging. Website. (April 2018 ). https:\/\/www.acm.org\/publications\/policies\/artifact-review-badging Retrieved September 10, 2018. ACM. 2018. Artifact Review and Badging. Website. (April 2018). https:\/\/www.acm.org\/publications\/policies\/artifact-review-badging Retrieved September 10, 2018."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1038\/533452a"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIPRO.2015.7160272"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2011.08.004"},{"key":"e_1_3_2_2_5_1","volume-title":"Ellis","author":"Glenn Begley C.","year":"2012","unstructured":"C. Glenn Begley and Lee M . Ellis . 2012 . Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-- 3. arXiv:arXiv:cond-mat\/9907372v1 C. Glenn Begley and Lee M. Ellis. 2012. Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-- 3. arXiv:arXiv:cond-mat\/9907372v1"},{"key":"e_1_3_2_2_6_1","volume-title":"The Research Object Suite of Ontologies: Sharing and Exchanging Research Data and Methods on the Open Web. arXiv preprint arXiv: 1401.4307","author":"Belhajjame Khalid","year":"2014","unstructured":"Khalid Belhajjame , Jun Zhao , Daniel Garijo , Kristina Hettne , Raul Palma , \u00d3scar Corcho , Jos\u00e9-Manuel G\u00f3mez-P\u00e9rez , Sean Bechhofer , Graham Klyne , and Carole Goble . 2014. The Research Object Suite of Ontologies: Sharing and Exchanging Research Data and Methods on the Open Web. arXiv preprint arXiv: 1401.4307 February 2014 (2014), 20. arXiv:1401.4307 http:\/\/arxiv.org\/abs\/1401.4307 Khalid Belhajjame, Jun Zhao, Daniel Garijo, Kristina Hettne, Raul Palma, \u00d3scar Corcho, Jos\u00e9-Manuel G\u00f3mez-P\u00e9rez, Sean Bechhofer, Graham Klyne, and Carole Goble. 2014. The Research Object Suite of Ontologies: Sharing and Exchanging Research Data and Methods on the Open Web. arXiv preprint arXiv: 1401.4307 February 2014 (2014), 20. arXiv:1401.4307 http:\/\/arxiv.org\/abs\/1401.4307"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2006.29"},{"key":"e_1_3_2_2_8_1","volume-title":"Proceedings of the 4th International WWW Conference","volume":"1","author":"Bentley Richard","year":"1995","unstructured":"Richard Bentley , Thilo Horstmann , Klaas Sikkel , and Jonathan Trevor . 1995 . Supporting collaborative information sharing with the World Wide Web: The BSCW shared workspace system . In Proceedings of the 4th International WWW Conference , Vol. 1 . 63--74. Richard Bentley, Thilo Horstmann, Klaas Sikkel, and Jonathan Trevor. 1995. Supporting collaborative information sharing with the World Wide Web: The BSCW shared workspace system. In Proceedings of the 4th International WWW Conference, Vol. 1. 63--74."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Tim Berners-Lee Robert Cailliau Jean-Francois Groff and Bernd Pollermann. 1992. World-wide web: The information universe. 52--58 pages.  Tim Berners-Lee Robert Cailliau Jean-Francois Groff and Bernd Pollermann. 1992. World-wide web: The information universe. 52--58 pages.","DOI":"10.1108\/eb047254"},{"volume-title":"Qualitative HCI Research: Going Behind the Scenes","author":"Blandford Ann","key":"e_1_3_2_2_10_1","unstructured":"Ann Blandford , Dominic Furniss , and Stephann Makri . 2016. Qualitative HCI Research: Going Behind the Scenes . Morgan & Claypool Publishers , 51--60. Ann Blandford, Dominic Furniss, and Stephann Makri. 2016. Qualitative HCI Research: Going Behind the Scenes. Morgan & Claypool Publishers, 51--60."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994031"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2034863.2034873"},{"volume-title":"Scholarship in the digital age: information, infrastructure, and the internet","author":"Borgman Christine L","key":"e_1_3_2_2_13_1","unstructured":"Christine L Borgman . 2007. Scholarship in the digital age: information, infrastructure, and the internet . MIT Press , Cambridge, MA . Christine L Borgman. 2007. Scholarship in the digital age: information, infrastructure, and the internet. MIT Press, Cambridge, MA."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025738"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/956863.956965"},{"key":"e_1_3_2_2_16_1","volume-title":"The birth of the web. Website. (Dec","author":"CERN.","year":"2013","unstructured":"CERN. 2013. The birth of the web. Website. (Dec 2013 ). http:\/\/cds.cern.ch\/record\/1998446 Retrieved March 15, 2018. CERN. 2013. The birth of the web. Website. (Dec 2013). http:\/\/cds.cern.ch\/record\/1998446 Retrieved March 15, 2018."},{"key":"e_1_3_2_2_17_1","volume-title":"CERN Annual Personnel Statistics","author":"CERN.","year":"2017","unstructured":"CERN. 2017. CERN Annual Personnel Statistics 2017 . (2017). https:\/\/cds.cern.ch\/record\/2317058 Personnel Statistics 2017. CERN. 2017. CERN Annual Personnel Statistics 2017. (2017). https:\/\/cds.cern.ch\/record\/2317058 Personnel Statistics 2017."},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/eScience.2015.68"},{"key":"e_1_3_2_2_19_1","volume-title":"International Conference on Theory and Practice of Digital Libraries. Springer, 347--356","author":"Chen Xiaoli","year":"2016","unstructured":"Xiaoli Chen , S\u00fcnje Dallmeier-Tiessen , Anxhela Dani , Robin Dasler , Javier Delgado Fern\u00e1ndez , Pamfilos Fokianos , Patricia Herterich , and Tibor Simko . 2016 . CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research . In International Conference on Theory and Practice of Digital Libraries. Springer, 347--356 . Xiaoli Chen, S\u00fcnje Dallmeier-Tiessen, Anxhela Dani, Robin Dasler, Javier Delgado Fern\u00e1ndez, Pamfilos Fokianos, Patricia Herterich, and Tibor Simko. 2016. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. In International Conference on Theory and Practice of Digital Libraries. Springer, 347--356."},{"key":"e_1_3_2_2_20_1","first-page":"1564","article-title":"Particle Physicists","volume":"333","author":"Cho Adrian","year":"2011","unstructured":"Adrian Cho . 2011 . Particle Physicists ' New Extreme Teams. Science 333 , 6049 (2011), 1564 -- 1567 . arXiv:http:\/\/science.sciencemag.org\/content\/333\/6049\/1564.full.pdf Adrian Cho. 2011. Particle Physicists' New Extreme Teams. Science 333, 6049 (2011), 1564--1567. arXiv:http:\/\/science.sciencemag.org\/content\/333\/6049\/1564.full.pdf","journal-title":"New Extreme Teams. Science"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691612462588"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"crossref","first-page":"928","DOI":"10.2307\/20159632","article-title":"Tie and network correlates of individual performance in knowledge-intensive work","volume":"47","author":"Cross Rob","year":"2004","unstructured":"Rob Cross and Jonathon N. Cummings . 2004 . Tie and network correlates of individual performance in knowledge-intensive work . Academy of Management Journal 47 , 6 (2004), 928 -- 937 . Rob Cross and Jonathon N. Cummings. 2004. Tie and network correlates of individual performance in knowledge-intensive work. Academy of Management Journal 47, 6 (2004), 928--937.","journal-title":"Academy of Management Journal"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/0306312716659373"},{"key":"e_1_3_2_2_24_1","volume-title":"Proceedings of the Evaluation Methods for Machine Learning Workshop 26th International Conference for Machine Learning 2005","author":"Drummond Chris","year":"2009","unstructured":"Chris Drummond . 2009 . Replicability is not reproducibility: Nor is it good science . Proceedings of the Evaluation Methods for Machine Learning Workshop 26th International Conference for Machine Learning 2005 (2009), 1--4. Chris Drummond. 2009. Replicability is not reproducibility: Nor is it good science. Proceedings of the Evaluation Methods for Machine Learning Workshop 26th International Conference for Machine Learning 2005 (2009), 1--4."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1316624.1316642"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1088\/1748-0221\/3\/08\/S08001"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1159733.1159755"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723872.2723875"},{"volume-title":"The FAIR data principles. Website. Retrieved","year":"2017","key":"e_1_3_2_2_29_1","unstructured":"FORCE11. 2014. The FAIR data principles. Website. Retrieved August 8, 2017 from https:\/\/www.force11.org\/group\/fairgroup\/fairprinciples. FORCE11. 2014. The FAIR data principles. Website. Retrieved August 8, 2017 from https:\/\/www.force11.org\/group\/fairgroup\/fairprinciples."},{"key":"e_1_3_2_2_30_1","volume-title":"CERN Data Centre passes the 200-petabyte milestone. (Jul","author":"Gaillard M\u00e9lissa","year":"2017","unstructured":"M\u00e9lissa Gaillard and Stefania Pandolfi . 2017. CERN Data Centre passes the 200-petabyte milestone. (Jul 2017 ). http:\/\/cds.cern.ch\/record\/ M\u00e9lissa Gaillard and Stefania Pandolfi. 2017. CERN Data Centre passes the 200-petabyte milestone. (Jul 2017). http:\/\/cds.cern.ch\/record\/"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783446.2783605"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-009-0111-1"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1852786.1852790"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3022198.3023264"},{"key":"e_1_3_2_2_35_1","volume-title":"CEUR Workshop Proceedings 976","author":"Greiffenhagen Christian","year":"2013","unstructured":"Christian Greiffenhagen and Stuart Reeves . 2013 . Is Replication important for HCI? CEUR Workshop Proceedings 976 (2013), 8--13. Christian Greiffenhagen and Stuart Reeves. 2013. Is Replication important for HCI? CEUR Workshop Proceedings 976 (2013), 8--13."},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nuclphysa.2006.06.056"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208704"},{"key":"e_1_3_2_2_38_1","volume-title":"Curtis Kennett, Agnieszka Slowik, Carina Sonnleitner, Chelsey Hess-Holden, Timothy M. Errington, Susann Fiedler, and Brian A. Nosek.","author":"Kidwell Mallory C.","year":"2016","unstructured":"Mallory C. Kidwell , Ljiljana B. Lazarevic , Erica Baranski , Tom E. Hardwicke , Sarah Piechowski , Lina Sophia Falkenberg , Curtis Kennett, Agnieszka Slowik, Carina Sonnleitner, Chelsey Hess-Holden, Timothy M. Errington, Susann Fiedler, and Brian A. Nosek. 2016 . Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biology ( 2016). Mallory C. Kidwell, Ljiljana B. Lazarevic, Erica Baranski, Tom E. Hardwicke, Sarah Piechowski, Lina Sophia Falkenberg, Curtis Kennett, Agnieszka Slowik, Carina Sonnleitner, Chelsey Hess-Holden, Timothy M. Errington, Susann Fiedler, and Brian A. Nosek. 2016. Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. PLoS Biology (2016)."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1421412111"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240840"},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Zeeya Merali. 2010. The large human collider: social scientists have embedded themselves at CERN to study the world's biggest research collaboration. Zeeya Merali reports on a 10 000-person physics project. Nature 464 7288 (2010) 482--485.  Zeeya Merali. 2010. The large human collider: social scientists have embedded themselves at CERN to study the world's biggest research collaboration. Zeeya Merali reports on a 10 000-person physics project. Nature 464 7288 (2010) 482--485.","DOI":"10.1038\/464482a"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971485.2971561"},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145376"},{"key":"e_1_3_2_2_44_1","volume-title":"Vision","author":"Piwowar Heather A.","year":"2013","unstructured":"Heather A. Piwowar and Todd J . Vision . 2013 . Data reuse and the open data citation advantage. PeerJ 1 (Oct. 2013), e175. Heather A. Piwowar and Todd J. Vision. 2013. Data reuse and the open data citation advantage. PeerJ 1 (Oct. 2013), e175."},{"key":"e_1_3_2_2_45_1","volume-title":"Believe it or not: how much can we rely on published data on potential drug targets? Nature reviews. Drug discovery 10, 9","author":"Prinz Florian","year":"2011","unstructured":"Florian Prinz , Thomas Schlange , and Khusru Asadullah . 2011. Believe it or not: how much can we rely on published data on potential drug targets? Nature reviews. Drug discovery 10, 9 ( 2011 ), 712. arXiv:arXiv:cond-mat\/9907372v1 Florian Prinz, Thomas Schlange, and Khusru Asadullah. 2011. Believe it or not: how much can we rely on published data on potential drug targets? Nature reviews. Drug discovery 10, 9 (2011), 712. arXiv:arXiv:cond-mat\/9907372v1"},{"key":"e_1_3_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518712"},{"key":"e_1_3_2_2_47_1","volume-title":"An incentive-based approach for improving data reproducibility. Science Translational Medicine 8, 336","author":"Rosenblatt Michael","year":"2016","unstructured":"Michael Rosenblatt . 2016. An incentive-based approach for improving data reproducibility. Science Translational Medicine 8, 336 ( 2016 ), 336ed5--336ed5. arXiv:http:\/\/stm.sciencemag.org\/content\/8\/336\/336ed5.full.pdf Michael Rosenblatt. 2016. An incentive-based approach for improving data reproducibility. Science Translational Medicine 8, 336 (2016), 336ed5--336ed5. arXiv:http:\/\/stm.sciencemag.org\/content\/8\/336\/336ed5.full.pdf"},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1186\/s41073-017-0028-9"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173606"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1038\/496007a"},{"key":"e_1_3_2_2_51_1","volume-title":"Data Sharing Effect on Article Citation Rate in Paleoceanography. AGU Fall Meeting Abstracts (Dec.","author":"Sears Jonathan R. L.","year":"2011","unstructured":"Jonathan R. L. Sears . 2011 . Data Sharing Effect on Article Citation Rate in Paleoceanography. AGU Fall Meeting Abstracts (Dec. 2011). Jonathan R. L. Sears. 2011. Data Sharing Effect on Article Citation Rate in Paleoceanography. AGU Fall Meeting Abstracts (Dec. 2011)."},{"key":"e_1_3_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958868"},{"key":"e_1_3_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1316624.1316681"},{"key":"e_1_3_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.5334\/jors.ay"},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2851581.2892549"},{"key":"e_1_3_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441827"},{"volume-title":"Theories of group behavior","author":"Wegner Daniel M","key":"e_1_3_2_2_57_1","unstructured":"Daniel M Wegner . 1987. Transactive memory: A contemporary analysis of the group mind . In Theories of group behavior . Springer , 185--208. Daniel M Wegner. 1987. Transactive memory: A contemporary analysis of the group mind. In Theories of group behavior. Springer, 185--208."},{"key":"e_1_3_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.18"},{"key":"e_1_3_2_2_59_1","volume-title":"RepliCHI. CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13","author":"Wilson Max L. L.","year":"2013","unstructured":"Max L. L. Wilson , Paul Resnick , David Coyle , and Ed H. Chi . 2013 . RepliCHI. CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13 ( 2013 ), 3159. Max L. L. Wilson, Paul Resnick, David Coyle, and Ed H. Chi. 2013. RepliCHI. CHI '13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA '13 (2013), 3159."},{"key":"e_1_3_2_2_60_1","doi-asserted-by":"crossref","unstructured":"Daniel J Worden. 2017. Emerging Technologies for Data Research: Implications for Bias Curation and Reproducible Results. In Human Capital and Assets in the Networked World.  Daniel J Worden. 2017. Emerging Technologies for Data Research: Implications for Bias Curation and Reproducible Results. In Human Capital and Assets in the Networked World.","DOI":"10.1108\/978-1-78714-827-720171003"}],"event":{"name":"CHI '19: CHI Conference on Human Factors in Computing Systems","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Glasgow Scotland Uk","acronym":"CHI '19"},"container-title":["Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3290605.3300685","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3290605.3300685","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:27Z","timestamp":1750204407000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3290605.3300685"}},"subtitle":["A Qualitative Study of Challenges and Opportunities in High Energy Physics"],"short-title":[],"issued":{"date-parts":[[2019,5,2]]},"references-count":60,"alternative-id":["10.1145\/3290605.3300685","10.1145\/3290605"],"URL":"https:\/\/doi.org\/10.1145\/3290605.3300685","relation":{},"subject":[],"published":{"date-parts":[[2019,5,2]]},"assertion":[{"value":"2019-05-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}