{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:16:37Z","timestamp":1750306597053,"version":"3.41.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2015,1,20]],"date-time":"2015-01-20T00:00:00Z","timestamp":1421712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGOPS Oper. Syst. Rev."],"published-print":{"date-parts":[[2015,1,20]]},"abstract":"<jats:p>Compared to more traditional disciplines, such as the natural sciences, computer science is said to have a somewhat sloppy relationship with the external repeatability of published results. However, from our experience the problem starts even earlier: In many cases, authors are not even able to replicate their own results a year later, or to explain how exactly that number on page three of the paper was ncomputed. Because of constant time pressure and strict submission deadlines, the successful researcher has to favor timely results over experiment documentation and data traceability.<\/jats:p>\n          <jats:p>We consider internal repeatability to be one of the most important prerequisites for external replicability and the scientific process. We describe our approach to foster internal repeatability in our own research projects with the help of dedicated tools for the automation of traceable experimental setups and for data presentation in scientific papers. By employing these tools, measures for ensuring internal repeatability no longer waste valuable working time and pay off quickly: They save time by eliminating recurring, and therefore error-prone, manual work steps, and at the same time increase confidence in experimental results.<\/jats:p>","DOI":"10.1145\/2723872.2723880","type":"journal-article","created":{"date-parts":[[2015,1,28]],"date-time":"2015-01-28T14:05:51Z","timestamp":1422453951000},"page":"51-60","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["The dataref versuchung"],"prefix":"10.1145","volume":"49","author":[{"given":"Christian","family":"Dietrich","sequence":"first","affiliation":[{"name":"Friedrich-Alexander University (FAU), Erlangen-Nuremberg, Germany"}]},{"given":"Daniel","family":"Lohmann","sequence":"additional","affiliation":[{"name":"Friedrich-Alexander University (FAU), Erlangen-Nuremberg, Germany"}]}],"member":"320","published-online":{"date-parts":[[2015,1,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2013.6575308"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/502034.502042"},{"volume-title":"The University of Arizona","year":"2014","author":"Collberg Christian","key":"e_1_2_1_3_1"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2362536.2362544"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2162024.2162030"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723872.2723875"},{"key":"e_1_2_1_7_1","unstructured":"Christian Feuers\u00e4nger. The Pgfplots package. Create normal\/ logarithmic plots in two and three dimensions. Version 1.11. URL: http:\/\/www.ctan.org\/pkg\/pgfplots (visited on 09\/12\/2014).  Christian Feuers\u00e4nger. The Pgfplots package. Create normal\/ logarithmic plots in two and three dimensions. Version 1.11. URL: http:\/\/www.ctan.org\/pkg\/pgfplots (visited on 09\/12\/2014)."},{"key":"e_1_2_1_8_1","unstructured":"Martin Fowler. Continuous Integration. 2006. URL: http:\/\/martinfowler.com\/articles\/continuousIntegration.html.  Martin Fowler. Continuous Integration. 2006. URL: http:\/\/martinfowler.com\/articles\/continuousIntegration.html."},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Juliana Freire David Koop Fernando Seabra Chirigati and Cl\u00e1udio T Silva. \"Reproducibility using VisTrails\". In: Implementing Reproducible Research (2014) p. 33. 10See https:\/\/github.com\/stettberger\/versuchung and http:\/\/www.ctan.org\/pkg\/dataref for details.  Juliana Freire David Koop Fernando Seabra Chirigati and Cl\u00e1udio T Silva. \"Reproducibility using VisTrails\". In: Implementing Reproducible Research (2014) p. 33. 10See https:\/\/github.com\/stettberger\/versuchung and http:\/\/www.ctan.org\/pkg\/dataref for details.","DOI":"10.1201\/9781315373461-2"},{"key":"e_1_2_1_10_1","first-page":"637","volume-title":"Proceedings of the International Conference on Computational Science, ICCS 2011","author":"Gavish Matan","year":"1877"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2012.36"},{"volume-title":"2nd Int. W'shop on Softw.-Based Methods for Robust Emb. Sys. (SOBRES '13). LNCS.","year":"2013","author":"Hoffmann Martin","key":"e_1_2_1_12_1"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC.2013.22"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/HASE.2014.14"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISORC.2014.26"},{"key":"e_1_2_1_16_1","article-title":"Experiences with Software-based Soft-Error Mitigation using AN-Codes","author":"Hoffmann Martin","year":"2015","journal-title":"Software Quality Journal ("},{"key":"e_1_2_1_17_1","unstructured":"Shriram Krishnamurthi. Examining \"Reproducibility in Computer Science\". {Accessed: 5. November 2014}. 2014. URL: http:\/\/cs.brown.edu\/~sk\/Memos\/Examining-Reproducibility\/.  Shriram Krishnamurthi. Examining \"Reproducibility in Computer Science\". {Accessed: 5. November 2014}. 2014. URL: http:\/\/cs.brown.edu\/~sk\/Memos\/Examining-Reproducibility\/."},{"key":"e_1_2_1_18_1","first-page":"575","volume-title":"Physica Verlag","author":"Leisch Friedrich","year":"2002"},{"key":"e_1_2_1_19_1","first-page":"111","volume-title":"USA: IEEE","author":"Nadi Sarah","year":"2013"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1950365.1950401"},{"key":"e_1_2_1_21_1","unstructured":"Kaspar Rufibach. reporttools: Generate LaTeX tables of descriptive statistics. Loads rounds formats and postprocesses numerical tables. Version 1.1.1. URL: http:\/\/cran.rproject.org\/web\/packages\/reporttools\/ (visited on 09\/12\/2014).  Kaspar Rufibach. reporttools: Generate LaTeX tables of descriptive statistics. Loads rounds formats and postprocesses numerical tables. Version 1.1.1. URL: http:\/\/cran.rproject.org\/web\/packages\/reporttools\/ (visited on 09\/12\/2014)."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2658761.2658767"},{"volume":"200","volume-title":"25th Int. Conf. on Architecture of Computing Systems (ARCS '12), Workshop Proceedings.","author":"Schirmeier Horst","key":"e_1_2_1_23_1"},{"key":"e_1_2_1_24_1","first-page":"217","volume-title":"SEP-89: Stanford Exploration Project","author":"Schwab Matthias","year":"1995"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868294.1868300"},{"key":"e_1_2_1_26_1","unstructured":"Nicola Talbot. The Datatool package. Tools to load and manipulate data. Version 2.22. URL: http:\/\/www.ctan.org\/pkg\/datatool (visited on 09\/12\/2014).  Nicola Talbot. The Datatool package. Tools to load and manipulate data. Version 2.22. URL: http:\/\/www.ctan.org\/pkg\/datatool (visited on 09\/12\/2014)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-012-0225-2"},{"key":"e_1_2_1_28_1","first-page":"421","volume-title":"USA: USENIX","author":"Tartler Reinhard","year":"2014"},{"key":"e_1_2_1_29_1","first-page":"w4","author":"Variability VAMOS","year":"2012","journal-title":"FAU Erlangen-Nuremberg"}],"container-title":["ACM SIGOPS Operating Systems Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2723872.2723880","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2723872.2723880","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:16:43Z","timestamp":1750227403000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2723872.2723880"}},"subtitle":["Saving Time through Better Internal Repeatability"],"short-title":[],"issued":{"date-parts":[[2015,1,20]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2015,1,20]]}},"alternative-id":["10.1145\/2723872.2723880"],"URL":"https:\/\/doi.org\/10.1145\/2723872.2723880","relation":{},"ISSN":["0163-5980"],"issn-type":[{"type":"print","value":"0163-5980"}],"subject":[],"published":{"date-parts":[[2015,1,20]]},"assertion":[{"value":"2015-01-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}