{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T16:07:04Z","timestamp":1753891624253,"version":"3.41.2"},"reference-count":32,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,7,22]],"date-time":"2024-07-22T00:00:00Z","timestamp":1721606400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000272","name":"National Institute for Health and Care Research","doi-asserted-by":"publisher","award":["1R24MH117295","2P41EB019936"],"award-info":[{"award-number":["1R24MH117295","2P41EB019936"]}],"id":[{"id":"10.13039\/501100000272","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"<jats:p>The value of research articles is increasingly contingent on complex data analysis results which substantiate their claims. Compared to data production, data analysis more readily lends itself to a higher standard of transparency and repeated operator-independent execution. This higher standard can be approached via fully reexecutable research outputs, which contain the entire instruction set for automatic end-to-end generation of an entire article from the earliest feasible provenance point. In this study, we make use of a peer-reviewed neuroimaging article which provides complete but fragile reexecution instructions, as a starting point to draft a new reexecution system which is both robust and portable. We render this system modular as a core design aspect, so that reexecutable article code, data, and environment specifications could potentially be substituted or adapted. In conjunction with this system, which forms the demonstrative product of this study, we detail the core challenges with full article reexecution and specify a number of best practices which permitted us to mitigate them. We further show how the capabilities of our system can subsequently be used to provide reproducibility assessments, both via simple statistical metrics and by visually highlighting divergent elements for human inspection. We argue that fully reexecutable articles are thus a feasible best practice, which can greatly enhance the understanding of data analysis variability and the trust in results. Lastly, we comment at length on the outlook for reexecutable research outputs and encourage re-use and derivation of the system produced herein.<\/jats:p>","DOI":"10.3389\/fninf.2024.1376022","type":"journal-article","created":{"date-parts":[[2024,7,22]],"date-time":"2024-07-22T04:38:51Z","timestamp":1721623131000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Neuroimaging article reexecution and reproduction assessment system"],"prefix":"10.3389","volume":"18","author":[{"given":"Horea-Ioan","family":"Ioanas","sequence":"first","affiliation":[]},{"given":"Austin","family":"Macdonald","sequence":"additional","affiliation":[]},{"given":"Yaroslav O.","family":"Halchenko","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,7,22]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"14","DOI":"10.3389\/fninf.2014.00014","article-title":"Machine learning for neuroimaging with scikit-learn","volume":"8","author":"Abraham","year":"2014","journal-title":"Front. Neuroinformat"},{"volume-title":"Portage: Bringing Hackers' Wisdom to Science","year":"2016","author":"Amadio","key":"B2"},{"key":"B3","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1016\/j.neuroimage.2010.09.025","article-title":"A reproducible evaluation of ANTs similarity metric performance in brain image registration","volume":"54","author":"Avants","year":"2011","journal-title":"NeuroImage"},{"key":"B4","unstructured":"BennettS. P.\n            FaulhammerC.\n            McCreeshC.\n            M\u00fcllerU.\n          Sanford, NCGentoo FoundationPackage Manager Specification2017"},{"key":"B5","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1006\/cbmr.1996.0014","article-title":"AFNI: software for analysis and visualization of functional magnetic resonance neuroimages","volume":"29","author":"Cox","year":"1996","journal-title":"Comput. Biomed. Res"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1101\/619254","author":"Dar","year":"2019","journal-title":"REMoDNaV: Robust Eye-Movement Classification for Dynamic Stimulation"},{"key":"B7","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1006\/nimg.1995.1018","article-title":"Characterizing evoked hemodynamics with fMRI","volume":"2","author":"Friston","year":"1995","journal-title":"NeuroImage"},{"key":"B8","doi-asserted-by":"publisher","first-page":"13","DOI":"10.3389\/fninf.2011.00013","article-title":"Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in Python","volume":"5","author":"Gorgolewski","year":"2011","journal-title":"Front. Neuroinform"},{"key":"B9","doi-asserted-by":"publisher","first-page":"3262","DOI":"10.21105\/joss.03262","article-title":"DataLad: distributed system for joint management of code, data, and their relationship","volume":"6","author":"Halchenko","year":"","journal-title":"Open J"},{"key":"B10","unstructured":"HalchenkoY. O.\n            NastaseS.\n            WagnerA.\n            ChengC. P.\n            BuccigrossiR.\n            GhoshS.\n          ReproNim Reproducible Basics Module"},{"year":"2018","author":"Hanke","key":"B11"},{"key":"B12","unstructured":"IoanasH.-I.\n            MacdonaldA.\n            HalchenkoY. O.\n          San Francisco, CAGitHubNeuroimaging Article Reexecution and Reproduction Assesment System. Apache License, Verison 2.0"},{"key":"B13","unstructured":"IoanasH.-I.\n            MacdonaldA.\n            HalchenkoY. O.\n          San Francisco, CAGitHubNeuroimaging Article Reexecution and Reproduction Assesment System. Apache License, Verison 2.0"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.6084\/m9.figshare.7247339","author":"Ioanas","year":"2018","journal-title":"Reproducible Self-publishing for Python-Based Research"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3234918","author":"Ioanas","year":"","journal-title":"SAMRI\u2014Small Animal Magnetic Resonance Imaging"},{"key":"B16","doi-asserted-by":"publisher","first-page":"118386","DOI":"10.1016\/j.neuroimage.2021.118386","article-title":"An optimized registration workflow and standard geometric space for small animal brain imaging","volume":"241","author":"Ioanas","year":"2021","journal-title":"Neuroimage"},{"key":"B17","doi-asserted-by":"publisher","first-page":"e12095","DOI":"10.3897\/rio.3.e12095","article-title":"Gentoo linux for neuroscience\u2014a replicable, flexible, scalable, rolling-release environment that provides direct access to development software","volume":"3","author":"Ioanas","year":"2017","journal-title":"Res. Ideas Outcomes"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3236930","author":"Ioanas","year":"","journal-title":"BIDS Data for \u201cA Whole-Brain Map and Assay Parameter Analysis of Mouse VTA Dopaminergic Activation\u201d"},{"key":"B19","doi-asserted-by":"crossref","first-page":"3236930","DOI":"10.1038\/s41398-022-01812-5","article-title":"Whole-brain opto-fMRI map of mouse VTA dopaminergic activation reflects structural projections with small but significant deviations","volume":"12","author":"Ioanas","year":"2022","journal-title":"Transl. Psychiat"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.8214739","author":"Ioanas","year":"","journal-title":"Mouse Brain Templates\u2014Generator Workflows and Data Links"},{"year":"","author":"Ioanas","key":"B21","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.2545838"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e124","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLoS Med"},{"key":"B23","doi-asserted-by":"publisher","first-page":"782","DOI":"10.1016\/j.neuroimage.2011.09.015","article-title":"FSL","volume":"62","author":"Jenkinson","year":"2012","journal-title":"Neuroimage"},{"volume-title":"NeuroLibre: A Preprint Server for Full-Fledged Reproducible Neuroscience","year":"2022","author":"Karakuzu","key":"B24"},{"key":"B25","unstructured":"MaciocciG.\n            AufreiterM.\n            BentleyN.\n          Introducing Elife's First Computationally Reproducible Article2019"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1037102","DOI":"10.1038\/sdata.2016.1037102","article-title":"Sharing brain mapping statistical results with the neuroimaging data model","volume":"3","author":"Maumet","year":"2016","journal-title":"Sci. Data"},{"volume-title":"Reproducibility and Replicability in Science","year":"2019","key":"B27"},{"key":"B28","doi-asserted-by":"publisher","first-page":"aac4716","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","year":"2015","journal-title":"Science"},{"key":"B29","first-page":"13","article-title":"What's GNU: Bash-the GNU Shell","volume":"1994","author":"Ramey","year":"1994","journal-title":"Linux J"},{"key":"B30","doi-asserted-by":"publisher","first-page":"giad113","DOI":"10.1050\/gigascience\/giad113","article-title":"Computational reproducibility of jupyter notebooks from biomedical publications","volume":"13","year":"2024","journal-title":"GigaScience"},{"volume-title":"Free software, free society: Selected essays of Richard M. Stallman","year":"2002","author":"Stallman","key":"B31"},{"key":"B32","unstructured":"TorvaldsL.\n            HamanoJ.\n          Git: Fast Version Control System2024"}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2024.1376022\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,22]],"date-time":"2024-07-22T04:38:58Z","timestamp":1721623138000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2024.1376022\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,22]]},"references-count":32,"alternative-id":["10.3389\/fninf.2024.1376022"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2024.1376022","relation":{},"ISSN":["1662-5196"],"issn-type":[{"type":"electronic","value":"1662-5196"}],"subject":[],"published":{"date-parts":[[2024,7,22]]},"article-number":"1376022"}}