{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T17:08:38Z","timestamp":1785604118146,"version":"3.56.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA1","license":[{"start":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T00:00:00Z","timestamp":1714348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ACT-X, Japan Science and Technology Agency","award":["JPMJAX210F"],"award-info":[{"award-number":["JPMJAX210F"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2024,4,29]]},"abstract":"<jats:p>Computational notebook environments are popular and de facto standard  \ntools for programming in data science, whereas computational notebooks are notorious in  \nsoftware engineering. The criticism there stems from the characteristic  \nof facilitating unrestricted dynamic patching of running programs, which  \nmakes exploratory coding quick but the resultant code messy and  \ninconsistent. In this work, we first reveal that dynamic patching is a  \nnatural demand rather than a mere bad practice in data science  \nprogramming on Kaggle. We then develop Multiverse Notebook, a  \ncomputational notebook engine for time-traveling exploration. It  \nenables users to time-travel to any past state and restart with new code  \nfrom there under state isolation. We present an approach to efficiently  \nimplementing time-traveling exploration. We empirically evaluate  \nMultiverse Notebook on ten real-world tasks from Kaggle. Our experiments  \nshow that time-traveling exploration on Multiverse Notebook is  \nreasonably efficient.<\/jats:p>","DOI":"10.1145\/3649838","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T17:53:50Z","timestamp":1714413230000},"page":"754-783","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Multiverse Notebook: Shifting Data Scientists to Time Travelers"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1496-1422","authenticated-orcid":false,"given":"Shigeyuki","family":"Sato","sequence":"first","affiliation":[{"name":"The University of Electro-Communications, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9451-5595","authenticated-orcid":false,"given":"Tomoki","family":"Nakamaru","sequence":"additional","affiliation":[{"name":"The University of Tokyo, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2660193.2660209"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3317550.3321435"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1180475.1180477"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/349299.349339"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3092255.3092272"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376729"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","unstructured":"Perry Cheng Robert Harper and Peter Lee. 1998. Generational Stack Collection and Profile-Driven Pretenuring. In roceedings of the ACM SIGPLAN 1998 Conference on Programming Language Design and Implementation (PLDI \u201998). ACM 162\u2013173. https:\/\/doi.org\/10.1145\/277650.277718 10.1145\/277650.277718","DOI":"10.1145\/277650.277718"},{"key":"e_1_2_1_8_1","unstructured":"Maur\u00edci Cordeiro. 2021. Why Data Scientists Should use Jupyter Notebooks with Moderation. https:\/\/towardsdatascience.com\/why-data-scientists-should-use-jupyter-notebooks-with-moderation-808900a69eff"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3641539"},{"key":"e_1_2_1_10_1","unstructured":"Jake Edge. 2013. The failure of pysandbox. https:\/\/lwn.net\/Articles\/574215\/"},{"key":"e_1_2_1_11_1","unstructured":"Eddie Elizondo. 2023. PEP683: Immortal Objects: Updates. https:\/\/discuss.python.org\/t\/pep683-immortal-objects-updates\/23382"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/68210.69226"},{"key":"e_1_2_1_13_1","unstructured":"Laura Fink. 2021. Signal where are you? https:\/\/www.kaggle.com\/code\/allunia\/signal-where-are-you"},{"key":"e_1_2_1_14_1","volume-title":"Tempe: An Interactive Data Science Environment for Exploration of Temporal and Streaming Data. Microsoft Research.","author":"Fisher Danyel","year":"2014","unstructured":"Danyel Fisher, Badrish Chandramouli, Robert DeLIne, Jonathan Goldstein, Andrei Aron, Mike Barnett, John Platt, James Terwilliger, and John Wernsing. 2014. Tempe: An Interactive Data Science Environment for Exploration of Temporal and Streaming Data. Microsoft Research."},{"key":"e_1_2_1_15_1","volume-title":"I Don\u2019t Like Notebooks. JupyterCon","author":"Grus Joel","year":"2018","unstructured":"Joel Grus. 2018. I Don\u2019t Like Notebooks. JupyterCon 2018. https:\/\/docs.google.com\/presentation\/d\/1n2RlMdmv1p25Xy5thJUhkKGvjtV-dkAIsUXP-AL4ffI\/edit?usp=sharing"},{"key":"e_1_2_1_16_1","unstructured":"Alena Guzharina. 2021. Revamped Reactive Mode and How It Makes Your Notebooks Reproducible. https:\/\/blog.jetbrains.com\/datalore\/2021\/10\/11\/revamped-reactive-mode-and-how-it-makes-your-notebooks-reproducible\/"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629460"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2384616.2384635"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300500"},{"key":"e_1_2_1_20_1","volume-title":"Dynamic Software Updating. Ph. D. Dissertation","author":"Hicks Michael","unstructured":"Michael Hicks. 2001. Dynamic Software Updating. Ph. D. Dissertation. University of Pennsylvania."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/378795.378798"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1108970.1108971"},{"key":"e_1_2_1_23_1","unstructured":"HN. 2020. Augmentations Data Cleaning and Bounding Boxes. https:\/\/www.kaggle.com\/code\/reighns\/augmentations-data-cleaning-and-bounding-boxes"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111652"},{"key":"e_1_2_1_25_1","unstructured":"Naman Jaswani. 2020. [New Baseline] Pytorch | MoA. https:\/\/www.kaggle.com\/code\/namanj27\/new-baseline-pytorch-moa"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.22152\/programming-journal.org"},{"key":"e_1_2_1_27_1","unstructured":"Umut Karakulak. 2017. Using stage 1 test results for stage 2 training. https:\/\/www.kaggle.com\/code\/umutto\/using-stage-1-test-results-for-stage-2-training"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767867"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300322"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2017.8103446"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2018.8506576"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173748"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2930987"},{"key":"e_1_2_1_36_1","unstructured":"Zekun Li. 2017. Copy-on-write friendly Python garbage collection. https:\/\/instagram-engineering.com\/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf"},{"key":"e_1_2_1_37_1","unstructured":"Andrew Lukyanenko. 2020. Which bird is it? https:\/\/www.kaggle.com\/code\/artgor\/which-bird-is-it"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360569"},{"key":"e_1_2_1_39_1","unstructured":"Vitalii Mokin. 2020. Ion Switching - AdvFE LGB Wavenet ConfMatrix. https:\/\/www.kaggle.com\/code\/vbmokin\/ion-switching-advfe-lgb-wavenet-confmatrix"},{"key":"e_1_2_1_40_1","unstructured":"Rob Mulla. 2019. IEEE Fraud Detection - First Look and EDA. https:\/\/www.kaggle.com\/code\/robikscube\/ieee-fraud-detection-first-look-and-eda"},{"key":"e_1_2_1_41_1","unstructured":"Rob Mulla. 2020. OpenVaccine: COVID-19 mRNA Starter EDA. https:\/\/www.kaggle.com\/code\/robikscube\/openvaccine-covid-19-mrna-starter-eda"},{"key":"e_1_2_1_42_1","unstructured":"Peter Parente. 2024. Estimate of Public Jupyter Notebooks on GitHub. https:\/\/github.com\/parente\/nbestimate"},{"key":"e_1_2_1_43_1","unstructured":"Mark Peng. 2020. DeepInsight: Transforming Non-image data to Images. https:\/\/www.kaggle.com\/code\/markpeng\/deepinsight-transforming-non-image-data-to-images"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-07196-1"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00077"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-09961-9"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304063"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2660193.2660220"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00072"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3572863"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580997"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173606"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580726"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.7656049"},{"key":"e_1_2_1_55_1","unstructured":"Eric Snow and Eddie Elizondo. 2022. PEP 683 \u2013Immortal Objects Using a Fixed Refcount. https:\/\/peps.python.org\/pep-0683\/"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00066"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519653"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22655-7_9"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445527"},{"key":"e_1_2_1_60_1","unstructured":"xhlulu. 2019. IEEE Fraud: XGBoost with GPU (Fit in 40s). https:\/\/www.kaggle.com\/code\/xhlulu\/ieee-fraud-xgboost-with-gpu-fit-in-40s"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3649838","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3649838","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:06Z","timestamp":1750287246000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3649838"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,29]]},"references-count":60,"journal-issue":{"issue":"OOPSLA1","published-print":{"date-parts":[[2024,4,29]]}},"alternative-id":["10.1145\/3649838"],"URL":"https:\/\/doi.org\/10.1145\/3649838","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,29]]},"assertion":[{"value":"2024-04-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}