{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T18:01:14Z","timestamp":1793815274874,"version":"4.6.1"},"reference-count":79,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW1","license":[{"start":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T00:00:00Z","timestamp":1648598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,3,30]]},"abstract":"<jats:p>Despite the widespread adoption of computational notebooks, little is known about best practices for their usage in collaborative contexts. In this paper, we fill this gap by eliciting a catalog of best practices for collaborative data science with computational notebooks. With this aim, we first look for best practices through a multivocal literature review. Then, we conduct interviews with professional data scientists to assess their awareness of these best practices. Finally, we assess the adoption of best practices through the analysis of 1,380 Jupyter notebooks retrieved from the Kaggle platform. Findings reveal that experts are mostly aware of the best practices and tend to adopt them in their daily work. Nonetheless, they do not consistently follow all the recommendations as, depending on specific contexts, some are deemed unfeasible or counterproductive due to the lack of proper tool support. As such, we envision the design of notebook solutions that allow data scientists not to have to prioritize exploration and rapid prototyping over writing code of quality.<\/jats:p>","DOI":"10.1145\/3512934","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T16:54:59Z","timestamp":1649350499000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":28,"title":["Eliciting Best Practices for Collaboration with Computational Notebooks"],"prefix":"10.1145","volume":"6","author":[{"given":"Luigi","family":"Quaranta","sequence":"first","affiliation":[{"name":"University of Bari, Bari, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Calefato","sequence":"additional","affiliation":[{"name":"University of Bari, Bari, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Filippo","family":"Lanubile","sequence":"additional","affiliation":[{"name":"University of Bari, Bari, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818052.2855518"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2021.3052101"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2017.8103446"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2011.6080832"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2008.4658076"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025694"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 4th international workshop on End-user software engineering (WEUSE '08)","author":"Brandt Joel","unstructured":"Joel Brandt , Philip J. Guo , Joel Lewenstein , and Scott R. Klemmer . 2008. Opportunistic programming: how rapid ideation and prototyping occur in practice . In Proceedings of the 4th international workshop on End-user software engineering (WEUSE '08) . Association for Computing Machinery, Leipzig, Germany, 1--5. https:\/\/doi.org\/10.1145\/1370847.1370848 10.1145\/1370847.1370848 Joel Brandt, Philip J. Guo, Joel Lewenstein, and Scott R. Klemmer. 2008. Opportunistic programming: how rapid ideation and prototyping occur in practice. In Proceedings of the 4th international workshop on End-user software engineering (WEUSE '08). Association for Computing Machinery, Leipzig, Germany, 1--5. https:\/\/doi.org\/10.1145\/1370847.1370848"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84882-854-4_12"},{"key":"e_1_2_1_10_1","unstructured":"Burak Karakan. 2020. Jupyter Notebook Best Practices. https:\/\/levelup.gitconnected.com\/jupyter-notebook-best-practices-fc326eb5cd22  Burak Karakan. 2020. Jupyter Notebook Best Practices. https:\/\/levelup.gitconnected.com\/jupyter-notebook-best-practices-fc326eb5cd22"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.2874668"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3311957.3358609"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376729"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3415250"},{"key":"e_1_2_1_15_1","volume-title":"Thematic analysis. Qualitative psychology: A practical guide to research methods","author":"Clarke Victoria","year":"2015","unstructured":"Victoria Clarke , Virginia Braun , and Nikki Hayfield . 2015. Thematic analysis. Qualitative psychology: A practical guide to research methods ( 2015 ), 222--248. Victoria Clarke, Virginia Braun, and Nikki Hayfield. 2015. Thematic analysis. Qualitative psychology: A practical guide to research methods (2015), 222--248."},{"key":"e_1_2_1_16_1","volume-title":"Data scientist: The Sexiest Job of the 21st Century. Harvard business review","author":"Davenport Thomas H","year":"2012","unstructured":"Thomas H Davenport and DJ Patil . 2012. Data scientist: The Sexiest Job of the 21st Century. Harvard business review , Vol. 90 , 5 ( 2012 ), 70--76. Thomas H Davenport and DJ Patil. 2012. Data scientist: The Sexiest Job of the 21st Century. Harvard business review, Vol. 90, 5 (2012), 70--76."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134673"},{"key":"e_1_2_1_18_1","unstructured":"Dominik Haitz. 2019. Jupyter Notebook Best Practices - Concise advice to use Jupyter notebooks more effectively. http:\/\/towardsdatascience.com\/jupyter-notebook-best-practices-f430a6ba8c69  Dominik Haitz. 2019. Jupyter Notebook Best Practices - Concise advice to use Jupyter notebooks more effectively. http:\/\/towardsdatascience.com\/jupyter-notebook-best-practices-f430a6ba8c69"},{"key":"e_1_2_1_19_1","unstructured":"Florian Wilhelm. 2018. Working efficiently with Jupyter Notebooks. https:\/\/www.inovex.de\/blog\/working-efficiently-with-jupyter-notebooks\/  Florian Wilhelm. 2018. Working efficiently with Jupyter Notebooks. https:\/\/www.inovex.de\/blog\/working-efficiently-with-jupyter-notebooks\/"},{"key":"e_1_2_1_20_1","volume-title":"Refactoring: Improving the Design of Existing Code","author":"Fowler Martin","year":"2018","unstructured":"Martin Fowler . 2018 . Refactoring: Improving the Design of Existing Code . Addison-Wesley Professional. Google-Books-ID : 2H1_DwAAQBAJ. Martin Fowler. 2018. Refactoring: Improving the Design of Existing Code. Addison-Wesley Professional. Google-Books-ID: 2H1_DwAAQBAJ."},{"key":"e_1_2_1_21_1","volume-title":"Provenance for computational tasks: A survey. Computing in Science & Engineering","author":"Freire Juliana","year":"2008","unstructured":"Juliana Freire , David Koop , Emanuele Santos , and Cl\u00e1udio T Silva . 2008. Provenance for computational tasks: A survey. Computing in Science & Engineering , Vol. 10 , 3 ( 2008 ), 11--21. Publisher : IEEE. Juliana Freire, David Koop, Emanuele Santos, and Cl\u00e1udio T Silva. 2008. Provenance for computational tasks: A survey. Computing in Science & Engineering, Vol. 10, 3 (2008), 11--21. Publisher: IEEE."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering - EASE '16. ACM Press","author":"Garousi Vahid","unstructured":"Vahid Garousi , Michael Felderer , and Mika V. M\u00e4ntyl\u00e4 . 2016. The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature . In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering - EASE '16. ACM Press , Limerick, Ireland, 1--6. https:\/\/doi.org\/10.1145\/2915970.2916008 10.1145\/2915970.2916008 Vahid Garousi, Michael Felderer, and Mika V. M\u00e4ntyl\u00e4. 2016. The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering - EASE '16. ACM Press, Limerick, Ireland, 1--6. https:\/\/doi.org\/10.1145\/2915970.2916008"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.09.006"},{"key":"e_1_2_1_24_1","volume-title":"Contemporary empirical methods in software engineering","author":"Garousi Vahid","unstructured":"Vahid Garousi , Michael Felderer , Mika V. M\u00e4ntyl\u00e4 , and Austen Rainer . 2020. Benefitting from the grey literature in software engineering research . In Contemporary empirical methods in software engineering , Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing , Cham , 385--413. https:\/\/doi.org\/10.1007\/978--3-030--32489--6_14 10.1007\/978--3-030--32489--6_14 Vahid Garousi, Michael Felderer, Mika V. M\u00e4ntyl\u00e4, and Austen Rainer. 2020. Benefitting from the grey literature in software engineering research. In Contemporary empirical methods in software engineering, Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing, Cham, 385--413. https:\/\/doi.org\/10.1007\/978--3-030--32489--6_14"},{"key":"e_1_2_1_25_1","unstructured":"Joel Grus. 2018. I don't like notebooks. https:\/\/conferences.oreilly.com\/jupyter\/jup-ny\/public\/schedule\/detail\/68282.html  Joel Grus. 2018. I don't like notebooks. https:\/\/conferences.oreilly.com\/jupyter\/jup-ny\/public\/schedule\/detail\/68282.html"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300500"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.11"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3134688"},{"key":"e_1_2_1_29_1","unstructured":"Jeremy Howard. 2019. nbdev: use Jupyter Notebooks for everything. https:\/\/www.fast.ai\/2019\/12\/02\/nbdev\/  Jeremy Howard. 2019. nbdev: use Jupyter Notebooks for everything. https:\/\/www.fast.ai\/2019\/12\/02\/nbdev\/"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368308.3415397"},{"key":"e_1_2_1_31_1","unstructured":"Jonathan Whitmore. 2016. Jupyter Notebook Best Practices for Data Science. https:\/\/www.kdnuggets.com\/2016\/10\/jupyter-notebook-best-practices-data-science.html  Jonathan Whitmore. 2016. Jupyter Notebook Best Practices for Data Science. https:\/\/www.kdnuggets.com\/2016\/10\/jupyter-notebook-best-practices-data-science.html"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070547"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.219"},{"key":"e_1_2_1_34_1","unstructured":"Karlijn Willems. 2019. Jupyter Notebook Tutorial: The Definitive Guide. https:\/\/www.datacamp.com\/community\/tutorials\/tutorial-jupyter-notebook  Karlijn Willems. 2019. Jupyter Notebook Tutorial: The Definitive Guide. https:\/\/www.datacamp.com\/community\/tutorials\/tutorial-jupyter-notebook"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025626"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2018.8506576"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173748"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884783"},{"key":"#cr-split#-e_1_2_1_39_1.1","unstructured":"Thomas Kluyver Benjamin Ragan-Kelley Fernando P\u00e9rez Brian E Granger Matthias Bussonnier Jonathan Frederic Kyle Kelley Jessica B Hamrick Jason Grout Sylvain Corlay and others. 2016. Jupyter Notebooks-a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players Agents and Agendas. 87--90. https:\/\/doi.org\/10.3233\/978--1--61499--649--1--87 10.3233\/978--1--61499--649--1--87"},{"key":"#cr-split#-e_1_2_1_39_1.2","unstructured":"Thomas Kluyver Benjamin Ragan-Kelley Fernando P\u00e9rez Brian E Granger Matthias Bussonnier Jonathan Frederic Kyle Kelley Jessica B Hamrick Jason Grout Sylvain Corlay and others. 2016. Jupyter Notebooks-a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players Agents and Agendas. 87--90. https:\/\/doi.org\/10.3233\/978--1--61499--649--1--87"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/27.2.97"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC50065.2020.9127202"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2012.167"},{"key":"e_1_2_1_43_1","volume-title":"The measurement of observer agreement for categorical data. biometrics","author":"Richard Landis J","year":"1977","unstructured":"J Richard Landis and Gary G Koch . 1977. The measurement of observer agreement for categorical data. biometrics ( 1977 ), 159--174. Publisher : JSTOR. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159--174. Publisher: JSTOR."},{"key":"e_1_2_1_44_1","unstructured":"Atma Mani. 2018. Coding Standards for Jupyter Notebook. https:\/\/www.esri.com\/about\/newsroom\/arcuser\/coding-standards-for-jupyter-notebook\/  Atma Mani. 2018. Coding Standards for Jupyter Notebook. https:\/\/www.esri.com\/about\/newsroom\/arcuser\/coding-standards-for-jupyter-notebook\/"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441794"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2745390"},{"key":"e_1_2_1_47_1","unstructured":"Michael Cheng and Viacheslav Kovalevskyi. 2019. Jupyter Notebook Manifesto: Best practices that can improve the life of any developer using Jupyter notebooks. https:\/\/cloud.google.com\/blog\/products\/ai-machine-learning\/best-practices-that-can-improve-the-life-of-any-developer-using-jupyter-notebooks  Michael Cheng and Viacheslav Kovalevskyi. 2019. Jupyter Notebook Manifesto: Best practices that can improve the life of any developer using Jupyter notebooks. https:\/\/cloud.google.com\/blog\/products\/ai-machine-learning\/best-practices-that-can-improve-the-life-of-any-developer-using-jupyter-notebooks"},{"key":"e_1_2_1_48_1","unstructured":"Lester James V. Miranda. 2020. How to use Jupyter Notebooks in 2020 (Part 2: Ecosystem growth). https:\/\/ljvmiranda921.github.io\/notebook\/2020\/03\/16\/jupyter-notebooks-in-2020-part-2\/  Lester James V. Miranda. 2020. How to use Jupyter Notebooks in 2020 (Part 2: Ecosystem growth). https:\/\/ljvmiranda921.github.io\/notebook\/2020\/03\/16\/jupyter-notebooks-in-2020-part-2\/"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300356"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"2","author":"Passi Samir","year":"2018","unstructured":"Samir Passi and Steven J. Jackson . 2018. Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects . Proceedings of the ACM on Human-Computer Interaction , Vol. 2 , CSCW ( Nov. 2018 ), 1--28. https:\/\/doi.org\/10.1145\/3274405 10.1145\/3274405 Samir Passi and Steven J. Jackson. 2018. Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (Nov. 2018), 1--28. https:\/\/doi.org\/10.1145\/3274405"},{"key":"e_1_2_1_51_1","volume-title":"Nature","volume":"563","author":"Perkel Jeffrey M","year":"2018","unstructured":"Jeffrey M Perkel . 2018 . Why Jupyter is data scientists' computational notebook of choice . Nature , Vol. 563 , 7732 (2018), 145--147. Jeffrey M Perkel. 2018. Why Jupyter is data scientists' computational notebook of choice. Nature, Vol. 563, 7732 (2018), 145--147."},{"key":"e_1_2_1_52_1","unstructured":"Alex Perrier. 2020. Best practices when sharing your data analysis - Jupyter Notebooks. https:\/\/alexisperrier.com\/datascience\/2020\/02\/15\/jupyter_notebooks_sharing_best_practices.html  Alex Perrier. 2020. Best practices when sharing your data analysis - Jupyter Notebooks. https:\/\/alexisperrier.com\/datascience\/2020\/02\/15\/jupyter_notebooks_sharing_best_practices.html"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00077"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2007.53"},{"key":"e_1_2_1_55_1","volume-title":"Granger","author":"P\u00e9rez Fernando","year":"2015","unstructured":"Fernando P\u00e9rez and Brian E . Granger . 2015 . Project Jupyter : Computational Narratives as the Engine of Collaborative Data Science. Technical Report. UC Berkeley and Cal Poly . 24 pages. http:\/\/archive.ipython.org\/JupyterGrantNarrative-2015.pdf Fernando P\u00e9rez and Brian E. Granger. 2015. Project Jupyter: Computational Narratives as the Engine of Collaborative Data Science. Technical Report. UC Berkeley and Cal Poly. 24 pages. http:\/\/archive.ipython.org\/JupyterGrantNarrative-2015.pdf"},{"key":"e_1_2_1_56_1","unstructured":"Roman Kierzkowski. 2017. 10 tips on using Jupyter Notebook. https:\/\/medium.com\/@r_kierzkowski\/10-tips-on-using-jupyter-notebook-abc0ba7028a4  Roman Kierzkowski. 2017. 10 tips on using Jupyter Notebook. https:\/\/medium.com\/@r_kierzkowski\/10-tips-on-using-jupyter-notebook-abc0ba7028a4"},{"key":"e_1_2_1_57_1","volume-title":"Fernando P\u00e9rez, and others.","author":"Rule Adam","year":"2019","unstructured":"Adam Rule , Amanda Birmingham , Cristal Zuniga , Ilkay Altintas , Shih-Cheng Huang , Rob Knight , Niema Moshiri , Mai H Nguyen , Sara Brin Rosenthal , Fernando P\u00e9rez, and others. 2019 . Ten simple rules for writing and sharing computational analyses in Jupyter Notebooks. PLOS Computational Biology , Vol. 15 , 7 (2019). Publisher : Public Library of Science . Adam Rule, Amanda Birmingham, Cristal Zuniga, Ilkay Altintas, Shih-Cheng Huang, Rob Knight, Niema Moshiri, Mai H Nguyen, Sara Brin Rosenthal, Fernando P\u00e9rez, and others. 2019. Ten simple rules for writing and sharing computational analyses in Jupyter Notebooks. PLOS Computational Biology, Vol. 15, 7 (2019). Publisher: Public Library of Science."},{"key":"e_1_2_1_58_1","volume-title":"Fernando P\u00e9rez, and Peter W. Rose.","author":"Rule Adam","year":"2018","unstructured":"Adam Rule , Amanda Birmingham , Cristal Zuniga , Ilkay Altintas , Shih-Cheng Huang , Rob Knight , Niema Moshiri , Mai H. Nguyen , Sara Brin Rosenthal , Fernando P\u00e9rez, and Peter W. Rose. 2018 a. Ten Simple Rules for Reproducible Research in Jupyter Notebooks . arXiv:1810.08055 [cs] (Oct. 2018). http:\/\/arxiv.org\/abs\/1810.08055 arXiv: 1810.08055. Adam Rule, Amanda Birmingham, Cristal Zuniga, Ilkay Altintas, Shih-Cheng Huang, Rob Knight, Niema Moshiri, Mai H. Nguyen, Sara Brin Rosenthal, Fernando P\u00e9rez, and Peter W. Rose. 2018a. Ten Simple Rules for Reproducible Research in Jupyter Notebooks. arXiv:1810.08055 [cs] (Oct. 2018). http:\/\/arxiv.org\/abs\/1810.08055 arXiv: 1810.08055."},{"key":"e_1_2_1_59_1","volume-title":"Aiding Collaborative Reuse of Computational Notebooks with Annotated Cell Folding. Proceedings of the ACM on Human-Computer Interaction","volume":"2","author":"Rule Adam","year":"2018","unstructured":"Adam Rule , Ian Drosos , Aur\u00e9lien Tabard , and James D. Hollan . 2018b . Aiding Collaborative Reuse of Computational Notebooks with Annotated Cell Folding. Proceedings of the ACM on Human-Computer Interaction , Vol. 2 , CSCW ( Nov. 2018 ), 1--12. https:\/\/doi.org\/10.1145\/3274419 10.1145\/3274419 Adam Rule, Ian Drosos, Aur\u00e9lien Tabard, and James D. Hollan. 2018b. Aiding Collaborative Reuse of Computational Notebooks with Annotated Cell Folding. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (Nov. 2018), 1--12. https:\/\/doi.org\/10.1145\/3274419"},{"key":"e_1_2_1_60_1","volume-title":"Proc. of the 2018 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10","author":"Rule Adam","unstructured":"Adam Rule , Aur\u00e9lien Tabard , and James D. Hollan . 2018c. Exploration and Explanation in Computational Notebooks . In Proc. of the 2018 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10 .1145\/3173574.3173606 10.1145\/3173574.3173606 Adam Rule, Aur\u00e9lien Tabard, and James D. Hollan. 2018c. Exploration and Explanation in Computational Notebooks. In Proc. of the 2018 CHI Conference on Human Factors in Computing Systems. https:\/\/doi.org\/10.1145\/3173574.3173606"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICGSE.2016.35"},{"key":"e_1_2_1_62_1","unstructured":"Saturn Cloud Dev Team. 2020. Best Practices for Jupyter Notebook. http:\/\/site.saturncloud.io\/s\/best-practices-for-jupyter-notebooks  Saturn Cloud Dev Team. 2020. Best Practices for Jupyter Notebook. http:\/\/site.saturncloud.io\/s\/best-practices-for-jupyter-notebooks"},{"key":"e_1_2_1_63_1","unstructured":"Matthew Seal Kyle Kelley and Michelle Ufford. 2018. Part 2: Scheduling Notebooks at Netflix. https:\/\/medium.com\/netflix-techblog\/scheduling-notebooks-348e6c14cfd6  Matthew Seal Kyle Kelley and Michelle Ufford. 2018. Part 2: Scheduling Notebooks at Netflix. https:\/\/medium.com\/netflix-techblog\/scheduling-notebooks-348e6c14cfd6"},{"key":"e_1_2_1_64_1","volume-title":"Jupyter for data science: Exploratory analysis, statistical modeling, machine learning, and data visualization with Jupyter","author":"Toomey Dan","unstructured":"Dan Toomey . 2017. Jupyter for data science: Exploratory analysis, statistical modeling, machine learning, and data visualization with Jupyter . Packt Publishing Ltd . Dan Toomey. 2017. Jupyter for data science: Exploratory analysis, statistical modeling, machine learning, and data visualization with Jupyter. Packt Publishing Ltd."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.59"},{"key":"e_1_2_1_66_1","volume-title":"Beyond Interactive: Notebook Innovation at Netflix. https:\/\/medium.com\/netflix-techblog\/notebook-innovation-591ee3221233","author":"Ufford Michelle","year":"2018","unstructured":"Michelle Ufford , M. Pacer , Matthew Seal , and Kyle Kelley . 2018 . Beyond Interactive: Notebook Innovation at Netflix. https:\/\/medium.com\/netflix-techblog\/notebook-innovation-591ee3221233 Michelle Ufford, M. Pacer, Matthew Seal, and Kyle Kelley. 2018. Beyond Interactive: Notebook Innovation at Netflix. https:\/\/medium.com\/netflix-techblog\/notebook-innovation-591ee3221233"},{"key":"e_1_2_1_67_1","volume-title":"Python data science handbook: Essential tools for working with data","unstructured":"VanderPlas, Jake. 2016. IPython Magic Commands . In Python data science handbook: Essential tools for working with data . O'Reilly Media, Inc. , 10--30. VanderPlas, Jake. 2016. IPython Magic Commands. In Python data science handbook: Essential tools for working with data. O'Reilly Media, Inc., 10--30."},{"key":"e_1_2_1_68_1","volume-title":"Cambridge advanced learner's dictionary","author":"Walter Elizabeth","unstructured":"Elizabeth Walter . 2008. Cambridge advanced learner's dictionary . Cambridge university press . Elizabeth Walter. 2008. Cambridge advanced learner's dictionary .Cambridge university press."},{"key":"e_1_2_1_69_1","volume-title":"How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction","volume":"3","author":"Wang April Yi","year":"2019","unstructured":"April Yi Wang , Anant Mittal , Christopher Brooks , and Steve Oney . 2019 a . How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction , Vol. 3 , CSCW (Nov. 2019), 39:1--39:30. https:\/\/doi.org\/10.1145\/3359141 10.1145\/3359141 April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019 a. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (Nov. 2019), 39:1--39:30. https:\/\/doi.org\/10.1145\/3359141"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376740"},{"key":"e_1_2_1_71_1","volume-title":"Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI. Proceedings of the ACM on Human-Computer Interaction","volume":"3","author":"Wang Dakuo","year":"2019","unstructured":"Dakuo Wang , Justin D. Weisz , Michael Muller , Parikshit Ram , Werner Geyer , Casey Dugan , Yla Tausczik , Horst Samulowitz , and Alexander Gray . 2019 b . Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI. Proceedings of the ACM on Human-Computer Interaction , Vol. 3 , CSCW (Nov. 2019), 1--24. https:\/\/doi.org\/10.1145\/3359313 10.1145\/3359313 Dakuo Wang, Justin D. Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019 b. Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (Nov. 2019), 1--24. https:\/\/doi.org\/10.1145\/3359313"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377812.3390803"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377816.3381724"},{"key":"e_1_2_1_74_1","first-page":"4","article-title":"Program slicing","volume":"10","author":"Weiser Mark","year":"1984","unstructured":"Mark Weiser . 1984 . Program slicing . IEEE Transactions on Software Engineering , Vol. 10 , 4 (July 1984), 352--357. https:\/\/doi.org\/10.1109\/TSE.1984.5010248 10.1109\/TSE.1984.5010248 Mark Weiser. 1984. Program slicing. IEEE Transactions on Software Engineering, Vol. 10, 4 (July 1984), 352--357. https:\/\/doi.org\/10.1109\/TSE.1984.5010248","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_2_1_75_1","unstructured":"Jonathan Whitmore. 2015. Jupyter Notebook Best Practices for Data Science. https:\/\/www.svds.com\/jupyter-notebook-best-practices-for-data-science\/  Jonathan Whitmore. 2015. Jupyter Notebook Best Practices for Data Science. https:\/\/www.svds.com\/jupyter-notebook-best-practices-for-data-science\/"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_2_1_77_1","unstructured":"Mark Woodbridge. 2017. Jupyter Notebooks and Reproducible Data Science. https:\/\/markwoodbridge.com\/2017\/03\/05\/jupyter-reproducible-science.html  Mark Woodbridge. 2017. Jupyter Notebooks and Reproducible Data Science. https:\/\/markwoodbridge.com\/2017\/03\/05\/jupyter-reproducible-science.html"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392826"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512934","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3512934","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:18Z","timestamp":1750188678000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512934"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,30]]},"references-count":79,"journal-issue":{"issue":"CSCW1","published-print":{"date-parts":[[2022,3,30]]}},"alternative-id":["10.1145\/3512934"],"URL":"https:\/\/doi.org\/10.1145\/3512934","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,30]]},"assertion":[{"value":"2022-04-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}