{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T23:09:21Z","timestamp":1740179361050,"version":"3.37.3"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,23]],"date-time":"2021-10-23T00:00:00Z","timestamp":1634947200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,23]],"date-time":"2021-10-23T00:00:00Z","timestamp":1634947200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"deutsche forschungsgemeinschaft","doi-asserted-by":"publisher","award":["INST 217\/835-1 FUGG"],"award-info":[{"award-number":["INST 217\/835-1 FUGG"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008131","name":"rheinische friedrich-wilhelms-universit\u00e4t bonn","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100008131","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008131","name":"Rheinische Friedrich-Wilhelms-Universit\u00e4t Bonn","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008131","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Softw Big Sci"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years Jupyter notebooks have conquered class rooms and some scientists also enjoy their convenience to quickly evaluate ideas and check whether a more detailed study is justified. To lower the threshold for getting started with Jupyter notebooks and to ease sharing and collaborative use, offering a JupyterHub service is tempting. However, offering such a service for a larger science class also requires a compute backend with sufficient resources such that hundreds of notebooks can be run simultaneously. Since resource usage for teaching activities typically fluctuates significantly over the year, dedicated compute resources seem inefficient. In this paper we present an alternative by exploiting an existing high throughput computing cluster (BAF2) at the University of Bonn, which comes with the additional advantage that scientific users may use the very same software and data environment they also select for their batch jobs. To implement this, we used a novel approach which allowed us to integrate BAF2 execute nodes although they do not have inbound network connectivity. Therefore, it does not touch the security concept of the cluster. The very same technique can be used to integrate any compute resources without inbound network connectivity and thus allows one to overcome usual firewall restrictions. This design also simplifies exploiting remote resources e.g. offered by resource federations or cloud providers.<\/jats:p>","DOI":"10.1007\/s41781-021-00063-1","type":"journal-article","created":{"date-parts":[[2021,10,23]],"date-time":"2021-10-23T04:02:45Z","timestamp":1634961765000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Unleashing JupyterHub: Exploiting Resources Without Inbound Network Connectivity Using HTCondor"],"prefix":"10.1007","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8053-6283","authenticated-orcid":false,"given":"Oliver","family":"Freyermuth","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katrin","family":"Kohl","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4094-6493","authenticated-orcid":false,"given":"Peter","family":"Wienemann","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,10,23]]},"reference":[{"key":"63_CR1","unstructured":"Jupyter. https:\/\/jupyter.org. Accessed on 27 May 2021"},{"key":"63_CR2","unstructured":"Kubernetes. https:\/\/kubernetes.io. Accessed on 27 May 2021"},{"key":"63_CR3","unstructured":"Zero to JupyterHub with Kubernetes. https:\/\/zero-to-jupyterhub.readthedocs.io. Accessed on 27 May 2021"},{"key":"63_CR4","unstructured":"JupyterHub Batch Spawner. https:\/\/github.com\/jupyterhub\/batchspawner. Accessed on 27 May 2021"},{"key":"63_CR5","unstructured":"TORQUE. http:\/\/adaptivecomputing.com\/cherry-services\/torque-resource-manager. Accessed on 27 May 2021"},{"key":"63_CR6","unstructured":"Slurm Workload Manager. https:\/\/slurm.schedmd.com. Accessed on 27 May 2021"},{"key":"63_CR7","unstructured":"Altair Grid Engine. https:\/\/www.altair.com\/grid-engine. Accessed on 27 May 2021"},{"key":"63_CR8","unstructured":"HTCondor. https:\/\/research.cs.wisc.edu\/htcondor. Accessed on 27 May 2021"},{"key":"63_CR9","unstructured":"IBM Spectrum LSF Suites. https:\/\/www.ibm.com\/products\/hpc-workload-management. Accessed on 27 May 2021"},{"issue":"1","key":"63_CR10","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1007\/s41781-021-00058-y","volume":"5","author":"J Reppin","year":"2021","unstructured":"Reppin J, Beyer C, Hartmann T, Schluenzen F, Flemming M, Sternberger S, Kemp Y (2021) Interactive analysis notebooks on DESY batch resources: Bringing Juypter to HTCondor and Maxwell at DESY. Comput Softw Big Sci 5(1):16. https:\/\/doi.org\/10.1007\/s41781-021-00058-y","journal-title":"Comput Softw Big Sci"},{"issue":"1","key":"63_CR11","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/s41781-020-00050-y","volume":"5","author":"O Freyermuth","year":"2021","unstructured":"Freyermuth O, Wienemann P, Bechtle P, Desch K (2021) Operating an HPC\/HTC Cluster with Fully Containerized Jobs Using HTCondor, Singularity, CephFS and CVMFS. Comput Softw Big Sci 5(1):9. https:\/\/doi.org\/10.1007\/s41781-020-00050-y","journal-title":"Comput Softw Big Sci"},{"key":"63_CR12","unstructured":"JupyterHub Spawners documentation. https:\/\/jupyterhub.readthedocs.io\/en\/stable\/reference\/spawners.html. Accessed on 27 May 2021"},{"key":"63_CR13","unstructured":"JupyterHub Spawner development documentation. https:\/\/jupyterhub-tutorial.readthedocs.io\/en\/latest\/spawners.html. Accessed on 27 May 2021"},{"key":"63_CR14","unstructured":"JupyterHub Spawner base class. https:\/\/github.com\/jupyterhub\/jupyterhub\/blob\/main\/jupyterhub\/spawner.py. Accessed on 27 May 2021"},{"key":"63_CR15","unstructured":"configurable-http-proxy. https:\/\/github.com\/jupyterhub\/configurable-http-proxy. Accessed on 27 May 2021"},{"key":"63_CR16","unstructured":"node-http-proxy. https:\/\/github.com\/http-party\/node-http-proxy. Accessed on 27 May 2021"},{"key":"63_CR17","unstructured":"Litzkow M (1987) Remote Unix-turning idle workstations into cycle servers. In: Proceedings of usenix summer conference, pp 381\u2013384. https:\/\/research.cs.wisc.edu\/htcondor\/doc\/remoteunix.pdf"},{"key":"63_CR18","doi-asserted-by":"crossref","unstructured":"Litzkow M, Livny M, Mutka MW (1988) Condor \u2014 a hunter of idle workstations. In: Proceedings of the 8th international conference of distributed computing systems, pp 104\u2013111. https:\/\/research.cs.wisc.edu\/htcondor\/doc\/condor-hunter.pdf","DOI":"10.1109\/DCS.1988.12507"},{"key":"63_CR19","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/0167-739X(95)00035-Q","volume":"12","author":"D Epema","year":"1996","unstructured":"Epema D, Livny M, van Dantzig R, Evers X, Pruyne J (1996) A worldwide flock of condors: Load sharing among workstation clusters. Future Gener Comput Syst 12:53","journal-title":"Future Gener Comput Syst"},{"key":"63_CR20","unstructured":"Livny M, Basney J, Raman R, Tannenbaum T (1997) Mechanisms for high throughput computing, SPEEDUP 11. https:\/\/research.cs.wisc.edu\/htcondor\/doc\/htc_mech.pdf"},{"key":"63_CR21","unstructured":"Basney J, Livny M (1999) High performance cluster computing: architectures and systems. In: Buyya R (ed) Prentice Hall PTR, vol 1, ISBN-13: 978-0130137845. https:\/\/research.cs.wisc.edu\/htcondor\/doc\/hpcc-chapter.pdf"},{"key":"63_CR22","unstructured":"Tannenbaum T, Wright D, Miller K, Livny M (2001) In: Sterling T (ed) Beowulf cluster computing with Linux, MIT Press, ISBN-13: 978-0262692748. https:\/\/research.cs.wisc.edu\/htcondor\/doc\/beowulf-chapter-rev1.pdf"},{"issue":"2\u20134","key":"63_CR23","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1002\/cpe.938","volume":"17","author":"D Thain","year":"2005","unstructured":"Thain D, Tannenbaum T, Livny M (2005) Distributed computing in practice: the condor experience. Concur Pract Exp 17(2\u20134):323","journal-title":"Concur Pract Exp"},{"key":"63_CR24","doi-asserted-by":"publisher","unstructured":"Raman R, Livny M, Solomon M (1998) Matchmaking: distributed resource management for high throughput computing. In: Proceedings of the seventh IEEE international symposium on high performance distributed computing (HPDC7), 98, Chicago, Illinois, USA, pp 140\u2013146, IEEE Computer Society. https:\/\/doi.org\/10.1109\/HPDC.1998.709966","DOI":"10.1109\/HPDC.1998.709966"},{"key":"63_CR25","unstructured":"OpenSSH. https:\/\/www.openssh.com. Accessed on 27 May 2021"},{"key":"63_CR26","unstructured":"Python 3: Asynchronous i\/o. https:\/\/docs.python.org\/3\/library\/asyncio.html. Accessed on 27 May 2021"},{"key":"63_CR27","unstructured":"AsyncIO Futures. https:\/\/docs.python.org\/3\/library\/asyncio-future.html. Accessed on 27 May 2021"},{"key":"63_CR28","unstructured":"Apache HTTP Server Project. https:\/\/httpd.apache.org. Accessed on 27 May 2021"},{"key":"63_CR29","unstructured":"Jupyter serverproxy. https:\/\/jupyter-server-proxy.readthedocs.io. Accessed on 27 May 2021"},{"key":"63_CR30","unstructured":"Jinja Templating Engine. https:\/\/jinja.palletsprojects.com. Accessed on 27 May 2021"},{"key":"63_CR31","unstructured":"JupyterHub: working with templates and UI. https:\/\/jupyterhub.readthedocs.io\/en\/stable\/reference\/templates.html. Accessed on 27 May 2021"},{"key":"63_CR32","unstructured":"JupyterHub Project, Issue #3414: implementing login page customization. https:\/\/github.com\/jupyterhub\/jupyterhub\/issues\/3414. Accessed on 27 May 2021"},{"key":"63_CR33","unstructured":"FormSpawner with customization for site-specifics. https:\/\/github.com\/unibonn\/ubnjupyterspawner. Accessed on 27 May 2021"},{"key":"63_CR34","unstructured":"CVMFS. https:\/\/cernvm.cern.ch\/portal\/filesystem. Accessed on 27 May 2021"},{"key":"63_CR35","unstructured":"IJulia. https:\/\/julialang.github.io\/IJulia.jl\/stable\/. Accessed on 27 May 2021"},{"key":"63_CR36","doi-asserted-by":"publisher","unstructured":"Brun R, Rademakers F, Canal P, Naumann A, Couet O, Moneta L, Vassilev V, Linev S, Piparo D, Ganis G, Bellenot B, Guiraud E, Amadio G, Verkerke W, Mato P, Timur P, Tadel M, Wlav, Tejedor E, Blomer J, Gheata A, Hageboeck S, Roiser S, Marsupial, Wunsch S, Shadura O, Bose A, Cristescu C, Valls X, Isemann R (2019) root-project\/root: v6.18\/02. https:\/\/doi.org\/10.5281\/zenodo.848818","DOI":"10.5281\/zenodo.848818"},{"key":"63_CR37","unstructured":"PyROOT. https:\/\/root.cern\/manual\/python\/. Accessed 27 May 2021"},{"key":"63_CR38","unstructured":"XPRA. https:\/\/xpra.org\/. Accessed on 27 May 2021"},{"key":"63_CR39","unstructured":"Anaconda. https:\/\/www.anaconda.com\/. Accessed on 27 May 2021"},{"key":"63_CR40","unstructured":"Python Package Index. https:\/\/pypi.org. Accessed on 27 May 2021"},{"key":"63_CR41","unstructured":"Linux PAM (Pluggable Authentication Modules for Linux) project. https:\/\/github.com\/linux-pam\/linux-pam. Accessed on 27 May 2021"},{"key":"63_CR42","unstructured":"sssd: Open Source Client for Enterprise Identity Management. https:\/\/sssd.io\/. Accessed on 27 May 2021"},{"key":"63_CR43","unstructured":"JupyterHub Batchspawner Project, Pull Request #200. https:\/\/github.com\/jupyterhub\/batchspawner\/pull\/200. Accessed on 27 May 2021"},{"key":"63_CR44","unstructured":"HTCondor Pool Flocking. https:\/\/htcondor.readthedocs.io\/en\/latest\/grid-computing\/connecting-pools-with-flocking.html. Accessed on 27 May 2021"},{"key":"63_CR45","doi-asserted-by":"publisher","unstructured":"Fischer M, Kuehn E, Giffels M (2021) COBalD\u2014the opportunistic balancing Daemon (2018). https:\/\/doi.org\/10.5281\/zenodo.1887872.. Accessed on 27 May 2021","DOI":"10.5281\/zenodo.1887872."},{"key":"63_CR46","unstructured":"HTMap Library. https:\/\/htmap.readthedocs.io. Accessed on 27 May 2021"}],"container-title":["Computing and Software for Big Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-021-00063-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41781-021-00063-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-021-00063-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,10]],"date-time":"2024-09-10T16:25:12Z","timestamp":1725985512000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41781-021-00063-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,23]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["63"],"URL":"https:\/\/doi.org\/10.1007\/s41781-021-00063-1","relation":{},"ISSN":["2510-2036","2510-2044"],"issn-type":[{"type":"print","value":"2510-2036"},{"type":"electronic","value":"2510-2044"}],"subject":[],"published":{"date-parts":[[2021,10,23]]},"assertion":[{"value":"14 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"24"}}