{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T23:09:33Z","timestamp":1740179373796,"version":"3.37.3"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T00:00:00Z","timestamp":1716336000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T00:00:00Z","timestamp":1716336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["SFB1258","EXC 2094"],"award-info":[{"award-number":["SFB1258","EXC 2094"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["I5420-N"],"award-info":[{"award-number":["I5420-N"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Austrian research promotion agency","award":["ML4CPD"],"award-info":[{"award-number":["ML4CPD"]}]},{"DOI":"10.13039\/501100012650","name":"Technische Universit\u00e4t Wien Bibliothek","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012650","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005357","name":"Slovak Research and Development Agency","doi-asserted-by":"crossref","award":["APVV-15-0576","APVV-21-0377"],"award-info":[{"award-number":["APVV-15-0576","APVV-21-0377"]}],"id":[{"id":"10.13039\/501100005357","id-type":"DOI","asserted-by":"crossref"}]},{"name":"German Federal Ministry of Education and Research","award":["05A20WO1","05A20VTA"],"award-info":[{"award-number":["05A20WO1","05A20VTA"]}]},{"name":"\u00d6sterreichische Akademie der Wissenschaften"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Softw Big Sci"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Cryogenic phonon detectors with transition-edge sensors achieve the best sensitivity to sub-GeV\/c<jats:inline-formula><jats:alternatives><jats:tex-math>$$^2$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mmultiscripts>\n                    <mml:mrow\/>\n                    <mml:mrow\/>\n                    <mml:mn>2<\/mml:mn>\n                  <\/mml:mmultiscripts>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> dark matter interactions with nuclei in current direct detection experiments. In such devices, the temperature of the thermometer and the bias current in its readout circuit need careful optimization to achieve optimal detector performance. This task is not trivial and is typically done manually by an expert. In our work, we automated the procedure with reinforcement learning in two settings. First, we trained on a simulation of the response of three Cryogenic Rare Event Search with Superconducting Thermometers (CRESST) detectors used as a virtual reinforcement learning environment. Second, we trained live on the same detectors operated in the CRESST underground setup. In both cases, we were able to optimize a standard detector as fast and with comparable results as human experts. Our method enables the tuning of large-scale cryogenic detector setups with minimal manual interventions.<\/jats:p>","DOI":"10.1007\/s41781-024-00119-y","type":"journal-article","created":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T17:01:36Z","timestamp":1716397296000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Optimal Operation of Cryogenic Calorimeters Through Deep Reinforcement Learning"],"prefix":"10.1007","volume":"8","author":[{"given":"G.","family":"Angloher","sequence":"first","affiliation":[]},{"given":"S.","family":"Banik","sequence":"additional","affiliation":[]},{"given":"G.","family":"Benato","sequence":"additional","affiliation":[]},{"given":"A.","family":"Bento","sequence":"additional","affiliation":[]},{"given":"A.","family":"Bertolini","sequence":"additional","affiliation":[]},{"given":"R.","family":"Breier","sequence":"additional","affiliation":[]},{"given":"C.","family":"Bucci","sequence":"additional","affiliation":[]},{"given":"J.","family":"Burkhart","sequence":"additional","affiliation":[]},{"given":"L.","family":"Canonica","sequence":"additional","affiliation":[]},{"given":"A.","family":"D\u2019Addabbo","sequence":"additional","affiliation":[]},{"given":"S.","family":"Di Lorenzo","sequence":"additional","affiliation":[]},{"given":"L.","family":"Einfalt","sequence":"additional","affiliation":[]},{"given":"A.","family":"Erb","sequence":"additional","affiliation":[]},{"given":"F.","family":"v. Feilitzsch","sequence":"additional","affiliation":[]},{"given":"S.","family":"Fichtinger","sequence":"additional","affiliation":[]},{"given":"D.","family":"Fuchs","sequence":"additional","affiliation":[]},{"given":"A.","family":"Garai","sequence":"additional","affiliation":[]},{"given":"V. M.","family":"Ghete","sequence":"additional","affiliation":[]},{"given":"P.","family":"Gorla","sequence":"additional","affiliation":[]},{"given":"P. V.","family":"Guillaumon","sequence":"additional","affiliation":[]},{"given":"S.","family":"Gupta","sequence":"additional","affiliation":[]},{"given":"D.","family":"Hauff","sequence":"additional","affiliation":[]},{"given":"M.","family":"Je\u0161kovsk\u00fd","sequence":"additional","affiliation":[]},{"given":"J.","family":"Jochum","sequence":"additional","affiliation":[]},{"given":"M.","family":"Kaznacheeva","sequence":"additional","affiliation":[]},{"given":"A.","family":"Kinast","sequence":"additional","affiliation":[]},{"given":"S.","family":"Kuckuk","sequence":"additional","affiliation":[]},{"given":"H.","family":"Kluck","sequence":"additional","affiliation":[]},{"given":"H.","family":"Kraus","sequence":"additional","affiliation":[]},{"given":"A.","family":"Langenk\u00e4mper","sequence":"additional","affiliation":[]},{"given":"M.","family":"Mancuso","sequence":"additional","affiliation":[]},{"given":"L.","family":"Marini","sequence":"additional","affiliation":[]},{"given":"B.","family":"Mauri","sequence":"additional","affiliation":[]},{"given":"L.","family":"Meyer","sequence":"additional","affiliation":[]},{"given":"V.","family":"Mokina","sequence":"additional","affiliation":[]},{"given":"K.","family":"Niedermayer","sequence":"additional","affiliation":[]},{"given":"M.","family":"Olmi","sequence":"additional","affiliation":[]},{"given":"T.","family":"Ortmann","sequence":"additional","affiliation":[]},{"given":"C.","family":"Pagliarone","sequence":"additional","affiliation":[]},{"given":"L.","family":"Pattavina","sequence":"additional","affiliation":[]},{"given":"F.","family":"Petricca","sequence":"additional","affiliation":[]},{"given":"W.","family":"Potzel","sequence":"additional","affiliation":[]},{"given":"P.","family":"Povinec","sequence":"additional","affiliation":[]},{"given":"F.","family":"Pr\u00f6bst","sequence":"additional","affiliation":[]},{"given":"F.","family":"Pucci","sequence":"additional","affiliation":[]},{"given":"F.","family":"Reindl","sequence":"additional","affiliation":[]},{"given":"J.","family":"Rothe","sequence":"additional","affiliation":[]},{"given":"K.","family":"Sch\u00e4ffner","sequence":"additional","affiliation":[]},{"given":"J.","family":"Schieck","sequence":"additional","affiliation":[]},{"given":"S.","family":"Sch\u00f6nert","sequence":"additional","affiliation":[]},{"given":"C.","family":"Schwertner","sequence":"additional","affiliation":[]},{"given":"M.","family":"Stahlberg","sequence":"additional","affiliation":[]},{"given":"L.","family":"Stodolsky","sequence":"additional","affiliation":[]},{"given":"C.","family":"Strandhagen","sequence":"additional","affiliation":[]},{"given":"R.","family":"Strauss","sequence":"additional","affiliation":[]},{"given":"I.","family":"Usherov","sequence":"additional","affiliation":[]},{"given":"F.","family":"Wagner","sequence":"additional","affiliation":[]},{"given":"V.","family":"Wagner","sequence":"additional","affiliation":[]},{"given":"M.","family":"Willers","sequence":"additional","affiliation":[]},{"given":"V.","family":"Zema","sequence":"additional","affiliation":[]},{"given":"C.","family":"Heitzinger","sequence":"additional","affiliation":[]},{"given":"W.","family":"Waltenberger","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,5,22]]},"reference":[{"key":"119_CR1","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1007\/10933596_3","volume-title":"Cryogenic particle detection","author":"K Irwin","year":"2005","unstructured":"Irwin K, Hilton G (2005) Transition-edge sensors. In: Enss C (ed) Cryogenic particle detection. Springer, Berlin, pp 63\u2013150. https:\/\/doi.org\/10.1007\/10933596_3 (ISBN 978-3-540-31478-3)"},{"key":"119_CR2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.100.102002","volume":"100","author":"AH Abdelhameed","year":"2019","unstructured":"Abdelhameed AH et al (2019) First results from the cresst-iii low-mass dark matter program. Phys Rev D 100:102002. https:\/\/doi.org\/10.1103\/PhysRevD.100.102002","journal-title":"Phys Rev D"},{"key":"119_CR3","unstructured":"Cresst homepage. https:\/\/cresst-experiment.org\/. Accessed 10 Apr 2024."},{"key":"119_CR4","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.107.122003","volume":"107","author":"G Angloher","year":"2023","unstructured":"Angloher G et al (2023) Results on sub-gev dark matter from a 10 ev threshold cresst-iii silicon detector. Phys Rev D 107:122003. https:\/\/doi.org\/10.1103\/PhysRevD.107.122003","journal-title":"Phys Rev D"},{"issue":"5","key":"119_CR5","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6633\/ac5754","volume":"85","author":"J Billard","year":"2022","unstructured":"Billard J et al (2022) Direct detection of dark matter\u2014APPEC committee report*. Rep Progr Phys 85(5):056201. https:\/\/doi.org\/10.1088\/1361-6633\/ac5754","journal-title":"Rep Progr Phys"},{"key":"119_CR6","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press. https:\/\/inst.eecs.berkeley.edu\/~cs188\/sp20\/assets\/files\/SuttonBartoIPRLBook2ndEd.pdf"},{"key":"119_CR7","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.24.104601","volume":"24","author":"John J St","year":"2021","unstructured":"St John J et al (2021) Real-time artificial intelligence for accelerator control: a study at the fermilab booster. Phys Rev Accel Beams 24:104601. https:\/\/doi.org\/10.1103\/PhysRevAccelBeams.24.104601","journal-title":"Phys Rev Accel Beams"},{"key":"119_CR8","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevAccelBeams.23.124801","volume":"23","author":"V Kain","year":"2020","unstructured":"Kain V et al (2020) Sample-efficient reinforcement learning for cern accelerator control. Phys Rev Accel Beams 23:124801. https:\/\/doi.org\/10.1103\/PhysRevAccelBeams.23.124801","journal-title":"Phys Rev Accel Beams"},{"issue":"2","key":"119_CR9","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/acce21","volume":"4","author":"FM Velotti","year":"2023","unstructured":"Velotti FM et al (2023) Towards automatic setup of 18 mev electron beamline using machine learning. Mach Learn Sci Technol 4(2):025016. https:\/\/doi.org\/10.1088\/2632-2153\/acce21","journal-title":"Mach Learn Sci Technol"},{"issue":"7897","key":"119_CR10","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1038\/s41586-021-04301-9","volume":"602","author":"J Degrave","year":"2022","unstructured":"Degrave J et al (2022) Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602(7897):414\u2013419. https:\/\/doi.org\/10.1038\/s41586-021-04301-9. (ISSN 1476-4687)","journal-title":"Nature"},{"key":"119_CR11","doi-asserted-by":"publisher","first-page":"215","DOI":"10.22331\/q-2019-12-16-215","volume":"3","author":"HP Nautrup","year":"2019","unstructured":"Nautrup HP et al (2019) Optimizing quantum error correction codes with reinforcement learning. Quantum 3:215. https:\/\/doi.org\/10.22331\/q-2019-12-16-215","journal-title":"Quantum"},{"key":"119_CR12","doi-asserted-by":"publisher","unstructured":"Haarnoja T, et\u00a0al (2018) Soft actor-critic algorithms and applications. ArXiv, 1812.05905. https:\/\/doi.org\/10.48550\/arXiv.1812.05905","DOI":"10.48550\/arXiv.1812.05905"},{"key":"119_CR13","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.106.092008","volume":"106","author":"G Angloher","year":"2022","unstructured":"Angloher G et al (2022) Testing spin-dependent dark matter interactions with lithium aluminate targets in cresst-III. Phys Rev D 106:092008. https:\/\/doi.org\/10.1103\/PhysRevD.106.092008","journal-title":"Phys Rev D"},{"key":"119_CR14","doi-asserted-by":"publisher","unstructured":"Brockman G et al (2016) Openai gym. arXiv:1606.01540. https:\/\/doi.org\/10.48550\/arXiv.1606.01540","DOI":"10.48550\/arXiv.1606.01540"},{"issue":"1","key":"119_CR15","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/BF00753837","volume":"100","author":"F Pr\u00f6bst","year":"1995","unstructured":"Pr\u00f6bst F et al (1995) Model for cryogenic particle detectors with superconducting phase transition thermometers. J Low Temp Phys 100(1):69\u2013104. https:\/\/doi.org\/10.1007\/BF00753837. (ISSN 1573-7357)","journal-title":"J Low Temp Phys"},{"key":"119_CR16","unstructured":"Tinkham M (June 2004) Introduction to superconductivity. Dover Publications, 2 edition. ISBN 0486435032. http:\/\/www.worldcat.org\/isbn\/0486435032"},{"issue":"8","key":"119_CR17","doi-asserted-by":"publisher","first-page":"4856","DOI":"10.1063\/1.1559000","volume":"93","author":"M Galeazzi","year":"2003","unstructured":"Galeazzi M, McCammon D (2003) Microcalorimeter and bolometer model. J Appl Phys 93(8):4856\u20134869. https:\/\/doi.org\/10.1063\/1.1559000. (ISSN 0021-8979)","journal-title":"J Appl Phys"},{"key":"119_CR18","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","volume":"17","author":"P Virtanen","year":"2020","unstructured":"Virtanen P et al (2020) Fundamental algorithms for scientific computing in python SciPy 1.0. Nat Methods 17:261\u2013272. https:\/\/doi.org\/10.1038\/s41592-019-0686-2","journal-title":"Nat Methods"},{"issue":"12","key":"119_CR19","doi-asserted-by":"publisher","first-page":"1982","DOI":"10.1016\/j.cpc.2010.09.003","volume":"181","author":"M Carrettoni","year":"2010","unstructured":"Carrettoni M, Cremonesi O (2010) Generation of noise time series with arbitrary power spectrum. Comput Phys Commun 181(12):1982\u20131985. https:\/\/doi.org\/10.1016\/j.cpc.2010.09.003","journal-title":"Comput Phys Commun"},{"key":"119_CR20","unstructured":"Farama gymnasium homepage. https:\/\/gymnasium.farama.org\/index.html. Accessed 10 Apr 2024"},{"key":"119_CR21","unstructured":"Mqtt homepage. https:\/\/docs.oasis-open.org\/mqtt\/mqtt\/v5.0\/mqtt-v5.0.html. Accessed 10 Apr 2024"},{"key":"119_CR22","doi-asserted-by":"publisher","DOI":"10.21468\/SciPostPhysProc.12.013","author":"G Angloher","year":"2023","unstructured":"Angloher G et al (2023) Latest observations on the low energy excess in CRESST-III. SciPost Phys Proc. https:\/\/doi.org\/10.21468\/SciPostPhysProc.12.013","journal-title":"SciPost Phys Proc"},{"issue":"1","key":"119_CR23","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1140\/epjp\/s13360-023-03674-2","volume":"138","author":"G Angloher","year":"2023","unstructured":"Angloher G et al (2023) Towards an automated data cleaning with deep learning in CRESST. Eur Phys J Plus 138(1):100. https:\/\/doi.org\/10.1140\/epjp\/s13360-023-03674-2. (ISSN 2190-5444)","journal-title":"Eur Phys J Plus"},{"issue":"1","key":"119_CR24","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/s41781-022-00092-4","volume":"6","author":"F Wagner","year":"2022","unstructured":"Wagner F et al (2022) Cait: analysis toolkit for cryogenic particle detectors in python. Compu Softw Big Sci 6(1):19. https:\/\/doi.org\/10.1007\/s41781-022-00092-4. (ISSN 2510-2044)","journal-title":"Compu Softw Big Sci"},{"issue":"1063\/1","key":"119_CR25","first-page":"4812323","volume":"10","author":"A Jain","year":"2013","unstructured":"Jain A et al (2013) Commentary: The Materials Project: a materials genome approach to accelerating materials innovation. APL Mater 10(1063\/1):4812323 (ISSN 2166-532X)","journal-title":"APL Mater"},{"key":"119_CR26","unstructured":"Panti\u0107 E (2008) Performance of Cryogenic Light detectors in the CRESST-II Dark Matter Search. PhD thesis, TU Munich. https:\/\/mediatum.ub.tum.de\/doc\/668368\/668368.pdf"},{"key":"119_CR27","unstructured":"Kittel C (2004) Introduction to solid state physics. Wiley, 8 edn. ISBN 9780471415268. http:\/\/www.amazon.com\/Introduction-Solid-Physics-Charles-Kittel\/dp\/047141526X\/ref=dp_ob_title_bk"},{"key":"119_CR28","doi-asserted-by":"publisher","unstructured":"Paszke A, et\u00a0al (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems 32. pp 8024\u201335. Curran Associates, Inc., URL https:\/\/doi.org\/10.48550\/arXiv.1912.01703","DOI":"10.48550\/arXiv.1912.01703"},{"key":"119_CR29","unstructured":"Kingma D\u00a0P, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings to the 3rd international conference on learning representations. arXiv:1412.6980"},{"issue":"1","key":"119_CR30","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1140\/epjc\/s10052-016-3877-3","volume":"76","author":"G Angloher","year":"2016","unstructured":"Angloher G et al (2016) Results on light dark matter particles with a low-threshold CRESST-II detector. Eur Phys J C 76(1):25. https:\/\/doi.org\/10.1140\/epjc\/s10052-016-3877-3. (ISSN 1434-6052)","journal-title":"Eur Phys J C"},{"key":"119_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.nima.2022.167532","volume":"1045","author":"G Angloher","year":"2023","unstructured":"Angloher G et al (2023) First measurements of remoTES cryogenic calorimeters: easy-to-fabricate particle detectors for a wide choice of target materials. Nucl Instrum Methods Phys Res Sect A Accelerators Spectrometers Detectors Assoc Equip 1045:167532. https:\/\/doi.org\/10.1016\/j.nima.2022.167532. (ISSN 0168-9002)","journal-title":"Nucl Instrum Methods Phys Res Sect A Accelerators Spectrometers Detectors Assoc Equip"}],"container-title":["Computing and Software for Big Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-024-00119-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41781-024-00119-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-024-00119-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,12]],"date-time":"2024-11-12T11:13:24Z","timestamp":1731410004000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41781-024-00119-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,22]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["119"],"URL":"https:\/\/doi.org\/10.1007\/s41781-024-00119-y","relation":{},"ISSN":["2510-2036","2510-2044"],"issn-type":[{"type":"print","value":"2510-2036"},{"type":"electronic","value":"2510-2044"}],"subject":[],"published":{"date-parts":[[2024,5,22]]},"assertion":[{"value":"18 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest. The datasets generated during and\/or analyzed during the current study are available from the corresponding author on reasonable request.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"10"}}