{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:43:36Z","timestamp":1787017416843,"version":"3.56.0"},"publisher-location":"New York, NY, USA","reference-count":39,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T00:00:00Z","timestamp":1597881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Institute of Health award","award":["NIH R01 1R01NS107291-01 and R56HL138415"],"award-info":[{"award-number":["NIH R01 1R01NS107291-01 and R56HL138415"]}]},{"name":"National Science Foundation award","award":["IIS-1418511 CCF-1533768 and IIS-1838042"],"award-info":[{"award-number":["IIS-1418511 CCF-1533768 and IIS-1838042"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,8,23]]},"DOI":"10.1145\/3394486.3403212","type":"proceedings-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T19:03:57Z","timestamp":1597950237000},"page":"1614-1624","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":79,"title":["HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units"],"prefix":"10.1145","author":[{"given":"Shenda","family":"Hong","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanbo","family":"Xu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alind","family":"Khare","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Satria","family":"Priambada","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"Maher","sequence":"additional","affiliation":[{"name":"Childrens Healthcare of Atlanta, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alaa","family":"Aljiffry","sequence":"additional","affiliation":[{"name":"Childrens Healthcare of Atlanta, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jimeng","family":"Sun","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Champaign, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexey","family":"Tumanov","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,8,20]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"What's new in ICU in 2050: big data and machine learning. Intensive care medicine 44, 9","author":"Bailly S\u00e9bastien","year":"2018","unstructured":"S\u00e9bastien Bailly , Geert Meyfroidt , and Jean-Fran\u00e7ois Timsit . 2018. What's new in ICU in 2050: big data and machine learning. Intensive care medicine 44, 9 ( 2018 ),1524--1527. S\u00e9bastien Bailly, Geert Meyfroidt, and Jean-Fran\u00e7ois Timsit. 2018. What's new in ICU in 2050: big data and machine learning. Intensive care medicine 44, 9 (2018),1524--1527."},{"key":"e_1_3_2_1_2_1","first-page":"281","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio . 2012 . Random search for hyper-parameter optimization . Journal of machine learning research 13 , Feb (2012), 281 -- 305 . James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of machine learning research 13, Feb (2012), 281--305.","journal-title":"Journal of machine learning research 13"},{"key":"e_1_3_2_1_3_1","unstructured":"James Bergstra Daniel Yamins and David Daniel Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. JMLR(2013).  James Bergstra Daniel Yamins and David Daniel Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. JMLR(2013)."},{"key":"e_1_3_2_1_4_1","unstructured":"James S Bergstra R\u00e9mi Bardenet Yoshua Bengio and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. In Advances in neural information processing systems. 2546--2554.  James S Bergstra R\u00e9mi Bardenet Yoshua Bengio and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. In Advances in neural information processing systems. 2546--2554."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018054314350"},{"key":"e_1_3_2_1_6_1","volume-title":"Random forests. Machine learning 45, 1","author":"Breiman Leo","year":"2001","unstructured":"Leo Breiman . 2001. Random forests. Machine learning 45, 1 ( 2001 ), 5--32. Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5--32."},{"key":"e_1_3_2_1_7_1","volume-title":"Proxylessnas: Direct neural architecture search on target task and hardware.arXiv:1812.00332(2018).","author":"Cai Han","year":"2018","unstructured":"Han Cai , Ligeng Zhu , and Song Han . 2018 . Proxylessnas: Direct neural architecture search on target task and hardware.arXiv:1812.00332(2018). Han Cai, Ligeng Zhu, and Song Han. 2018. Proxylessnas: Direct neural architecture search on target task and hardware.arXiv:1812.00332(2018)."},{"key":"e_1_3_2_1_8_1","volume-title":"Closing the data loop.American journal of respiratory and critical care medicine 187, 11","author":"Celi Leo Anthony","year":"2013","unstructured":"Leo Anthony Celi , Roger G Mark , David J Stone , and Robert A Montgomery . 2013. \" Big data\" in the intensive care unit. Closing the data loop.American journal of respiratory and critical care medicine 187, 11 ( 2013 ), 1157. Leo Anthony Celi, Roger G Mark, David J Stone, and Robert A Montgomery. 2013. \"Big data\" in the intensive care unit. Closing the data loop.American journal of respiratory and critical care medicine 187, 11 (2013), 1157."},{"key":"e_1_3_2_1_9_1","volume-title":"Clipper: A Low-Latency Online Prediction Serving System. CoRRabs\/1612.03079","author":"Crankshaw Daniel","year":"2016","unstructured":"Daniel Crankshaw , Xin Wang , Giulio Zhou , Michael J. Franklin , Joseph E. Gonzalez , and Ion Stoica . 2016 . Clipper: A Low-Latency Online Prediction Serving System. CoRRabs\/1612.03079 (2016). arXiv:1612.03079 http:\/\/arxiv.org\/abs\/1612.03079 Daniel Crankshaw, Xin Wang, Giulio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. 2016. Clipper: A Low-Latency Online Prediction Serving System. CoRRabs\/1612.03079 (2016). arXiv:1612.03079 http:\/\/arxiv.org\/abs\/1612.03079"},{"key":"e_1_3_2_1_10_1","volume-title":"Jan Hendrik Metzen, and Frank Hutter","author":"Elsken Thomas","year":"2018","unstructured":"Thomas Elsken , Jan Hendrik Metzen, and Frank Hutter . 2018 . Neural architecture search: A survey. arXiv preprint arXiv:1808.05377(2018). Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2018. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377(2018)."},{"key":"e_1_3_2_1_11_1","volume-title":"Patterns of daily costs differ for medical and surgical intensive care unit patients.Annals of the American Thoracic Society 12, 12","author":"Gershengorn Hayley B","year":"2015","unstructured":"Hayley B Gershengorn , Allan Garland , and Michelle N Gong . 2015. Patterns of daily costs differ for medical and surgical intensive care unit patients.Annals of the American Thoracic Society 12, 12 ( 2015 ), 1831--1836. Hayley B Gershengorn, Allan Garland, and Michelle N Gong. 2015. Patterns of daily costs differ for medical and surgical intensive care unit patients.Annals of the American Thoracic Society 12, 12 (2015), 1831--1836."},{"key":"e_1_3_2_1_12_1","volume-title":"Webster","author":"Gulshan Varun","year":"2016","unstructured":"Varun Gulshan , Lily Peng , Marc Coram , Martin C. Stumpe , Derek Wu , Arunachalam Narayanaswamy , Subhashini Venugopalan , Kasumi Widner , Tom Madams , Jorge Cuadros , Ramasamy Kim , Rajiv Raman , Philip C. Nelson , Jessica L. Mega ,and Dale R . Webster . 2016 . Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316, 22 (12 2016), 2402--2410. https:\/\/doi.org\/10.1001\/jama.2016.17216arXiv:https:\/\/jamanetwork.com\/journals\/jama\/articlepdf\/2588763\/joi160132.pdf 10.1001\/jama.2016.17216arXiv:https: Varun Gulshan, Lily Peng, Marc Coram, Martin C. Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams,Jorge Cuadros, Ramasamy Kim, Rajiv Raman, Philip C. Nelson, Jessica L. Mega,and Dale R. Webster. 2016. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316, 22 (12 2016), 2402--2410. https:\/\/doi.org\/10.1001\/jama.2016.17216arXiv:https:\/\/jamanetwork.com\/journals\/jama\/articlepdf\/2588763\/joi160132.pdf"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1097\/CCM.0000000000001227"},{"key":"e_1_3_2_1_14_1","volume-title":"Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms usinga deep neural network. Nature medicine 25, 1","author":"Hannun Awni Y","year":"2019","unstructured":"Awni Y Hannun , Pranav Rajpurkar , Masoumeh Haghpanahi , Geoffrey H Tison , Codie Bourn , Mintu P Turakhia , and Andrew Y Ng. 2019. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms usinga deep neural network. Nature medicine 25, 1 ( 2019 ), 65. Awni Y Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H Tison, Codie Bourn, Mintu P Turakhia, and Andrew Y Ng. 2019. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms usinga deep neural network. Nature medicine 25, 1 (2019), 65."},{"key":"e_1_3_2_1_15_1","volume-title":"Greg Ver Steeg, and Aram Galstyan","author":"Harutyunyan Hrayr","year":"2017","unstructured":"Hrayr Harutyunyan , Hrant Khachatrian , David C Kale , Greg Ver Steeg, and Aram Galstyan . 2017 . Multitask learning and benchmarking with clinical time series data. arXiv preprint arXiv:1703.07771(2017). Hrayr Harutyunyan, Hrant Khachatrian, David C Kale, Greg Ver Steeg, and Aram Galstyan. 2017. Multitask learning and benchmarking with clinical time series data. arXiv preprint arXiv:1703.07771(2017)."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Shenda Hong Yuxi Zhou Junyuan Shang Cao Xiao and Jimeng Sun. 2020. Opportunities and challenges of deep learning methods for electrocardiogram data: A systematic review. Computers in Biology and Medicine(2020) 103801.  Shenda Hong Yuxi Zhou Junyuan Shang Cao Xiao and Jimeng Sun. 2020. Opportunities and challenges of deep learning methods for electrocardiogram data: A systematic review. Computers in Biology and Medicine(2020) 103801.","DOI":"10.1016\/j.compbiomed.2020.103801"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"e_1_3_2_1_18_1","volume-title":"Surrogate-based modeling and optimization","author":"Koziel Slawomir","unstructured":"Slawomir Koziel and Leifur Leifsson . 2013. Surrogate-based modeling and optimization . Springer . Slawomir Koziel and Leifur Leifsson. 2013. Surrogate-based modeling and optimization. Springer."},{"key":"e_1_3_2_1_19_1","volume-title":"Deep learning. nature 521, 7553","author":"LeCun Yann","year":"2015","unstructured":"Yann LeCun , Yoshua Bengio , and Geoffrey Hinton . 2015. Deep learning. nature 521, 7553 ( 2015 ), 436. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.5555\/3020948.3020994"},{"key":"e_1_3_2_1_21_1","unstructured":"Zachary C Lipton David C Kale Charles Elkan and Randall Wetzel. 2016. Learning to diagnose with LSTM recurrent neural networks. ICLR.  Zachary C Lipton David C Kale Charles Elkan and Randall Wetzel. 2016. Learning to diagnose with LSTM recurrent neural networks. ICLR."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Chenxi Liu Barret Zoph Maxim Neumann Jonathon Shlens Wei Hua Li-Jia Li Li Fei-Fei Alan Yuille Jonathan Huang and Kevin Murphy. 2018. Progressive neural architecture search. In ECCV. 19--34.  Chenxi Liu Barret Zoph Maxim Neumann Jonathon Shlens Wei Hua Li-Jia Li Li Fei-Fei Alan Yuille Jonathan Huang and Kevin Murphy. 2018. Progressive neural architecture search. In ECCV. 19--34.","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"e_1_3_2_1_23_1","unstructured":"Xiangrui Meng Joseph Bradley Burak Yavuz Evan Sparks Shivaram Venkataraman Davies Liu Jeremy Freeman DB Tsai Manish Amde Sean Owen Doris Xin Reynold Xin Michael J. Franklin Reza Zadeh Matei Zaharia and Ameet Talwalkar. 2015. MLlib: Machine Learning in Apache Spark. arXiv:1505.06807 [cs.LG]  Xiangrui Meng Joseph Bradley Burak Yavuz Evan Sparks Shivaram Venkataraman Davies Liu Jeremy Freeman DB Tsai Manish Amde Sean Owen Doris Xin Reynold Xin Michael J. Franklin Reza Zadeh Matei Zaharia and Ameet Talwalkar. 2015. MLlib: Machine Learning in Apache Spark. arXiv:1505.06807 [cs.LG]"},{"key":"e_1_3_2_1_24_1","volume-title":"theory and applications","author":"Mockus Jonas","unstructured":"Jonas Mockus . 2012. Bayesian approach to global optimization : theory and applications . Vol. 37 . Springer Science & Business Media . Jonas Mockus. 2012.Bayesian approach to global optimization: theory and applications. Vol. 37. Springer Science & Business Media."},{"key":"e_1_3_2_1_25_1","volume-title":"13th USENIX Symposium on Operating Systems Design and Implementation (OSDI'18)","author":"Moritz Philipp","year":"2018","unstructured":"Philipp Moritz , Robert Nishihara , Stephanie Wang , Alexey Tumanov , RichardLiaw, Eric Liang , Melih Elibol , Zongheng Yang , William Paul , Michael I Jordan , 2018 . Ray: A distributed framework for emerging AI applications . In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI'18) . 561--577. Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, RichardLiaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan,et al. 2018. Ray: A distributed framework for emerging AI applications. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI'18). 561--577."},{"key":"e_1_3_2_1_26_1","unstructured":"Phuoc Nguyen Truyen Tran and Svetha Venkatesh. 2017. Deep learning to attend to risk in ICU. arXiv preprint arXiv: 1707.05010(2017).  Phuoc Nguyen Truyen Tran and Svetha Venkatesh. 2017. Deep learning to attend to risk in ICU. arXiv preprint arXiv: 1707.05010(2017)."},{"key":"e_1_3_2_1_27_1","volume-title":"Tensorflow-serving: Flexible, high-performance ml serving. arXiv:1712.06139(2017).","author":"Olston Christopher","year":"2017","unstructured":"Christopher Olston , Noah Fiedel , Kiril Gorovoy , Jeremiah Harmsen , Li Lao , Fang-wei Li, Vinu Rajashekhar , Sukriti Ramesh , and Jordan Soyke . 2017 . Tensorflow-serving: Flexible, high-performance ml serving. arXiv:1712.06139(2017). Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fang-wei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. 2017. Tensorflow-serving: Flexible, high-performance ml serving. arXiv:1712.06139(2017)."},{"key":"e_1_3_2_1_28_1","unstructured":"Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen Zeming Lin Natalia Gimelshein Luca Antiga etal 2019. PyTorch: An imperative style high-performance deep learning library. In Advances in Neural Information Processing Systems. 8024--8035.  Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen Zeming Lin Natalia Gimelshein Luca Antiga et al. 2019. PyTorch: An imperative style high-performance deep learning library. In Advances in Neural Information Processing Systems. 8024--8035."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-018-0029-1"},{"key":"e_1_3_2_1_30_1","volume-title":"Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225(2017).","author":"Rajpurkar Pranav","year":"2017","unstructured":"Pranav Rajpurkar , Jeremy Irvin , Kaylie Zhu , Brandon Yang , Hershel Mehta , Tony Duan , Daisy Ding , Aarti Bagul , Curtis Langlotz , Katie Shpanskaya , 2017 . Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225(2017). Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, et al. 2017. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225(2017)."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2494218"},{"key":"e_1_3_2_1_32_1","unstructured":"Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951--2959.  Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951--2959."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219961"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Darrell Whitley. 1994. A genetic algorithm tutorial.Statistics and computing 4 2(1994) 65--85.  Darrell Whitley. 1994. A genetic algorithm tutorial.Statistics and computing 4 2(1994) 65--85.","DOI":"10.1007\/BF00175354"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocy068"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220051"},{"key":"e_1_3_2_1_38_1","volume-title":"Ensemble methods: foundations and algorithms","author":"Zhou Zhi-Hua","unstructured":"Zhi-Hua Zhou . 2012. Ensemble methods: foundations and algorithms . Chapman and Hall\/CRC. Zhi-Hua Zhou. 2012. Ensemble methods: foundations and algorithms. Chapman and Hall\/CRC."},{"key":"e_1_3_2_1_39_1","unstructured":"Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578(2016).  Barret Zoph and Quoc V Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578(2016)."}],"event":{"name":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event CA USA","acronym":"KDD '20","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403212","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394486.3403212","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:01:46Z","timestamp":1750183306000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,20]]},"references-count":39,"alternative-id":["10.1145\/3394486.3403212","10.1145\/3394486"],"URL":"https:\/\/doi.org\/10.1145\/3394486.3403212","relation":{},"subject":[],"published":{"date-parts":[[2020,8,20]]},"assertion":[{"value":"2020-08-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}