{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T09:02:49Z","timestamp":1773824569847,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2025,5,17]],"date-time":"2025-05-17T00:00:00Z","timestamp":1747440000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Arm and by EPSRC","award":["EP\/S023046\/1"],"award-info":[{"award-number":["EP\/S023046\/1"]}]},{"name":"EPSRC Centre for Doctoral Training in Sensor Technologies and Applications"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2025,7,31]]},"abstract":"<jats:p>Machine Learning models typically assume that time series are regularly spaced; however, this is often unrealistic in healthcare, where missing data recordings are common. In this context, uncertainty estimates play a pivotal role, as they can enable confident and non-confident predictions to be distinguished. We propose SQUIREDL, a novel uncertainty-aware sequence-to-sequence prediction method for sparse healthcare time series. Specifically, we enhance the state-of-the-art evidential regression framework, widely used for uncertainty estimation, to handle missing data. Following data imputation with an Akima spline-based method, we modify the loss function of evidential regression by assigning different weights to imputed and observed data points, to offer more reliable uncertainty estimates. Additionally, we examine a variety of metrics for assessing the success of uncertainty estimations on sequence-to-sequence predictions, providing a reliable way to evaluate the models in a medical setting. Our proposal is demonstrated in two clinical applications. In continuous glucose monitoring, we use sequence-to-sequence prediction to obtain the hypoglycaemia risk from glucose sensor readings. Our approach captures the ground truth risk values 30% more accurately, bringing consistent improvements in both uncertainty-aware and accuracy-based metrics. Similarly, in COVID-19 hospital admissions data, we achieve a 22% improvement in the accuracy of uncertainty-aware predictions, enabling better resource planning.<\/jats:p>","DOI":"10.1145\/3723049","type":"journal-article","created":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T14:58:36Z","timestamp":1741705116000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["SQUIREDL: Sparse Sequence-to-Sequence Uncertainty Estimation in Evidential Deep Learning"],"prefix":"10.1145","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-8106-1634","authenticated-orcid":false,"given":"Sotirios","family":"Vavaroutas","sequence":"first","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3806-1493","authenticated-orcid":false,"given":"Ting","family":"Dang","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6450-0878","authenticated-orcid":false,"given":"Emma","family":"Rocheteau","sequence":"additional","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9614-4380","authenticated-orcid":false,"given":"Cecilia","family":"Mascolo","sequence":"additional","affiliation":[{"name":"University of Cambridge, Cambridge, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,5,17]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-29642-0"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/355780.355786"},{"key":"e_1_3_1_4_2","series-title":"Proceedings of Machine Learning Research","first-page":"175","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Alaa Ahmed","year":"2020","unstructured":"Ahmed Alaa and Mihaela Van Der Schaar. 2020. Frequentist uncertainty in recurrent neural networks via blockwise influence functions. In Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, Vol. 119, 175\u2013190. Retrieved from https:\/\/proceedings.mlr.press\/v119\/alaa20b.html"},{"key":"e_1_3_1_5_2","unstructured":"Alexander Amini Wilko Schwarting Ava Soleimany and Daniela Rus. 2019. Deep evidential regression. arXiv:1910.02600. Retrieved from http:\/\/arxiv.org\/abs\/1910.02600"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/math8081374"},{"key":"e_1_3_1_7_2","doi-asserted-by":"crossref","unstructured":"Nicolas Baca\u00ebr. 2011. Lotka Volterra and the predator-prey system (1920\u20131926). In A Short History of Mathematical Population Dynamics 71\u201376. Retrieved from https:\/\/link.springer.com\/chapter\/10.1007\/978-0-85729-115-8_13","DOI":"10.1007\/978-0-85729-115-8_13"},{"key":"e_1_3_1_8_2","unstructured":"Peter A. N. Bosman and Dirk Thierens. 2000. Negative Log-Likelihood and Statistical Hypothesis Testing as the Basis of Model Selection in IDEAs. Technical Report. Retrieved from https:\/\/homepages.cwi.nl\/~bosman\/publications\/2000_negativeloglikelihood.pdf"},{"key":"e_1_3_1_9_2","unstructured":"Margaret Burke. 2022. What Is the Life Expectancy for Type 1 Diabetes? EndocrineWeb. Retrieved from https:\/\/www.endocrineweb.com\/conditions\/type-1-diabetes\/type-1-diabetes-life-expectancy"},{"key":"e_1_3_1_10_2","unstructured":"Ludovic Charleux Emile Roux Thibault Goyallon Giovanni Feverati and Pierre Nagorny. 2021. Python Examples for Lotka-Volterra Equations. Scientific Python. Retrieved from https:\/\/scientific-python.readthedocs.io\/en\/latest\/notebooks_rst\/3_Ordinary_Differential_Equations\/02_Examples\/Lotka_Volterra_model.html"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-24271-9"},{"key":"e_1_3_1_12_2","unstructured":"Tian Qi Chen Yulia Rubanova Jesse Bettencourt and David Duvenaud. 2018. Neural ordinary differential equations. arXiv:1806.07366. Retrieved from http:\/\/arxiv.org\/abs\/1806.07366"},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Jan Chorowski and Navdeep Jaitly. 2016. Towards better decoding and language model integration in sequence to sequence models. arXiv:1612.02695. Retrieved from http:\/\/arxiv.org\/abs\/1612.02695","DOI":"10.21437\/Interspeech.2017-343"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.2337\/dc12-2058"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.3233\/WOR-213458"},{"key":"e_1_3_1_16_2","volume-title":"Bayesian Methods for Nonlinear Classification and Regression","author":"Denison David G. T.","year":"2002","unstructured":"David G. T. Denison, Christopher C. Holmes, Bani K. Mallick, and Adrian F. M. Smith. 2002. Bayesian Methods for Nonlinear Classification and Regression. John Wiley & Sons."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2022.e11648"},{"key":"e_1_3_1_18_2","unstructured":"Leonid Erlygin Vladimir Zholobov Valeriia Baklanova Evgeny Sokolovskiy and Alexey Zaytsev. 2023. Uncertainty estimation for time series forecasting via Gaussian process regression surrogates. arXiv:2302.02834. Retrieved from https:\/\/arxiv.org\/abs\/2302.02834"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-023-10562-9"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40471-018-0136-1"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_22_2","first-page":"6696","volume-title":"Advances in Neural Information Processing Systems","volume":"33","author":"Kidger Patrick","year":"2020","unstructured":"Patrick Kidger, James Morrill, James Foster, and Terry Lyons. 2020. Neural controlled differential equations for irregular time series. In Advances in Neural Information Processing Systems, Vol. 33, 6696\u20136707. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/4a5876b450b45371f6cfe5047ac8cd45-Paper.pdf"},{"key":"e_1_3_1_23_2","first-page":"2796","volume-title":"Proceedings of the 35th International Conference on Machine Learning (PMLR)","volume":"80","author":"Kuleshov Volodymyr","year":"2018","unstructured":"Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. 2018. Accurate uncertainties for deep learning using calibrated regression. In Proceedings of the 35th International Conference on Machine Learning (PMLR), Vol. 80, 2796\u20132804. Retrieved from https:\/\/proceedings.mlr.press\/v80\/kuleshov18a.html"},{"key":"e_1_3_1_24_2","unstructured":"Wesley J. Maddox Timur Garipov Pavel Izmailov Dmitry P. Vetrov and Andrew Gordon Wilson. 2019. A simple baseline for Bayesian uncertainty in deep learning. arXiv:1902.02476. Retrieved from http:\/\/arxiv.org\/abs\/1902.02476"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.o2289"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1177\/1932296813514502"},{"key":"e_1_3_1_27_2","first-page":"15682","volume-title":"Advances in Neural Information Processing Systems","volume":"34","author":"Minderer Matthias","year":"2021","unstructured":"Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. 2021. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems, Vol. 34, 15682\u201315694. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2021\/file\/8420d359404024567b5aefda1231af24-Paper.pdf"},{"key":"e_1_3_1_28_2","unstructured":"Jeremy Nixon Mike Dusenberry Linchuan Zhang Ghassen Jerfel and Dustin Tran. 2019. Measuring calibration in deep learning. arXiv:1904.01685. Retrieved from http:\/\/arxiv.org\/abs\/1904.01685"},{"key":"e_1_3_1_29_2","unstructured":"Office for National Statistics. 2023. Coronavirus (COVID-19) Latest Insights. UK Office for National Statistics. Retrieved January 2023 from https:\/\/www.ons.gov.uk\/peoplepopulationandcommunity\/healthandsocialcare\/conditionsanddiseases\/articles\/coronaviruscovid19\/latestinsights"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Myle Ott Michael Auli David Grangier and Marc\u2019Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. arXiv:1803.00047. Retrieved from http:\/\/arxiv.org\/abs\/1803.00047","DOI":"10.18653\/v1\/W18-6301"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450439.3451860"},{"key":"e_1_3_1_32_2","first-page":"5320","volume-title":"Advances in Neural Information Processing Systems","volume":"32","author":"Rubanova Yulia","year":"2019","unstructured":"Yulia Rubanova, Ricky T. Q. Chen, and David K. Duvenaud. 2019. Latent ordinary differential equations for irregularly-sampled time series. In Advances in Neural Information Processing Systems, Vol. 32, 5320\u20135330. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/42a6845a557bef704ad8ac9cb4461d43-Paper.pdf"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2009.11.009"},{"key":"e_1_3_1_34_2","unstructured":"Murat Sensoy Melih Kandemir and Lance M. Kaplan. 2018. Evidential deep learning to quantify classification uncertainty. arXiv:1806.01768. Retrieved from http:\/\/arxiv.org\/abs\/1806.01768"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1186\/2251-6581-11-17"},{"key":"e_1_3_1_36_2","unstructured":"Satya Narayan Shukla and Benjamin M. Marlin. 2020. A survey on principles models and methods for learning from irregularly sampled time series. arXiv:2012.00168. Retrieved from https:\/\/arxiv.org\/abs\/2012.00168"},{"key":"e_1_3_1_37_2","unstructured":"SKLearn Developers. 2007. API Reference: Paired Euclidean Distances. Scikit-learn. Retrieved from https:\/\/scikit-learn.org\/1.5\/modules\/generated\/sklearn.metrics.pairwise.paired_euclidean_distances.html"},{"key":"e_1_3_1_38_2","unstructured":"SKLearn Developers. 2007. API Reference: SKLearn Metrics. Scikit-learn. Retrieved from https:\/\/scikit-learn.org\/stable\/api\/sklearn.metrics.html"},{"key":"e_1_3_1_39_2","unstructured":"SKLearn Developers. 2010. API Reference: Explained Variance Score. Scikit-learn. Retrieved from https:\/\/scikit-learn.org\/0.17\/modules\/generated\/sklearn.metrics.explained_variance_score.html"},{"key":"e_1_3_1_40_2","unstructured":"Ilya Sutskever Oriol Vinyals and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. arXiv:1409.3215. Retrieved from http:\/\/arxiv.org\/abs\/1409.3215"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13063-021-05284-8"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3025813"},{"key":"e_1_3_1_43_2","unstructured":"The SciPy Community. 2019. Akima1DInterpolator. Retrieved from https:\/\/docs.scipy.org\/doc\/scipy\/reference\/generated\/scipy.interpolate.Akima1DInterpolator.html"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1089\/dia.2013.0377"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22208016"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2020.3042637"},{"key":"e_1_3_1_47_2","unstructured":"Dongxia Wu Liyao Gao Xinyue Xiong Matteo Chinazzi Alessandro Vespignani Yi-An Ma and Rose Yu. 2021. Quantifying uncertainty in deep spatiotemporal forecasting. arXiv:2105.11982. Retrieved from https:\/\/arxiv.org\/abs\/2105.11982"},{"key":"e_1_3_1_48_2","volume-title":"Proceedings of the Workshop on Trustworthy and Socially Responsible ML (NeurIPS \u201922)","author":"Xia Tong","year":"2022","unstructured":"Tong Xia, Jing Han, Lorena Qendro, Ting Dang, and Cecilia Mascolo. 2022. Hybrid-EDL: Improving evidential deep learning for uncertainty quantification on imbalanced data. In Proceedings of the Workshop on Trustworthy and Socially Responsible ML (NeurIPS \u201922). Retrieved from https:\/\/openreview.net\/pdf?id=Nr1RDmAx-Qm"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-444-53858-1.00009-0"},{"key":"e_1_3_1_50_2","unstructured":"Jinyu Xie. 2018. Simglucose. Python Package. Retrieved from https:\/\/github.com\/jxx123\/simglucose"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3723049","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3723049","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:56:42Z","timestamp":1750298202000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3723049"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,17]]},"references-count":49,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,7,31]]}},"alternative-id":["10.1145\/3723049"],"URL":"https:\/\/doi.org\/10.1145\/3723049","relation":{},"ISSN":["2637-8051"],"issn-type":[{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,17]]},"assertion":[{"value":"2024-07-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}