{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:30:10Z","timestamp":1772253010213,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,4]],"date-time":"2021-02-04T00:00:00Z","timestamp":1612396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100011878","name":"Vlaamse regering","doi-asserted-by":"publisher","award":["Onderzoeksprogramma Artifici\u00eble Intelligentie (AI) Vlaanderen"],"award-info":[{"award-number":["Onderzoeksprogramma Artifici\u00eble Intelligentie (AI) Vlaanderen"]}],"id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014419","name":"EIT Health","doi-asserted-by":"publisher","award":["19263 - SeizeIT2: Discreet Personalized Epileptic Seizure Detection Device"],"award-info":[{"award-number":["19263 - SeizeIT2: Discreet Personalized Epileptic Seizure Detection Device"]}],"id":[{"id":"10.13039\/100014419","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wearable technology will become available and allow prolonged electroencephalography (EEG) monitoring in the home environment of patients with epilepsy. Neurologists analyse the EEG visually and annotate all seizures, which patients often under-report. Visual analysis of a 24-h EEG recording typically takes one to two hours. Reliable automated seizure detection algorithms will be crucial to reduce this analysis. We investigated such algorithms on a dataset of behind-the-ear EEG measurements. Our first aim was to develop a methodology where part of the data is deferred to a human expert, who performs perfectly, with the goal of obtaining an (almost) perfect detection sensitivity (DS). Prediction confidences are determined by temperature scaling of the classification model outputs and trust scores. A DS of approximately 90% (99%) can be achieved when deferring around 10% (40%) of the data. Perfect DS can be achieved when deferring 50% of the data. Our second contribution demonstrates that a common modelling strategy, where predictions from several short EEG segments are combined to obtain a final prediction, can be improved by filtering out untrustworthy segments with low trust scores. The false detection rate shows a relative decrease between 21% and 43%, and the DS shows a small increase or decrease.<\/jats:p>","DOI":"10.3390\/s21041046","type":"journal-article","created":{"date-parts":[[2021,2,4]],"date-time":"2021-02-04T03:15:28Z","timestamp":1612408528000},"page":"1046","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Classification with a Deferral Option and Low-Trust Filtering for Automated Seizure Detection"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3432-783X","authenticated-orcid":false,"given":"Thijs","family":"Becker","sequence":"first","affiliation":[{"name":"I-Biostat, Data Science Institute, Hasselt University, 3500 Hasselt, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9888-577X","authenticated-orcid":false,"given":"Kaat","family":"Vandecasteele","sequence":"additional","affiliation":[{"name":"STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, Department of Electrical Engineering (ESAT), KU Leuven, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9054-5340","authenticated-orcid":false,"given":"Christos","family":"Chatzichristos","sequence":"additional","affiliation":[{"name":"STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, Department of Electrical Engineering (ESAT), KU Leuven, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8535-1699","authenticated-orcid":false,"given":"Wim","family":"Van Paesschen","sequence":"additional","affiliation":[{"name":"Department of Neurology, UZ Leuven, 3001 Leuven, Belgium"},{"name":"Laboratory of Epilepsy Research, KU Leuven, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1877-3496","authenticated-orcid":false,"given":"Dirk","family":"Valkenborg","sequence":"additional","affiliation":[{"name":"I-Biostat, Data Science Institute, Hasselt University, 3500 Hasselt, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5939-0996","authenticated-orcid":false,"given":"Sabine","family":"Van Huffel","sequence":"additional","affiliation":[{"name":"STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, Department of Electrical Engineering (ESAT), KU Leuven, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3482-5145","authenticated-orcid":false,"given":"Maarten","family":"De Vos","sequence":"additional","affiliation":[{"name":"STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, Department of Electrical Engineering (ESAT), KU Leuven, 3001 Leuven, Belgium"},{"name":"Department of Development and Regeneration, KU Leuven, 3001 Leuven, Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1212\/WNL.0000000000003509","article-title":"Prevalence and incidence of epilepsy: A systematic review and meta-analysis of international studies","volume":"88","author":"Fiest","year":"2017","journal-title":"Neurology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1111\/j.1528-1167.2007.00992.x","article-title":"Refractory Epilepsy: Clinical Overview","volume":"48","author":"French","year":"2007","journal-title":"Epilepsia"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1016\/S1474-4422(13)70092-9","article-title":"Seizure prediction and documentation\u2014two important problems","volume":"6","author":"Elger","year":"2013","journal-title":"Lancet Neurol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Brunnhuber, F., Slater, J., Goyal, S., Amin, D., Thorvardsson, G., Freestone, D.R., and Richardson, M.P. (2020). Past, Present and Future of Home video-electroencephalographic telemetry: A review of the development of in-home video-electroencephalographic recordings. Epilepsia.","DOI":"10.1111\/epi.16578"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/j.yebeh.2012.04.128","article-title":"Seizure diaries for clinical research and practice: Limitations and future prospects","volume":"24","author":"Fisher","year":"2012","journal-title":"Epilepsy Behav."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1111\/epi.14052","article-title":"Seizure detection using scalp-EEG","volume":"59","author":"Baumgartner","year":"2018","journal-title":"Epilepsia"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Beniczky, S., Karoly, P., Nurse, E., Ryvlin, P., and Cook, M. (2020). Machine learning and wearable devices of the future. Epilepsia.","DOI":"10.1111\/epi.16555"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1111\/epi.14047","article-title":"Multimodal seizure detection: A review","volume":"59","author":"Leijten","year":"2018","journal-title":"Epilepsia"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"De Cooman, T., Varon, C., Van de Vel, A., Ceulemans, B., Lagae, L., and Van Huffel, S. (2018, January 4\u20137). Comparison and combination of electrocardiogram, electromyogram and accelerometry for tonic-clonic seizure detection in children. Proceedings of the 2018 IEEE EMBS International Conference on Biomedical Health Informatics (BHI), Las Vegas, NV, USA.","DOI":"10.1109\/BHI.2018.8333462"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"701","DOI":"10.3389\/fneur.2020.00701","article-title":"Epileptic Seizure Detection and Experimental Treatment: A Review","volume":"11","author":"Kim","year":"2020","journal-title":"Front. Neurol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40708-020-00105-1","article-title":"A review of epileptic seizure detection using machine learning classifiers","volume":"7","author":"Siddiqui","year":"2020","journal-title":"Brain Inform."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shoeibi, A., Ghassemi, N., Khodatars, M., Jafari, M., Hussain, S., Alizadehsani, R., Moridian, P., Khosravi, A., Hosseini-Nejad, H., and Rouhani, M. (2020). Epileptic seizure detection using deep learning techniques: A Review. arXiv.","DOI":"10.3390\/ijerph18115780"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1016\/j.clinph.2013.12.104","article-title":"Automatic seizure detection in long-term scalp EEG using an adaptive thresholding technique: A validation study for clinical routine","volume":"125","author":"Kasper","year":"2014","journal-title":"Clin. Neurophysiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1016\/j.clinph.2014.09.023","article-title":"Prospective multi-center study of an automatic online seizure detection system for epilepsy monitoring units","volume":"126","author":"Ossenblok","year":"2015","journal-title":"Clin. Neurophysiol."},{"key":"ref_15","first-page":"1","article-title":"Automatic seizure detection based on imaged-EEG signals through fully convolutional networks","volume":"10","author":"Navarrete","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_16","first-page":"e1540","article-title":"Automated seizure detection accuracy for ambulatory EEG recordings","volume":"92","author":"Bachman","year":"2019","journal-title":"Neurology"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"83","DOI":"10.3389\/fninf.2018.00083","article-title":"The Temple University Hospital Seizure Detection Corpus","volume":"12","author":"Shah","year":"2018","journal-title":"Front. Neuroinform."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chatzichristos, C., Dan, J., Narayanan, A., Seeuws, N., Vandecasteele, K., De Vos, M., Bertrand, A., and Van Huffel, S. (2020, January 5). Epileptic Seizure Detection in EEG via Fusion of Multi-View Attention-Gated U-net Deep Neural Networks. Proceedings of the IEEE Signal Processing in Medicine and Biology Symposium (SPMB), Philadelphia, PA, USA.","DOI":"10.1109\/SPMB50085.2020.9353630"},{"key":"ref_19","unstructured":"(2020, December 14). SeizeIT1. Available online: https:\/\/www.imec-int.com\/en\/what-we-offer\/research-portfolio\/seizeit."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Boeckx, S., van Paesschen, W., Bonte, B., and Dan, J. (2018, January 17\u201319). Live Demonstration: SeizeIT\u2014A wearable multimodal epileptic seizure detection device. Proceedings of the 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), Cleveland, OH, USA.","DOI":"10.1109\/BIOCAS.2018.8584738"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2454","DOI":"10.1016\/j.clinph.2017.09.115","article-title":"Ear-EEG detects ictal and interictal abnormalities in focal and generalized epilepsy\u2014A comparison with scalp EEG monitoring","volume":"128","author":"Zibrandtsen","year":"2017","journal-title":"Clin. Neurophysiol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gu, Y., Cleeren, E., Dan, J., Claes, K., Van Paesschen, W., Van Huffel, S., and Hunyadi, B. (2018). Comparison between Scalp EEG and Behind-the-Ear EEG for Development of a Wearable Seizure Detection System for Patients with Focal Epilepsy. Sensors, 18.","DOI":"10.3390\/s18010029"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"766","DOI":"10.1111\/epi.16470","article-title":"Visual seizure annotation and automated seizure detection using behind-the-ear electroencephalographic channels","volume":"61","author":"Vandecasteele","year":"2020","journal-title":"Epilepsia"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"105472","DOI":"10.1016\/j.cmpb.2020.105472","article-title":"Unsupervised automatic seizure detection for focal-onset seizures recorded with behind-the-ear EEG using an anomaly-detecting generative adversarial network","volume":"193","author":"You","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Clarke, S., Karoly, P.J., Nurse, E., Seneviratne, U., Taylor, J., Knight-Sadler, R., Kerr, R., Moore, B., Hennessy, P., and Mendis, D. (2019). Computer-assisted EEG diagnostic review for idiopathic generalized epilepsy. Epilepsy Behav., 106556.","DOI":"10.1101\/682112"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/TEC.1957.5222035","article-title":"An optimum character recognition system using decision functions","volume":"EC-6","author":"Chow","year":"1957","journal-title":"IRE Trans. Electron. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/TIT.1970.1054406","article-title":"On optimum recognition error and reject tradeoff","volume":"16","author":"Chow","year":"1970","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","first-page":"1823","article-title":"Classification with a Reject Option using a Hinge Loss","volume":"9","author":"Bartlett","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ortner, R., Simon, H.U., and Zilles, S. (2016). Learning with Rejection. Algorithmic Learning Theory, Springer International Publishing.","DOI":"10.1007\/978-3-319-46379-7"},{"key":"ref_30","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). To Trust Or Not To Trust A Classifier. Advances in Neural Information Processing Systems 31, Curran Associates, Inc."},{"key":"ref_31","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer. Advances in Neural Information Processing Systems 31, Curran Associates, Inc."},{"key":"ref_32","unstructured":"Raghu, M., Blumer, K., Corrado, G., Kleinberg, J., Obermeyer, Z., and Mullainathan, S. (2019). The Algorithmic Automation Problem: Prediction, Triage, and Human Effort. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"De, A., Okati, N., Zarezade, A., and Gomez-Rodriguez, M. (2020). Classification Under Human Assistance. arXiv.","DOI":"10.1609\/aaai.v35i7.16738"},{"key":"ref_34","unstructured":"Mozannar, H., and Sontag, D. (2020). Consistent Estimators for Learning to Defer to an Expert. arXiv."},{"key":"ref_35","first-page":"2619","article-title":"Epilepsyecosystem.org: Crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG","volume":"141","author":"Kuhlmann","year":"2018","journal-title":"Brain"},{"key":"ref_36","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nat. Methods"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"16743","DOI":"10.1038\/srep16743","article-title":"Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear","volume":"5","author":"Debener","year":"2015","journal-title":"Sci. Rep."},{"key":"ref_39","unstructured":"EIT Health (2020, December 14). SeizeIT2. Available online: https:\/\/eithealth.eu\/project\/seizeit2\/."},{"key":"ref_40","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017, January 6\u201311). On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_41","unstructured":"Platt, J.C. (1999). Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. Advances in Large Margin Classifiers, MIT Press."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.2517-6161.1974.tb00994.x","article-title":"Cross-Validatory Choice and Assessment of Statistical Predictions","volume":"36","author":"Stone","year":"1974","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_43","unstructured":"Wainer, J., and Cawley, G. (2018). Nested cross-validation when selecting classifiers is overzealous for most practical applications. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.yebeh.2016.02.039","article-title":"Epileptic seizure onset detection based on EEG and ECG data fusion","volume":"58","author":"Qaraqe","year":"2016","journal-title":"Epilepsy Behav."},{"key":"ref_45","first-page":"5998","article-title":"Attention is All you Need","volume":"Volume 30","author":"Guyon","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_46","unstructured":"Dzeroski, S., and De Raedt, L. (2005, January 7\u201311). Predicting Good Probabilities with Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany."},{"key":"ref_47","unstructured":"Cohen, W., McCallum, A., and Roweis, S. (2008, January 5\u20139). An Empirical Evaluation of Supervised Learning in High Dimensions. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Fabianinkatu."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Rousseau, A.J., Becker, T., Bertels, J., Blaschko, M.B., and Valkenborg, D. (2020). Post Training Uncertainty Calibration of Deep Networks For Medical Image Segmentation. arXiv.","DOI":"10.1109\/ISBI48211.2021.9434131"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Vandecasteele, K., De Cooman, T., Gu, Y., Cleeren, E., Claes, K., Van Paesschen, W., Van Huffel, S., and Hunyadi, B. (2017). Automated Epileptic Seizure Detection Based on Wearable ECG and PPG in a Hospital Environment. Sensors, 17.","DOI":"10.3390\/s17102338"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1046\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:19:50Z","timestamp":1760159990000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1046"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,4]]},"references-count":49,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041046"],"URL":"https:\/\/doi.org\/10.3390\/s21041046","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202012.0527.v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,4]]}}}