{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T01:47:44Z","timestamp":1776822464612,"version":"3.51.2"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007437","name":"TrygFonden","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100007437","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003491","name":"Herlev Hospital","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003491","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001734","name":"K\u00f8benhavns Universitet","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001734","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012774","name":"Innovationsfonden","doi-asserted-by":"publisher","award":["0153-00167B"],"award-info":[{"award-number":["0153-00167B"]}],"id":[{"id":"10.13039\/100012774","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012774","name":"Innovationsfonden","doi-asserted-by":"publisher","award":["2051-00015B"],"award-info":[{"award-number":["2051-00015B"]}],"id":[{"id":"10.13039\/100012774","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100009708","name":"Novo Nordisk Fonden","doi-asserted-by":"publisher","award":["NNF18OC0031840"],"award-info":[{"award-number":["NNF18OC0031840"]}],"id":[{"id":"10.13039\/501100009708","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Advanced stroke treatment is time-dependent and, therefore, relies on recognition by call-takers at prehospital telehealth services to ensure fast hospitalisation. This study aims to develop and assess the potential of machine learning in improving prehospital stroke recognition during medical helpline calls. We used calls from 1 January 2015 to 31 December 2020 in Copenhagen to develop a machine learning-based classification pipeline. Calls from 2021 are used for testing. Calls are first transcribed using an automatic speech recognition model and then categorised as stroke or non-stroke using a text classification model. Call-takers achieve a sensitivity of 52.7% (95% confidence interval 49.2\u201356.4%) with a positive predictive value (PPV) of 17.1% (15.5\u201318.6%). The machine learning framework performs significantly better (<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.0001) with a sensitivity of 63.0% (62.0\u201364.1%) and a PPV of 24.9% (24.3\u201325.5%). Thus, a machine learning framework for recognising stroke in prehospital medical helpline calls may become a supportive tool for call-takers, aiding in early and accurate stroke recognition.<\/jats:p>","DOI":"10.1038\/s41746-023-00980-y","type":"journal-article","created":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T13:02:02Z","timestamp":1702990922000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["A retrospective study on machine learning-assisted stroke recognition for medical helpline calls"],"prefix":"10.1038","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9604-4638","authenticated-orcid":false,"given":"Jonathan","family":"Wenstrup","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4849-0817","authenticated-orcid":false,"given":"Jakob Drachmann","family":"Havtorn","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3562-8442","authenticated-orcid":false,"given":"Lasse","family":"Borgholt","sequence":"additional","affiliation":[]},{"given":"Stig Nikolaj","family":"Blomberg","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2030-520X","authenticated-orcid":false,"given":"Lars","family":"Maaloe","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0322-3181","authenticated-orcid":false,"given":"Michael R.","family":"Sayre","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7472-3194","authenticated-orcid":false,"given":"Hanne","family":"Christensen","sequence":"additional","affiliation":[]},{"given":"Christina","family":"Kruuse","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3302-7149","authenticated-orcid":false,"given":"Helle Collatz","family":"Christensen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,12,19]]},"reference":[{"key":"980_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S1474-4422(21)00252-0","volume":"20","author":"VL Feigin","year":"2021","unstructured":"Feigin, V. L. et al. Global, regional, and national burden of stroke and its risk factors, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Neurol. 20, 1\u201326 (2021).","journal-title":"Lancet Neurol."},{"key":"980_CR2","doi-asserted-by":"publisher","first-page":"1859","DOI":"10.1016\/S0140-6736(18)32335-3","volume":"392","author":"HH Kyu","year":"2018","unstructured":"Kyu, H. H. et al. Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 392, 1859\u20131922 (2018).","journal-title":"Lancet"},{"key":"980_CR3","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1055\/s-0038-1649503","volume":"38","author":"M Katan","year":"2018","unstructured":"Katan, M. & Luft, A. Global burden of stroke. Semin. Neurol. 38, 208\u2013211 (2018).","journal-title":"Semin. Neurol."},{"key":"980_CR4","doi-asserted-by":"publisher","first-page":"I","DOI":"10.1177\/2396987321989865","volume":"6","author":"E Berge","year":"2021","unstructured":"Berge, E. et al. European Stroke Organisation (ESO) guidelines on intravenous thrombolysis for acute ischaemic stroke. Eur. Stroke J. 6, I\u2013LXII (2021).","journal-title":"Eur. Stroke J."},{"key":"980_CR5","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1136\/neurintsurg-2018-014568","volume":"11","author":"G Turc","year":"2019","unstructured":"Turc, G. et al. European Stroke Organisation (ESO) - European Society for Minimally Invasive Neurological Therapy (ESMINT) guidelines on mechanical thrombectomy in acute ischemic stroke. J. Neurointerv. Surg. 11, 535\u2013538 (2019).","journal-title":"J. Neurointerv. Surg."},{"key":"980_CR6","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1007\/s11910-022-01170-1","volume":"22","author":"P Hariharan","year":"2022","unstructured":"Hariharan, P. et al. Mobile stroke units: current evidence and impact. Curr. Neurol. Neurosci. Rep. 22, 71\u201381 (2022).","journal-title":"Curr. Neurol. Neurosci. Rep."},{"key":"980_CR7","doi-asserted-by":"publisher","first-page":"2103","DOI":"10.1161\/STROKEAHA.121.037376","volume":"53","author":"BB Navi","year":"2022","unstructured":"Navi, B. B. et al. Mobile stroke units: evidence, gaps, and next steps. Stroke 53, 2103\u20132113 (2022).","journal-title":"Stroke"},{"key":"980_CR8","doi-asserted-by":"publisher","first-page":"776","DOI":"10.1161\/STROKEAHA.111.634980","volume":"43","author":"S Krebes","year":"2012","unstructured":"Krebes, S. et al. Development and validation of a dispatcher identification algorithm for stroke emergencies. Stroke 43, 776\u2013781 (2012).","journal-title":"Stroke"},{"key":"980_CR9","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1136\/emermed-2011-200150","volume":"29","author":"P Govindarajan","year":"2012","unstructured":"Govindarajan, P. et al. Feasibility study to assess the use of the Cincinnati stroke scale by emergency medical dispatchers: a pilot study. Emerg. Med. J. 29, 848\u2013850 (2012).","journal-title":"Emerg. Med. J."},{"key":"980_CR10","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1159\/000447459","volume":"42","author":"JA Oostema","year":"2016","unstructured":"Oostema, J. A. et al. Dispatcher stroke recognition using a stroke screening tool: a systematic review. Cerebrovasc. Dis. 42, 370\u2013377 (2016).","journal-title":"Cerebrovasc. Dis."},{"key":"980_CR11","doi-asserted-by":"publisher","unstructured":"Viereck S. et al. Medical dispatchers recognise substantial amount of acute stroke during emergency calls. Scand. J. Trauma Resusc. Emerg. Med. https:\/\/doi.org\/10.1186\/S13049-016-0277-5 (2016).","DOI":"10.1186\/S13049-016-0277-5"},{"key":"980_CR12","doi-asserted-by":"publisher","unstructured":"Bohm K., Kurland L. The accuracy of medical dispatch - a systematic review. Scand. J. Trauma Resusc. Emerg. Med. https:\/\/doi.org\/10.1186\/S13049-018-0528-8 (2018).","DOI":"10.1186\/S13049-018-0528-8"},{"key":"980_CR13","doi-asserted-by":"crossref","unstructured":"Watkins C. L., et al. Training emergency services\u2019 dispatchers to recognise stroke: an interrupted time-series analysis. BMC Health Serv. Res. 13, 318 (2013)","DOI":"10.1186\/1472-6963-13-318"},{"key":"980_CR14","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1016\/j.resuscitation.2019.01.015","volume":"138","author":"SN Blomberg","year":"2019","unstructured":"Blomberg, S. N. et al. Machine learning as a supportive tool to recognize cardiac arrest in emergency calls. Resuscitation 138, 322\u2013329 (2019).","journal-title":"Resuscitation"},{"key":"980_CR15","doi-asserted-by":"crossref","unstructured":"Blomberg S. N., et al. Effect of machine learning on dispatcher recognition of out-of-hospital cardiac arrest during calls to emergency medical services: a randomized clinical trial. JAMA Netw. Open. 4, e2032320 (2021).","DOI":"10.1001\/jamanetworkopen.2020.32320"},{"key":"980_CR16","doi-asserted-by":"crossref","unstructured":"Carcel C. et al. Sex matters in stroke: a review of recent evidence on the differences between women and men. Front. Neuroendocrinol. 59, 100870 (2020).","DOI":"10.1016\/j.yfrne.2020.100870"},{"key":"980_CR17","doi-asserted-by":"publisher","unstructured":"Eddelien H. S. et al. Sex and age differences in patient-reported acute stroke symptoms. Front. Neurol. https:\/\/doi.org\/10.3389\/FNEUR.2022.846690 (2022).","DOI":"10.3389\/FNEUR.2022.846690"},{"key":"980_CR18","doi-asserted-by":"crossref","unstructured":"Han K. J. et al. Deep learning-based telephony speech recognition in the wild. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH https:\/\/www.isca-speech.org\/archive\/pdfs\/interspeech_2017\/han17_interspeech.pdf 1323\u20131327 (2017).","DOI":"10.21437\/Interspeech.2017-1695"},{"key":"980_CR19","unstructured":"A. Radford, et al. Robust speech recognition via large-scale weak supervision. In: Proceedings of the 40th International Conference on Machine Learning. 28492\u201328518 (2023)."},{"key":"980_CR20","unstructured":"Hvingelby, R. et al. DaNE: a named entity resource for Danish. In: Proceedings of the LREC 2022 Workshop of the 1st Annual Meeting of the ELRA\/ISCA Special Interest Group on Under-Resourced Languages. 4597\u20134604 (2020)."},{"key":"980_CR21","unstructured":"Mohamed, A. et al. Self-supervised speech representation learning: a review. IEEE J Sel Top Signal Process https:\/\/arxiv.org\/abs\/2205.10643 (2022)."},{"key":"980_CR22","doi-asserted-by":"crossref","unstructured":"Gururangan, S. et al. Don\u2019t stop pretraining: adapt language models to domains and tasks. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 8342\u20138360 (2020).","DOI":"10.18653\/v1\/2020.acl-main.740"},{"key":"980_CR23","first-page":"4038","volume":"1","author":"L Galke","year":"2022","unstructured":"Galke, L. et al. Bag-of-words vs. graph vs. sequence in text classification: questioning the necessity of text-graphs and the surprising strength of a wide MLP. Assoc. Comput. Linguist. 1, 4038\u20134051 (2022).","journal-title":"Assoc. Comput. Linguist."},{"key":"980_CR24","unstructured":"Danmarks Statistik (Statistics Denmark) (2023). FOLK1: population quarterly database. https:\/\/www.statistikbanken.dk\/FOLK1A (2023)."},{"key":"980_CR25","doi-asserted-by":"publisher","unstructured":"Zinger N. D. et al. Impact of integrating out-of-hours services into Emergency Medical Services Copenhagen: a descriptive study of transformational years. Int. J. Emerg. Med. https:\/\/doi.org\/10.1186\/S12245-022-00442-4 (2022).","DOI":"10.1186\/S12245-022-00442-4"},{"key":"980_CR26","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1007\/s10654-014-9930-3","volume":"29","author":"M Schmidt","year":"2014","unstructured":"Schmidt, M., Pedersen, L. & S\u00f8rensen, H. T. The Danish Civil Registration System as a tool in epidemiology. Eur. J. Epidemiol. 29, 541\u2013549 (2014).","journal-title":"Eur. J. Epidemiol."},{"key":"980_CR27","doi-asserted-by":"publisher","first-page":"697","DOI":"10.2147\/CLEP.S103662","volume":"8","author":"S Johnsen","year":"2016","unstructured":"Johnsen, S. et al. The Danish Stroke Registry. Clin. Epidemiol. 8, 697\u2013702 (2016).","journal-title":"Clin. Epidemiol."},{"key":"980_CR28","unstructured":"Blauenfeldt, R., Wienecke T. National Neurologisk Behandlingsvejledning: Isk\u00e6misk apopleksi - akut udredning og behandling, https:\/\/nnbv.dk\/iskaemisk-apopleksi-akut-udredning-og-behandling\/ (accessed 19 September 2023)."},{"key":"980_CR29","doi-asserted-by":"publisher","first-page":"27","DOI":"10.2147\/CLEP.S50449","volume":"6","author":"C Wildenschild","year":"2013","unstructured":"Wildenschild, C. et al. Registration of acute stroke: validity in the Danish Stroke Registry and the Danish National Registry of Patients. Clin. Epidemiol. 6, 27\u201336 (2013).","journal-title":"Clin. Epidemiol."},{"key":"980_CR30","unstructured":"Dansk Indeks for Akuthj\u00e6lp. Landsudgaven, version 1.10\u2014revideret april 2022., https:\/\/www.ph.rm.dk\/siteassets\/prahospitalet\/fagfolk\/dansk-indeks\/dansk%0A-indeks-version-1.10---landsudgaven.pdf (accessed 27 March 2023)."},{"key":"980_CR31","doi-asserted-by":"crossref","unstructured":"Borgholt L. et al. Do end-to-end speech recognition models care about context? In: Proceedings of the Annual Conference of the International Speech Communication, 4352\u20134356 https:\/\/arxiv.org\/abs\/2102.09928 (2020).","DOI":"10.21437\/Interspeech.2020-1750"},{"key":"980_CR32","unstructured":"Lecun Y., Bengio Y. Convolutional networks for images, speech, and time-series, https:\/\/nyuscholars.nyu.edu\/en\/publications\/convolutional-networks-for-images-speech-and-time-series (1995, accessed 20 April 2023)."},{"key":"980_CR33","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput. 9, 1735\u20131780 (1997).","journal-title":"Neural Comput."},{"key":"980_CR34","doi-asserted-by":"crossref","unstructured":"Graves A. et al. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. 369\u2013376 (2006).","DOI":"10.1145\/1143844.1143891"},{"key":"980_CR35","doi-asserted-by":"publisher","first-page":"993","DOI":"10.1109\/34.58871","volume":"12","author":"LK Hansen","year":"1990","unstructured":"Hansen, L. K. & Salamon, P. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12, 993\u20131001 (1990).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"980_CR36","doi-asserted-by":"publisher","first-page":"386","DOI":"10.1037\/h0042519","volume":"65","author":"F Rosenblatt","year":"1958","unstructured":"Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386\u2013408 (1958).","journal-title":"Psychol. Rev."},{"key":"980_CR37","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1214\/aoms\/1177707045","volume":"28","author":"M Dwass","year":"1957","unstructured":"Dwass, M. Modified randomization tests for nonparametric hypotheses. Ann. Math. Stat. 28, 181\u2013187 (1957).","journal-title":"Ann. Math. Stat."},{"key":"980_CR38","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1017\/S0021859600052862","volume":"23","author":"T Eden","year":"1933","unstructured":"Eden, T. & Yates, F. On the validity of Fisher\u2019s z test when applied to an actual example of non-normal data. (With five text-figures.). J. Agric. Sci. 23, 6\u201317 (1933).","journal-title":"J. Agric. Sci."},{"key":"980_CR39","unstructured":"Danish Data Protection Agency. https:\/\/www.datatilsynet.dk\/english (accessed 14 November 2023)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00980-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00980-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00980-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T01:02:15Z","timestamp":1703034135000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-023-00980-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["980"],"URL":"https:\/\/doi.org\/10.1038\/s41746-023-00980-y","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"assertion":[{"value":"30 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"J.D.H. and L.B. received funding from the Innovation Fund Denmark. J.D.H. and L.B. used Corti and held stock warrants. L.M. is a co-founder, stockholder, and the Chief Technology Officer of Corti. J.W. received funding from Trygfonden. S.N.F.B. has no conflicts of interest to declare. H.C. has received funding from the Velux Foundation, Tv\u00e6rsfonden, Helsefonden, Hartmann Fonden, Lundbeck Foundation, and Novo Nordisk Foundation; royalties from Gyldendal; honoraria from Bayer and Bristol Meyers Squibb, and is chair of Action Plan for stroke in Europe Implementation, Co-chair of the Scientific Stroke Panel EAN and Senior Guest Editor of AHA Stroke. M.S. has no conflicts of interest to declare. H.C.C. has no conflicts of interest to declare. C.K. received funding from the Novo Nordisk Foundation and is the chair of the Danish Resuscitation Council and vice chair of the Danish Stroke Society. Both positions are unpaid.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"235"}}