{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T04:20:08Z","timestamp":1772166008146,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T00:00:00Z","timestamp":1689033600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T00:00:00Z","timestamp":1689033600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>We aimed to select and externally validate a benchmark method for emergency ambulance services to use to forecast the daily number of calls that result in the dispatch of one or more ambulances.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The study was conducted using standard methods known to the UK\u2019s NHS to aid implementation in practice. We selected our benchmark model from a naive benchmark and 14 standard forecasting methods. Mean absolute scaled error and 80 and 95% prediction interval coverage over a 84 day horizon were evaluated using time series cross validation across eight time series from the South West of England. External validation was conducted by time series cross validation across 13 time series from London, Yorkshire and Welsh Ambulance Services.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>A model combining a simple average of Facebook\u2019s prophet and regression with ARIMA errors (1, 1, 3)(1, 0, 1, 7) was selected. Benchmark MASE, 80 and 95% prediction intervals were 0.68 (95% CI 0.67 - 0.69), 0.847 (95% CI 0.843 - 0.851), and 0.965 (95% CI 0.949 - 0.977), respectively. Performance in the validation set was within expected ranges for MASE, 0.73 (95% CI 0.72 - 0.74) 80% coverage (0.833; 95% CI 0.828-0.838), and 95% coverage (0.965; 95% CI 0.963-0.967).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We provide a robust externally validated benchmark for future ambulance demand forecasting studies to improve on. Our benchmark forecasting model is high quality and usable by ambulance services. We provide a simple python framework to aid its implementation in practice. The results of this study were implemented in the South West of England.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12911-023-02218-z","type":"journal-article","created":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T06:02:29Z","timestamp":1689055349000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Forecasting the daily demand for emergency medical ambulances in England and Wales: a benchmark model and external validation"],"prefix":"10.1186","volume":"23","author":[{"given":"Thomas","family":"Monks","sequence":"first","affiliation":[]},{"given":"Alison","family":"Harper","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Allen","sequence":"additional","affiliation":[]},{"given":"Lucy","family":"Collins","sequence":"additional","affiliation":[]},{"given":"Andrew","family":"Mayne","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,7,11]]},"reference":[{"key":"2218_CR1","doi-asserted-by":"publisher","unstructured":"Lee DW, Moon HJ, Heo NH. Association between ambulance response time and neurologic outcome in patients with cardiac arrest. Am J Emerg Med. 2019;37(11):1999\u20132003. https:\/\/doi.org\/10.1016\/j.ajem.2019.02.021. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0735675719300993","DOI":"10.1016\/j.ajem.2019.02.021"},{"issue":"4","key":"2218_CR2","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1161\/STROKEAHA.113.002910","volume":"45","author":"A Meretoja","year":"2014","unstructured":"Meretoja A, Keshtkaran M, Saver JL, Tatlisumak T, Parsons MW, Kaste M, et al. Stroke thrombolysis: Save a minute, save a day. Stroke. 2014;45(4):1053\u20138. https:\/\/doi.org\/10.1161\/STROKEAHA.113.002910.","journal-title":"Stroke."},{"key":"2218_CR3","doi-asserted-by":"publisher","unstructured":"Byrne JP, Mann NC, Dai M, Mason SA, Karanicolas P, Rizoli S, et\u00a0al. Association Between Emergency Medical Service Response Time and Motor Vehicle Crash Mortality in the United States. JAMA Surg. 2019;154(4):286\u2013293. Publisher: American Medical Association. https:\/\/doi.org\/10.1001\/jamasurg.2018.5097. https:\/\/jamanetwork.com\/journals\/jamasurgery\/fullarticle\/2723267.","DOI":"10.1001\/jamasurg.2018.5097"},{"key":"2218_CR4","doi-asserted-by":"crossref","unstructured":"Baker JR, Fitzpatrick KE. Determination of an Optimal Forecast Model for Ambulance Demand Using Goal Programming. J Oper Res Soc. 1986;37(11):1047\u20131059. http:\/\/www.jstor.org\/stable\/2582689.","DOI":"10.1057\/jors.1986.182"},{"issue":"11","key":"2218_CR5","doi-asserted-by":"publisher","first-page":"1556","DOI":"10.1057\/jors.2011.160","volume":"63","author":"JL Vile","year":"2012","unstructured":"Vile JL, Gillard JW, Harper PR, Knight VA. Predicting ambulance demand using singular spectrum analysis. J Oper Res Soc. 2012;63(11):1556\u201365. https:\/\/doi.org\/10.1057\/jors.2011.160.","journal-title":"J Oper Res Soc."},{"key":"2218_CR6","doi-asserted-by":"publisher","unstructured":"Setzler H, Saydam C, Park S. EMS call volume predictions: A comparative study. Comput Oper Res. 2009;36(6):1843\u201351. https:\/\/doi.org\/10.1016\/j.cor.2008.05.010. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0305054808001068","DOI":"10.1016\/j.cor.2008.05.010"},{"issue":"2","key":"2218_CR7","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1080\/10903120701204797","volume":"11","author":"LH Brown","year":"2007","unstructured":"Brown LH, Lerner EB, Larmon B, LeGassick T, Taigman M. Are EMS Call Volume Predictions Based on Demand Pattern Analysis Accurate? Prehospital Emerg Care. 2007;11(2):199\u2013203. https:\/\/doi.org\/10.1080\/10903120701204797.","journal-title":"Prehospital Emerg Care."},{"issue":"4","key":"2218_CR8","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1080\/20476965.2020.1783190","volume":"10","author":"MAK Al-Azzani","year":"2021","unstructured":"Al-Azzani MAK, Davari S, England TJ. An empirical investigation of forecasting methods for ambulance calls - a case study. Health Syst. 2021;10(4):268\u201385. https:\/\/doi.org\/10.1080\/20476965.2020.1783190.","journal-title":"Health Syst."},{"key":"2218_CR9","doi-asserted-by":"publisher","unstructured":"Bayisa FL, \u00c5dahl M, Ryd\u00e9n P, Cronie O. Large-scale modelling and forecasting of ambulance calls in northern Sweden using spatio-temporal log-Gaussian Cox processes. Spat Stat. 2020;39:100471. https:\/\/doi.org\/10.1016\/j.spasta.2020.100471. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S2211675320300658.","DOI":"10.1016\/j.spasta.2020.100471"},{"key":"2218_CR10","doi-asserted-by":"publisher","unstructured":"Mapuwei TW, Bodhlyera O, Mwambi H. Univariate Time Series Analysis of Short-Term Forecasting Horizons Using Artificial Neural Networks: The Case of Public Ambulance Emergency Preparedness. J Appl Math. 2020;2020:e2408698. Publisher: Hindawi. https:\/\/doi.org\/10.1155\/2020\/2408698. https:\/\/www.hindawi.com\/journals\/jam\/2020\/2408698\/.","DOI":"10.1155\/2020\/2408698"},{"issue":"4","key":"2218_CR11","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1109\/JBHI.2015.2443799","volume":"20","author":"AY Chen","year":"2016","unstructured":"Chen AY, Lu TY, Ma MHM, Sun WZ. Demand Forecast Using Data Analytics for the Preallocation of Ambulances. IEEE J Biomed Health Inform. 2016;20(4):1178\u201387. https:\/\/doi.org\/10.1109\/JBHI.2015.2443799.","journal-title":"IEEE J Biomed Health Inform."},{"key":"2218_CR12","doi-asserted-by":"publisher","unstructured":"Martin RJ, Mousavi R, Saydam C. Predicting emergency medical service call demand: A modern spatiotemporal machine learning approach. Oper Res Health Care. 2021;28:100285. https:\/\/doi.org\/10.1016\/j.orhc.2021.100285. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S2211692321000011","DOI":"10.1016\/j.orhc.2021.100285"},{"key":"2218_CR13","doi-asserted-by":"crossref","unstructured":"Hyndman RJ, Athanasopoulos G. Forecasting: principles and practice, 2nd edition. Melbourne, Australia: OTexts; 2018. http:\/\/OTexts.com\/fpp2.\u00a0Accessed 10 July 2023.","DOI":"10.32614\/CRAN.package.fpp2"},{"key":"2218_CR14","doi-asserted-by":"publisher","unstructured":"Hyndman RJ, Koehler AB. Another look at measures of forecast accuracy. Int J Forecast. 2006;22(4):679\u201388. https:\/\/doi.org\/10.1016\/j.ijforecast.2006.03.001. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0169207006000239","DOI":"10.1016\/j.ijforecast.2006.03.001"},{"key":"2218_CR15","unstructured":"Smith TG, et\u00a0al. pmdarima: ARIMA estimators for Python. 2017. http:\/\/www.alkaline-ml.com\/pmdarima.\u00a0Accessed 10 July 2023."},{"key":"2218_CR16","doi-asserted-by":"publisher","unstructured":"Taylor SJ, Letham B. Forecasting at Scale.\u00a0Am Stat. 2018;72(1):37\u201345. https:\/\/doi.org\/10.1080\/00031305.2017.1380080.","DOI":"10.1080\/00031305.2017.1380080"},{"key":"2218_CR17","doi-asserted-by":"publisher","unstructured":"Seabold S, Perktold J. statsmodels: Econometric and statistical modeling with python. In: 9th Python in Science Conference. Austin: Scipy; 2010. https:\/\/doi.org\/10.25080\/Majora-92bf1922-011.","DOI":"10.25080\/Majora-92bf1922-011"},{"key":"2218_CR18","unstructured":"Skorupa G. BATS and TBATS time series forecasting. 2020. https:\/\/pypi.org\/project\/tbats\/.\u00a0Accessed 10 July 2023."},{"key":"2218_CR19","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3969789","author":"T Monks","year":"2020","unstructured":"Monks T. forecast-tools: fundamental tools to support the forecasting process in python. Zenodo. 2020. https:\/\/doi.org\/10.5281\/zenodo.3969789.","journal-title":"Zenodo"},{"key":"2218_CR20","doi-asserted-by":"crossref","unstructured":"Korobeynikov A. Computation- and space-efficient implementation of SSA. Stat Interface. 2010;3(3):357\u2013368. R package version 1.0.2.","DOI":"10.4310\/SII.2010.v3.n3.a9"},{"key":"2218_CR21","unstructured":"McKinney W. pandas: a foundational Python library for data analysis and statistics. Python High Perform Sci Comput. 2011;14:1\u20139."},{"issue":"2","key":"2218_CR22","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/MCSE.2011.37","volume":"13","author":"S van der Walt","year":"2011","unstructured":"van der Walt S, Colbert SC, Varoquaux G. The NumPy Array: A Structure for Efficient Numerical Computation. Comput Sci Eng. 2011;13(2):22\u201330. https:\/\/doi.org\/10.1109\/MCSE.2011.37.","journal-title":"Comput Sci Eng."},{"issue":"3","key":"2218_CR23","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","volume":"9","author":"JD Hunter","year":"2007","unstructured":"Hunter JD. Matplotlib: A 2D graphics environment. Comput Sci Eng. 2007;9(3):90\u20135. https:\/\/doi.org\/10.1109\/MCSE.2007.55.","journal-title":"Comput Sci Eng."},{"key":"2218_CR24","doi-asserted-by":"publisher","unstructured":"The Turing Way Community. The Turing Way: A handbook for reproducible, ethical and collaborative research. Zenodo; 2021. This work was supported by The UKRI Strategic Priorities Fund under the EPSRC Grant EP\/T001569\/1, particularly the \u201cTools, Practices and Systems\u201d theme within that grant, and by The Alan Turing Institute under the EPSRC grant EP\/N510129\/1. https:\/\/doi.org\/10.5281\/zenodo.5671094.","DOI":"10.5281\/zenodo.5671094"},{"key":"2218_CR25","doi-asserted-by":"publisher","unstructured":"Monks T, Allen M, Harper A, Mayne A, Collins L. TomMonks\/swast-benchmarking: v1.0.0. Zenodo; 2021. https:\/\/doi.org\/10.5281\/zenodo.4850149.","DOI":"10.5281\/zenodo.4850149"},{"key":"2218_CR26","doi-asserted-by":"publisher","unstructured":"Monks T, Allen M, Harper A, Mayne A, Collins L. TomMonks\/swast-forecast-tool: v0.1.0. Zenodo; 2020. https:\/\/doi.org\/10.5281\/zenodo.4277723.","DOI":"10.5281\/zenodo.4277723"},{"issue":"496","key":"2218_CR27","doi-asserted-by":"publisher","first-page":"1513","DOI":"10.1198\/jasa.2011.tm09771","volume":"106","author":"AMD Livera","year":"2011","unstructured":"Livera AMD, Hyndman RJ, Snyder RD. Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing. J Am Stat Assoc. 2011;106(496):1513\u201327. https:\/\/doi.org\/10.1198\/jasa.2011.tm09771.","journal-title":"J Am Stat Assoc."},{"key":"2218_CR28","doi-asserted-by":"publisher","unstructured":"Makridakis S, Spiliotis E, Assimakopoulos V. The M4 Competition: 100,000 time series and 61 forecasting methods. Int J Forecast. 2020;36(1):54 \u2013 74. M4 Competition. https:\/\/doi.org\/10.1016\/j.ijforecast.2019.04.014. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0169207019301128.","DOI":"10.1016\/j.ijforecast.2019.04.014"},{"key":"2218_CR29","doi-asserted-by":"publisher","unstructured":"Tashman LJ. Out-of-sample tests of forecasting accuracy: an analysis and review. Int J Forecast. 2000;16(4):437 \u2013 450. The M3- Competition. https:\/\/doi.org\/10.1016\/S0169-2070(00)00065-0. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0169207000000650.","DOI":"10.1016\/S0169-2070(00)00065-0"},{"key":"2218_CR30","doi-asserted-by":"publisher","unstructured":"Vile JL, Gillard JW, Harper PR, Knight VA. Time-dependent stochastic methods for managing and scheduling Emergency Medical Services. Oper Res Health Care. 2016;8:42\u201352. https:\/\/doi.org\/10.1016\/j.orhc.2015.07.002. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S2211692314200518","DOI":"10.1016\/j.orhc.2015.07.002"},{"key":"2218_CR31","doi-asserted-by":"publisher","unstructured":"Winkler RL. A Decision-Theoretic Approach to Interval Estimation. J Am Stat Assoc. 1972;67(337):187\u201391. https:\/\/doi.org\/10.1080\/01621459.1972.10481224. http:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/01621459.1972.10481224","DOI":"10.1080\/01621459.1972.10481224"},{"key":"2218_CR32","unstructured":"NHS England. NHS England Advanced forecasting techniques. 2020. https:\/\/www.england.nhs.uk\/publication\/advanced-forecasting-techniques\/.\u00a0Accessed 10 July 2023."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02218-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-023-02218-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02218-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T19:27:20Z","timestamp":1729711640000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02218-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,11]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2218"],"URL":"https:\/\/doi.org\/10.1186\/s12911-023-02218-z","relation":{"has-preprint":[{"id-type":"doi","id":"10.31219\/osf.io\/a6nu5","asserted-by":"object"}]},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,11]]},"assertion":[{"value":"14 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 June 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable. No human data were used in this research. Data used are counts of vehicle dispatches over time.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"117"}}