{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T04:24:50Z","timestamp":1750739090583,"version":"3.37.3"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T00:00:00Z","timestamp":1618790400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T00:00:00Z","timestamp":1618790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Stat"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Given noisy, partial observations of a time-homogeneous, finite-statespace Markov chain, conceptually simple, direct statistical inference is available, in theory, via its rate matrix, or infinitesimal generator, <jats:inline-formula><jats:alternatives><jats:tex-math>$${\\mathsf {Q}}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>Q<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>, since <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\exp ({\\mathsf {Q}}t)$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mo>exp<\/mml:mo>\n                    <mml:mo>(<\/mml:mo>\n                    <mml:mi>Q<\/mml:mi>\n                    <mml:mi>t<\/mml:mi>\n                    <mml:mo>)<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> is the transition matrix over time <jats:italic>t<\/jats:italic>. However, perhaps because of inadequate tools for matrix exponentiation in programming languages commonly used amongst statisticians or a belief that the necessary calculations are prohibitively expensive, statistical inference for continuous-time Markov chains with a large but finite state space is typically conducted via particle MCMC or other relatively complex inference schemes. When, as in many applications <jats:inline-formula><jats:alternatives><jats:tex-math>$${\\mathsf {Q}}$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>Q<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> arises from a reaction network, it is usually sparse. We describe variations on known algorithms which allow fast, robust and accurate evaluation of the product of a non-negative vector with the exponential of a large, sparse rate matrix. Our implementation uses relatively recently developed, efficient, linear algebra tools that take advantage of such sparsity. We demonstrate the straightforward statistical application of the key algorithm on a model for the mixing of two alleles in a population and on the Susceptible-Infectious-Removed epidemic model.\n<\/jats:p>","DOI":"10.1007\/s00180-021-01102-6","type":"journal-article","created":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T17:03:34Z","timestamp":1618851814000},"page":"2863-2887","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Direct statistical inference for finite Markov jump processes via the matrix exponential"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2429-3157","authenticated-orcid":false,"given":"Chris","family":"Sherlock","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,4,19]]},"reference":[{"issue":"2","key":"1102_CR1","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1137\/100788860","volume":"33","author":"AH Al-Mohy","year":"2011","unstructured":"Al-Mohy AH, Higham NJ (2011) Computing the action of a matrix exponential with an application to exponential integrators. SIAM J. Sci. Comput. 33(2):488\u2013511","journal-title":"SIAM J. Sci. Comput."},{"key":"1102_CR2","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1007\/s40300-019-00151-8","volume":"77","author":"R Amoros","year":"2019","unstructured":"Amoros R, King R, Toyoda H, Kumada T, Johnson PJ, Bird TG (2019) A continuous-time hidden Markov model for cancer surveillance using serum biomarkers with application to hepatocellular carcinoma. METRON 77:67\u201386","journal-title":"METRON"},{"key":"1102_CR3","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1126\/science.7063839","volume":"215","author":"RM Anderson","year":"1982","unstructured":"Anderson RM, May RM (1982) Directly transmitted infectious diseases: control by vaccination. Science 215:1053\u20131060","journal-title":"Science"},{"key":"1102_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1158-7","volume-title":"Stochastic epidemic models and their statistical analysis","author":"H Andersson","year":"2000","unstructured":"Andersson H, Britton T (2000) Stochastic epidemic models and their statistical analysis. Springer, New York"},{"key":"1102_CR5","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/978-3-642-04107-5_3","volume-title":"Monte Carlo and Quasi-Monte Carlo Methods 2008","author":"C Andrieu","year":"2009","unstructured":"Andrieu C, Doucet A, Holenstein R (2009) Particle Markov chain Monte Carlo for efficient numerical simulation. In: L\u2019Ecuyer P, Owen AB (eds) Monte Carlo and Quasi-Monte Carlo Methods 2008. Spinger, Berlin, pp 45\u201360"},{"issue":"3","key":"1102_CR6","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1111\/j.1467-9868.2009.00736.x","volume":"72","author":"C Andrieu","year":"2010","unstructured":"Andrieu C, Doucet A, Holenstein R (2010) Particle Markov chain Monte Carlo methods. J. R. Statist. Soc. B 72(3):269\u2013342","journal-title":"J. R. Statist. Soc. B"},{"issue":"4","key":"1102_CR7","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1007\/s11222-018-9827-1","volume":"29","author":"AJ Black","year":"2019","unstructured":"Black AJ (2019) Importance sampling for partially observed temporal epidemic models. Stat Comput 29(4):617\u2013630","journal-title":"Stat Comput"},{"key":"1102_CR8","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199535255.001.0001","volume-title":"Concentration inequalities : a nonasymptotic theory of independence","author":"S Boucheron","year":"2013","unstructured":"Boucheron S, Lugosi G, Massart P (2013) Concentration inequalities\u202f: a nonasymptotic theory of independence. Oxford University Press, Oxford"},{"key":"1102_CR9","unstructured":"Broadfoot K, Keeling M (2015) Measles epidemics in vaccinated populations. Accessed on 1 Feb 2020"},{"key":"1102_CR10","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1093\/biomet\/asu075","volume":"102","author":"A Doucet","year":"2015","unstructured":"Doucet A, Pitt MK, Kohn R (2015) Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator. Biometrika 102:295\u2013313","journal-title":"Biometrika"},{"key":"1102_CR11","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1111\/biom.12449","volume":"72","author":"CC Drovandi","year":"2016","unstructured":"Drovandi CC, McCutchan R (2016) Alive SMC$$^2$$: Bayesian model selction for low-count time series models with intractable likelihoods. Biometrics 72:344\u2013353","journal-title":"Biometrics"},{"key":"1102_CR12","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1016\/j.csda.2013.02.005","volume":"71","author":"D Eddelbuettel","year":"2014","unstructured":"Eddelbuettel D, Sanderson C (2014) Rcpparmadillo: Accelerating r with high-performance c++ linear algebra. Comput. Stat. Data Anal. 71:1054\u20131063","journal-title":"Comput. Stat. Data Anal."},{"key":"1102_CR13","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1111\/biom.12152","volume":"70","author":"P Fearnhead","year":"2014","unstructured":"Fearnhead P, Giagos V, Sherlock C (2014) Inference for reaction networks using the Linear Noise Approximation. Biometrics 70:457\u2013466","journal-title":"Biometrics"},{"issue":"5","key":"1102_CR14","first-page":"1236","volume":"13","author":"E Gallopoulos","year":"1992","unstructured":"Gallopoulos E, Saad Y (1992) Efficient solution of parabolic equations by Krylov approximation methods. J. Appl. Stat. 13(5):1236\u20131264","journal-title":"J. Appl. Stat."},{"issue":"4","key":"1102_CR15","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1007\/s11222-016-9667-9","volume":"27","author":"A Georgoulas","year":"2017","unstructured":"Georgoulas A, Hillston J, Sanguinetti G (2017) Unbiased bayesian inference for population markov jump processes via random truncations. Stat. Comput. 27(4):991\u20131002","journal-title":"Stat. Comput."},{"issue":"5","key":"1102_CR16","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.1007\/s11222-019-09861-5","volume":"29","author":"A Golightly","year":"2019","unstructured":"Golightly A, Sherlock C (2019) Efficient sampling of conditioned Markov jump processes. Stat. Comput. 29(5):1149\u20131163","journal-title":"Stat. Comput."},{"issue":"3","key":"1102_CR17","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1111\/j.1541-0420.2005.00345.x","volume":"61","author":"A Golightly","year":"2005","unstructured":"Golightly A, Wilkinson DJ (2005) Bayesian inference for stochastic kinetic models using a diffusion approximation. Biometrics 61(3):781\u2013788","journal-title":"Biometrics"},{"issue":"6","key":"1102_CR18","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1098\/rsfs.2011.0047","volume":"1","author":"A Golightly","year":"2011","unstructured":"Golightly A, Wilkinson DJ (2011) Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo. Interface Focus 1(6):807\u2013820","journal-title":"Interface Focus"},{"issue":"2","key":"1102_CR19","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1515\/sagmb-2014-0070","volume":"14","author":"A Golightly","year":"2015","unstructured":"Golightly A, Wilkinson DJ (2015) Bayesian inference for Markov jump processes with informative observations. Stat. Appl. Genet. Mol. Biol. 14(2):169\u2013188","journal-title":"Stat. Appl. Genet. Mol. Biol."},{"issue":"3","key":"1102_CR20","doi-asserted-by":"publisher","first-page":"1993","DOI":"10.1214\/18-AOAS1141","volume":"12","author":"LST Ho","year":"2018","unstructured":"Ho LST, Crawford FW, Suchard MA (2018) Direct likelihood-based inference for discretely observed stochastic compartmental models of infectious disease. Ann. Appl. Stat. 12(3):1993\u20132021","journal-title":"Ann. Appl. Stat."},{"key":"1102_CR21","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1016\/S0140-6736(13)60908-3","volume":"381","author":"Z Jakab","year":"2013","unstructured":"Jakab Z, Salisbury DM (2013) Back to basics: the miracle and tragedy of measles vaccine. The Lancet 381:1433\u20131434","journal-title":"The Lancet"},{"issue":"5","key":"1102_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0036160","volume":"7","author":"G Jenkinson","year":"2012","unstructured":"Jenkinson G, Goutsias J (2012) Numerical integration of the master equation in some models of stochastic epidemiology. PLOS ONE 7(5):1\u20139","journal-title":"PLOS ONE"},{"issue":"3","key":"1102_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1006046","volume":"14","author":"T Kinyanjui","year":"2018","unstructured":"Kinyanjui T, Middleton J, G\u00fcttel S, Cassell J, Ross J, House T (2018) Scabies in residential care homes: Modelling, inference and interventions for well-connected population sub-units. PLOS Comput. Biol. 14(3):1\u201324","journal-title":"PLOS Comput. Biol."},{"issue":"2","key":"1102_CR24","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1007\/s11222-013-9440-2","volume":"25","author":"E Koblents","year":"2015","unstructured":"Koblents E, Miguez J (2015) A population Monte Carlo scheme with transformed weights and its application to stochastic kinetic models. Stat. Comput. 25(2):407\u2013425","journal-title":"Stat. Comput."},{"key":"1102_CR25","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.mbs.2016.07.001","volume":"287","author":"T Kypraios","year":"2017","unstructured":"Kypraios T, Neal P, Prangle D (2017) A tutorial introduction to bayesian inference for stochastic epidemic models using approximate bayesian computation. Math. Biosci. 287:42\u201353","journal-title":"Math. Biosci."},{"key":"1102_CR26","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1016\/j.csda.2012.12.012","volume":"71","author":"TJ McKinley","year":"2014","unstructured":"McKinley TJ, Ross JV, Deardon R, Cook AR (2014) Simulation-based Bayesian inference for epidemic models. Comput. Stat. Data Anal. 71:434\u2013447","journal-title":"Comput. Stat. Data Anal."},{"issue":"1","key":"1102_CR27","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1137\/S00361445024180","volume":"45","author":"C Moler","year":"2003","unstructured":"Moler C, Van Loan C (2003) Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Rev. 45(1):3\u201349","journal-title":"SIAM Rev."},{"issue":"1","key":"1102_CR28","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1017\/S0305004100033193","volume":"54","author":"P Moran","year":"1958","unstructured":"Moran P (1958) Random processes in genetics. Math. Proc. Camb. Philos. Soc. 54(1):60\u201371","journal-title":"Math. Proc. Camb. Philos. Soc."},{"key":"1102_CR29","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511810633","volume-title":"Markov chains","author":"JR Norris","year":"1997","unstructured":"Norris JR (1997) Markov chains. Cambridge University Press, Cambridge"},{"issue":"2","key":"1102_CR30","first-page":"189","volume":"14","author":"J Owen","year":"2015","unstructured":"Owen J, Wilkinson DJ, Gillespie CS (2015) Likelihood free inference for Markov processes: a comparison. Stat. Appl. Gen. Mol. Biol. 14(2):189\u2013209","journal-title":"Stat. Appl. Gen. Mol. Biol."},{"key":"1102_CR31","unstructured":"Public Health Wales (2013) Vaccine uptake in children in wales: October to december 2012"},{"key":"1102_CR32","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Core Team","year":"2018","unstructured":"R Core Team (2018) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna"},{"issue":"2","key":"1102_CR33","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1080\/02664768200000021","volume":"9","author":"G Raggett","year":"1982","unstructured":"Raggett G (1982) A stochastic model of the Eyam plague. J. Appl. Stat. 9(2):212\u2013225","journal-title":"J. Appl. Stat."},{"key":"1102_CR34","first-page":"3295","volume":"14","author":"V Rao","year":"2013","unstructured":"Rao V, Teh YW (2013) Fast mcmc sampling for markov jump processes and extensions. J. Mach. Learn. Res. 14:3295\u20133320","journal-title":"J. Mach. Learn. Res."},{"issue":"1","key":"1102_CR35","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/0305-0548(88)90026-3","volume":"15","author":"A Reibman","year":"1988","unstructured":"Reibman A, Trivedi K (1988) Numerical transient analysis of markov models. Comput. Oper. Res. 15(1):19\u201336","journal-title":"Comput. Oper. Res."},{"key":"1102_CR36","volume-title":"Stochastic processes","author":"SM Ross","year":"1996","unstructured":"Ross SM (1996) Stochastic processes. Wiley, New York"},{"issue":"1","key":"1102_CR37","first-page":"209","volume":"29","author":"Y Saad","year":"1992","unstructured":"Saad Y (1992) Analysis of some Krylov subspace approximations to the matrix exponential operator. J. Appl. Stat. 29(1):209\u2013228","journal-title":"J. Appl. Stat."},{"key":"1102_CR38","doi-asserted-by":"publisher","first-page":"26","DOI":"10.21105\/joss.00026","volume":"1","author":"C Sanderson","year":"2016","unstructured":"Sanderson C, Curtin R (2016) Armadillo: a template-based C++ library for linear algebra. J. Open Source Softw. 1:26","journal-title":"J. Open Source Softw."},{"key":"1102_CR39","first-page":"422","volume":"10931","author":"C Sanderson","year":"2018","unstructured":"Sanderson C, Curtin R (2018) A user-friendly hybrid sparse matrix class in C++. LNCS 10931:422\u2013430","journal-title":"LNCS"},{"key":"1102_CR40","unstructured":"Sherlock C,Golightly A (2019). Exact bayesian inference for discretely observed markov jump processes using finite rate matrices"},{"issue":"1","key":"1102_CR41","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1214\/14-AOS1278","volume":"43","author":"C Sherlock","year":"2015","unstructured":"Sherlock C, Thiery A, Roberts GO, Rosenthal JS (2015) On the effciency of pseudo-marginal random walk Metropolis algorithms. Ann. Stat. 43(1):238\u2013275","journal-title":"Ann. Stat."},{"issue":"3","key":"1102_CR42","first-page":"1","volume":"2013","author":"M Short","year":"2013","unstructured":"Short M (2013) Improved inequalities for the Poisson and binomial distribution and upper tail quantile functions. ISRN Probabil. Stat. 2013(3):1\u20136","journal-title":"ISRN Probabil. Stat."},{"issue":"1","key":"1102_CR43","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1145\/285861.285868","volume":"24","author":"RB Sidje","year":"1998","unstructured":"Sidje RB (1998) EXPOKIT: a software package for computing matrix exponentials. ACM Trans. Math. Soft. 24(1):130\u2013156","journal-title":"ACM Trans. Math. Soft."},{"issue":"3","key":"1102_CR44","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1016\/S0167-9473(98)00062-0","volume":"29","author":"RB Sidje","year":"1999","unstructured":"Sidje RB, Stewart WJ (1999) A numerical study of large sparse matrix exponentials arising in Markov chains. Comput. Stat. Data Anal. 29(3):345\u2013368","journal-title":"Comput. Stat. Data Anal."},{"issue":"39","key":"1102_CR45","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1098\/rsif.2008.0476","volume":"6","author":"M Vellela","year":"2009","unstructured":"Vellela M, Qian H (2009) Stochastic dynamics and non-equilibrium thermodynamics of a bistable chemical system: the Schl\u00f6gel model revisited. J. Royal Soc. Interface 6(39):925\u2013940","journal-title":"J. Royal Soc. Interface"},{"issue":"52","key":"1102_CR46","doi-asserted-by":"publisher","first-page":"7219","DOI":"10.1016\/j.vaccine.2009.09.116","volume":"27","author":"WHO","year":"2009","unstructured":"WHO (2009) Who position on measles vaccines. Vaccine 27(52):7219\u20137221","journal-title":"Vaccine"},{"key":"1102_CR47","volume-title":"Stochastic modelling for systems biology","author":"DJ Wilkinson","year":"2012","unstructured":"Wilkinson DJ (2012) Stochastic modelling for systems biology. CRC Press, Boca Raton"},{"key":"1102_CR48","doi-asserted-by":"crossref","unstructured":"Wise J (2013). Largest group of children affected by measles outbreak in wales is 10\u201318 year olds. BMJ, 346","DOI":"10.1136\/bmj.f2545"}],"container-title":["Computational Statistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-021-01102-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00180-021-01102-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-021-01102-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,29]],"date-time":"2022-10-29T15:04:27Z","timestamp":1667055867000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00180-021-01102-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,19]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1102"],"URL":"https:\/\/doi.org\/10.1007\/s00180-021-01102-6","relation":{},"ISSN":["0943-4062","1613-9658"],"issn-type":[{"type":"print","value":"0943-4062"},{"type":"electronic","value":"1613-9658"}],"subject":[],"published":{"date-parts":[[2021,4,19]]},"assertion":[{"value":"20 July 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}