{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T02:18:47Z","timestamp":1778725127781,"version":"3.51.4"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T00:00:00Z","timestamp":1714435200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T00:00:00Z","timestamp":1714435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004543","name":"China Scholarships Council","doi-asserted-by":"crossref","award":["201906310134"],"award-info":[{"award-number":["201906310134"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","award":["EP\/N510129\/1"],"award-info":[{"award-number":["EP\/N510129\/1"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000288","name":"Royal Society","doi-asserted-by":"publisher","award":["IES\\R2\\19220"],"award-info":[{"award-number":["IES\\R2\\19220"]}],"id":[{"id":"10.13039\/501100000288","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Netw Sci"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Reconstructing dynamics of complex systems from sparse, incomplete time series data is a challenging problem with applications in various domains. Here, we develop an iterative heuristic method to infer the underlying network structure and parameters governed by Ising dynamics from incomplete spin configurations based on sparse and small-sized samples. Our method iterates between imputing missing spin states given current coupling strengths and re-estimating couplings from completed spin state data. Central to our approach is the novel application of adaptive <jats:inline-formula><jats:alternatives><jats:tex-math>$$l_1$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:msub>\n                    <mml:mi>l<\/mml:mi>\n                    <mml:mn>1<\/mml:mn>\n                  <\/mml:msub>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> regularization on updating coupling strengths, which features an automatic adjustment of the regularization strength throughout the iterative inference process. By doing so, we aim at preventing over-fitting and enforcing the sparsity of couplings without access to ground truth parameters. We demonstrate that this approach accurately recovers parameters and imputes missing spins even with substantial missing data and short time series, providing improvements in the inference of Ising model parameters even for relatively small sample sizes.<\/jats:p>","DOI":"10.1007\/s41109-024-00621-7","type":"journal-article","created":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T10:01:58Z","timestamp":1714471318000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Enhanced network inference from sparse incomplete time series through automatically adapted $$L_1$$ regularization"],"prefix":"10.1007","volume":"9","author":[{"given":"Zhongqi","family":"Cai","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Enrico","family":"Gerding","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Brede","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,4,30]]},"reference":[{"key":"621_CR1","doi-asserted-by":"publisher","first-page":"P06013","DOI":"10.1088\/1742-5468\/2014\/06\/P06013","volume":"2014","author":"L Bachschmid-Romano","year":"2014","unstructured":"Bachschmid-Romano L, Opper M (2014) Inferring hidden states in a random kinetic ising model: replica analysis. J Stat Mech: Theory Exp 2014:P06013","journal-title":"J Stat Mech: Theory Exp"},{"key":"621_CR2","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1126\/science.286.5439.509","volume":"286","author":"AL Barab\u00e1si","year":"1999","unstructured":"Barab\u00e1si AL, Albert R (1999) Emergence of scaling in random networks. Science 286:509\u2013512","journal-title":"Science"},{"key":"621_CR3","doi-asserted-by":"publisher","first-page":"P05021","DOI":"10.1088\/1742-5468\/2015\/05\/P05021","volume":"2015","author":"C Battistin","year":"2015","unstructured":"Battistin C, Hertz J, Tyrcha J, Roudi Y (2015) Belief propagation and replicas for inference and learning in a kinetic ising model with hidden spins. J Stat Mech: Theory Exp 2015:P05021","journal-title":"J Stat Mech: Theory Exp"},{"key":"621_CR4","doi-asserted-by":"publisher","first-page":"1698","DOI":"10.3389\/fpsyg.2015.01698","volume":"6","author":"C Bisconti","year":"2015","unstructured":"Bisconti C, Corallo A, Fortunato L, Gentile AA, Massafra A, Pell\u00e8 P (2015) Reconstruction of a real world social network using the potts model and loopy belief propagation. Front Psychol 6:1698","journal-title":"Front Psychol"},{"key":"621_CR5","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.99.062138","volume":"99","author":"C Campajola","year":"2019","unstructured":"Campajola C, Lillo F, Tantari D (2019) Inference of the kinetic ising model with heterogeneous missing data. Phys Rev E 99:062138","journal-title":"Phys Rev E"},{"key":"621_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.mbs.2021.108678","volume":"341","author":"E Cresswell-Clay","year":"2021","unstructured":"Cresswell-Clay E, Periwal V (2021) Genome-wide covariation in sars-cov-2. Math Biosci 341:108678. https:\/\/doi.org\/10.1016\/j.mbs.2021.108678","journal-title":"Math Biosci"},{"key":"621_CR7","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.96.062104","volume":"96","author":"C Donner","year":"2017","unstructured":"Donner C, Opper M (2017) Inverse ising problem in continuous time: a latent variable approach. Phys Rev E 96:062104. https:\/\/doi.org\/10.1103\/PhysRevE.96.062104","journal-title":"Phys Rev E"},{"key":"621_CR8","doi-asserted-by":"publisher","DOI":"10.1088\/1751-8121\/aa59dc","volume":"50","author":"B Dunn","year":"2017","unstructured":"Dunn B, Battistin C (2017) The appropriateness of ignorance in the inverse kinetic ising model. J Phys A: Math Theor 50:124002","journal-title":"J Phys A: Math Theor"},{"key":"621_CR9","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.87.022127","volume":"87","author":"B Dunn","year":"2013","unstructured":"Dunn B, Roudi Y (2013) Learning and inference in a nonequilibrium ising model with hidden nodes. Phys Rev E 87:022127","journal-title":"Phys Rev E"},{"key":"621_CR10","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.103.062302","volume":"103","author":"B Gemao","year":"2021","unstructured":"Gemao B, Lai PY et al (2021) Effects of hidden nodes on noisy network dynamics. Phys Rev E 103:062302","journal-title":"Phys Rev E"},{"key":"621_CR11","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1063\/1.1703954","volume":"4","author":"RJ Glauber","year":"1963","unstructured":"Glauber RJ (1963) Time-dependent statistics of the ising model. J Math Phys 4:294\u2013307","journal-title":"J Math Phys"},{"key":"621_CR12","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.114.028701","volume":"114","author":"X Han","year":"2015","unstructured":"Han X, Shen Z, Wang WX, Di Z (2015) Robust reconstruction of complex networks from sparse data. Phys Rev Lett 114:028701","journal-title":"Phys Rev Lett"},{"key":"621_CR13","doi-asserted-by":"crossref","unstructured":"Hastie T, Tibshirani R, Friedman JH, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, vol 2. Springer","DOI":"10.1007\/978-0-387-84858-7"},{"key":"621_CR14","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.99.042114","volume":"99","author":"DT Hoang","year":"2019","unstructured":"Hoang DT, Jo J, Periwal V (2019a) Data-driven inference of hidden nodes in networks. Phys Rev E 99:042114. https:\/\/doi.org\/10.1103\/PhysRevE.99.042114","journal-title":"Phys Rev E"},{"key":"621_CR15","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.99.023311","volume":"99","author":"DT Hoang","year":"2019","unstructured":"Hoang DT, Song J, Periwal V, Jo J (2019b) Network inference in stochastic systems from neurons to currencies: improved performance at small sample size. Phys Rev E 99:023311. https:\/\/doi.org\/10.1103\/PhysRevE.99.023311","journal-title":"Phys Rev E"},{"key":"621_CR16","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1016\/j.socnet.2005.07.002","volume":"28","author":"G Kossinets","year":"2006","unstructured":"Kossinets G (2006) Effects of missing data in social networks. Soc Netw 28:247\u2013268","journal-title":"Soc Netw"},{"key":"621_CR17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.104.024119","volume":"104","author":"S Lee","year":"2021","unstructured":"Lee S, Periwal V, Jo J (2021) Inference of stochastic time series with missing data. Phys Rev E 104:024119. https:\/\/doi.org\/10.1103\/PhysRevE.104.024119","journal-title":"Phys Rev E"},{"key":"621_CR18","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1049\/iet-its.2013.0052","volume":"8","author":"Y Li","year":"2014","unstructured":"Li Y, Li Z, Li L (2014) Missing traffic data: comparison of imputation methods. IET Intel Transport Syst 8:51\u201357","journal-title":"IET Intel Transport Syst"},{"key":"621_CR19","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1186\/s12918-018-0630-6","volume":"12","author":"L Liao","year":"2018","unstructured":"Liao L, Li K, Li K, Yang C, Tian Q (2018) A multiple kernel density clustering algorithm for incomplete datasets in bioinformatics. BMC Syst Biol 12:99\u2013116","journal-title":"BMC Syst Biol"},{"key":"621_CR20","unstructured":"Liu Q, Ihler A (2011) Learning scale free networks by reweighted $$l_1$$ regularization. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, pp 40\u201348"},{"key":"621_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.jedc.2020.104022","volume":"121","author":"P Mazzarisi","year":"2020","unstructured":"Mazzarisi P, Zaoli S, Campajola C, Lillo F (2020) Tail granger causalities and where to find them: extreme risk spillovers vs spurious linkages. J Econ Dyn Control 121:104022","journal-title":"J Econ Dyn Control"},{"key":"621_CR22","doi-asserted-by":"publisher","first-page":"L07001","DOI":"10.1088\/1742-5468\/2011\/07\/L07001","volume":"2011","author":"M M\u00e9zard","year":"2011","unstructured":"M\u00e9zard M, Sakellariou J (2011) Exact mean-field inference in asymmetric kinetic ising systems. J Stat Mech: Theory Exp 2011:L07001","journal-title":"J Stat Mech: Theory Exp"},{"key":"621_CR23","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0033721","volume":"7","author":"A Mirshahvalad","year":"2012","unstructured":"Mirshahvalad A, Lindholm J, Derl\u00e9n M, Rosvall M (2012) Significant communities in large sparse networks. PLoS ONE 7:e33721","journal-title":"PLoS ONE"},{"key":"621_CR24","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.64.026118","volume":"64","author":"ME Newman","year":"2001","unstructured":"Newman ME, Strogatz SH, Watts DJ (2001) Random graphs with arbitrary degree distributions and their applications. Phys Rev E 64:026118","journal-title":"Phys Rev E"},{"key":"621_CR25","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1080\/00018732.2017.1341604","volume":"66","author":"HC Nguyen","year":"2017","unstructured":"Nguyen HC, Zecchina R, Berg J (2017) Inverse statistical problems: from the inverse ising problem to data science. Adv Phys 66:197\u2013261. https:\/\/doi.org\/10.1080\/00018732.2017.1341604","journal-title":"Adv Phys"},{"key":"621_CR26","first-page":"428","volume":"95","author":"J Pearl","year":"2000","unstructured":"Pearl J (2000) Causal inference without counterfactuals: comment. J Am Stat Assoc 95:428\u2013431","journal-title":"J Am Stat Assoc"},{"key":"621_CR27","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.conb.2014.10.011","volume":"32","author":"Y Roudi","year":"2015","unstructured":"Roudi Y, Dunn B, Hertz J (2015) Multi-neuronal activity and functional connectivity in cell assemblies. Curr Opin Neurobiol 32:38\u201344. https:\/\/doi.org\/10.1016\/j.conb.2014.10.011","journal-title":"Curr Opin Neurobiol"},{"key":"621_CR28","doi-asserted-by":"publisher","first-page":"P03031","DOI":"10.1088\/1742-5468\/2011\/03\/P03031","volume":"2011","author":"Y Roudi","year":"2011","unstructured":"Roudi Y, Hertz J (2011a) Dynamical tap equations for non-equilibrium ising spin glasses. J Stat Mech: Theory Exp 2011:P03031","journal-title":"J Stat Mech: Theory Exp"},{"key":"621_CR29","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.106.048702","volume":"106","author":"Y Roudi","year":"2011","unstructured":"Roudi Y, Hertz J (2011b) Mean field theory for nonequilibrium network reconstruction. Phys Rev Lett 106:048702. https:\/\/doi.org\/10.1103\/PhysRevLett.106.048702","journal-title":"Phys Rev Lett"},{"key":"621_CR30","doi-asserted-by":"publisher","first-page":"8828","DOI":"10.1038\/srep08828","volume":"5","author":"A Singh","year":"2015","unstructured":"Singh A, Humphries MD (2015) Finding communities in sparse networks. Sci Rep 5:8828","journal-title":"Sci Rep"},{"key":"621_CR31","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004464","volume":"11","author":"D Soudry","year":"2015","unstructured":"Soudry D, Keshri S, Stinson P, Oh MH, Iyengar G, Paninski L (2015) Efficient \u201cshotgun\u2019\u2019 inference of neural connectivity from highly sub-sampled activity data. PLoS Comput Biol 11:e1004464","journal-title":"PLoS Comput Biol"},{"key":"621_CR32","doi-asserted-by":"publisher","first-page":"149","DOI":"10.3934\/mbe.2014.11.149","volume":"11","author":"J Tyrcha","year":"2014","unstructured":"Tyrcha J, Hertz J (2014) Network inference with hidden units. Math Biosci Eng 11:149\u2013165","journal-title":"Math Biosci Eng"},{"key":"621_CR33","doi-asserted-by":"publisher","first-page":"P03005","DOI":"10.1088\/1742-5468\/2013\/03\/P03005","volume":"2013","author":"J Tyrcha","year":"2013","unstructured":"Tyrcha J, Roudi Y, Marsili M, Hertz J (2013) The effect of nonstationarity on models inferred from neural data. J Stat Mech: Theory Exp 2013:P03005. https:\/\/doi.org\/10.1088\/1742-5468\/2013\/03\/P03005","journal-title":"J Stat Mech: Theory Exp"},{"key":"621_CR34","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1038\/30918","volume":"393","author":"DJ Watts","year":"1998","unstructured":"Watts DJ, Strogatz SH (1998) Collective dynamics of \u201csmall-world\u2019\u2019 networks. Nature 393:440\u2013442","journal-title":"Nature"},{"key":"621_CR35","doi-asserted-by":"crossref","unstructured":"Wu TT, Lange K (2008) Coordinate descent algorithms for lasso penalized regression. Ann Appl Stat","DOI":"10.1214\/07-AOAS147"},{"key":"621_CR36","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1080\/00018732.2016.1211393","volume":"65","author":"L Zdeborov\u00e1","year":"2016","unstructured":"Zdeborov\u00e1 L, Krzakala F (2016) Statistical physics of inference: thresholds and algorithms. Adv Phys 65:453\u2013552. https:\/\/doi.org\/10.1080\/00018732.2016.1211393","journal-title":"Adv Phys"},{"key":"621_CR37","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.83.041135","volume":"83","author":"HL Zeng","year":"2011","unstructured":"Zeng HL, Aurell E, Alava M, Mahmoudi H (2011) Network inference using asynchronously updated kinetic ising model. Phys Rev E 83:041135. https:\/\/doi.org\/10.1103\/PhysRevE.83.041135","journal-title":"Phys Rev E"},{"key":"621_CR38","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1007\/s10955-012-0547-1","volume":"148","author":"P Zhang","year":"2012","unstructured":"Zhang P (2012) Inference of kinetic ising model on sparse graphs. J Stat Phys 148:502\u2013512","journal-title":"J Stat Phys"},{"key":"621_CR39","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1017\/S0956792515000492","volume":"27","author":"JR Zipkin","year":"2016","unstructured":"Zipkin JR, Schoenberg FP, Coronges K, Bertozzi AL (2016) Point-process models of social network interactions: parameter estimation and missing data recovery. Eur J Appl Math 27:502\u2013529","journal-title":"Eur J Appl Math"}],"container-title":["Applied Network Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41109-024-00621-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41109-024-00621-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41109-024-00621-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T10:51:37Z","timestamp":1714474297000},"score":1,"resource":{"primary":{"URL":"https:\/\/appliednetsci.springeropen.com\/articles\/10.1007\/s41109-024-00621-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,30]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["621"],"URL":"https:\/\/doi.org\/10.1007\/s41109-024-00621-7","relation":{},"ISSN":["2364-8228"],"issn-type":[{"value":"2364-8228","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,30]]},"assertion":[{"value":"13 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 April 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 April 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"13"}}