{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:53:15Z","timestamp":1760233995219,"version":"build-2065373602"},"reference-count":56,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,3,15]],"date-time":"2021-03-15T00:00:00Z","timestamp":1615766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100018976","name":"Department of Telecommunications, Ministry of Communications, India","doi-asserted-by":"publisher","award":["DOTC-0001"],"award-info":[{"award-number":["DOTC-0001"]}],"id":[{"id":"10.13039\/501100018976","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Centre for Networked Intelligence (a Cisco CSR initiative), IISc, Bangalore","award":["Student Fellowship"],"award-info":[{"award-number":["Student Fellowship"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Samples from a high-dimensional first-order auto-regressive process generated by an independently and identically distributed random innovation sequence are observed by a sender which can communicate only finitely many bits per unit time to a receiver. The receiver seeks to form an estimate of the process value at every time instant in real-time. We consider a time-slotted communication model in a slow-sampling regime where multiple communication slots occur between two sampling instants. We propose a successive update scheme which uses communication between sampling instants to refine estimates of the latest sample and study the following question: Is it better to collect communication of multiple slots to send better refined estimates, making the receiver wait more for every refinement, or to be fast but loose and send new information in every communication opportunity? We show that the fast but loose successive update scheme with ideal spherical codes is universally optimal asymptotically for a large dimension. However, most practical quantization codes for fixed dimensions do not meet the ideal performance required for this optimality, and they typically will have a bias in the form of a fixed additive error. Interestingly, our analysis shows that the fast but loose scheme is not an optimal choice in the presence of such errors, and a judiciously chosen frequency of updates outperforms it.<\/jats:p>","DOI":"10.3390\/e23030347","type":"journal-article","created":{"date-parts":[[2021,3,15]],"date-time":"2021-03-15T11:38:58Z","timestamp":1615808338000},"page":"347","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Tracking an Auto-Regressive Process with Limited Communication per Unit Time"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7150-2081","authenticated-orcid":false,"given":"Rooji","family":"Jinan","sequence":"first","affiliation":[{"name":"Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science, Bangalore 560012, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3757-904X","authenticated-orcid":false,"given":"Parimal","family":"Parag","sequence":"additional","affiliation":[{"name":"Department of ECE, Indian Institute of Science, Bangalore 560012, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Himanshu","family":"Tyagi","sequence":"additional","affiliation":[{"name":"Department of ECE, Indian Institute of Science, Bangalore 560012, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2111","DOI":"10.1002\/j.1538-7305.1967.tb04246.x","article-title":"Random packings and coverings of the unit n-sphere","volume":"46","author":"Wyner","year":"1967","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/18.567644","article-title":"On the role of mismatch in rate distortion theory","volume":"43","author":"Lapidoth","year":"1997","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1002\/j.1538-7305.1979.tb02263.x","article-title":"On the Structure of Real-Time Source Coders","volume":"58","author":"Witsenhausen","year":"1979","journal-title":"Bell Syst. Tech. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4017","DOI":"10.1109\/TIT.2006.880067","article-title":"On the Structure of Optimal Real-Time Encoders and Decoders in Noisy Communication","volume":"52","author":"Teneketzis","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mahajan, A., and Teneketzis, D. (2007, January 2\u20136). On Real-Time Communication Systems with Noisy Feedback. Proceedings of the 2007 IEEE Information Theory Workshop, Tahoe City, CA, USA.","DOI":"10.1109\/ITW.2007.4313088"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1109\/TIT.1983.1056760","article-title":"Optimal causal coding-decoding problems","volume":"29","author":"Walrand","year":"1983","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1109\/TIT.2012.2212874","article-title":"On optimal causal coding of partially observed Markov sources in single and multiterminal settings","volume":"59","author":"Yuksel","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5975","DOI":"10.1109\/TIT.2014.2346780","article-title":"On optimal zero-delay coding of vector Markov sources","volume":"60","author":"Linder","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_9","first-page":"5968","article-title":"Optimal zero delay coding of Markov sources: Stationary and finite memory codes","volume":"63","author":"Wood","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/TIT.1962.1057738","article-title":"Information transmission with additional noise","volume":"8","author":"Dobrushin","year":"1962","journal-title":"IRE Trans. Inf. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1109\/TIT.1970.1054469","article-title":"Transmission of noisy information to a noisy receiver with minimum distortion","volume":"16","author":"Wolf","year":"1970","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1109\/TIT.1980.1056251","article-title":"Indirect rate distortion problems","volume":"26","author":"Witsenhausen","year":"1980","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"6833","DOI":"10.1109\/TIT.2017.2735960","article-title":"Sampling and Distortion Tradeoffs for Indirect Source Retrieval","volume":"63","author":"Mohammadi","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/18.370112","article-title":"Rate-distortion performance in coding bandlimited sources by sampling and dithered quantization","volume":"41","author":"Zamir","year":"1995","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mashiach, A., and Zamir, R. (2012, January 14\u201317). Entropy-coded quantization of periodic nonuniform samples. Proceedings of the 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel, Eilat, Israel.","DOI":"10.1109\/EEEI.2012.6376953"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mashiach, A., and Zamir, R. (2013, January 7\u201312). Noise-shaped quantization for nonuniform sampling. Proceedings of the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey.","DOI":"10.1109\/ISIT.2013.6620414"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1109\/TIT.2015.2485271","article-title":"Distortion rate function of sub-Nyquist sampled Gaussian sources","volume":"62","author":"Kipnis","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1294","DOI":"10.1109\/9.623096","article-title":"Systems with finite communication bandwidth constraints. I. State estimation problems","volume":"42","author":"Wong","year":"1997","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_19","unstructured":"Nair, G.N., and Evans, R.J. (1997, January 12). State estimation via a capacity-limited communication channel. Proceedings of the 36th IEEE Conference on Decision and Control, San Diego, CA, USA."},{"key":"ref_20","unstructured":"Nair, G.N., and Evans, R.J. (1998, January 18). State estimation under bit-rate constraints. Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171), Tampa, FL, USA."},{"key":"ref_21","unstructured":"Dokuchaev, N.G., and Savkin, A.V. (1999, January 7\u201310). Recursive state estimation via limited capacity communication channels. Proceedings of the 38th IEEE Conference on Decision and Control, Phoenix, AZ, USA."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2163","DOI":"10.1109\/TAC.2003.820140","article-title":"Estimation with lossy measurements: Jump estimators for jump systems","volume":"48","author":"Smith","year":"2003","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1109\/TAC.2003.809771","article-title":"The problem of state estimation via asynchronous communication channels with irregular transmission times","volume":"48","author":"Matveev","year":"2003","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2013","DOI":"10.1109\/TAC.2011.2139370","article-title":"Remote state estimation with communication costs for first-order LTI systems","volume":"56","author":"Lipsa","year":"2011","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1109\/TAC.2016.2580589","article-title":"Fundamental Limits of Remote Estimation of Autoregressive Markov Processes Under Communication Constraints","volume":"62","author":"Chakravorty","year":"2017","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2246","DOI":"10.1109\/TAC.2013.2254615","article-title":"Optimal Strategies for Communication and Remote Estimation With an Energy Harvesting Sensor","volume":"58","author":"Nayyar","year":"2013","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sun, Y., Polyanskiy, Y., and Uysal-Biyikoglu, E. (2017, January 25\u201330). Remote estimation of the Wiener process over a channel with random delay. Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006542"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2533","DOI":"10.1109\/18.945263","article-title":"A zero-delay sequential scheme for lossy coding of individual sequences","volume":"47","author":"Linder","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1109\/18.986015","article-title":"On Limited-Delay Lossy Coding and Filtering of Individual Sequences","volume":"48","author":"Weissman","year":"2002","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2151","DOI":"10.1109\/18.945240","article-title":"Universal prediction of individual binary sequences in the presence of noise","volume":"47","author":"Weisman","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5240","DOI":"10.1109\/TIT.2006.885537","article-title":"Universal Zero-Delay Joint Source-Channel Coding","volume":"52","author":"Matloub","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_32","first-page":"184","article-title":"Nonanticipatory and Prognostic Epsilon Entropies and Message Generation Rates","volume":"9","author":"Gorbunov","year":"1973","journal-title":"Probl. Inform. Transm."},{"key":"ref_33","first-page":"93","article-title":"Prognostic Epsilon Entropy of a Gaussian Message and a Gaussian Source","volume":"10","author":"Gorbunov","year":"1974","journal-title":"Probl. Inform. Transm."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Stavrou, P., Kourtellaris, C.K., and Charalambous, C.D. (2014). Information Nonanticipative Rate Distortion Function and Its Applications. CoRR.","DOI":"10.1109\/ISIT.2014.6875397"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1109\/JSTSP.2018.2855046","article-title":"Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources","volume":"12","author":"Stavrou","year":"2018","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3731","DOI":"10.1137\/17M1116349","article-title":"Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss\u2013Markov Processes","volume":"56","author":"Stavrou","year":"2018","journal-title":"SIAM J. Control Optim."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/18.817521","article-title":"Sequential coding of correlated sources","volume":"46","author":"Viswanathan","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3763","DOI":"10.1109\/TIT.2011.2132990","article-title":"On Delayed Sequential Coding of Correlated Sources","volume":"57","author":"Ma","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1109\/TCNS.2018.2850225","article-title":"Tracking and Control of Gauss\u2013Markov Processes over Packet-Drop Channels with Acknowledgments","volume":"6","author":"Khina","year":"2019","journal-title":"IEEE Trans. Control. Netw. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kipnis, A., and Reeves, G. (2019, January 7\u201312). Gaussian Approximation of Quantization Error for Estimation from Compressed Data. Proceedings of the 2019 IEEE International Symposium on Information Theory (ISIT), Paris, France.","DOI":"10.1109\/ISIT.2019.8849826"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/0167-6911(89)90102-3","article-title":"Extracting state information from a quantized output record","volume":"13","author":"Delchamps","year":"1989","journal-title":"Syst. Control. Lett."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Borkar, V.S., and Mitter, S.K. (1997). LQG Control with Communication Constraints. Communications, Computation, Control, and Signal Processing: A Tribute to Thomas Kailath, Springer.","DOI":"10.1007\/978-1-4615-6281-8_21"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1109\/9.763226","article-title":"Systems with finite communication bandwidth constraints. II. Stabilization with limited information feedback","volume":"44","author":"Wong","year":"1999","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_44","unstructured":"Nair, G.N., and Evans, R.J. (2000, January 12\u201315). Communication-limited stabilization of linear systems. Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No. 00CH37187), Sydney, Australia."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1109\/TAC.2002.808487","article-title":"On stabilization of linear systems with limited information","volume":"48","author":"Liberzon","year":"2003","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2373","DOI":"10.1109\/TAC.2010.2054890","article-title":"Minimum data rate for mean square stabilization of discrete LTI systems over lossy channels","volume":"55","author":"You","year":"2010","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1109\/TAC.2010.2079510","article-title":"Stochastic stabilization of noisy linear systems with fixed-rate limited feedback","volume":"55","author":"Yuksel","year":"2010","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/TAC.2012.2204157","article-title":"Random-time, state-dependent stochastic drift for Markov chains and application to stochastic stabilization over erasure channels","volume":"58","author":"Yuksel","year":"2012","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"6332","DOI":"10.1109\/TIT.2012.2204033","article-title":"Characterization of information channels for asymptotic mean stationarity and stochastic stability of nonstationary\/unstable linear systems","volume":"58","author":"Yuksel","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2844","DOI":"10.1137\/140989686","article-title":"Stationary and ergodic properties of stochastic nonlinear systems controlled over communication channels","volume":"54","author":"Yuksel","year":"2016","journal-title":"SIAM J. Control Optim."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TCOM.1975.1092824","article-title":"Quantization error in predictive coders","volume":"23","author":"Arnstein","year":"1975","journal-title":"IEEE Trans. Commun."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1109\/TIT.1985.1057040","article-title":"Rate-distortion performance of DPCM schemes for autoregressive sources","volume":"31","author":"Farvardin","year":"1985","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_53","unstructured":"Gersho, A., and Gray, R.M. (2012). Vector Quantization and Signal Compression, Springer Science & Business Media."},{"key":"ref_54","unstructured":"Mayekar, P., and Tyagi, H. (2019). RATQ: A Universal Fixed-Length Quantizer for Stochastic Optimization. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Khina, A., Khisti, A., Kostina, V., and Hassibi, B. (2017, January 6\u201310). Sequential coding of Gauss-Markov sources with packet erasures and feedback. Proceedings of the 2017 IEEE Information Theory Workshop (ITW), Kaohsiung, Taiwan.","DOI":"10.1109\/ITW.2017.8277955"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Cover, T., and Thomas, J. (2006). Elements of Information Theory, Wiley.","DOI":"10.1002\/047174882X"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/3\/347\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:35:54Z","timestamp":1760160954000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/3\/347"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,15]]},"references-count":56,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["e23030347"],"URL":"https:\/\/doi.org\/10.3390\/e23030347","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,3,15]]}}}