{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T06:46:35Z","timestamp":1767854795478,"version":"3.49.0"},"reference-count":80,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,6,28]],"date-time":"2019-06-28T00:00:00Z","timestamp":1561680000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The Predictive Rate\u2013Distortion curve quantifies the trade-off between compressing information about the past of a stochastic process and predicting its future accurately. Existing estimation methods for this curve work by clustering finite sequences of observations or by utilizing analytically known causal states. Neither type of approach scales to processes such as natural languages, which have large alphabets and long dependencies, and where the causal states are not known analytically. We describe Neural Predictive Rate\u2013Distortion (NPRD), an estimation method that scales to such processes, leveraging the universal approximation capabilities of neural networks. Taking only time series data as input, the method computes a variational bound on the Predictive Rate\u2013Distortion curve. We validate the method on processes where Predictive Rate\u2013Distortion is analytically known. As an application, we provide bounds on the Predictive Rate\u2013Distortion of natural language, improving on bounds provided by clustering sequences. Based on the results, we argue that the Predictive Rate\u2013Distortion curve is more useful than the usual notion of statistical complexity for characterizing highly complex processes such as natural language.<\/jats:p>","DOI":"10.3390\/e21070640","type":"journal-article","created":{"date-parts":[[2019,6,28]],"date-time":"2019-06-28T11:20:26Z","timestamp":1561720826000},"page":"640","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Estimating Predictive Rate\u2013Distortion Curves via Neural Variational Inference"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4828-4834","authenticated-orcid":false,"given":"Michael","family":"Hahn","sequence":"first","affiliation":[{"name":"Department of Linguistics, Stanford University, Stanford, CA 94305, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richard","family":"Futrell","sequence":"additional","affiliation":[{"name":"Department of Language Science, University of California, Irvine, CA 92697, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"968","DOI":"10.3390\/e16020968","article-title":"Information Bottleneck Approach to Predictive Inference","volume":"16","author":"Still","year":"2014","journal-title":"Entropy"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1312","DOI":"10.1007\/s10955-016-1520-1","article-title":"Predictive Rate-Distortion for Infinite-Order Markov Processes","volume":"163","author":"Marzen","year":"2016","journal-title":"J. Stat. Phys."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Creutzig, F., Globerson, A., and Tishby, N. (2009). Past-future information bottleneck in dynamical systems. Phys. Rev. E, 79.","DOI":"10.1103\/PhysRevE.79.041925"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Amir, N., Tiomkin, S., and Tishby, N. (2015, January 15\u201318). Past-future Information Bottleneck for linear feedback systems. Proceedings of the 54th IEEE Conference on Decision and Control (CDC), Osaka, Japan.","DOI":"10.1109\/CDC.2015.7403120"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"27","DOI":"10.3389\/frobt.2015.00027","article-title":"Bounded rationality, abstraction, and hierarchical decision-making: An information-theoretic optimality principle","volume":"2","author":"Genewein","year":"2015","journal-title":"Front. Robot. AI"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"037111","DOI":"10.1063\/1.3489885","article-title":"Optimal causal inference: Estimating stored information and approximating causal architecture","volume":"20","author":"Still","year":"2010","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_7","unstructured":"J\u00f3zefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., and Wu, Y. (2016). Exploring the Limits of Language Modeling. arXiv."},{"key":"ref_8","unstructured":"Merity, S., Keskar, N.S., and Socher, R. (2018). An analysis of neural language modeling at multiple scales. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Dai, Z., Yang, Z., Yang, Y., Cohen, W.W., Carbonell, J., Le, Q.V., and Salakhutdinov, R. (2019). Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. arXiv.","DOI":"10.18653\/v1\/P19-1285"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Takahashi, S., and Tanaka-Ishii, K. (2018). Cross Entropy of Neural Language Models at Infinity\u2014A New Bound of the Entropy Rate. Entropy, 20.","DOI":"10.3390\/e20110839"},{"key":"ref_11","unstructured":"Ogunmolu, O., Gu, X., Jiang, S., and Gans, N. (2016). Nonlinear systems identification using deep dynamic neural networks. arXiv."},{"key":"ref_12","unstructured":"Laptev, N., Yosinski, J., Li, L.E., and Smyl, S. (2017, January 11). Time-series extreme event forecasting with neural networks at uber. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.compbiomed.2018.05.018","article-title":"Survey on deep learning for radiotherapy","volume":"98","author":"Meyer","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Soderstrom, T. (2018, January 19\u201323). Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3219845"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"White, G., Palade, A., and Clarke, S. (2018, January 8\u201313). Forecasting qos attributes using lstm networks. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489052"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.apor.2018.06.011","article-title":"Dynamic model identification of unmanned surface vehicles using deep learning network","volume":"78","author":"Woo","year":"2018","journal-title":"Appl. Ocean Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sirignano, J., and Cont, R. (2018). Universal features of price formation in financial markets: perspectives from Deep Learning. arXiv.","DOI":"10.2139\/ssrn.3141294"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mohajerin, N., and Waslander, S.L. (2019). Multistep Prediction of Dynamic Systems With Recurrent Neural Networks. IEEE Transa. Neural Netw. Learn. Syst.","DOI":"10.1109\/TNNLS.2019.2891257"},{"key":"ref_19","unstructured":"Rezende, D.J., and Mohamed, S. (2015, January 6\u201311). Variational inference with normalizing flows. Proceedings of the 32nd International Conference on International Conference on Machine Learning, Lille, France."},{"key":"ref_20","unstructured":"Huang, C.W., Krueger, D., Lacoste, A., and Courville, A. (2018, January 10\u201315). Neural Autoregressive Flows. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_21","unstructured":"Tishby, N., Pereira, F.C., and Bialek, W. (1999, January 22\u201324). The Information Bottleneck Method. Proceedings of the Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Harremo\u00ebs, P., and Tishby, N. (2007, January 24\u201329). The information bottleneck revisited or how to choose a good distortion measure. Proceedings of the IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557285"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1142\/S0219525904000196","article-title":"Synchronizing to Periodicity: The Transient Information and Synchronization Time of Periodic Sequences","volume":"7","author":"Feldman","year":"2004","journal-title":"Adv. Complex Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1103\/PhysRevLett.63.105","article-title":"Inferring statistical complexity","volume":"63","author":"Crutchfield","year":"1989","journal-title":"Phys. Rev. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1007\/BF00668821","article-title":"Toward a quantitative theory of self-generated complexity","volume":"25","author":"Grassberger","year":"1986","journal-title":"Int. J. Theor. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"385","DOI":"10.3390\/e110300385","article-title":"Properties of the Statistical Complexity Functional and Partially Deterministic HMMs","volume":"11","year":"2009","journal-title":"Entropy"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"016203","DOI":"10.1103\/PhysRevE.67.016203","article-title":"Application of computational mechanics to the analysis of natural data: An example in geomagnetism","volume":"67","author":"Clarke","year":"2003","journal-title":"Phys. Rev. E"},{"key":"ref_28","unstructured":"Singh, S.P., Littman, M.L., Jong, N.K., Pardoe, D., and Stone, P. (2003, January 21\u201324). Learning predictive state representations. Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA."},{"key":"ref_29","unstructured":"Singh, S., James, M.R., and Rudary, M.R. (2004). Predictive state representations: A new theory for modeling dynamical systems. Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, AUAI Press."},{"key":"ref_30","unstructured":"Jaeger, H. (1998). Discrete-Time, Discrete-Valued Observable Operator Models: A Tutorial, GMD-Forschungszentrum Informationstechnik."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Rubin, J., Shamir, O., and Tishby, N. (2012). Trading value and information in MDPs. Decision Making with Imperfect Decision Makers, Springer.","DOI":"10.1007\/978-3-642-24647-0_3"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_33","unstructured":"Kingma, D.P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M. (2016). Improved variational inference with inverse autoregressive flow. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_34","unstructured":"Papamakarios, G., Pavlakou, T., and Murray, I. (2017). Masked autoregressive flow for density estimation. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_35","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_36","unstructured":"Kingma, D.P., and Welling, M. (2014, January 14\u201316). Auto-Encoding Variational Bayes. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada."},{"key":"ref_37","unstructured":"McAllester, D., and Statos, K. (2018). Formal Limitations on the Measurement of Mutual Information. arXiv."},{"key":"ref_38","unstructured":"Alemi, A.A., Fischer, I., Dillon, J.V., and Murphy, K. (2017, January 24\u201326). Deep Variational Information Bottleneck. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France."},{"key":"ref_39","unstructured":"Grathwohl, W., and Wilson, A. (2016). Disentangling space and time in video with hierarchical variational auto-encoders. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Walker, J., Doersch, C., Gupta, A., and Hebert, M. (2016). An uncertain future: Forecasting from static images using variational autoencoders. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46478-7_51"},{"key":"ref_41","unstructured":"Fraccaro, M., Kamronn, S., Paquet, U., and Winther, O. (2017). A disentangled recognition and nonlinear dynamics model for unsupervised learning. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"062412","DOI":"10.1103\/PhysRevE.97.062412","article-title":"Variational encoding of complex dynamics","volume":"97","author":"Sultan","year":"2018","journal-title":"Phys. Rev. E"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Bowman, S.R., Vilnis, L., Vinyals, O., Dai, A.M., Jozefowicz, R., and Bengio, S. (2016, January 11\u201312). Generating Sentences from a Continuous Space. Proceedings of the CoNLL, Berlin, Germany.","DOI":"10.18653\/v1\/K16-1002"},{"key":"ref_44","unstructured":"Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. (2017, January 24\u201326). \u03b2-VAE: Learning basic visual concepts with a constrained variational framework. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_45","unstructured":"Burgess, C., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. (2018). Understanding disentangling in \u03b2-VAE. arXiv."},{"key":"ref_46","unstructured":"Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2019, June 28). Automatic Differentiation in PyTorch 2017. Available online: https:\/\/openreview.net\/forum?id=BJJsrmfCZ."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1002\/j.1538-7305.1951.tb01366.x","article-title":"Prediction and entropy of printed English","volume":"30","author":"Shannon","year":"1951","journal-title":"Bell Syst. Tech. J."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Takahira, R., Tanaka-Ishii, K., and D\u0119bowski, \u0141. (2016). Entropy rate estimates for natural language\u2014A new extrapolation of compressed large-scale corpora. Entropy, 18.","DOI":"10.3390\/e18100364"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Bentz, C., Alikaniotis, D., Cysouw, M., and Ferrer-i Cancho, R. (2017). The entropy of words\u2014Learnability and expressivity across more than 1000 languages. Entropy, 19.","DOI":"10.20944\/preprints201704.0180.v1"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hale, J. (2001, January 1\u20137). A Probabilistic Earley Parser as a Psycholinguistic Model. Proceedings of the NAACL, Pittsburgh, PA, USA.","DOI":"10.3115\/1073336.1073357"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1016\/j.cognition.2007.05.006","article-title":"Expectation-based syntactic comprehension","volume":"106","author":"Levy","year":"2008","journal-title":"Cognition"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.cognition.2013.02.013","article-title":"The effect of word predictability on reading time is logarithmic","volume":"128","author":"Smith","year":"2013","journal-title":"Cognition"},{"key":"ref_53","unstructured":"Frank, S.L., Otten, L.J., Galli, G., and Vigliocco, G. (2013, January 4\u20139). Word surprisal predicts N400 amplitude during reading. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1080\/23273798.2015.1102299","article-title":"What do we mean by prediction in language comprehension?","volume":"31","author":"Kuperberg","year":"2016","journal-title":"Lang. Cogn. Neurosci."},{"key":"ref_55","first-page":"400","article-title":"Konstanz im Kurzzeitged\u00e4chtnis\u2014Konstanz im sprachlichen Informationsflu\u00df","volume":"27","author":"Fenk","year":"1980","journal-title":"Z. Exp. Angew. Psychol."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Genzel, D., and Charniak, E. (2002, January 7\u201312). Entropy rate constancy in text. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073117"},{"key":"ref_57","unstructured":"Jaeger, T.F., and Levy, R.P. (2007, January 3\u20136). Speakers optimize information density through syntactic reduction. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1142\/S0218348X93000083","article-title":"Long range correlation in human writings","volume":"1","author":"Schenkel","year":"1993","journal-title":"Fractals"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1209\/0295-5075\/26\/4\/001","article-title":"Entropy and long-range correlations in literary English","volume":"26","author":"Ebeling","year":"1994","journal-title":"EPL (Europhys. Lett.)"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/0378-4371(95)00025-3","article-title":"Long-range correlations between letters and sentences in texts","volume":"215","author":"Ebeling","year":"1995","journal-title":"Phys. A Stat. Mech. Appl."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"11582","DOI":"10.1073\/pnas.1117723109","article-title":"On the origin of long-range correlations in texts","volume":"109","author":"Altmann","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Yang, T., Gu, C., and Yang, H. (2016). Long-range correlations in sentence series from A Story of the Stone. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0162423"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"9362468","DOI":"10.1155\/2018\/9362468","article-title":"Quantifying evolution of short and long-range correlations in Chinese narrative texts across 2000 years","volume":"2018","author":"Chen","year":"2018","journal-title":"Complexity"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"D\u0119bowski, \u0141. (2018). Is natural language a perigraphic process? The theorem about facts and words revisited. Entropy, 20.","DOI":"10.3390\/e20020085"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Koplenig, A., Meyer, P., Wolfer, S., and Mueller-Spitzer, C. (2017). The statistical trade-off between word order and word structure\u2013Large-scale evidence for the principle of least effort. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0173614"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0010-0277(98)00034-1","article-title":"Linguistic complexity: locality of syntactic dependencies","volume":"68","author":"Gibson","year":"1998","journal-title":"Cognition"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Futrell, R., and Levy, R. (2017, January 3\u20137). Noisy-context surprisal as a human sentence processing cost model. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain.","DOI":"10.18653\/v1\/E17-1065"},{"key":"ref_68","unstructured":"Petrov, S., Das, D., and McDonald, R.T. (2012, January 23\u201325). A Universal Part-of-Speech Tagset. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012), Istanbul, Turkey."},{"key":"ref_69","unstructured":"Nivre, J., Agic, Z., Ahrenberg, L., Antonsen, L., Aranzabe, M.J., Asahara, M., Ateyah, L., Attia, M., Atutxa, A., and Augustinus, L. (2019, June 28). Universal Dependencies 2.1. Available online: https:\/\/lindat.mff.cuni.cz\/repository\/xmlui\/handle\/11234\/1-2515."},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Kim, Y., Jernite, Y., Sontag, D., and Rush, A.M. (2016, January 12\u201317). Character-aware neural language models. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10362"},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Luong, M.T., and Manning, C.D. (2016). Achieving open vocabulary neural machine translation with hybrid word-character models. arXiv.","DOI":"10.18653\/v1\/P16-1100"},{"key":"ref_72","first-page":"313","article-title":"Building a large annotated corpus of English: The Penn Treebank","volume":"19","author":"Marcus","year":"1993","journal-title":"Comput. Linguist."},{"key":"ref_73","unstructured":"Nivre, J., de Marneffe, M.C., Ginter, F., Goldberg, Y., Hajic, J., Manning, C.D., McDonald, R.T., Petrov, S., Pyysalo, S., and Silveira, N. (2016, January 23\u201328). Universal Dependencies v1: A Multilingual Treebank Collection. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portoro\u017e, Slovenia."},{"key":"ref_74","unstructured":"Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004, January 27\u201329). The penn arabic treebank: Building a large-scale annotated arabic corpus. Proceedings of the NEMLAR Conference on Arabic Language Resources and Tools, Cairo, Egypt."},{"key":"ref_75","unstructured":"Hajic, J., Smrz, O., Zem\u00e1nek, P., \u0160naidauf, J., and Be\u0161ka, E. (2004, January 22\u201323). Prague Arabic dependency treebank: Development in data and tools. Proceedings of the NEMLAR Internaional Conference on Arabic Language Resources and Tools, Cairo, Egypt."},{"key":"ref_76","first-page":"272","article-title":"Sovremennoe sostoyanie gluboko annotirovannogo korpusa tekstov russkogo yazyka (SinTagRus)","volume":"10","author":"Dyachenko","year":"2015","journal-title":"Trudy Instituta Russkogo Yazyka im. VV Vinogradova"},{"key":"ref_77","unstructured":"Che, W., Li, Z., and Liu, T. (2012). Chinese Dependency Treebank 1.0 LDC2012T05, Linguistic Data Consortium. Web Download."},{"key":"ref_78","unstructured":"Graff, D., and Wu, Z. (1995). Japanese bUsiness News Text, Linguistic Data Consortium. LDC95T8."},{"key":"ref_79","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_80","unstructured":"Bradbury, J., Merity, S., Xiong, C., and Socher, R. (2017, January 24\u201326). Quasi-recurrent neural networks. Proceedings of the ICLR 2017, Toulon, France."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/7\/640\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:02:13Z","timestamp":1760187733000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/7\/640"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,28]]},"references-count":80,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["e21070640"],"URL":"https:\/\/doi.org\/10.3390\/e21070640","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,28]]}}}