{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,26]],"date-time":"2025-12-26T07:13:32Z","timestamp":1766733212985,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T00:00:00Z","timestamp":1737936000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["52474270"],"award-info":[{"award-number":["52474270"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Traditional entropy source evaluation methods rely on statistical analysis and are hard to deploy on-chip or online. However, online detection of entropy source quality is necessary in some applications with high encryption levels. To address these issues, our experimental results demonstrate a significant negative correlation between minimum entropy values and prediction accuracy, with a Pearson correlation coefficient of \u22120.925 (p-value = 1.07 \u00d7 10\u22127). This finding offers a novel approach for assessing entropy source quality, achieving an accurate rate in predicting the next bit of a random sequence using neural networks. To further improve prediction capabilities, we also propose a novel deep learning architecture, Fast Fourier Transform-Attention Mechanism-Long Short-Term Memory Network (FFT-ATT-LSTM), that integrates a simplified soft attention mechanism with Fast Fourier Transform (FFT), enabling effective fusion of time-domain and frequency-domain features. The FFT-ATT-LSTM improves prediction accuracy by 4.46% and 8% over baseline networks when predicting random numbers. Additionally, FFT-ATT-LSTM maintains a compact parameter size of 33.90 KB, significantly smaller than Temporal Convolutional Networks (TCN) at 41.51 KB and Transformers at 61.51 KB, while retaining comparable prediction performance. This optimal balance between accuracy and resource efficiency makes FFT-ATT-LSTM suitable for online deployment, demonstrating considerable application potential.<\/jats:p>","DOI":"10.3390\/e27020136","type":"journal-article","created":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T11:38:49Z","timestamp":1737977929000},"page":"136","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["An Online Evaluation Method for Random Number Entropy Sources Based on Time-Frequency Feature Fusion"],"prefix":"10.3390","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-3128-7264","authenticated-orcid":false,"given":"Qian","family":"Sun","sequence":"first","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1379-591X","authenticated-orcid":false,"given":"Kainan","family":"Ma","sequence":"additional","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Yiheng","family":"Zhou","sequence":"additional","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Zhaoyuxuan","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Chaoxing","family":"You","sequence":"additional","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"}]},{"given":"Ming","family":"Liu","sequence":"additional","affiliation":[{"name":"Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,27]]},"reference":[{"key":"ref_1","first-page":"595","article-title":"Optical quantum random number generator","volume":"47","author":"Stefanov","year":"2000","journal-title":"J. Mod. Opt."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hastings, M., Fried, J., and Heninger, N. (2016, January 16). Weak keys remain widespread in network devices. Proceedings of the 2016 Internet Measurement Conference, Piscataway, NJ, USA.","DOI":"10.1145\/2987443.2987486"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ghafoor, I., Jattala, I., Durrani, S., and Tahir, C.M. (2014, January 8\u201310). Analysis of OpenSSL Heartbleed vulnerability for embedded systems. Proceedings of the 17th IEEE International Multi Topic Conference 2014, Karachi, Pakistan.","DOI":"10.1109\/INMIC.2014.7097358"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1007\/s10836-019-05818-8","article-title":"Security analysis and improvement of the pseudo-random number generator based on piecewise logistic map","volume":"35","year":"2019","journal-title":"J. Electron. Test."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/j.future.2019.08.034","article-title":"ECDSA weak randomness in Bitcoin","volume":"102","author":"Wang","year":"2020","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_6","first-page":"79","article-title":"Implementation of a digital TRNG using jitter based multiple entropy source on FPGA","volume":"49","author":"Garipcan","year":"2019","journal-title":"Inf. MIDEM"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Huang, M., Chen, Z., Zhang, Y., and Guo, H. (2020). A Gaussian-distributed quantum random number generator using vacuum shot noise. Entropy, 22.","DOI":"10.3390\/e22060618"},{"key":"ref_8","first-page":"1","article-title":"Real-time 2.5-Gb\/s correlated random bit generation using synchronized chaos induced by a common laser with dispersive feedback","volume":"56","author":"Wang","year":"2019","journal-title":"IEEE J. Quantum Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3686","DOI":"10.1364\/OE.382234","article-title":"Entropy evaluation of white chaos generated by optical heterodyne for certifying physical random number generators","volume":"28","author":"Yoshiya","year":"2020","journal-title":"Opt. Express"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1109\/TIFS.2018.2850770","article-title":"Machine learning cryptanalysis of a quantum random number generator","volume":"14","author":"Truong","year":"2018","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rukhin, A., Soto, J., Nechvatal, J., Smid, M., Barker, E., Leigh, S., Levenson, M., Vangel, M., Banks, D., and Heckert, A. (2001). A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications, US Department of Commerce, Technology Administration, National Institute of Standards and Technology.","DOI":"10.6028\/NIST.SP.800-22"},{"key":"ref_12","unstructured":"Killmann, W., and Schindler, W. (2001). AIS 31: Functionality Classes and Evaluation Methodology for True (Physical) Random Number Generators, version 3.1, Bundesamt fur Sicherheit in der Informationstechnik (BSI)."},{"key":"ref_13","unstructured":"Brown, R.G., Eddelbuettel, D., and Bauer, D. (2018). Dieharder, Duke University Physics Department Durham."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1268776.1268777","article-title":"TestU01: AC library for empirical testing of random number generators","volume":"33","author":"Simard","year":"2007","journal-title":"ACM Trans. Math. Softw. (TOMS)"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Maksutov, A.A., Goryushkin, P.N., Gerasimov, A.A., and Orlov, A.A. (February, January 29). PRNG assessment tests based on neural networks. Proceedings of the 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), Moscow, Russia.","DOI":"10.1109\/EIConRus.2018.8317101"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"22987","DOI":"10.1109\/ACCESS.2018.2826448","article-title":"Learning from pseudo-randomness with an artificial neural network\u2013does god play pseudo-dice?","volume":"6","author":"Fan","year":"2018","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, J., Zhu, S., Chen, T., Ma, Y., Lv, N., and Lin, J. (2018, January 8\u201310). Neural network based min-entropy estimation for random number generators. Proceedings of the International Conference on Security and Privacy in Communication Systems, Singapore.","DOI":"10.1007\/978-3-030-01704-0_13"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/TNNLS.2017.2716952","article-title":"Broad learning system: An effective and efficient incremental learning system without the need for deep architecture","volume":"29","author":"Chen","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/TCYB.2018.2857815","article-title":"Fuzzy broad learning system: A novel neuro-fuzzy model for regression and classification","volume":"50","author":"Feng","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1809","DOI":"10.1109\/TKDE.2018.2866149","article-title":"Structured manifold broad learning system: A manifold perspective for large-scale chaotic time series analysis and prediction","volume":"31","author":"Han","year":"2018","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1405","DOI":"10.1109\/TCYB.2018.2863020","article-title":"Recurrent broad learning systems for time series prediction","volume":"50","author":"Xu","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1049\/el.2019.0485","article-title":"Machine learning\u2014Resistant pseudo\u2014Random number generator","volume":"55","author":"Wen","year":"2019","journal-title":"Electron. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, C., Zhang, J., Sang, L., Gong, L., Wang, L., Wang, A., and Wang, Y. (2020). Deep learning-based security verification for a random number generator using white chaos. Entropy, 22.","DOI":"10.3390\/e22101134"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Nagy, I., and Suciu, A. (2021, January 28\u201330). Randomness testing with neural networks. Proceedings of the 2021 IEEE 17th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.","DOI":"10.1109\/ICCP53602.2021.9733481"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, Z., Feng, B., Cui, L., Wang, H., Bian, Y., Piao, G., and Zhou, X. (2023, January 20\u201322). Quantify Randomness of Quantum Random Number with Transformer Network. Proceedings of the 2023 3rd International Conference on Intelligent Power and Systems (ICIPS), Shenzhen, China.","DOI":"10.1109\/ICIPS59254.2023.10405286"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"104785","DOI":"10.1016\/j.knosys.2019.05.028","article-title":"EA-LSTM: Evolutionary attention-based LSTM for time series prediction","volume":"181","author":"Li","year":"2019","journal-title":"Knowl. Based Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yuan, Y., Jia, K., Ma, F., Xun, G., Wang, Y., Su, L., and Zhang, A. (2019). A hybrid self-attention deep learning framework for multivariate sleep stage classification. BMC Bioinform., 20.","DOI":"10.1186\/s12859-019-3075-z"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"117081","DOI":"10.1016\/j.energy.2020.117081","article-title":"Wind power forecasting using attention-based gated recurrent unit network","volume":"196","author":"Niu","year":"2020","journal-title":"Energy"},{"key":"ref_30","first-page":"74","article-title":"FPGA based true random number generators for embedded cryptographic applications","volume":"1","author":"Varchola","year":"2008","journal-title":"Semant. Sch."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1754386.1754390","article-title":"Improving the Robustness of Ring Oscillator TRNGs","volume":"3","author":"Yoo","year":"2010","journal-title":"ACM Trans. Reconfig. Technol. Syst."},{"key":"ref_32","unstructured":"S\u00f6nmez Turan, M., Barker, E., Kelsey, J., McKay, K., Baish, M., and Boyle, M. (2016). Recommendation for the Entropy Sources Used for Random Bit Generation, National Institute of Standards and Technology. No. NIST Special Publication (SP) 800-90B (Draft)."},{"key":"ref_33","first-page":"102","article-title":"Recommendation for the entropy sources used for random bit generation","volume":"800","author":"Turan","year":"2018","journal-title":"NIST Spec. Publ."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3168","DOI":"10.1109\/TII.2019.2902129","article-title":"Nonlinear dynamic soft sensor modeling with supervised long short-term memory network","volume":"16","author":"Yuan","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pienaar, S.W., and Malekian, R. (2019, January 19). Human activity recognition using LSTM-RNN deep neural network architecture. Proceedings of the 2019 IEEE 2nd wireless africa conference (WAC), Pretoria, South Africa.","DOI":"10.1109\/AFRICA.2019.8843403"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"872","DOI":"10.1038\/s41467-017-00839-3","article-title":"Bypassing the Kohn-Sham equations with machine learning","volume":"8","author":"Brockherde","year":"2017","journal-title":"Nat. Commun."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/BF02551274","article-title":"Approximation by superpositions of a sigmoidal function","volume":"2","author":"Cybenko","year":"1989","journal-title":"Math. Control Signals Syst."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/2\/136\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:37:21Z","timestamp":1759919841000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/2\/136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,27]]},"references-count":37,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["e27020136"],"URL":"https:\/\/doi.org\/10.3390\/e27020136","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2025,1,27]]}}}