{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T21:51:54Z","timestamp":1767649914367,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T00:00:00Z","timestamp":1543881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Deep belief networks (DBNs) of deep learning technology have been successfully used in many fields. However, the structure of a DBN is difficult to design for different datasets. Hence, a DBN structure design algorithm based on information entropy and reconstruction error is proposed. Unlike previous algorithms, we innovatively combine network depth and node number and optimizes them simultaneously. First, the mathematical model of the structural design problem is established, and the boundary constraint for node number based on information entropy is derived by introducing the idea of information compression. Moreover, the optimization objective of the network performance based on reconstruction error is proposed by deriving the fact that network energy is proportional to reconstruction error. Finally, the improved simulated annealing (ISA) algorithm is used to adjust the DBN network layers and nodes simultaneously. Experiments were carried out on three public datasets (MNIST, Cifar-10 and Cifar-100). The results show that the proposed algorithm can design its proper structure to different datasets, yielding a trained DBN which has the lowest reconstruction error and prediction error rate. The proposed algorithm is shown to have the best performance compared with other algorithms and can be used to assist the setting of DBN structural parameters for different datasets.<\/jats:p>","DOI":"10.3390\/e20120927","type":"journal-article","created":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T11:56:18Z","timestamp":1543924578000},"page":"927","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["DBN Structure Design Algorithm for Different Datasets Based on Information Entropy and Reconstruction Error"],"prefix":"10.3390","volume":"20","author":[{"given":"Jianjun","family":"Jiang","sequence":"first","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]},{"given":"Jing","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]},{"given":"Lijia","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]},{"given":"Xiaomin","family":"Ran","sequence":"additional","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]},{"given":"Jun","family":"Jiang","sequence":"additional","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]},{"given":"Yifan","family":"Wu","sequence":"additional","affiliation":[{"name":"National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China"}]}],"member":"1968","published-online":{"date-parts":[[2018,12,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1162\/neco.2008.04-07-510","article-title":"Representational power of restricted Boltzmann machines and deep belief networks","volume":"20","author":"Nicolas","year":"2008","journal-title":"Neural Comput."},{"key":"ref_2","unstructured":"Haykin, S. (2011). Neural Network and Machine Learning, China Machine Press."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_4","unstructured":"Hinton, G.E. (2012). Neural Networks: Tricks of the Trade, Springer."},{"key":"ref_5","first-page":"567","article-title":"Deep learning","volume":"521","author":"Yann","year":"2015","journal-title":"Nature"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Taigman, Y., Yang, M., Marc\u2019Aurelio, R., and Wolf, L. (2014, January 23\u201328). Deepface: Closing the gap to human-level performance in face verification. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.220"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Toshev, A., and Szegedy, C. (2014, January 23\u201328). Deeppose: Human pose estimation via deep neural networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.214"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Denton, E., Weston, J., Paluri, M., Bourdev, L., and Fergus, R. (2015, January 10\u201313). User conditional hashtag prediction for images. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.","DOI":"10.1145\/2783258.2788576"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.patcog.2016.05.028","article-title":"Discriminant deep belief network for high-resolution SAR image classification","volume":"61","author":"Zhiqiang","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wu, Z., Chen, T., Chen, Y., Zhang, Z., and Liu, G. (2017). NIRExpNet: Three-stream 3D convolutional neural network for near infrared facial expression recognition. Appl. Sci., 7.","DOI":"10.3390\/app7111184"},{"key":"ref_11","first-page":"2972","article-title":"Research on low probability of intercept radar signal recognition using deep belief network and bispectra diagonal slice","volume":"38","author":"Xing","year":"2016","journal-title":"J. Electron. Inf. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhu, L., Chen, L., Zhao, D., Zhou, J., and Zhang, W. (2017). Emotion recognition from Chinese speech for smart affective services using a combination of SVM and DBN. Sensors, 17.","DOI":"10.3390\/s17071694"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Su, R., Chen, X., Cao, S., and Zhang, X. (2016). Random forest-based recognition of isolated sign language subwords using data from accelerometers and surface electromyographic sensors. Sensors, 16.","DOI":"10.3390\/s16010100"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tang, Q., Chai, Y., Qu, J., and Ren, H. (2018). Fisher discriminative sparse representation based on DBN for fault diagnosis of complex system. Appl. Sci., 8.","DOI":"10.3390\/app8050795"},{"key":"ref_15","first-page":"256","article-title":"Calculation for depth of deep belief network","volume":"30","author":"Guangyuan","year":"2015","journal-title":"Control Decis."},{"key":"ref_16","first-page":"234","article-title":"Research and application of the level of the deep belief betwork (DBN)","volume":"16","author":"Qiang","year":"2016","journal-title":"Sci. Technol. Eng."},{"key":"ref_17","first-page":"843","article-title":"Alternating update layers for DBN-DNN fast training method","volume":"33","author":"Xuan","year":"2016","journal-title":"Appl. Res. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2133","DOI":"10.1080\/01431160802549278","article-title":"How many hidden layers and nodes?","volume":"30","author":"Stathakis","year":"2009","journal-title":"Int. J. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sermanet, P., Kavukcuoglu, K., Chintala, S., and Lecun, Y. (2013, January 23\u201328). Pedestrian detection with unsupervised multi-stage feature learning. Proceedings of the IEEE 2013 Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.465"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2014","journal-title":"Neural Netw."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1007\/978-3-642-35289-8_26","article-title":"Practical recommendations for gradient-based training of deep architectures","volume":"Volume 7700","author":"Bengio","year":"2012","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_22","first-page":"1042","article-title":"Algorithm of depth neural network training based on multi-GPU","volume":"36","author":"Naijie","year":"2015","journal-title":"J. Chin. Comput. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kim, S.K., McAfee, L.C., McMahon, P.L., and Olukotun, K. (September, January 31). A highly scalable Restricted Boltzmann Machine FPGA implementation. Proceedings of the 2009 International Conference on Field Programmable Logic and Applications, Prague, Czech Republic.","DOI":"10.1109\/FPL.2009.5272262"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1162\/neco.2008.04-07-510","article-title":"Representational power of restricted Boltzmann machines and deep belief networks","volume":"20","author":"Roux","year":"2008","journal-title":"Neural Comput."},{"key":"ref_25","first-page":"159","article-title":"Restricted Boltzmann Machine","volume":"3","author":"Chunxia","year":"2015","journal-title":"Chin. J. Eng. Math."},{"key":"ref_26","first-page":"129","article-title":"Simulated Annealing Algorithm of Global Search for Critical Sliding Surface of Slope and Improvements","volume":"37","author":"Hao","year":"2007","journal-title":"J. Jilin Univ."},{"key":"ref_27","unstructured":"Lishan, K., Yun, X., and Shiyong, Y. (1997). Non-Numerical Parallel Computing: Simulated Annealing Algorithm, Science Press."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000006","article-title":"Learning Deep Architectures for AI","volume":"2","author":"Bengio","year":"2009","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_29","unstructured":"Asuncion, A., and Newman, D. (2018, October 10). UCI Machine Learning Repository. Available online: http:\/\/archive.ics.uci.edu\/ml."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/12\/927\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:31:06Z","timestamp":1760196666000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/12\/927"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,4]]},"references-count":29,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2018,12]]}},"alternative-id":["e20120927"],"URL":"https:\/\/doi.org\/10.3390\/e20120927","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,12,4]]}}}