{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:22:25Z","timestamp":1750220545604,"version":"3.41.0"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2016YFB1100205"],"award-info":[{"award-number":["2016YFB1100205"]}]},{"name":"Beijing Science and Technology Special Project","award":["Z161100004916009"],"award-info":[{"award-number":["Z161100004916009"]}]},{"name":"Beijing Science and Technology Planning Project Support","award":["Z161100001516007"],"award-info":[{"award-number":["Z161100001516007"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2021,9,30]]},"abstract":"<jats:p>The Internet of Things and big data are currently hot concepts and research fields. The mining, classification, and recognition of big data in the Internet of Things system are the key links that are widely of concern at present. The artificial neural network is beneficial for multi-dimensional data classification and recognition because of its strong feature extraction and self-learning ability. Pre-training is an effective method to address the gradient diffusion problem in deep neural networks and could result in better generalization. This article focuses on the performance of supervised pre-training that uses labelled data. In particular, this pre-training procedure is a simulation that shows the changes in judgment patterns as they progress from primary to mature within the human brain. In this article, the state-of-the-art of neural network pre-training is reviewed. Then, the principles of the auto-encoder and supervised pre-training are introduced in detail. Furthermore, an extended structure of supervised pre-training is proposed. A set of experiments are carried out to compare the performances of different pre-training methods. These experiments include a comparison between the original and pre-trained networks as well as a comparison between the networks with two types of sub-network structures. In addition, a homemade database is established to analyze the influence of pre-training on the generalization ability of neural networks. Finally, an ordinary convolutional neural network is used to verify the applicability of supervised pre-training.<\/jats:p>","DOI":"10.1145\/3433539","type":"journal-article","created":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T21:12:38Z","timestamp":1626901958000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Research On Pre-Training Method and Generalization Ability of Big Data Recognition Model of the Internet of Things"],"prefix":"10.1145","volume":"20","author":[{"given":"Junyang","family":"Tan","sequence":"first","affiliation":[{"name":"National Key Laboratory for Remanufacturing, Beijing, China and The Department of 63926 Troops, Beijing, China"}]},{"given":"Dan","family":"Xia","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Remanufacturing, Beijing, China"}]},{"given":"Shiyun","family":"Dong","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Remanufacturing, Beijing, China"}]},{"given":"Honghao","family":"Zhu","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Remanufacturing, Beijing, China"}]},{"given":"Binshi","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Remanufacturing, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2021,7,21]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"1533","article-title":"Big data effective information filtering mining of Internet of Things based on SVM","volume":"23","author":"Li Y. W.","year":"2016","unstructured":"Y. W. Li and L. Ma . 2016 . Big data effective information filtering mining of Internet of Things based on SVM . Control Engineering of China 23 , 10 (2016), 1533 \u2013 1537 . Y. W. Li and L. Ma. 2016. Big data effective information filtering mining of Internet of Things based on SVM. Control Engineering of China 23, 10 (2016), 1533\u20131537.","journal-title":"Control Engineering of China"},{"key":"e_1_2_1_2_1","first-page":"149","article-title":"Differential epidemic model of virus and worms in computer network","volume":"14","author":"Bimal K. M.","year":"2012","unstructured":"K. M. Bimal and M. A. Gholam . 2012 . Differential epidemic model of virus and worms in computer network . International Journal of Network Security 14 , 3 (2012), 149 \u2013 155 . K. M. Bimal and M. A. Gholam. 2012. Differential epidemic model of virus and worms in computer network. International Journal of Network Security 14, 3 (2012), 149\u2013155.","journal-title":"International Journal of Network Security"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1093\/clinchem\/40.9.1692"},{"volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917)","author":"Zhou S.","key":"e_1_2_1_4_1","unstructured":"S. Zhou , J. Wang , and J. Wang et al. 2017. Point to set similarity based deep feature learning for person re-identification . In IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917) , Honolulu, HI. IEEE Computer Society, 3741\u20133750. S. Zhou, J. Wang, and J. Wang et al. 2017. Point to set similarity based deep feature learning for person re-identification. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917), Honolulu, HI. IEEE Computer Society, 3741\u20133750."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2010.938364"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1756025"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504\u2013507.  G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313 5786 (2006) 504\u2013507.","DOI":"10.1126\/science.1127647"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog0901_7"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3125486.3125492"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390294"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1953039"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104482.3104587"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/2029556.2029563"},{"key":"e_1_2_1_15_1","volume-title":"USA","author":"Schlegl T.","year":"2014","unstructured":"T. Schlegl , J. Ofner , and G. Langs . 2014. Unsupervised pre-training across image domains improves lung tissue classification. In Medical Computer Vision (MCV\u201914): Algorithms for Big Data, Cambridge, MA , USA , September 18, 2014 . Springer International Publishing, 82--93. T. Schlegl, J. Ofner, and G. Langs. 2014. Unsupervised pre-training across image domains improves lung tissue classification. In Medical Computer Vision (MCV\u201914): Algorithms for Big Data, Cambridge, MA, USA, September 18, 2014. Springer International Publishing, 82--93."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00429-013-0687-3"},{"key":"e_1_2_1_17_1","volume-title":"Roubaix","author":"Glauner P. O.","year":"2015","unstructured":"P. O. Glauner . 2015 . Deep convolutional neural networks for smile recognition. In Deep Learning For Smile Recognition[C]\/\/ 12th Conference on Uncertainty Modelling in Knowledge Engineering and Decision Making (FLINS'16) , Roubaix , France , August 24-26, 2016. 986--989. P. O. Glauner. 2015. Deep convolutional neural networks for smile recognition. In Deep Learning For Smile Recognition[C]\/\/ 12th Conference on Uncertainty Modelling in Knowledge Engineering and Decision Making (FLINS'16), Roubaix, France, August 24-26, 2016. 986--989."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2015.2458702"},{"key":"e_1_2_1_19_1","unstructured":"A. Santara D. Maji D. Tejas etal 2016. Faster learning of deep stacked autoencoders on multi-core systems using synchronized layer-wise pre-training. arXiv preprint arXiv:1603.02836.  A. Santara D. Maji D. Tejas et al. 2016. Faster learning of deep stacked autoencoders on multi-core systems using synchronized layer-wise pre-training. arXiv preprint arXiv:1603.02836."},{"key":"e_1_2_1_20_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17)","author":"Zhang Z.","year":"2017","unstructured":"Z. Zhang , Y. Song , and H. Qi . 2017. Age progression\/regression by conditional adversarial autoencoder . In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17) , Hawaii, USA July 21-26, 2017 . 287--291. Z. Zhang, Y. Song, and H. Qi. 2017. Age progression\/regression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17), Hawaii, USA July 21-26, 2017. 287--291."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2016.2536638"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11220-019-0227-1"},{"key":"e_1_2_1_23_1","first-page":"3","article-title":"Deep learning: Methods and applications","volume":"7","author":"Deng L.","year":"2013","unstructured":"L. Deng and D. Yu . 2013 . Deep learning: Methods and applications . Foundations and Trends in Signal Processing 7 , 3 . L. Deng and D. Yu. 2013. Deep learning: Methods and applications. Foundations and Trends in Signal Processing 7, 3.","journal-title":"Foundations and Trends in Signal Processing"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-015-1939-3"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/1296061"},{"key":"e_1_2_1_26_1","volume-title":"IEEE Information Theory and Applications Workshop (ITA\u201918)","author":"Achille A.","year":"2018","unstructured":"A. Achille and S. Soatto . 2018. Emergence of invariance and disentanglement in deep representations . In IEEE Information Theory and Applications Workshop (ITA\u201918) , San Diego, CA, February 11\u201316 , 2018 . arXiv: 1706.01350v2 [cs.LG] 16, 1\u20139. A. Achille and S. Soatto. 2018. Emergence of invariance and disentanglement in deep representations. In IEEE Information Theory and Applications Workshop (ITA\u201918), San Diego, CA, February 11\u201316, 2018. arXiv: 1706.01350v2 [cs.LG] 16, 1\u20139."},{"key":"e_1_2_1_27_1","unstructured":"P. Koprinkova-Hristova V. Mladenov and K. N. Kasabov. 2015. How to Pretrain Deep Boltzmann Machines in Two Stages[J]. Springer International Publishing. 10.1007\/978-3-319-09903-3 (Chapter 10) 201--219.  P. Koprinkova-Hristova V. Mladenov and K. N. Kasabov. 2015. How to Pretrain Deep Boltzmann Machines in Two Stages[J]. Springer International Publishing. 10.1007\/978-3-319-09903-3 (Chapter 10) 201--219."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of International Joint Conference on Neural Networks","author":"Eastwood M.","year":"2013","unstructured":"M. Eastwood and C. Jayne . 2013. Restricted Boltzmann machines for pre-training deep Gaussian networks . In Proceedings of International Joint Conference on Neural Networks , Dallas, USA , August 4-9, 2013 . 1501--1507. M. Eastwood and C. Jayne. 2013. Restricted Boltzmann machines for pre-training deep Gaussian networks. In Proceedings of International Joint Conference on Neural Networks, Dallas, USA, August 4-9, 2013. 1501--1507."},{"key":"e_1_2_1_29_1","unstructured":"G. Desjardins A. Courville and Y. Bengio. 2012. On training deep Boltzmann machines. arXiv: 1203.4416v1 [cs.NE] 20 1\u20137.  G. Desjardins A. Courville and Y. Bengio. 2012. On training deep Boltzmann machines. arXiv: 1203.4416v1 [cs.NE] 20 1\u20137."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/104279.104293"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3433539","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3433539","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:10Z","timestamp":1750195690000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3433539"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2021,9,30]]}},"alternative-id":["10.1145\/3433539"],"URL":"https:\/\/doi.org\/10.1145\/3433539","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2021,7,21]]},"assertion":[{"value":"2020-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}