{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T23:38:18Z","timestamp":1775000298108,"version":"3.50.1"},"reference-count":50,"publisher":"American Association for the Advancement of Science (AAAS)","funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["12004266"],"award-info":[{"award-number":["12004266"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004826","name":"Natural Science Foundation of Beijing Municipality","doi-asserted-by":"publisher","award":["1232025"],"award-info":[{"award-number":["1232025"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012580","name":"Natural Science Foundation of Tianjin-Science and Technology Correspondent Project","doi-asserted-by":"publisher","award":["20JCYBJC00500"],"award-info":[{"award-number":["20JCYBJC00500"]}],"id":[{"id":"10.13039\/501100012580","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["spj.science.org"],"crossmark-restriction":true},"short-container-title":["Intell Comput"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:p>\n            The neural network (NN) designed for challenging machine learning tasks is in general a highly nonlinear mapping that contains numerous variational parameters. The complexity of NNs, if unbounded or unconstrained, might unpredictably cause severe issues including overfitting, loss of generalization power, and excessive cost of hardware. In this study, we propose a general compression scheme that considerably reduces the variational parameters of NNs, regardless of their specific types (linear, convolutional, etc.), by encoding them into deep automatically differentiable tensor networks (ADTNs) that contain exponentially fewer free parameters. The superior compression performance of our scheme is demonstrated on several widely recognized NNs (FC-2, LeNet-5, AlexNet, ZFNet, and VGG-16) and datasets (MNIST, CIFAR-10, and CIFAR-100). For instance, we compress 2 linear layers in VGG-16 with approximately 10\n            <jats:sup>7<\/jats:sup>\n            parameters to 2 ADTNs with only 424 parameters, improving the testing accuracy on CIFAR-10 from 90.17% to 91.74%. We argue that the deep structure of the ADTN is an essential reason for its remarkable compression performance compared to existing compression schemes, which are mainly based on tensor decompositions\/factorization and shallow TNs. Our study suggests that the deep TN is an exceptionally efficient mathematical structure for representing the variational parameters of NNs, one that exhibits better compressibility than the commonly used matrices and multiway arrays.\n          <\/jats:p>","DOI":"10.34133\/icomputing.0123","type":"journal-article","created":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T17:00:26Z","timestamp":1744822826000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark_01","source":"Crossref","is-referenced-by-count":5,"title":["Compressing Neural Networks Using Tensor Networks with Exponentially Fewer Variational Parameters"],"prefix":"10.34133","volume":"4","author":[{"given":"Yong","family":"Qing","sequence":"first","affiliation":[{"name":"Center for Quantum Physics and Intelligent Sciences, Department of Physics, \rCapital Normal University, Beijing 100048, China."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ke","family":"Li","sequence":"additional","affiliation":[{"name":"Center for Quantum Physics and Intelligent Sciences, Department of Physics, \rCapital Normal University, Beijing 100048, China."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng-Fei","family":"Zhou","sequence":"additional","affiliation":[{"name":"Center for Quantum Physics and Intelligent Sciences, Department of Physics, \rCapital Normal University, Beijing 100048, China."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1844-7268","authenticated-orcid":true,"given":"Shi-Ju","family":"Ran","sequence":"additional","affiliation":[{"name":"Center for Quantum Physics and Intelligent Sciences, Department of Physics, \rCapital Normal University, Beijing 100048, China."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"221","published-online":{"date-parts":[[2025,5,15]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2014.09.003"},{"key":"e_1_3_4_3_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cagd.2018.10.005","article-title":"A geometric view of optimal transportation and generative model","volume":"68","author":"Lei N","year":"2019","unstructured":"Lei N, Su K, Cui L, Yau ST, Gu XD. A geometric view of optimal transportation and generative model. Comput Aided Geom Des. 2019;68:1\u201321.","journal-title":"Comput Aided Geom Des"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04086-x"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aag2302"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-017-00705-2"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1038\/nphys4035"},{"issue":"26","key":"e_1_3_4_8_2","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevLett.121.260601","article-title":"Neural network renormalization group","volume":"121","author":"Li SH","year":"2018","unstructured":"Li SH, Wang L. Neural network renormalization group. Phys Rev Lett. 2018;121(26): Article 260601.","journal-title":"Phys Rev Lett"},{"issue":"8","key":"e_1_3_4_9_2","doi-asserted-by":"crossref","first-page":"080602","DOI":"10.1103\/PhysRevLett.122.080602","article-title":"Solving statistical mechanics using variational autoregressive networks","volume":"122","author":"Wu D","year":"2019","unstructured":"Wu D, Wang L, Zhang P. Solving statistical mechanics using variational autoregressive networks. Phys Rev Lett. 2019;122(8):080602.","journal-title":"Phys Rev Lett"},{"issue":"4","key":"e_1_3_4_10_2","doi-asserted-by":"crossref","DOI":"10.1103\/RevModPhys.91.045002","article-title":"Machine learning and the physical sciences","volume":"91","author":"Carleo G","year":"2019","unstructured":"Carleo G, Cirac I, Cranmer K, Daudet L, Schuld M, Tishby N, Vogt-Maranto L, Zdeborov\u00e1 L. Machine learning and the physical sciences. Rev Mod Phys. 2019;91(4): Article 045002.","journal-title":"Rev Mod Phys"},{"issue":"11","key":"e_1_3_4_11_2","doi-asserted-by":"crossref","DOI":"10.1088\/0256-307X\/38\/11\/110301","article-title":"Deep learning quantum states for Hamiltonian estimation","volume":"38","author":"Ma X","year":"2021","unstructured":"Ma X, Tu ZC, Ran SJ. Deep learning quantum states for Hamiltonian estimation. Chin Phys Lett. 2021;38(11): Article 110301.","journal-title":"Chin Phys Lett"},{"key":"e_1_3_4_12_2","unstructured":"Ouyang L Wu J Jiang X Almeida D Wainwright CL Mishkin P Zhang C Agarwal S Slama K Ray A et\u00a0al. Training language models to follow instructions with human feedback. Paper presented at: Advances in Neural Information Processing Systems; 2022; New Orleans LA USA."},{"key":"e_1_3_4_13_2","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/s10846-023-01988-y","article-title":"Efficient deep learning-based semantic mapping approach using monocular vision for resource-limited mobile robots","volume":"109","author":"Singh A","year":"2023","unstructured":"Singh A, Raj K, Roy AM. Efficient deep learning-based semantic mapping approach using monocular vision for resource-limited mobile robots. J Intell Robot Syst. 2023;109:69.","journal-title":"J Intell Robot Syst"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.3390\/drones7020081"},{"issue":"2","key":"e_1_3_4_15_2","doi-asserted-by":"crossref","first-page":"686","DOI":"10.3390\/ai5020036","article-title":"Efficient paddy grain quality assessment approach utilizing affordable sensors","volume":"5","author":"Singh A","year":"2024","unstructured":"Singh A, Raj K, Meghwar T, Roy AM. Efficient paddy grain quality assessment approach utilizing affordable sensors. AI. 2024;5(2):686\u2013703.","journal-title":"AI"},{"key":"e_1_3_4_16_2","unstructured":"Brown T Mann B Ryder N Subbiah M Kaplan J Dhariwal P Neelakantan A Shyam P Satry G Askell A et\u00a0al. Language models are few-shot learners. Paper presented at: Advances in Neural Information Processing Systems; 2020; Virtual Conference."},{"key":"e_1_3_4_17_2","unstructured":"Le Cun Y Denker JS Solla SA. Optimal brain damage. Paper presented at: Proceedings of the 2nd International Conference on Neural Information Processing Systems; 1989; Denver CO USA."},{"key":"e_1_3_4_18_2","unstructured":"Liu Z Sun M Zhou T Huang G Darrell T. Rethinking the value of network pruning. Paper presented at: International Conference on Learning Representations; 2019; Orleans LA USA."},{"key":"e_1_3_4_19_2","doi-asserted-by":"crossref","unstructured":"Wang Z Li C Wang X. Convolutional neural network pruning with structural redundancy reduction. Paper presented at: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2021; Virtual Conference.","DOI":"10.1109\/CVPR46437.2021.01467"},{"key":"e_1_3_4_20_2","first-page":"9","article-title":"Distilling the knowledge in a neural network","volume":"1050","author":"Hinton G","year":"2015","unstructured":"Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. Stat. 2015;1050:9.","journal-title":"Stat"},{"key":"e_1_3_4_21_2","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge distillation: A survey","volume":"129","author":"Gou J","year":"2021","unstructured":"Gou J, Yu B, Maybank SJ, Tao D. Knowledge distillation: A survey. Int J Comput Vis. 2021;129:1789\u20131819.","journal-title":"Int J Comput Vis"},{"issue":"4","key":"e_1_3_4_22_2","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1162\/neco.1992.4.4.473","article-title":"Simplifying neural networks by soft weight-sharing","volume":"4","author":"Nowlan SJ","year":"1992","unstructured":"Nowlan SJ, Hinton GE. Simplifying neural networks by soft weight-sharing. Neural Comput. 1992;4(4):473\u2013493.","journal-title":"Neural Comput"},{"key":"e_1_3_4_23_2","doi-asserted-by":"crossref","unstructured":"Chu X Zhang B Xu R. FairNAS: Rethinking evaluation fairness of weight sharing neural architecture search. Paper presented at: 2021 IEEE\/CVF International Conference on Computer Vision (ICCV); 2021; Virtual Conference (Montreal Canada).","DOI":"10.1109\/ICCV48922.2021.01202"},{"key":"e_1_3_4_24_2","unstructured":"Hayashi K Yamaguchi T Sugawara Y Maeda Si. Exploring unexplored tensor network decompositions for convolutional neural networks. Paper presented at: Advances in Neural Information Processing Systems; 2019; Vancouver Canada."},{"key":"e_1_3_4_25_2","doi-asserted-by":"crossref","DOI":"10.1016\/j.knosys.2022.108171","article-title":"Deep neural network compression by Tucker decomposition with nonlinear response","volume":"241","author":"Liu Y","year":"2022","unstructured":"Liu Y, Ng MK. Deep neural network compression by Tucker decomposition with nonlinear response. Knowl-Based Syst. 2022;241: Article 108171.","journal-title":"Knowl-Based Syst"},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"Gusak J Kholiavchenko M Ponomarev E Markeeva L Blagoveschensky P Cichocki A Oseledets I. Automated multi-stage compression of neural networks. Paper presented at: 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW); 2019; Seoul South Korea.","DOI":"10.1109\/ICCVW.2019.00306"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","unstructured":"Liang CC Lee CR. Automatic selection of tensor decomposition for compressing convolutional neural networks a case study on VGG-type networks. Paper presented at: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW); 2021; Portland OR USA.","DOI":"10.1109\/IPDPSW52791.2021.00115"},{"key":"e_1_3_4_28_2","unstructured":"Hu EJ Shen Y Wallis P Allen-Zhu Z Li Y Wang S Wang L Chen W. LoRA: Low-rank adaptation of large language models. arXiv. 2021. https:\/\/doi.org\/10.48550\/arXiv.2106.09685"},{"key":"e_1_3_4_29_2","first-page":"401","article-title":"Matrix product state representations","volume":"7","author":"Perez-Garc\u0131a D","year":"2007","unstructured":"Perez-Garc\u0131a D, Verstraete F, Wolf MM, Cirac JI. Matrix product state representations. Quant Inf Comput. 2007;7:401\u2013430.","journal-title":"Quant Inf Comput"},{"issue":"3","key":"e_1_3_4_30_2","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevB.95.035129","article-title":"Generic construction of efficient matrix product operators","volume":"95","author":"Hubig C","year":"2017","unstructured":"Hubig C, McCulloch IP, Schollwock U. Generic construction of efficient matrix product operators. Phys Rev B. 2017;95(3): Article 035129.","journal-title":"Phys Rev B"},{"issue":"5","key":"e_1_3_4_31_2","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.1137\/090752286","article-title":"Tensor-train decomposition","volume":"33","author":"Oseledets IV","year":"2011","unstructured":"Oseledets IV. Tensor-train decomposition. SIAM J Sci Comput. 2011;33(5):2295\u20132317.","journal-title":"SIAM J Sci Comput"},{"key":"e_1_3_4_32_2","doi-asserted-by":"crossref","DOI":"10.1088\/1367-2630\/12\/2\/025012","article-title":"Matrix product operator representations","volume":"12","author":"Pirvu B","year":"2010","unstructured":"Pirvu B, Murg V, Cirac JI, Verstraete F. Matrix product operator representations. New J Phys. 2010;12: Article 025012.","journal-title":"New J Phys"},{"key":"e_1_3_4_33_2","unstructured":"Novikov A Podoprikhin D Osokin A Vetrov DP. Tensorizing neural networks. Paper presented at: Advances in Neural Information Processing Systems; 2015; Montreal Canada."},{"key":"e_1_3_4_34_2","doi-asserted-by":"crossref","unstructured":"Kim YD Park E Yoo S Choi T Yang L Shin D. Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv. 2015. https:\/\/doi.org\/10.48550\/arXiv.1511.06530","DOI":"10.14257\/astl.2016.140.36"},{"issue":"2","key":"e_1_3_4_35_2","article-title":"Compressing deep neural networks by matrix product operators","volume":"2","author":"Gao ZF","year":"2020","unstructured":"Gao ZF, Cheng S, He RQ, Xie ZY, Zhao HH, Lu ZY, Xiang T. Compressing deep neural networks by matrix product operators. Phys Rev Res. 2020;2(2): Article 023300.","journal-title":"Phys Rev Res"},{"key":"e_1_3_4_36_2","doi-asserted-by":"crossref","first-page":"2837","DOI":"10.1109\/TASLP.2020.3030495","article-title":"A model compression method with matrix product operators for speech enhancement","volume":"28","author":"Sun X","year":"2020","unstructured":"Sun X, Gao ZF, Lu ZY, Li J, Yan Y. A model compression method with matrix product operators for speech enhancement. IEEE\/ACM Trans Audio Speech Lang Process. 2020;28:2837\u20132847.","journal-title":"IEEE\/ACM Trans Audio Speech Lang Process"},{"key":"e_1_3_4_37_2","unstructured":"Wang W Sun Y Eriksson B Wang W Aggarwal V. Wide compression: Tensor ring nets. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2018; Salt Lake City UT USA."},{"key":"e_1_3_4_38_2","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1016\/j.neunet.2021.08.028","article-title":"Nonlinear tensor train format for deep neural network compression","volume":"144","author":"Wang D","year":"2021","unstructured":"Wang D, Zhao G, Chen H, Liu Z, Deng L, Li G. Nonlinear tensor train format for deep neural network compression. Neural Netw. 2021;144:320\u2013333.","journal-title":"Neural Netw"},{"key":"e_1_3_4_39_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-34489-4","volume-title":"Tensor network contractions: Methods and applications to quantum many-body","author":"Ran SJ","year":"2020","unstructured":"Ran SJ, Tirrito E, Peng C, Chen X, Tagliacozzo L, Su G, Lewesnstein M. Tensor network contractions: Methods and applications to quantum many-body systems. Cham: Springer; 2020."},{"issue":"3","key":"e_1_3_4_40_2","article-title":"Differentiable programming tensor networks","volume":"9","author":"Liao HJ","year":"2019","unstructured":"Liao HJ, Liu JG, Wang L, Xiang T. Differentiable programming tensor networks. Phys Rev X. 2019;9(3): Article 031041.","journal-title":"Phys Rev X"},{"issue":"4","key":"e_1_3_4_41_2","article-title":"Automatically differentiable quantum circuit for many-qubit state preparation","volume":"104","author":"Zhou PF","year":"2021","unstructured":"Zhou PF, Hong R, Ran SJ. Automatically differentiable quantum circuit for many-qubit state preparation. Phys Rev A. 2021;104(4): Article 042601.","journal-title":"Phys Rev A"},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_4_43_2","unstructured":"Simonyan K Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv. 2014. https:\/\/doi.org\/10.48550\/arXiv.1409.1556"},{"key":"e_1_3_4_44_2","unstructured":"http:\/\/yann.lecun.com\/exdb\/mnist."},{"key":"e_1_3_4_45_2","unstructured":"https:\/\/www.cs.toronto.edu\/~kriz\/cifar.html."},{"key":"e_1_3_4_46_2","unstructured":"Nagel M Fournarakis M Amjad RA Bondarenko Y Van Baalen M Blankevoort T. A white paper on neural network quantization. arXiv. 2021. https:\/\/doi.org\/10.48550\/arXiv.2106.08295"},{"issue":"3","key":"e_1_3_4_47_2","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevLett.100.030504","article-title":"Entropy scaling and simulability by matrix product states","volume":"100","author":"Schuch N","year":"2008","unstructured":"Schuch N, Wolf MM, Verstraete F, Cirac JI. Entropy scaling and simulability by matrix product states. Phys Rev Lett. 2008;100(3): Article 030504.","journal-title":"Phys Rev Lett"},{"issue":"12","key":"e_1_3_4_48_2","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevB.87.125139","article-title":"Area law and real-space renormalization","volume":"87","author":"Ferris AJ","year":"2013","unstructured":"Ferris AJ. Area law and real-space renormalization. Phys Rev B. 2013;87(12): Article 125139.","journal-title":"Phys Rev B"},{"key":"e_1_3_4_49_2","article-title":"Area laws and efficient descriptions of quantum many-body states","volume":"18","author":"Ge Y","year":"2016","unstructured":"Ge Y, Eisert J. Area laws and efficient descriptions of quantum many-body states. New J Phys. 2016;18: Article 083026.","journal-title":"New J Phys"},{"key":"e_1_3_4_50_2","doi-asserted-by":"publisher","DOI":"10.34133\/icomputing.0061"},{"issue":"12","key":"e_1_3_4_51_2","doi-asserted-by":"crossref","DOI":"10.1103\/PhysRevB.103.125117","article-title":"Supervised learning with projected entangled pair states","volume":"103","author":"Cheng S","year":"2021","unstructured":"Cheng S, Wang L, Zhang P. Supervised learning with projected entangled pair states. Phys Rev B. 2021;103(12): Article 125117.","journal-title":"Phys Rev B"}],"container-title":["Intelligent Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/spj.science.org\/doi\/pdf\/10.34133\/icomputing.0123","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,15]],"date-time":"2025-05-15T11:46:54Z","timestamp":1747309614000},"score":1,"resource":{"primary":{"URL":"https:\/\/spj.science.org\/doi\/10.34133\/icomputing.0123"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":50,"alternative-id":["10.34133\/icomputing.0123"],"URL":"https:\/\/doi.org\/10.34133\/icomputing.0123","relation":{},"ISSN":["2771-5892"],"issn-type":[{"value":"2771-5892","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"2024-04-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-22","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"0123"}}