{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T23:35:34Z","timestamp":1769124934342,"version":"3.49.0"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,2,18]],"date-time":"2021-02-18T00:00:00Z","timestamp":1613606400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"},{"start":{"date-parts":[[2021,2,18]],"date-time":"2021-02-18T00:00:00Z","timestamp":1613606400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springer.com\/tdm"}],"funder":[{"name":"Science and Technology Development Fund of Tianjin Education Commission for Higher Education","award":["2018KJ217"],"award-info":[{"award-number":["2018KJ217"]}]},{"name":"Tianjin Natural Science Foundation of China","award":["20JCYBJC00500"],"award-info":[{"award-number":["20JCYBJC00500"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"published-print":{"date-parts":[[2021,4]]},"DOI":"10.1007\/s11063-021-10445-6","type":"journal-article","created":{"date-parts":[[2021,2,19]],"date-time":"2021-02-19T16:33:32Z","timestamp":1613752412000},"page":"1385-1400","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Information Bottleneck Theory on Convolutional Neural Networks"],"prefix":"10.1007","volume":"53","author":[{"given":"Junjie","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ding","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,2,18]]},"reference":[{"key":"10445_CR1","unstructured":"Advani MS, Saxe AM (2017) High-dimensional dynamics of generalization error in neural networks. Preprint arXiv:1710.03667"},{"key":"10445_CR2","doi-asserted-by":"publisher","first-page":"2225","DOI":"10.1109\/TPAMI.2019.2909031","volume":"42","author":"RA Amjad","year":"2019","unstructured":"Amjad RA, Geiger BC (2019) Learning representations for neural network-based classification using the information bottleneck principle. IEEE Trans Pattern Anal Mach Intell 42:2225\u20132239","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"Jan","key":"10445_CR3","first-page":"165","volume":"6","author":"G Chechik","year":"2005","unstructured":"Chechik G, Globerson A, Tishby N, Weiss Y (2005) Information bottleneck for Gaussian variables. J Mach Learn Res 6(Jan):165\u2013188","journal-title":"J Mach Learn Res"},{"key":"10445_CR4","unstructured":"Dai B, Zhu C, Wipf D (2018) Compressing neural networks using the variational information bottleneck. Preprint arXiv:1802.10399"},{"issue":"Jan","key":"10445_CR5","first-page":"81","volume":"6","author":"G Elidan","year":"2005","unstructured":"Elidan G, Friedman N (2005) Learning hidden variable networks: the information bottleneck approach. J Mach Learn Res 6(Jan):81\u2013127","journal-title":"J Mach Learn Res"},{"key":"10445_CR6","unstructured":"Friedman N, Mosenzon O, Slonim N, Tishby N (2013) Multivariate information bottleneck. Preprint arXiv:1301.2270"},{"key":"10445_CR7","unstructured":"Gabri\u00e9 M, Manoel A, Luneau C, Macris N, Krzakala F, Zdeborov\u00e1 L et\u00a0al (2018) Entropy and mutual information in models of deep neural networks. In: Advances in neural information processing systems, pp 1821\u20131831"},{"key":"10445_CR8","unstructured":"Goldfeld Z, Berg E, Greenewald K, Melnyk I, Nguyen N, Kingsbury B, Polyanskiy Y (2018) Estimating information flow in deep neural networks. Preprint arXiv:1810.05728"},{"key":"10445_CR9","unstructured":"Goldfeld Z, Van Den Berg E, Greenewald K, Melnyk I, Nguyen N, Kingsbury B, Polyanskiy Y (2019) Estimating information flow in deep neural networks. In: Proceedings of the 36th international conference on machine learning, vol 97, pp 2299\u20132308"},{"key":"10445_CR10","volume-title":"Deep learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, New York"},{"issue":"5","key":"10445_CR11","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1145\/3236009","volume":"51","author":"R Guidotti","year":"2019","unstructured":"Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D (2019) A survey of methods for explaining black box models. ACM Comput Surv (CSUR) 51(5):93","journal-title":"ACM Comput Surv (CSUR)"},{"key":"10445_CR12","doi-asserted-by":"crossref","unstructured":"Hsu WH, Kennedy LS, Chang SF (2006) Video search reranking via information bottleneck principle. In: Proceedings of the 14th ACM international conference on multimedia, pp 35\u201344","DOI":"10.1145\/1180639.1180654"},{"key":"10445_CR13","doi-asserted-by":"crossref","unstructured":"J\u00f3nsson H, Cherubini G, Eleftheriou E (2019) Convergence of DNNS with mutual-information-based regularization. In: Proceedings of the Bayesian deep learning@ advances in neural information processing systems (NeurIPS), Vancouver","DOI":"10.3390\/e22070727"},{"key":"10445_CR14","unstructured":"Kadmon J, Sompolinsky H (2016) Optimal architectures in a solvable model of deep networks. In: Advances in neural information processing systems, pp 4781\u20134789"},{"issue":"7","key":"10445_CR15","doi-asserted-by":"publisher","first-page":"361","DOI":"10.3390\/e19070361","volume":"19","author":"A Kolchinsky","year":"2017","unstructured":"Kolchinsky A, Tracey B (2017) Estimating mixture entropy with pairwise distances. Entropy 19(7):361","journal-title":"Entropy"},{"issue":"12","key":"10445_CR16","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.3390\/e21121181","volume":"21","author":"A Kolchinsky","year":"2019","unstructured":"Kolchinsky A, Tracey BD, Wolpert DH (2019) Nonlinear information bottleneck. Entropy 21(12):1181","journal-title":"Entropy"},{"key":"10445_CR17","unstructured":"Krizhevsky A, Hinton G et\u00a0al (2009) Learning multiple layers of features from tiny images"},{"issue":"1","key":"10445_CR18","first-page":"7908","volume":"18","author":"A Painsky","year":"2017","unstructured":"Painsky A, Tishby N (2017) Gaussian lower bound for the information bottleneck limit. J Mach Learn Res 18(1):7908\u20137936","journal-title":"J Mach Learn Res"},{"key":"10445_CR19","unstructured":"Poole B, Ozair S, Oord A, Alemi AA, Tucker G (2019) On variational bounds of mutual information. Preprint arXiv:1905.06922"},{"issue":"12","key":"10445_CR20","doi-asserted-by":"publisher","first-page":"124020","DOI":"10.1088\/1742-5468\/ab3985","volume":"2019","author":"AM Saxe","year":"2019","unstructured":"Saxe AM, Bansal Y, Dapello J, Advani M, Kolchinsky A, Tracey BD, Cox DD (2019) On the information bottleneck theory of deep learning. J Stat Mech Theory Exp 2019(12):124020","journal-title":"J Stat Mech Theory Exp"},{"key":"10445_CR21","unstructured":"Saxe AM, Mcclelland JL, Ganguli S (2014) Exact solutions to the nonlinear dynamics of learning in deep linear neural network. In: In International conference on learning representations. Citeseer, New York"},{"issue":"29\u201330","key":"10445_CR22","doi-asserted-by":"publisher","first-page":"2696","DOI":"10.1016\/j.tcs.2010.04.006","volume":"411","author":"O Shamir","year":"2010","unstructured":"Shamir O, Sabato S, Tishby N (2010) Learning and generalization with the information bottleneck. Theoret Comput Sci 411(29\u201330):2696\u20132711","journal-title":"Theoret Comput Sci"},{"key":"10445_CR23","unstructured":"Shwartz-Ziv R, Tishby N (2017) Opening the black box of deep neural networks via information. Preprint arXiv:1703.00810"},{"key":"10445_CR24","doi-asserted-by":"crossref","unstructured":"Slonim N, Tishby N (2000) Document clustering using word clusters via the information bottleneck method. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, pp 208\u2013215","DOI":"10.1145\/345508.345578"},{"issue":"6","key":"10445_CR25","doi-asserted-by":"publisher","first-page":"1611","DOI":"10.1162\/NECO_a_00961","volume":"29","author":"D Strouse","year":"2017","unstructured":"Strouse D, Schwab DJ (2017) The deterministic information bottleneck. Neural Comput 29(6):1611\u20131630","journal-title":"Neural Comput"},{"key":"10445_CR26","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818\u20132826","DOI":"10.1109\/CVPR.2016.308"},{"key":"10445_CR27","unstructured":"Tishby N (2000) The information bottleneck method. Computing Research Repository (CoRR)"},{"key":"10445_CR28","doi-asserted-by":"crossref","unstructured":"Tishby N, Zaslavsky N (2015) Deep learning and the information bottleneck principle. In: 2015 IEEE information theory workshop (ITW). IEEE, New York, pp 1\u20135","DOI":"10.1109\/ITW.2015.7133169"},{"issue":"9","key":"10445_CR29","doi-asserted-by":"publisher","first-page":"4376","DOI":"10.1109\/TIP.2019.2910667","volume":"28","author":"Q Wang","year":"2019","unstructured":"Wang Q, Gao J, Li X (2019) Weakly supervised adversarial domain adaptation for semantic segmentation in urban scenes. IEEE Trans Image Process 28(9):4376\u20134386","journal-title":"IEEE Trans Image Process"},{"issue":"1","key":"10445_CR30","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1109\/TGRS.2018.2849692","volume":"57","author":"Q Wang","year":"2018","unstructured":"Wang Q, Yuan Z, Du Q, Li X (2018) Getnet: a general end-to-end 2-D CNN framework for hyperspectral image change detection. IEEE Trans Geosci Remote Sens 57(1):3\u201313","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"10445_CR31","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.neunet.2019.05.003","volume":"117","author":"S Yu","year":"2019","unstructured":"Yu S, Principe JC (2019) Understanding autoencoders with information theoretic concepts. Neural Netw 117:104\u2013123","journal-title":"Neural Netw"},{"key":"10445_CR32","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.2968509","author":"S Yu","year":"2020","unstructured":"Yu S, Wickstr\u00f8m K, Jenssen R, Principe JC (2020) Understanding convolutional neural networks with information theory: an initial exploration. IEEE Trans Neural Netw Learn Syst. https:\/\/doi.org\/10.1109\/TNNLS.2020.2968509","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"10445_CR33","unstructured":"Yu Y, Chan KHR, You C, Song C, Ma Y (2020) Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Preprint arXiv:2006.08558"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-021-10445-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-021-10445-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-021-10445-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,13]],"date-time":"2021-04-13T13:06:56Z","timestamp":1618319216000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-021-10445-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,18]]},"references-count":33,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["10445"],"URL":"https:\/\/doi.org\/10.1007\/s11063-021-10445-6","relation":{},"ISSN":["1370-4621","1573-773X"],"issn-type":[{"value":"1370-4621","type":"print"},{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,18]]},"assertion":[{"value":"2 February 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 February 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}