{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:41:41Z","timestamp":1777696901277,"version":"3.51.4"},"reference-count":45,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T00:00:00Z","timestamp":1747180800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No.52105526"],"award-info":[{"award-number":["No.52105526"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No.51975394"],"award-info":[{"award-number":["No.51975394"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Postgraduate Research & Practice Innovation Program of Jiangsu Province","award":["KYCX24_3422"],"award-info":[{"award-number":["KYCX24_3422"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Intelligent Data Analysis: An International Journal"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:p>\n                    With the development of deep learning, many researches have been devoted to the explanatory nature of Neural Networks (NNs), among which the Information Bottleneck (IB) theory based on information theory has received the most attention. Previous IB work has focused on small MLP or several-layer CNN networks, and a small number of datasets, and the applicability of its conclusions in popular image classification models cannot be verified, especially lacking in transfer learning. To address the above issues, we experiment on an industrial surface defects classification task to investigate IB and performance variations in ResNet transfer learning. The local densities of feature parameters in ResNet transfer learning are calculated by adaptive bins and K-nearest neighbour method. The mutual information of input image information-model parameters and model parameters-output information is calculated, and it is found that there are two stages of fitting and compression of CNN features in transfer learning. The IB representation method\n                    <jats:inline-formula>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" display=\"inline\" overflow=\"scroll\">\n                        <mml:mi>D<\/mml:mi>\n                        <mml:mi>I<\/mml:mi>\n                        <mml:mi>B<\/mml:mi>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    is proposed, and the\n                    <jats:inline-formula>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" display=\"inline\" overflow=\"scroll\">\n                        <mml:mi>M<\/mml:mi>\n                        <mml:mi>a<\/mml:mi>\n                        <mml:mi>x<\/mml:mi>\n                        <mml:mo stretchy=\"false\">(<\/mml:mo>\n                        <mml:mi>D<\/mml:mi>\n                        <mml:mi>I<\/mml:mi>\n                        <mml:mi>B<\/mml:mi>\n                        <mml:mo stretchy=\"false\">)<\/mml:mo>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    value is found to be positively correlated with the CNN model performance, which provides an innovative idea to explain the transfer learning performance of the model from the IB perspective. Evaluating 81 transfer learning tasks with the derived IP conclusions captures 22.2% poor models for re-optimization. To our knowledge, we are the first study to analyze IB in transfer learning and generalize to the understanding of common CNN image classification models.\n                  <\/jats:p>","DOI":"10.1177\/1088467x251335121","type":"journal-article","created":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T19:43:22Z","timestamp":1747251802000},"page":"196-216","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":1,"title":["Research on the interpretability of surface defect detection transfer learning tasks based on information bottleneck theory"],"prefix":"10.1177","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-9318-2055","authenticated-orcid":false,"given":"Zengguang","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9587-4135","authenticated-orcid":false,"given":"Guizhong","family":"Fu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou, China"},{"name":"School of Automation, Southeast University, Nanjing, China"},{"name":"Borch Machinery Co., Ltd. (Borche), Guangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8917-719X","authenticated-orcid":false,"given":"Yehu","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1579-0207","authenticated-orcid":false,"given":"Jinbin","family":"Li","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electrical Engineering, Shihezi University, Shihezi, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4148-3438","authenticated-orcid":false,"given":"Quansheng","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5275-6657","authenticated-orcid":false,"given":"Qixin","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Suzhou University of Science and Technology, Suzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,5,14]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_3_4_3_2","unstructured":"Wang H Raj B. On the origin of deep learning. arXiv preprint arXiv:1702.07800 2017."},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000006"},{"key":"e_1_3_4_5_2","unstructured":"Alain G Bengio Y. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644 2016."},{"key":"e_1_3_4_6_2","unstructured":"Shwartz-Ziv R Tishby N. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810 2017."},{"key":"e_1_3_4_7_2","doi-asserted-by":"crossref","unstructured":"Tishby N Zaslavsky N. Deep learning and the information bottleneck principle. In: 2015 IEEE information theory workshop (itw) 2015 pp.1\u20135. IEEE.","DOI":"10.1109\/ITW.2015.7133169"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/ab3985"},{"key":"e_1_3_4_9_2","unstructured":"Abrol V Tanner J. Information-bottleneck under mean field initialization 2020."},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","unstructured":"Noshad M Zeng Y Hero AO. Scalable mutual information estimation using dependence graphs. In: ICASSP 2019-2019 IEEE international conference on acoustics speech and signal processing (ICASSP) 2019 pp.2962\u20132966. IEEE.","DOI":"10.1109\/ICASSP.2019.8683351"},{"key":"e_1_3_4_11_2","doi-asserted-by":"crossref","unstructured":"Torrey L Shavlik J. Transfer learning. In: Handbook of research on machine learning applications and trends: algorithms methods and techniques 2010 pp.242\u2013264. IGI global.","DOI":"10.4018\/978-1-60566-766-9.ch011"},{"key":"e_1_3_4_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2970461"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.optlaseng.2019.05.005"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compind.2021.103585"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.1883"},{"key":"e_1_3_4_16_2","unstructured":"Ganin Y Lempitsky V. Unsupervised domain adaptation by backpropagation. In: International conference on machine learning 2015 pp.1180\u20131189. PMLR."},{"key":"e_1_3_4_17_2","first-page":"1","article-title":"How transferable are features in deep neural networks","volume":"27","author":"Yosinski J","year":"2014","unstructured":"Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks? Adv Neural Inf Process Syst 2014; 27: 1\u20139.","journal-title":"Adv Neural Inf Process Syst"},{"key":"e_1_3_4_18_2","doi-asserted-by":"crossref","unstructured":"Zamir AR Sax A Shen W et al. Taskonomy: disentangling task transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2018 pp.3712\u20133722.","DOI":"10.1109\/CVPR.2018.00391"},{"key":"e_1_3_4_19_2","unstructured":"You K Liu Y Wang J et al. Logme: Practical assessment of pre-trained models for transfer learning. In: International conference on machine learning 2021 pp.12133\u201312143. PMLR."},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.715"},{"key":"e_1_3_4_21_2","doi-asserted-by":"crossref","unstructured":"He K Zhang X Ren S et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2016 pp.770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_4_22_2","unstructured":"Simonyan K Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 2014."},{"key":"e_1_3_4_23_2","doi-asserted-by":"crossref","unstructured":"Sandler M Howard A Zhu M et al. Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2018 pp.4510\u20134520.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cobme.2018.12.005"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSM.2019.2941752"},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"Deng J Dong W Socher R et al. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition 2009 pp.248\u2013255. Ieee.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","unstructured":"Cao Z You K Long M et al. Learning to transfer examples for partial domain adaptation. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition 2019 pp.2985\u20132994.","DOI":"10.1109\/CVPR.2019.00310"},{"key":"e_1_3_4_28_2","doi-asserted-by":"crossref","unstructured":"Wang Z Dai Z P\u00f3czos B et al. Characterizing and avoiding negative transfer. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition 2019 pp.11293\u201311302.","DOI":"10.1109\/CVPR.2019.01155"},{"key":"e_1_3_4_29_2","unstructured":"Tishby N Pereira FC Bialek W. The information bottleneck method. arXiv preprint physics\/0004057 2000."},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3089037"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.114"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.04.021"},{"key":"e_1_3_4_33_2","unstructured":"Kirsch A Lyle C Gal Y. Scalable training with information bottleneck objectives. In: Proceedings of the international conference on machine learning (icml): workshop on uncertainty and robustness in deep learning virtual 2020 pp.17\u201318."},{"key":"e_1_3_4_34_2","unstructured":"Hjelm RD Fedorov A Lavoie-Marchildon S et al. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 2018."},{"key":"e_1_3_4_35_2","unstructured":"Alemi AA Fischer I Dillon JV et al. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 2016."},{"key":"e_1_3_4_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.apsusc.2013.09.002"},{"key":"e_1_3_4_37_2","first-page":"8032","article-title":"A deep learning-based surface defect inspection system using multiscale and channel-compressed features","volume":"69","author":"Yang J","year":"2020","unstructured":"Yang J, Fu G, Zhu W, et al. A deep learning-based surface defect inspection system using multiscale and channel-compressed features. IEEE Trans Instrum Meas 2020; 69: 8032\u20138042.","journal-title":"IEEE Trans Instrum Meas"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3291598"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-018-1588-5"},{"key":"e_1_3_4_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2020.3045196"},{"key":"e_1_3_4_41_2","unstructured":"Dagm2007 dataset. https:\/\/hci.iwr.uni-heidelberg.de\/node\/3616 (accessed 10 October 2023)."},{"key":"e_1_3_4_42_2","volume-title":"Kylberg texture dataset v. 1.0","author":"Kylberg G","year":"2011","unstructured":"Kylberg G. Kylberg texture dataset v. 1.0. Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University, 2011."},{"key":"e_1_3_4_43_2","unstructured":"Fritz M Hayman E Caputo B et al. The kth-tips database 2004."},{"key":"e_1_3_4_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2018.11.015"},{"key":"e_1_3_4_45_2","doi-asserted-by":"crossref","unstructured":"Cui Y Song Y Sun C et al. Large scale fine-grained categorization and domain-specific transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2018 pp.4109\u20134118.","DOI":"10.1109\/CVPR.2018.00432"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.066138"}],"container-title":["Intelligent Data Analysis: An International Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1088467X251335121","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1088467X251335121","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1088467X251335121","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:21:16Z","timestamp":1777454476000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1088467X251335121"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,14]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["10.1177\/1088467X251335121"],"URL":"https:\/\/doi.org\/10.1177\/1088467x251335121","relation":{},"ISSN":["1088-467X","1571-4128"],"issn-type":[{"value":"1088-467X","type":"print"},{"value":"1571-4128","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,14]]}}}