{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T16:19:46Z","timestamp":1792599586612,"version":"4.5.0"},"reference-count":30,"publisher":"World Scientific Pub Co Pte Ltd","issue":"12","funder":[{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["390696704"],"award-info":[{"award-number":["390696704"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Neur. Syst."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p> It has been proven that, compared to using 32-bit floating-point numbers in the training phase, Deep Convolutional Neural Networks (DCNNs) can operate with low-precision during inference, thereby saving memory footprint and power consumption. However, neural network quantization is always accompanied by accuracy degradation. Here, we propose a quantization method called double-stage Squeeze-and-Threshold (double-stage ST) to close the accuracy gap with full-precision models. While accurate colors in pictures can be pleasing to the viewer, they are not necessary for distinguishing objects. The era of black and white television proves this idea. As long as the limited colors are filled reasonably for different objects, the objects can be well identified and distinguished. Our method utilizes the attention mechanism to adjust the activations and learn the thresholds to distinguish objects (features). We then divide the numerically rich activations into intervals (a limited variety of numerical values) by the learned thresholds. The proposed method supports both binarization and multi-bit quantization. Our method achieves state-of-the-art results. In binarization, ReActNet [Z. Liu, Z. Shen, S. Li, K. Helwegen, D. Huang and K. Cheng, arXiv:abs\/2106.11309 ] trained with our method outperforms the previous state-of-the-art result by 0.2 percentage points. Whereas in multi-bit quantization, the top-1 accuracy of the 3-bit ResNet-18 [K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, 2016 IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2016, 27\u201330 June 2016, Las Vegas, NV, USA (IEEE Computer Society, 2016), pp. 770\u2013778] model exceeds the top-1 accuracy of its full-precision baseline model by 0.4 percentage points. The double-stage ST activation quantization method is easy to apply by inserting it before the convolution. Besides, the double-stage ST is detachable after training and introducing no computational cost in inference. <\/jats:p>","DOI":"10.1142\/s0129065722500514","type":"journal-article","created":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T06:14:51Z","timestamp":1659334491000},"source":"Crossref","is-referenced-by-count":4,"title":["Convolutional Neural Networks Quantization with Double-Stage Squeeze-and-Threshold"],"prefix":"10.1142","volume":"32","author":[{"given":"Binyi","family":"Wu","sequence":"first","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, Technische Universit\u00e4t Dresden, Helmholtzstra\u00dfe 18, Dresden 01069, Germany"},{"name":"Division of Power and Sensor Systems, Infineon Technologies, K\u00f6nigsbr\u00fccker Stra\u00dfe 180, Dresden, 01099, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernd","family":"Waschneck","sequence":"additional","affiliation":[{"name":"Division of Power and Sensor Systems, Infineon Technologies, K\u00f6nigsbr\u00fccker Stra\u00dfe 180, Dresden, 01099, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian Georg","family":"Mayr","sequence":"additional","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, Centre for Tactile Internet with Human-in-the-loop (CeTI), Technische Universit\u00e4t Dresden, Helmholtzstra\u00dfe 18 Dresden 01069, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"219","published-online":{"date-parts":[[2022,9,26]]},"reference":[{"key":"S0129065722500514BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"S0129065722500514BIB002","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2017.09.017"},{"key":"S0129065722500514BIB003","first-page":"2704","volume-title":"2018 IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2018","author":"Jacob B.","year":"2018"},{"key":"S0129065722500514BIB005","unstructured":"Y. L. Cun,  J. S. Denker and  S. A. Solla ,  Optimal Brain Damage, Advances in Neural Information Processing Systems 2  (Morgan Kaufmann Publishers Inc.,  San Francisco, CA, USA,  1990),  pp. 598\u2013605."},{"key":"S0129065722500514BIB006","first-page":"1135","volume-title":"Proc. 28th Int. Conf. Neural Information Processing Systems, NIPS\u201915","volume":"1","author":"Han S.","year":"2015"},{"key":"S0129065722500514BIB007","first-page":"1314","volume-title":"2019 IEEE\/CVF Int. Conf. Computer Vision, ICCV 2019","author":"Howard A.","year":"2019"},{"key":"S0129065722500514BIB008","first-page":"265","volume-title":"Proc. 12th USENIX Conf. Operating Systems Design and Implementation, OSDI\u201916","author":"Abadi M.","year":"2016"},{"key":"S0129065722500514BIB009","first-page":"8024","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conf. Neural Information Processing Systems 2019, NeurIPS 2019","author":"Paszke A.","year":"2019"},{"key":"S0129065722500514BIB011","first-page":"1","volume-title":"8th Int. Conf. Learning Representations, ICLR 2020","author":"Esser S. K.","year":"2020"},{"key":"S0129065722500514BIB012","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/978-3-030-58568-6_9","volume-title":"Computer Vision \u2014 ECCV 2020","author":"Liu Z.","year":"2020"},{"key":"S0129065722500514BIB013","first-page":"232","volume-title":"Proc. 22nd Engineering Applications of Neural Networks Conf.","author":"Wu B.","year":"2021"},{"key":"S0129065722500514BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"S0129065722500514BIB016","first-page":"1","volume-title":"2014 IEEE Workshop on Signal Processing Systems (SiPS)","author":"Hwang K.","year":"2014"},{"key":"S0129065722500514BIB017","first-page":"4107","volume-title":"Advances in Neural Information Processing Systems 29: Annual Conf. Neural Information Processing Systems 2016","author":"Hubara I.","year":"2016"},{"key":"S0129065722500514BIB018","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"S0129065722500514BIB020","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1007\/978-3-319-46493-0_32","volume-title":"Computer Vision \u2014 ECCV 2016 \u2014 14th European Conf., Proc., Part IV","volume":"9908","author":"Rastegari M.","year":"2016"},{"key":"S0129065722500514BIB021","first-page":"62","volume-title":"30th British Machine Vision Conf. 2019, BMVC 2019","author":"Bulat A.","year":"2019"},{"issue":"1","key":"S0129065722500514BIB022","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1007\/s11263-019-01227-8","volume":"128","author":"Liu Z.","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"S0129065722500514BIB023","first-page":"1951","volume-title":"2019 IEEE\/CVF Int. Conf. Computer Vision Workshops, ICCV Workshops 2019","author":"Bethge J.","year":"2019"},{"key":"S0129065722500514BIB024","doi-asserted-by":"crossref","first-page":"1438","DOI":"10.1109\/WACV48630.2021.00148","volume-title":"IEEE Winter Conf. Applications of Computer Vision, WACV 2021","author":"Bethge J.","year":"2021"},{"key":"S0129065722500514BIB025","first-page":"2247","volume-title":"2020 IEEE\/CVF Conf. Computer Vision and Pattern Recognition, CVPR 2020","author":"Qin H.","year":"2020"},{"key":"S0129065722500514BIB027","first-page":"5406","volume-title":"2017 IEEE Conf. Computer Vision and Pattern Recognition, CVPR 2017","author":"Cai Z.","year":"2017"},{"key":"S0129065722500514BIB028","first-page":"1485","volume-title":"2020 IEEE\/CVF Conf. Computer Vision and Pattern Recognition (CVPR)","author":"Zhuang B.","year":"2020"},{"key":"S0129065722500514BIB029","first-page":"1","volume-title":"7th Int. Conf. Learning Representations, ICLR 2019","author":"Louizos C.","year":"2019"},{"key":"S0129065722500514BIB030","doi-asserted-by":"crossref","first-page":"102413","DOI":"10.1016\/j.media.2022.102413","volume":"78","author":"Zhao M.","year":"2022","journal-title":"Med. Image Anal."},{"key":"S0129065722500514BIB031","doi-asserted-by":"crossref","first-page":"2150038","DOI":"10.1142\/S0129065721500386","volume":"31","author":"Cosimo Ieracitano A. H.","year":"2021","journal-title":"Int. J. Neural Syst."},{"key":"S0129065722500514BIB032","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/j.ins.2022.01.046","volume":"591","author":"Wang F.","year":"2022","journal-title":"Inf. Sci."},{"key":"S0129065722500514BIB033","first-page":"12546","volume-title":"Proc. 38th Int. Conf. Machine Learning, ICML 2021, Virtual Event","author":"Zhang Z.","year":"2021"},{"key":"S0129065722500514BIB035","first-page":"4851","volume-title":"2019 IEEE\/CVF Int. Conf. Computer Vision, ICCV 2019","author":"Gong R.","year":"2019"},{"key":"S0129065722500514BIB036","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/978-3-030-01237-3_23","volume-title":"Computer Vision \u2014 ECCV 2018","author":"Zhang D.","year":"2018"}],"container-title":["International Journal of Neural Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129065722500514","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,26]],"date-time":"2022-11-26T07:43:56Z","timestamp":1669448636000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0129065722500514"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,26]]},"references-count":30,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1142\/S0129065722500514"],"URL":"https:\/\/doi.org\/10.1142\/s0129065722500514","relation":{},"ISSN":["0129-0657","1793-6462"],"issn-type":[{"value":"0129-0657","type":"print"},{"value":"1793-6462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,26]]},"article-number":"2250051"}}