{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T01:57:43Z","timestamp":1772762263390,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T00:00:00Z","timestamp":1639958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science Fund of the Republic of Serbia, 6527104, AI-Com-in-AI.","award":["6527104"],"award-info":[{"award-number":["6527104"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Driven by the need for the compression of weights in neural networks (NNs), which is especially beneficial for edge devices with a constrained resource, and by the need to utilize the simplest possible quantization model, in this paper, we study the performance of three-bit post-training uniform quantization. The goal is to put various choices of the key parameter of the quantizer in question (support region threshold) in one place and provide a detailed overview of this choice\u2019s impact on the performance of post-training quantization for the MNIST dataset. Specifically, we analyze whether it is possible to preserve the accuracy of the two NN models (MLP and CNN) to a great extent with the very simple three-bit uniform quantizer, regardless of the choice of the key parameter. Moreover, our goal is to answer the question of whether it is of the utmost importance in post-training three-bit uniform quantization, as it is in quantization, to determine the optimal support region threshold value of the quantizer to achieve some predefined accuracy of the quantized neural network (QNN). The results show that the choice of the support region threshold value of the three-bit uniform quantizer does not have such a strong impact on the accuracy of the QNNs, which is not the case with two-bit uniform post-training quantization, when applied in MLP for the same classification task. Accordingly, one can anticipate that due to this special property, the post-training quantization model in question can be greatly exploited.<\/jats:p>","DOI":"10.3390\/e23121699","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T08:43:32Z","timestamp":1639989812000},"page":"1699","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Whether the Support Region of Three-Bit Uniform Quantizer Has a Strong Impact on Post-Training Quantization for MNIST Dataset?"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3552-7211","authenticated-orcid":false,"given":"Jelena","family":"Nikoli\u0107","sequence":"first","affiliation":[{"name":"Faculty of Electronic Engineering, University of Nis, Aleksandra Medvedeva 14, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8267-9541","authenticated-orcid":false,"given":"Zoran","family":"Peri\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electronic Engineering, University of Nis, Aleksandra Medvedeva 14, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8660-6473","authenticated-orcid":false,"given":"Danijela","family":"Aleksi\u0107","sequence":"additional","affiliation":[{"name":"Department of Mobile Network Nis, Telekom Srbija, Vozdova 11, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefan","family":"Tomi\u0107","sequence":"additional","affiliation":[{"name":"School of Engineering and Technology, Al Dar University College, Dubai 35529, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aleksandra","family":"Jovanovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electronic Engineering, University of Nis, Aleksandra Medvedeva 14, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Vestias, M., Duarte, R., Sousa, J., and Neto, H. (2020). Moving Deep Learning to the Edge. Algorithms, 13.","DOI":"10.3390\/a13050125"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., and Keutzer, K. (2021). A Survey of Quantization Methods for Efficient Neural Network Inference. arXiv.","DOI":"10.1201\/9781003162810-13"},{"key":"ref_3","unstructured":"Liu, D., Kong, H., Luo, X., Liu, W., and Subramaniam, R. (2020). Bringing AI to Edge: From Deep Learning\u2019s Perspective. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/JMASS.2020.3034205","article-title":"A Review of Recent Advances of Binary Neural Networks for Edge Computing","volume":"2","author":"Zhao","year":"2021","journal-title":"IEEE J. Miniat. Air Space Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Novac, P.E., Hacene, G.B., Pegatoquet, A., Miramond, B., and Gripon, V. (2021). Quantization and Deployment of Deep Neural Networks on Microcontrollers. Sensors, 21.","DOI":"10.3390\/s21092984"},{"key":"ref_6","unstructured":"Guo, Y. (2018). A Survey on Methods and Theories of Quantized Neural Networks. arXiv."},{"key":"ref_7","unstructured":"(2021, November 01). Number of Internet of Things (IoT) Connected Devices Worldwide in 2018, 2025 and 2030. Available online: https:\/\/www.statista.com\/statistics\/802690\/worldwide-connected-devices-by-accesstechnology."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1109\/TPAMI.2018.2886192","article-title":"Deep Neural Network Compression by In-Parallel Pruning-Quantization","volume":"42","author":"Tung","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","unstructured":"Yang, Z., Wang, Y., Han, K., Xu, C., Xu, C., Tao, D., and Xu, C. (2020, January 6\u201312). Searching for Low-Bit Weights in Quantized Neural Networks. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada."},{"key":"ref_10","unstructured":"Han, S., Mao, H., and Dally, W.J. (2016, January 2\u20134). Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. Proceedings of the International Conference on Learning Representations, San Juan, PR, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sanghyun, S., and Juntae, K. (2019). Efficient Weights Quantization of Convolutional Neural Networks Using Kernel Density Estimation Based Non-Uniform Quantizer. Appl. Sci., 9.","DOI":"10.3390\/app9122559"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Peri\u0107, Z., Deni\u0107, B., Savi\u0107, M., and Despotovi\u0107, V. (2020). Design and Analysis of Binary Scalar Quantizer of Laplacian Source with Applications. Information, 11.","DOI":"10.3390\/info11110501"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"55","DOI":"10.5755\/j02.eie.28881","article-title":"Binary Quantization Analysis of Neural Networks Weights on MNIST Dataset","volume":"27","year":"2021","journal-title":"Elektron. Ir Elektrotechnika"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"47194","DOI":"10.1109\/ACCESS.2021.3067889","article-title":"Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer","volume":"9","author":"Pham","year":"2021","journal-title":"IEEE Access"},{"key":"ref_15","unstructured":"Banner, R., Nahshan, Y., and Soudry, D. (2019, January 8\u201314). Post-training 4-bit Quantization of Convolutional Networks for Rapid-Deployment. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada."},{"key":"ref_16","unstructured":"Choi, J., Venkataramani, S., Srinivasan, V., Gopalakrishnan, K., Wang, Z., and Chuang, P. (April, January 31). Accurate and Efficient 2-Bit Quantized Neural Networks. Proceedings of the 2nd SysML Conference, Stanford, CA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bhalgat, Y., Lee, J., Nagel, M., Blankevoort, T., and Kwak, N. (2020, January 14\u201319). LSQ+: Improving Low-Bit Quantization through Learnable Offsets and Better Initialization. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00356"},{"key":"ref_18","first-page":"6869","article-title":"Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations","volume":"18","author":"Hubara","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","unstructured":"Huang, K., Ni, B., and Yang, X. (February, January 27). Efficient Quantization for Neural Networks with Binary Weights and Low Bit Width Activations. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7839064","DOI":"10.1155\/2020\/7839064","article-title":"A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks","volume":"2020","author":"Long","year":"2020","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shlezinger, N., and Eldar, Y. (2021). Deep Task-Based Quantization. Entropy, 23.","DOI":"10.3390\/e23010104"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/18.915652","article-title":"Asymptotic Analysis of Optimal Fixed-Rate Uniform Scalar Quantization","volume":"47","author":"Hui","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1109\/LCOMM.2017.2749218","article-title":"A Rigorous Revisit to the Partial Distortion Theorem in the Case of a Laplacian Source","volume":"21","author":"Lee","year":"2017","journal-title":"IEEE Commun. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2626","DOI":"10.1109\/TIT.2017.2775615","article-title":"On the Convexity of the MSE Distortion of Symmetric Uniform Scalar Quantization","volume":"64","author":"Na","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1782","DOI":"10.1109\/TIT.2018.2867182","article-title":"Monotonicity of Step Sizes of MSE-Optimal Symmetric Uniform Scalar Quantizers","volume":"65","author":"Na","year":"2019","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_26","first-page":"615","article-title":"Analysis and Design of Robust Quasilogarithmic Quantizer for the Purpose of Traffic Optimisation","volume":"47","year":"2018","journal-title":"Inf. Technol. Control"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.dsp.2013.12.014","article-title":"Optimization of Multiple Region Quantizer for Laplacian Source","volume":"27","year":"2014","journal-title":"Digit. Signal Process."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1049\/iet-spr.2011.0213","article-title":"High-quality Laplacian Source Quantisation Using a Combination of Restricted and Unrestricted Logarithmic Quantisers","volume":"6","year":"2012","journal-title":"IET Signal Process."},{"key":"ref_29","first-page":"6647135","article-title":"Symmetric Quantile Quantizer Parameterization for the Laplacian Source: Qualification for Contemporary Quantization Solutions","volume":"2021","year":"2021","journal-title":"Math. Probl. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2972","DOI":"10.1109\/18.959274","article-title":"On the Support of MSE-optimal, Fixed-Rate, Scalar Quantizers","volume":"47","author":"Na","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_31","unstructured":"Jayant, S., and Noll, P. (1984). Digital Coding of Waveforms, Prentice Hall."},{"key":"ref_32","first-page":"403","article-title":"Quasilogarithmic Quantizer for Laplacian Source: Support Region Ubiquitous Optimization Task","volume":"64","year":"2019","journal-title":"Rev. Roum. Sci. Tech."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Fang, J., Shafiee, A., Abdel-Aziz, H., Thorsley, D., Georgiadis, G., and Hassoun, J. (2020, January 23\u201328). Post-Training Piecewise Linear Quantization for Deep Neural Networks. Proceedings of the European Conference on Computer Vision 2020, Glasgow, UK.","DOI":"10.1007\/978-3-030-58536-5_5"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tomi\u0107, S., Nikoli\u0107, J., Peri\u0107, Z., and Aleksi\u0107, D. Performance of Post-training Two-bits Uniform and Layer-wise Uniform Quantization for MNIST Dataset from the Perspective of Support Region Choice. Math. Probl. Eng., 2021. submitted.","DOI":"10.1155\/2022\/1463094"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1049\/cmu2.12114","article-title":"Iterative Algorithm for Designing Asymptotically Optimal Uniform Scalar Quantization of the One-Sided Rayleigh Density","volume":"15","year":"2021","journal-title":"IET Commun."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1683","DOI":"10.1109\/TIT.2009.2013020","article-title":"Analysis of Optimal High Resolution and Fixed Rate Scalar Quantization","volume":"55","author":"Bach","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","unstructured":"Salimans, T., and Kingma, D. (2018). Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.4316\/AECE.2021.03001","article-title":"Robust 2-bit Quantization of Weights in Neural Network Modeled by Laplacian Distribution","volume":"21","year":"2021","journal-title":"Adv. Electr. Comput. Eng."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Niu, H., Wei, J., and Chen, Y. (2021). Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions. Entropy, 23.","DOI":"10.3390\/e23010056"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Peri\u0107, Z., Savi\u0107, M., Simi\u0107, N., Deni\u0107, B., and Despotovi\u0107, V. (2021). Design of a 2-bit Neural Network Quantizer for Laplacian Source. Entropy, 23.","DOI":"10.3390\/e23080933"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]","volume":"29","author":"Deng","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_42","unstructured":"Agarap, A.F. (2019). Deep Learning Using Rectified Linear Units (ReLU). arXiv."},{"key":"ref_43","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, C.S., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale Machine Learning on Heterogeneous Distributed Systems. arXiv."},{"key":"ref_44","unstructured":"(2021, October 10). Available online: https:\/\/github.com\/zalandoresearch\/fashion-mnist."},{"key":"ref_45","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv."},{"key":"ref_46","unstructured":"Python Software Foundation (2021, September 01). Python Language Reference, Version 2.7. Available online: http:\/\/www.python.org."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"135059","DOI":"10.1109\/ACCESS.2021.3116418","article-title":"Network Compression via Mixed Precision Quantization Using a Multi-Layer Perceptron for the Bit-Width Allocation","volume":"9","author":"Soufleri","year":"2021","journal-title":"IEEE Access"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1699\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:52:25Z","timestamp":1760169145000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1699"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,20]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["e23121699"],"URL":"https:\/\/doi.org\/10.3390\/e23121699","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,20]]}}}