{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T01:57:41Z","timestamp":1772762261962,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,7,22]],"date-time":"2021-07-22T00:00:00Z","timestamp":1626912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100016047","name":"Science Fund of the Republic of Serbia","doi-asserted-by":"publisher","award":["6527104"],"award-info":[{"award-number":["6527104"]}],"id":[{"id":"10.13039\/501100016047","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Achieving real-time inference is one of the major issues in contemporary neural network applications, as complex algorithms are frequently being deployed to mobile devices that have constrained storage and computing power. Moving from a full-precision neural network model to a lower representation by applying quantization techniques is a popular approach to facilitate this issue. Here, we analyze in detail and design a 2-bit uniform quantization model for Laplacian source due to its significance in terms of implementation simplicity, which further leads to a shorter processing time and faster inference. The results show that it is possible to achieve high classification accuracy (more than 96% in the case of MLP and more than 98% in the case of CNN) by implementing the proposed model, which is competitive to the performance of the other quantization solutions with almost optimal precision.<\/jats:p>","DOI":"10.3390\/e23080933","type":"journal-article","created":{"date-parts":[[2021,7,22]],"date-time":"2021-07-22T22:06:48Z","timestamp":1626991608000},"page":"933","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Design of a 2-Bit Neural Network Quantizer for Laplacian Source"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8267-9541","authenticated-orcid":false,"given":"Zoran","family":"Peri\u0107","sequence":"first","affiliation":[{"name":"Faculty of Electronic Engineering, University of Nis, Aleksandra Medvedeva 14, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7131-132X","authenticated-orcid":false,"given":"Milan","family":"Savi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Sciences and Mathematics, University of Pristina in Kosovska Mitrovica, Ive Lole Ribara 29, 38220 Kosovska Mitrovica, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0748-4672","authenticated-orcid":false,"given":"Nikola","family":"Simi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovica 6, 21000 Novi Sad, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bojan","family":"Deni\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electronic Engineering, University of Nis, Aleksandra Medvedeva 14, 18000 Nis, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vladimir","family":"Despotovi\u0107","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg, Avenue de la Fonte 6, L-4364 Esch-sur-Alzette, Luxembourg"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Teerapittayanon, S., McDanel, B., and Kung, H.T. (2017, January 5\u20138). Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices. Proceedings of the 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA.","DOI":"10.1109\/ICDCS.2017.226"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5784","DOI":"10.1109\/TNNLS.2018.2808319","article-title":"Ristretto: A Framework for Empirical Study of Resource-Efficient Inference in Convolutional Neural Networks","volume":"29","author":"Gysel","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_3","unstructured":"Breiman, L., Friedman, J., Olshen, R., and Stone, C. (1984). Classification and Regression Trees, CRC Press."},{"key":"ref_4","unstructured":"Langley, P., Iba, W., and Thompson, K. (1992, January 12\u201316). An analysis of Bayesian classifiers. Proceedings of the 10th National Conference on Artificial Intelligence, San Jose, CA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/5326.669575","article-title":"Quantizability and learning complexity in multilayer neural networks","volume":"28","author":"Fu","year":"1998","journal-title":"IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.)"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sayood, K. (2017). Introduction to Data Compression, Morgan Kaufmann. [5th ed.].","DOI":"10.1016\/B978-0-12-809474-7.00019-7"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jayant, N.S., and Noll, P. (1984). Digital Coding of Waveforms: Principles and Applications to Speech and Video, Prentice Hall.","DOI":"10.1016\/0165-1684(85)90053-2"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5663","DOI":"10.1016\/j.jfranklin.2020.04.008","article-title":"Design of single and dual-mode companding scalar quantizers based on piecewise linear approximation of the Gaussian PDF","volume":"357","year":"2020","journal-title":"J. Frankl. Inst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.sigpro.2015.08.016","article-title":"Two forward adaptive dual-mode companding scalar quantizers for Gaussian source","volume":"120","author":"Nikolic","year":"2016","journal-title":"Signal Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3169","DOI":"10.1109\/TIT.2011.2179843","article-title":"Asymptotic MSE Distortion of Mismatched Uniform Scalar Quantization","volume":"58","author":"Na","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2626","DOI":"10.1109\/TIT.2017.2775615","article-title":"On the Convexity of the MSE Distortion of Symmetric Uniform Scalar Quantization","volume":"64","author":"Na","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1782","DOI":"10.1109\/TIT.2018.2867182","article-title":"Monotonicity of Step Sizes of MSE-Optimal Symmetric Uniform Scalar Quantizers","volume":"65","author":"Na","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","unstructured":"Banner, R., Hubara, I., Hoffer, E., and Soudry, D. (2018, January 2\u20138). Scalable Methods for 8-bit Training of Neural Networks. Proceedings of the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, QC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"47194","DOI":"10.1109\/ACCESS.2021.3067889","article-title":"Training Multi-Bit Quantized and Binarized Networks with a Learnable Symmetric Quantizer","volume":"9","author":"Pham","year":"2021","journal-title":"IEEE Access"},{"key":"ref_15","unstructured":"Banner, R., Nahshan, Y., and Soudry, D. (2019, January 8\u201310). Post Training 4-bit Quantization of Convolutional Networks for Rapid-Deployment. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_16","unstructured":"Choi, J., Venkataramani, S., Srinivasan, V., Gopalakrishnan, K., Wang, Z., and Chuang, P. (April, January 31). Accurate and Efficient 2-Bit Quantized Neural Networks. Proceedings of the 2nd SysML Conference, Stanford, CA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bhalgat, Y., Lee, J., Nagel, M., Blankevoort, T., and Kwak, N. (2020, January 14\u201319). LSQ+: Improving Low-Bit Quantization Through Learnable Offsets and Better Initialization. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00356"},{"key":"ref_18","first-page":"1","article-title":"Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations","volume":"18","author":"Hubara","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","unstructured":"Zamirai, P., Zhang, J., Aberger, C.R., and De Sa, C. (2020). Revisiting BFloat16 Training. arXiv."},{"key":"ref_20","unstructured":"Li, Y., Dong, X., and Wang, W. (May, January 26). Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Conference, Formerly Addis Ababa, Ethiopia."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3444943","article-title":"Uniq: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks","volume":"37","author":"Baskin","year":"2021","journal-title":"ACM Trans. Comput. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Simons, T., and Lee, D.-J. (2019). A Review of Binarized Neural Networks. Electronics, 8.","DOI":"10.3390\/electronics8060661"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107281","DOI":"10.1016\/j.patcog.2020.107281","article-title":"Binary Neural Networks: A Survey","volume":"105","author":"Qin","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"164858","DOI":"10.1109\/ACCESS.2020.3022902","article-title":"Fixed-Sign Binary Neural Network: An Efficient Design of Neural Network for Internet-of-Things Devices","volume":"8","author":"Li","year":"2018","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/JMASS.2020.3034205","article-title":"A Review of Recent Advances of Binary Neural Networks for Edge Computing","volume":"2","author":"Zhao","year":"2021","journal-title":"IEEE J. Miniat. Air Space Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Peri\u0107, Z., Deni\u0107, B., Savi\u0107, M., and Despotovi\u0107, V. (2020). Design and Analysis of Binary Scalar Quantizer of Laplacian Source with Applications. Information, 11.","DOI":"10.3390\/info11110501"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1109\/LSP.2003.813679","article-title":"Speech Probability Distribution","volume":"10","author":"Gazor","year":"2003","journal-title":"IEEE Signal Proc. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"703","DOI":"10.15388\/Informatica.2017.152","article-title":"Coding Algorithm for Grayscale Images\u2014Design of Piecewise Uniform Quantizer with Golomb\u2013Rice Code and Novel Analytical Model for Performance Analysis","volume":"28","year":"2017","journal-title":"Informatica"},{"key":"ref_29","unstructured":"Banner, R., Nahshan, Y., Hoffer, E., and Soudry, D. (2018). ACIQ: Analytical Clipping for Integer Quantization of Neural Networks. arXiv."},{"key":"ref_30","unstructured":"Zhang, A., Lipton, Z.C., Li, M., and Smola, A.J. (2020). Dive into Deep Learning. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1109\/OJCAS.2021.3083332","article-title":"FantastIC4: A Hardware-Software Co-Design Approach for Efficiently Running 4Bit-Compact Multilayer Perceptrons","volume":"2","author":"Wiedemann","year":"2021","journal-title":"IEEE Open J. Circuits Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1109\/TETC.2017.2673548","article-title":"A Power-Aware Digital Multilayer Perceptron Accelerator with On-Chip Training Based on Approximate Computing","volume":"5","author":"Kim","year":"2017","journal-title":"IEEE Trans. Emerg. Top. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.micpro.2010.12.001","article-title":"A Scalable Pipelined Architecture for Real-Time Computation of MLP-BP Neural Networks","volume":"36","author":"Savich","year":"2012","journal-title":"Microprocess. Microsyst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4403","DOI":"10.1109\/JIOT.2020.2976702","article-title":"FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/18.915652","article-title":"Asymptotic Analysis of Optimal Fixed-Rate Uniform Scalar Quantization","volume":"47","author":"Hui","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1109\/LSP.2007.910240","article-title":"Asymptotic Formulas for Mismatched Fixed-Rate Minimum MSE Laplacian Quantizers","volume":"15","author":"Na","year":"2008","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2437","DOI":"10.1109\/TSP.2011.2112354","article-title":"Asymptotic Formulas for Variance-Mismatched Fixed-Rate Scalar Quantization of a Gaussian source","volume":"59","author":"Na","year":"2011","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_38","unstructured":"Peric, Z., Denic, B., Savi\u0107, M., Dincic, M., and Mihajlov, D. (2012). Quantization of Weights of Neural Networks with Negligible Decreasing of Prediction Accuracy. Inf. Technol. Control, Accept."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Peric, Z., Savic, M., Dincic, M., Vucic, N., Djosic, D., and Milosavljevic, S. (2021, January 25\u201327). Floating Point and Fixed Point 32-bits Quantizers for Quantization of Weights of Neural Networks. Proceedings of the 12th International Symposium on Advanced Topics in Electrical Engineering (ATEE), Bucharest, Romania.","DOI":"10.1109\/ATEE52255.2021.9425265"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.dsp.2011.09.001","article-title":"An Adaptive Waveform Coding Algorithm and its Application in Speech Coding","volume":"22","author":"Peric","year":"2012","journal-title":"Digit. Signal Process."},{"key":"ref_41","unstructured":"LeCun, Y., Cortez, C., and Burges, C. (2021, May 15). The MNIST Handwritten Digit Database. Available online: yann.lecun.com\/exdb\/mnist\/."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/933\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:33:32Z","timestamp":1760164412000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/933"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,22]]},"references-count":41,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["e23080933"],"URL":"https:\/\/doi.org\/10.3390\/e23080933","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,22]]}}}