{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T21:08:29Z","timestamp":1772831309441,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T00:00:00Z","timestamp":1652400000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>In the paper, the possibility of combining deep neural network (DNN) model compression methods to achieve better compression results was considered. To compare the advantages and disadvantages of each method, all methods were applied to the ResNet18 model for pretraining to the NCT-CRC-HE-100K dataset while using CRC-VAL-HE-7K as the validation dataset. In the proposed method, quantization, pruning, weight clustering, QAT (quantization-aware training), preserve cluster QAT (hereinafter PCQAT), and distillation were performed for the compression of ResNet18. The final evaluation of the obtained models was carried out on a Raspberry Pi 4 device using the validation dataset. The greatest model compression result on the disk was achieved by applying the PCQAT method, whose application led to a reduction in size of the initial model by as much as 45 times, whereas the greatest model acceleration result was achieved via distillation on the MobileNetV2 model. All methods led to the compression of the initial size of the model, with a slight loss in the model accuracy or an increase in the model accuracy in the case of QAT and weight clustering. INT8 quantization and knowledge distillation also led to a significant decrease in the model execution time.<\/jats:p>","DOI":"10.3390\/axioms11050229","type":"journal-article","created":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T08:37:02Z","timestamp":1652431022000},"page":"229","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["The Possibility of Combining and Implementing Deep Neural Network Compression Methods"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3679-5058","authenticated-orcid":false,"given":"Bratislav","family":"Predi\u0107","sequence":"first","affiliation":[{"name":"Faculty of Electronic Engineering, University of Ni\u0161, Aleksandra Medvedeva 14, 18000 Ni\u0161, Serbia"}]},{"given":"Uro\u0161","family":"Vuki\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electronic Engineering, University of Ni\u0161, Aleksandra Medvedeva 14, 18000 Ni\u0161, Serbia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2577-7927","authenticated-orcid":false,"given":"Muzafer","family":"Sara\u010devi\u0107","sequence":"additional","affiliation":[{"name":"Department of Computer Sciences, University of Novi Pazar, Dimitrija Tucovi\u0107a bb, 36300 Novi Pazar, Serbia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5308-2503","authenticated-orcid":false,"given":"Darjan","family":"Karaba\u0161evi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Jevrejska 24, 11000 Belgrade, Serbia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6846-3074","authenticated-orcid":false,"given":"Dragi\u0161a","family":"Stanujki\u0107","sequence":"additional","affiliation":[{"name":"Technical Faculty in Bor, University of Belgrade, Vojske Jugoslavije 12, 19210 Bor, Serbia"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kotenko, I., Izrailov, K., and Buinevich, M. (2022). Static Analysis of Information Systems for IoT Cyber Security: A Survey of Machine Learning Approaches. Sensors, 22.","DOI":"10.3390\/s22041335"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"10357","DOI":"10.1007\/s11227-022-04310-z","article-title":"Application of machine learning and IoT to enable child safety at home environment","volume":"78","author":"Shenbagalakshmi","year":"2022","journal-title":"J. Supercomput."},{"key":"ref_3","unstructured":"Mohammadi, F.G., Shenavarmasouleh, F., and Arabnia, H.R. (2022). Applications of Machine Learning in Healthcare and Internet of Things (IOT): A Comprehensive Review. arXiv."},{"key":"ref_4","first-page":"42","article-title":"Machine learning for IoT healthcare applications: A review","volume":"5","author":"Mohammed","year":"2021","journal-title":"Int. J. Sci. Bus."},{"key":"ref_5","first-page":"92","article-title":"Machine Learning Powered IoT for Smart Applications","volume":"5","author":"Hamad","year":"2021","journal-title":"Int. J. Sci. Bus."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103741","DOI":"10.1016\/j.micpro.2020.103741","article-title":"A machine learning based IoT for providing an intrusion detection system for security","volume":"82","author":"Atul","year":"2021","journal-title":"Microprocess. Microsyst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"22071","DOI":"10.1073\/pnas.1900654116","article-title":"Definitions, methods, and applications in interpretable machine learning","volume":"116","author":"Murdoch","year":"2019","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-00327-4","article-title":"Selecting critical features for data classification based on machine learning methods","volume":"7","author":"Chen","year":"2020","journal-title":"J. Big Data"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Abdulhammed, R., Musafer, H., Alessa, A., Faezipour, M., and Abuzneid, A. (2019). Features dimensionality reduction ap-proaches for machine learning based network intrusion detection. Electronics, 8.","DOI":"10.3390\/electronics8030322"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., and Yu, B. (2019). Interpretable machine learning: Definitions, methods, and applications. arXiv.","DOI":"10.1073\/pnas.1900654116"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/s12178-020-09600-8","article-title":"Machine Learning and Artificial Intelligence: Definitions, Applications, and Future Directions","volume":"13","author":"Helm","year":"2020","journal-title":"Curr. Rev. Musculoskelet. Med."},{"key":"ref_12","unstructured":"Canziani, A., Paszke, A., and Culurciello, E. (2016). An analysis of deep neural network models for practical applications. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.1109\/JPROC.2017.2761740","article-title":"Efficient Processing of Deep Neural Networks: A Tutorial and Survey","volume":"105","author":"Sze","year":"2017","journal-title":"Proc. IEEE"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.comcom.2019.11.022","article-title":"Detection of flood disaster system based on IoT, big data and convolutional deep neural network","volume":"150","author":"Anbarasan","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_15","unstructured":"Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv."},{"key":"ref_16","first-page":"1","article-title":"Deep neural network-based application partitioning and scheduling for hospitals and medical enterprises using IoT assisted mobile fog cloud","volume":"15","author":"Lakhan","year":"2021","journal-title":"Enterp. Inf. Syst."},{"key":"ref_17","unstructured":"Sattler, F., Wiegand, T., and Samek, W. (2020). Trends and advancements in deep neural network communication. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MSSC.2017.2745818","article-title":"Embedded Deep Neural Network Processing: Algorithmic and Processor Techniques Bring Deep Learning to IoT and Edge Devices","volume":"9","author":"Verhelst","year":"2017","journal-title":"IEEE Solid-State Circuits Mag."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1016\/j.compag.2018.10.015","article-title":"IoT based hydroponics system using Deep Neural Networks","volume":"155","author":"Mehra","year":"2018","journal-title":"Comput. Electron. Agric."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"114800","DOI":"10.1016\/j.eswa.2021.114800","article-title":"A comprehensive survey on deep neural networks for stock market: The need, challenges, and future directions","volume":"177","author":"Thakkar","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"89","DOI":"10.5937\/jouproman2103089B","article-title":"Solving integer programming problems by using population-based beetle antennae search algorithm","volume":"9","year":"2021","journal-title":"J. Process Manag. New Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5791","DOI":"10.1007\/s00521-022-06960-9","article-title":"Literature review: Efficient deep neural networks techniques for medical image analysis","volume":"34","author":"Abdou","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Fang, X., Liu, H., Xie, G., Zhang, Y., and Liu, D. (2020, January 27\u201330). Deep Neural Network Compression Method Based on Product Quantization. Proceedings of the 39th Chinese Control Conference (CCC), Shenyang, China.","DOI":"10.23919\/CCC50068.2020.9188698"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3005348","article-title":"Structured Pruning of Deep Convolutional Neural Networks","volume":"13","author":"Anwar","year":"2017","journal-title":"ACM J. Emerg. Technol. Comput. Syst."},{"key":"ref_25","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning Filters for Efficient ConvNets. arXiv."},{"key":"ref_26","unstructured":"Chen, W., Wilson, J.T., Tyree, S., Weinberger, K.Q., and Chen, Y. (2015, January 6\u201311). Compressing Neural Networks with the Hashing Trick. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_27","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bucila, C., Caruana, R., and Niculescu-Mizil, A. (2006, January 20\u201323). Model compression. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD \u201906, New York, NY, USA.","DOI":"10.1145\/1150402.1150464"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge Distillation: A Survey","volume":"129","author":"Gou","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","unstructured":"Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2019). Once-for-All: Train One Network and Specialize It for Efficient Deployment. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Luo, J.-H., Wu, J., and Lin, W. (2017, January 22\u201329). ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.541"},{"key":"ref_32","first-page":"721","article-title":"Image Information Hiding Method Based on Image Compression and Deep Neural Network","volume":"124","author":"Duan","year":"2020","journal-title":"Comput. Model. Eng. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Chen, R., Chen, Y., and Su, J. (2018, January 30). Deep convolutional neural networks compression method based on linear representation of kernels. Proceedings of the Eleventh International Conference on Machine Vision (ICMV 2018), Munich, Germany.","DOI":"10.1117\/12.2522992"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sun, S., Chen, W., Bian, J., Liu, X., and Liu, T. Ensemble-Compression: A New Method for Parallel Training of Deep Neural Networks. Machine Learning and Knowledge Discovery in Databases. Lecture Notes in Artificial Intelligence, ECML PKDD 2017, PT I.","DOI":"10.1007\/978-3-319-71249-9_12"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Salehinejad, H., and Valaee, S. (2019, January 12\u201317). Ising-dropout: A Regularization Method for Training and Compression of Deep Neural Networks. Proceedings of the ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682914"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yamagiwa, S., Yang, W., and Wada, K. (2022). Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network. Electronics, 11.","DOI":"10.3390\/electronics11040504"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zeng, L., Chen, S., and Zeng, S. (2019, January 18\u201320). An Efficient End-to-End Channel Level Pruning Method for Deep Neural Networks Compression. Proceedings of the IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.","DOI":"10.1109\/ICSESS47205.2019.9040742"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ademola, O.A., Leier, M., and Petlenkov, E. (2021). Evaluation of Deep Neural Network Compression Methods for Edge Devices Using Weighted Score-Based Ranking Scheme. Sensors, 21.","DOI":"10.3390\/s21227529"},{"key":"ref_39","unstructured":"Kather, J.N., Halama, N., and Marx, A. (2021, December 12). Zenodo. Available online: https:\/\/zenodo.org\/record\/1214456#.YZkx57so9hF."},{"key":"ref_40","unstructured":"Macenko, M., Niethammer, M., Marron, J.S., Borland, D., Woosley, J.T., Guan, X., Schmitt, C., and Thomas, N.E. (July, January 28). A Method For Normalizing Histology Slides For Quantitative Analysis. Proceedings of the IEEE International Symposium on Biomedical Imaging, Boston, MA, USA."},{"key":"ref_41","unstructured":"Yakubovskiy, P. (2021, December 12). Classification Models Zoo\u2014Keras (and TensorFlow Keras). Available online: https:\/\/pypi.org\/project\/image-classifiers."},{"key":"ref_42","unstructured":"(2022, May 03). TensorFlow Model Optimization Toolkit. Available online: https:\/\/www.tensorflow.org\/model_optimization\/guide."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1109\/TCYB.2019.2928174","article-title":"A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks","volume":"51","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_44","first-page":"147","article-title":"Multicriteria decision-making in the selection of warships: A new approach to the ahp method","volume":"13","author":"Costa","year":"2021","journal-title":"Int. J. Anal. Hierarchy Process"}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/11\/5\/229\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:10:24Z","timestamp":1760137824000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/11\/5\/229"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,13]]},"references-count":44,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["axioms11050229"],"URL":"https:\/\/doi.org\/10.3390\/axioms11050229","relation":{},"ISSN":["2075-1680"],"issn-type":[{"value":"2075-1680","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,13]]}}}