{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,20]],"date-time":"2026-11-20T18:35:10Z","timestamp":1795199710652,"version":"4.10.3"},"reference-count":60,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,4,23]],"date-time":"2021-04-23T00:00:00Z","timestamp":1619136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Embedding Artificial Intelligence onto low-power devices is a challenging task that has been partly overcome with recent advances in machine learning and hardware design. Presently, deep neural networks can be deployed on embedded targets to perform different tasks such as speech recognition, object detection or Human Activity Recognition. However, there is still room for optimization of deep neural networks onto embedded devices. These optimizations mainly address power consumption, memory and real-time constraints, but also an easier deployment at the edge. Moreover, there is still a need for a better understanding of what can be achieved for different use cases. This work focuses on quantization and deployment of deep neural networks onto low-power 32-bit microcontrollers. The quantization methods, relevant in the context of an embedded execution onto a microcontroller, are first outlined. Then, a new framework for end-to-end deep neural networks training, quantization and deployment is presented. This framework, called MicroAI, is designed as an alternative to existing inference engines (TensorFlow Lite for Microcontrollers and STM32Cube.AI). Our framework can indeed be easily adjusted and\/or extended for specific use cases. Execution using single precision 32-bit floating-point as well as fixed-point on 8- and 16 bits integers are supported. The proposed quantization method is evaluated with three different datasets (UCI-HAR, Spoken MNIST and GTSRB). Finally, a comparison study between MicroAI and both existing embedded inference engines is provided in terms of memory and power efficiency. On-device evaluation is done using ARM Cortex-M4F-based microcontrollers (Ambiq Apollo3 and STM32L452RE).<\/jats:p>","DOI":"10.3390\/s21092984","type":"journal-article","created":{"date-parts":[[2021,4,25]],"date-time":"2021-04-25T02:12:57Z","timestamp":1619316777000},"page":"2984","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":218,"title":["Quantization and Deployment of Deep Neural Networks on Microcontrollers"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2468-8711","authenticated-orcid":false,"given":"Pierre-Emmanuel","family":"Novac","sequence":"first","affiliation":[{"name":"CNRS, LEAT, Universit\u00e9 C\u00f4te d\u2019Azur, 06903 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6855-5825","authenticated-orcid":false,"given":"Ghouthi","family":"Boukli Hacene","sequence":"additional","affiliation":[{"name":"IMT Atlantique, 29200 Brest, France"},{"name":"MILA, Montreal, QC H2S 3H1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alain","family":"Pegatoquet","sequence":"additional","affiliation":[{"name":"CNRS, LEAT, Universit\u00e9 C\u00f4te d\u2019Azur, 06903 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beno\u00eet","family":"Miramond","sequence":"additional","affiliation":[{"name":"CNRS, LEAT, Universit\u00e9 C\u00f4te d\u2019Azur, 06903 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4353-4542","authenticated-orcid":false,"given":"Vincent","family":"Gripon","sequence":"additional","affiliation":[{"name":"IMT Atlantique, 29200 Brest, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,23]]},"reference":[{"key":"ref_1","unstructured":"Wang, Y., Wei, G., and Brooks, D. (2019). Benchmarking TPU, GPU, and CPU Platforms for Deep Learning. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Lin, J., Chen, W.M., Lin, Y., Cohn, J., Gan, C., and Han, S. (2020, January 6\u201312). MCUNet: Tiny Deep Learning on IoT Devices. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Online.","DOI":"10.1109\/IPCCC50635.2020.9391558"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lai, L., and Suda, N. (2018, January 5\u20138). Enabling Deep Learning at the IoT Edge. Proceedings of the International Conference on Computer-Aided Design (ICCAD\u201918), San Diego, CA, USA.","DOI":"10.1145\/3240765.3243473"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kromes, R., Russo, A., Miramond, B., and Verdier, F. (2019, January 11\u201313). Energy consumption minimization on LoRaWAN sensor network by using an Artificial Neural Network based application. Proceedings of the 2019 IEEE Sensors Applications Symposium (SAS), Sophia Antipolis, France.","DOI":"10.1109\/SAS.2019.8705992"},{"key":"ref_5","unstructured":"Chaudhuri, K., and Salakhutdinov, R. (2019, January 9\u201315). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning (PMLR 2019), Long Beach, CA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Novac, P.E., Russo, A., Miramond, B., Pegatoquet, A., Verdier, F., and Castagnetti, A. (2020, January 26\u201328). Toward unsupervised Human Activity Recognition on Microcontroller Units. Proceedings of the 2020 23rd Euromicro Conference on Digital System Design (DSD), Kranj, Slovenia.","DOI":"10.1109\/DSD51259.2020.00090"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TVLSI.2016.2580142","article-title":"Hybrid Hardware\/Software Floating-Point Implementations for Optimized Area and Throughput Tradeoffs","volume":"25","author":"Pimentel","year":"2017","journal-title":"IEEE Trans. Very Large Scale Integr. (VLSI) Syst."},{"key":"ref_8","unstructured":"Choi, J., Chuang, P.I.J., Wang, Z., Venkataramani, S., Srinivasan, V., and Gopalakrishnan, K. (2018). Bridging the accuracy gap for 2-bit quantized neural networks (qnn). arXiv."},{"key":"ref_9","unstructured":"Esser, S.K., McKinstry, J.L., Bablani, D., Appuswamy, R., and Modha, D.S. (2019). Learned step size quantization. arXiv."},{"key":"ref_10","unstructured":"Nikoli\u0107, M., Hacene, G.B., Bannon, C., Lascorz, A.D., Courbariaux, M., Bengio, Y., Gripon, V., and Moshovos, A. (2020). Bitpruning: Learning bitlengths for aggressive and accurate quantization. arXiv."},{"key":"ref_11","unstructured":"Uhlich, S., Mauch, L., Yoshiyama, K., Cardinaux, F., Garcia, J.A., Tiedemann, S., Kemp, T., and Nakamura, A. (2019). Differentiable quantization of deep neural networks. arXiv."},{"key":"ref_12","unstructured":"Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (2016). Binarized Neural Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. (2016, January 8\u201316). Xnor-net: Imagenet classification using binary convolutional neural networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"ref_14","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_15","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_18","unstructured":"Han, S., Pool, J., Tran, J., and Dally, W. (2015, January 7\u201310). Learning both weights and connections for efficient neural network. Proceedings of the 28th International Conference on Neural Information Processing Systems\u2014Volume 1, Montreal, QC, Canada."},{"key":"ref_19","unstructured":"Yamamoto, K., and Maeno, K. (2018). PCAS: Pruning Channels with Attention Statistics. arXiv."},{"key":"ref_20","unstructured":"Hacene, G.B., Lassance, C., Gripon, V., Courbariaux, M., and Bengio, Y. (2019). Attention based pruning for shift networks. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ramakrishnan, R.K., Sari, E., and Nia, V.P. (2020, January 13\u201315). Differentiable Mask for Pruning Convolutional and Recurrent Networks. Proceedings of the 2020 17th Conference on Computer and Robot Vision (CRV), Ottawa, ON, Canada.","DOI":"10.1109\/CRV50864.2020.00037"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, Y., Ding, Y., Liu, P., Zhu, L., Zhang, H., and Yang, Y. (2020, January 14\u201319). Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00208"},{"key":"ref_23","unstructured":"Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.patrec.2020.07.028","article-title":"Deep k-means: Jointly clustering with k-means and learning representations","volume":"138","author":"Fard","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"860","DOI":"10.1109\/JSTSP.2020.3005030","article-title":"Iteratively training look-up tables for network quantization","volume":"14","author":"Cardinaux","year":"2020","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, Z., and Fan, D. (2019, January 16\u201320). Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network Using Truncated Gaussian Approximation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01170"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lee, E., and Hwang, Y. (2021). Layer-Wise Network Compression Using Gaussian Mixture Model. Electronics, 10.","DOI":"10.3390\/electronics10010072"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Vogel, S., Raghunath, R.B., Guntoro, A., Van Laerhoven, K., and Ascheid, G. (2019, January 28\u201330). Bit-Shift-Based Accelerator for CNNs with Selectable Accuracy and Throughput. Proceedings of the 2019 22nd Euromicro Conference on Digital System Design (DSD), Kallithea, Greece.","DOI":"10.1109\/DSD.2019.00106"},{"key":"ref_29","unstructured":"Courbariaux, M., Bengio, Y., and David, J.P. (2015). Training deep neural networks with low precision multiplications. arXiv."},{"key":"ref_30","unstructured":"Holt, J.L., and Baker, T.E. (1991, January 8\u201312). Back propagation simulations using limited precision calculations. Proceedings of the IJCNN-91-Seattle International Joint Conference on Neural Networks, Seattle, WA, USA."},{"key":"ref_31","unstructured":"Vanhoucke, V., Senior, A., and Mao, M.Z. (2011, January 12\u201317). Improving the speed of neural networks on CPUs. Proceedings of the Deep Learning and Unsupervised Feature Learning Workshop (NIPS 2011), Granada, Spain."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Garofalo, A., Tagliavini, G., Conti, F., Rossi, D., and Benini, L. (2020, January 9\u201313). XpulpNN: Accelerating Quantized Neural Networks on RISC-V Processors Through ISA Extensions. Proceedings of the 2020 Design, Automation & Test in Europe Conference & Exhibition, DATE 2020, Grenoble, France.","DOI":"10.23919\/DATE48585.2020.9116529"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cotton, N.J., Wilamowski, B.M., and Dundar, G. (2008, January 25\u201329). A Neural Network Implementation on an Inexpensive Eight Bit Microcontroller. Proceedings of the 2008 International Conference on Intelligent Engineering Systems, Miami, FL, USA.","DOI":"10.1109\/INES.2008.4481278"},{"key":"ref_34","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML\u201910), Haifa, Israel."},{"key":"ref_35","unstructured":"Zhang, Y., Suda, N., Lai, L., and Chandra, V. (2018). Hello Edge: Keyword Spotting on Microcontrollers. arXiv."},{"key":"ref_36","unstructured":"IEEE Standard for Floating-Point Arithmetic (2019). IEEE Std 754-2019 (Revision of IEEE 754-2008), IEEE."},{"key":"ref_37","unstructured":"Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., and Venkatesh, G. (May, January 30). Mixed Precision Training. Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_38","unstructured":"ARM (2001). ARM Developer Suite AXD and armsd Debuggers Guide, 4.7.9 Q-Format, Arm Ltd.. ARM DUI 0066D Version 1.2."},{"key":"ref_39","unstructured":"David, R., Duke, J., Jain, A., Reddi, V., Jeffries, N., Li, J., Kreeger, N., Nappier, I., Natraj, M., and Regev, S. (2020). TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems. arXiv."},{"key":"ref_40","unstructured":"STMicroelectronics (2021, March 19). STM32Cube.AI. Available online: https:\/\/www.st.com\/content\/st_com\/en\/stm32-ann.html."},{"key":"ref_41","unstructured":"Google (2021, March 22). TensorFlow Lite for Microcontrollers Supported Operations. Available online: https:\/\/github.com\/tensorflow\/tensorflow\/blob\/master\/tensorflow\/lite\/micro\/kernels\/micro_ops.h."},{"key":"ref_42","unstructured":"Google (2021, March 19). TensorFlow Lite 8-Bit Quantization Specification. Available online: https:\/\/www.tensorflow.org\/lite\/performance\/quantization_spec."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18\u201322). Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00286"},{"key":"ref_44","unstructured":"(2021, March 19). STMicroelectronics. Supported Deep Learning Toolboxes and Layers, Documentation Embedded in X-CUBE-AI Expansion Package 5.2.0, Available online: https:\/\/www.st.com\/en\/embedded-software\/x-cube-ai.html."},{"key":"ref_45","unstructured":"Nordby, J. (2021, February 18). Emlearn: Machine Learning Inference Engine for Microcontrollers and Embedded Devices. Available online: https:\/\/doi.org\/10.5281\/zenodo.2589394."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sakr, F., Bellotti, F., Berta, R., and De Gloria, A. (2020). Machine Learning on Mainstream Microcontrollers. Sensors, 20.","DOI":"10.3390\/s20092638"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Givargis, T. (2021, January 18\u201321). Gravity: An Artificial Neural Network Compiler for Embedded Applications. Proceedings of the 26th Asia and South Pacific Design Automation Conference (ASPDAC\u201921), Tokyo, Japan.","DOI":"10.1145\/3394885.3431514"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"4403","DOI":"10.1109\/JIOT.2020.2976702","article-title":"FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_49","unstructured":"(2021, March 19). Tom\u2019s Obvious Minimal Language. Available online: https:\/\/toml.io\/."},{"key":"ref_50","first-page":"448","article-title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","volume":"Volume 37","author":"Bach","year":"2015","journal-title":"Proceedings of the 32nd International Conference on Machine Learning"},{"key":"ref_51","unstructured":"(2021, March 19). Jinja2. Available online: https:\/\/palletsprojects.com\/p\/jinja\/."},{"key":"ref_52","unstructured":"Zhang, H., Ciss\u00e9, M., Dauphin, Y.N., and Lopez-Paz, D. (May, January 30). mixup: Beyond Empirical Risk Minimization. Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_53","unstructured":"Davide, A., Alessandro, G., Luca, O., Xavier, P., and Jorge, L.R.O. (2013, January 24\u201326). A Public Domain Dataset for Human Activity Recognition using Smartphones. Proceedings of the ESANN, Bruges, Belgium."},{"key":"ref_54","unstructured":"Khacef, L., Rodriguez, L., and Miramond, B. (2021, February 18). Written and Spoken Digits Database for Multimodal Learning. Available online: https:\/\/doi.org\/10.5281\/zenodo.3515935."},{"key":"ref_55","unstructured":"Warden, P. (2018). Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition. arXiv."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (August, January 31). The German Traffic Sign Recognition Benchmark: A multi-class classification competition. Proceedings of the 2011 International Joint Conference on Neural Networks, San Jose, CA, USA.","DOI":"10.1109\/IJCNN.2011.6033395"},{"key":"ref_57","first-page":"871","article-title":"CMix-NN: Mixed Low-Precision CNN Library for Memory-Constrained Edge Devices","volume":"67","author":"Capotondi","year":"2020","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Park, E., Kim, D., Kim, S., Kim, Y., Kim, G., Yoon, S., and Yoo, S. (2015, January 4\u20139). Big\/little deep neural network for ultra low power inference. Proceedings of the 2015 International Conference on Hardware\/Software Codesign and System Synthesis (CODES + ISSS), Amsterdam, The Netherlands.","DOI":"10.1109\/CODESISSS.2015.7331375"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3005348","article-title":"Structured Pruning of Deep Convolutional Neural Networks","volume":"13","author":"Anwar","year":"2017","journal-title":"J. Emerg. Technol. Comput. Syst."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Arcaya-Jordan, A., Pegatoquet, A., and Castagnetti, A. (2019, January 11\u201313). Smart Connected Glasses for Drowsiness Detection: A System-Level Modeling Approach. Proceedings of the 2019 IEEE Sensors Applications Symposium (SAS), Sophia Antipolis, France.","DOI":"10.1109\/SAS.2019.8706022"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/2984\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:52:13Z","timestamp":1760161933000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/2984"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,23]]},"references-count":60,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["s21092984"],"URL":"https:\/\/doi.org\/10.3390\/s21092984","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,23]]}}}