{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T14:35:01Z","timestamp":1775745301777,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"University of Illinois Chicago Discovery Partners Institute Seed Funding Program and NSF","award":["1739396 and 1934915"],"award-info":[{"award-number":["1739396 and 1934915"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2022,11,30]]},"abstract":"<jats:p>Convolution has been the core operation of modern deep neural networks. It is well known that convolutions can be implemented in the Fourier Transform domain. In this article, we propose to use binary block Walsh\u2013Hadamard transform (WHT) instead of the Fourier transform. We use WHT-based binary layers to replace some of the regular convolution layers in deep neural networks. We utilize both one-dimensional (1D) and 2D binary WHTs in this article. In both 1D and 2D layers, we compute the binary WHT of the input feature map and denoise the WHT domain coefficients using a nonlinearity that is obtained by combining soft-thresholding with the tanh function. After denoising, we compute the inverse WHT. We use 1D-WHT to replace the 1 \u00d7 1 convolutional layers, and 2D-WHT layers can replace the 3 \u00d7 3 convolution layers and Squeeze-and-Excite layers. 2D-WHT layers with trainable weights can be also inserted before the Global Average Pooling layers to assist the dense layers. In this way, we can reduce the number of trainable parameters significantly with a slight decrease in trainable parameters. In this article, we implement the WHT layers into MobileNet-V2, MobileNet-V3-Large, and ResNet to reduce the number of parameters significantly with negligible accuracy loss. Moreover, according to our speed test, the 2D-FWHT layer runs about 24 times as fast as the regular 3 \u00d7 3 convolution with 19.51% less RAM usage in an NVIDIA Jetson Nano experiment.<\/jats:p>","DOI":"10.1145\/3510026","type":"journal-article","created":{"date-parts":[[2022,1,26]],"date-time":"2022-01-26T18:13:20Z","timestamp":1643220800000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Block Walsh\u2013Hadamard Transform-based Binary Layers in Deep Neural Networks"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9421-3936","authenticated-orcid":false,"given":"Hongyi","family":"Pan","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diaa","family":"Badawi","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmet Enis","family":"Cetin","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,18]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 (2012), 1097\u20131105.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_1_3_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556.","journal-title":"arXiv:1409.1556"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.683"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2020.2976555"},{"key":"e_1_3_1_9_2","first-page":"1159722","volume-title":"Medical Imaging 2021: Computer-Aided Diagnosis","author":"Agarwal Chirag","year":"2021","unstructured":"Chirag Agarwal, Shahin Khobahi, Dan Schonfeld, and Mojtaba Soltanalian. 2021. CoroNet: A deep network architecture for enhanced identification of COVID-19 from chest x-ray images. In Medical Imaging 2021: Computer-Aided Diagnosis, Vol. 11597. International Society for Optics and Photonics, 1159722."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9054762"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240765.3240796"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-63007-2_63"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/GlobalSIP45357.2019.8969274"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8683629"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_20"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00069"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"e_1_3_1_19_2","article-title":"Fast-scnn: Fast semantic segmentation network","author":"Poudel Rudra P. K.","year":"2019","unstructured":"Rudra P. K. Poudel, Stephan Liwicki, and Roberto Cipolla. 2019. Fast-scnn: Fast semantic segmentation network. arXiv:1902.04502. Retrieved from https:\/\/arxiv.org\/abs\/1902.04502.","journal-title":"arXiv:1902.04502"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIEA.2019.8834159"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00523"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.521"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9054442"},{"key":"e_1_3_1_25_2","first-page":"2285","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Chen Wenlin","year":"2015","unstructured":"Wenlin Chen, James Wilson, Stephen Tyree, Kilian Weinberger, and Yixin Chen. 2015. Compressing neural networks with the hashing trick. In Proceedings of the International Conference on Machine Learning. 2285\u20132294."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20102891"},{"key":"e_1_3_1_27_2","article-title":"Gradiveq: Vector quantization for bandwidth-efficient gradient aggregation in distributed cnn training","author":"Yu Mingchao","year":"2018","unstructured":"Mingchao Yu, Zhifeng Lin, Krishna Narra, Songze Li, Youjie Li, Nam Sung Kim, Alexander Schwing, Murali Annavaram, and Salman Avestimehr. 2018. Gradiveq: Vector quantization for bandwidth-efficient gradient aggregation in distributed cnn training. arXiv:1811.03617. Retrieved from https:\/\/arxiv.org\/abs\/1811.03617.","journal-title":"arXiv:1811.03617"},{"key":"e_1_3_1_28_2","article-title":"Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding","author":"Han Song","year":"2019","unstructured":"Song Han, Huizi Mao, and William J. Dally. 2019. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv:1510.00149. Retrieved from https:\/\/arxiv.org\/abs\/1510.00149.","journal-title":"arXiv:1510.00149"},{"key":"e_1_3_1_29_2","article-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size","author":"Iandola Forrest N.","year":"2016","unstructured":"Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv:1602.07360. Retrieved from https:\/\/arxiv.org\/abs\/1602.07360.","journal-title":"arXiv:1602.07360"},{"key":"e_1_3_1_30_2","article-title":"Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1","author":"Courbariaux Matthieu","year":"2016","unstructured":"Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv:1602.02830. Retrieved from https:\/\/arxiv.org\/abs\/1602.02830.","journal-title":"arXiv:1602.02830"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2866051"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"e_1_3_1_33_2","article-title":"S2-BNN: Bridging the gap between self-supervised real and 1-bit neural networks via guided distribution calibration","author":"Shen Zhiqiang","year":"2021","unstructured":"Zhiqiang Shen, Zechun Liu, Jie Qin, Lei Huang, Kwang-Ting Cheng, and Marios Savvides. 2021. S2-BNN: Bridging the gap between self-supervised real and 1-bit neural networks via guided distribution calibration. arXiv:2102.08946. Retrieved from https:\/\/arxiv.org\/abs\/2102.08946.","journal-title":"arXiv:2102.08946"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58568-6_9"},{"key":"e_1_3_1_35_2","article-title":"Training binary neural networks with real-to-binary convolutions","author":"Martinez Brais","year":"2020","unstructured":"Brais Martinez, Jing Yang, Adrian Bulat, and Georgios Tzimiropoulos. 2020. Training binary neural networks with real-to-binary convolutions. arXiv:2003.11535. Retrieved from https:\/\/arxiv.org\/abs\/2003.11535.","journal-title":"arXiv:2003.11535"},{"key":"e_1_3_1_36_2","article-title":"Bats: Binary architecture search","author":"Bulat Adrian","year":"2020","unstructured":"Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos. 2020. Bats: Binary architecture search. arXiv:2003.01711. Retrieved from https:\/\/arxiv.org\/abs\/2003.01711.","journal-title":"arXiv:2003.01711"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157557"},{"key":"e_1_3_1_38_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Alizadeh Milad","year":"2018","unstructured":"Milad Alizadeh, Javier Fern\u00e1ndez-Marqu\u00e9s, Nicholas D. Lane, and Yarin Gal. 2018. An empirical study of binary neural networks\u2019 optimisation. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_1_39_2","article-title":"Larq compute engine: Design, benchmark and deploy state-of-the-art binarized neural networks","volume":"3","author":"Bannink Tom","year":"2021","unstructured":"Tom Bannink, Adam Hillier, Lukas Geiger, Tim de Bruin, Leon Overweel, Jelmer Neeven, and Koen Helwegen. 2021. Larq compute engine: Design, benchmark and deploy state-of-the-art binarized neural networks. Proc. Mach. Learn. Syst. 3 (2021), 680\u2013695.","journal-title":"Proc. Mach. Learn. Syst."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.456"},{"key":"e_1_3_1_41_2","unstructured":"Xiaofan Lin Cong Zhao and Wei Pan. 2017. Towards accurate binary convolutional neural network. Advances in Neural Information Processing Systems 30 (2017)."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00066"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01156"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSI.2021.3064033"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCWC47524.2020.9031131"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00497"},{"key":"e_1_3_1_47_2","article-title":"Energy efficient hadamard neural networks","author":"Deveci T. Ceren","year":"2018","unstructured":"T. Ceren Deveci, Serdar Cakir, and A. Enis Cetin. 2018. Energy efficient hadamard neural networks. arXiv:1805.05421. Retrieved from https:\/\/arxiv.org\/abs\/1805.05421.","journal-title":"arXiv:1805.05421"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/76.260200"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.2307\/2387224"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1976.1674569"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"e_1_3_1_53_2","first-page":"535","volume-title":"Computers in Cardiology 1999","author":"Agante P. M.","year":"1999","unstructured":"P. M. Agante and J. P. Marques De S\u00e1. 1999. ECG noise filtering using wavelets with soft-thresholding methods. In Computers in Cardiology 1999, Vol. 26 (Cat. No. 99CH37004). IEEE, 535\u2013538."},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/18.382009"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2000.838192"},{"key":"e_1_3_1_56_2","article-title":"FNet: Mixing tokens with fourier transforms","author":"Lee-Thorp James","year":"2021","unstructured":"James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, and Santiago Ontanon. 2021. FNet: Mixing tokens with fourier transforms. arXiv:2105.03824. Retrieved from https:\/\/arxiv.org\/abs\/2105.03824.","journal-title":"arXiv:2105.03824"},{"key":"e_1_3_1_57_2","unstructured":"Mobilenet in tensorflow\u2019s official github. Retrieved March 1 2021 https:\/\/github.com\/tensorflow\/models\/tree\/master\/research\/slim\/nets\/mobilenet."},{"key":"e_1_3_1_58_2","unstructured":"Transfer learning and fine-tuning. Retrieved March 1 2021 from https:\/\/www.tensorflow.org\/tutorials\/images\/transfer_learning."},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00140"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_3_1_61_2","unstructured":"https:\/\/github.com\/phy710\/Block-Walsh-Hadamard-Transform-Layer-Speed-Test Block walsh-hadamard transform layer speed test code"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3510026","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3510026","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:24Z","timestamp":1750191144000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3510026"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,18]]},"references-count":60,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,11,30]]}},"alternative-id":["10.1145\/3510026"],"URL":"https:\/\/doi.org\/10.1145\/3510026","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,18]]},"assertion":[{"value":"2021-07-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-30","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}