{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T14:27:23Z","timestamp":1785248843304,"version":"3.55.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,5,26]],"date-time":"2022-05-26T00:00:00Z","timestamp":1653523200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1909900"],"award-info":[{"award-number":["1909900"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Meas. Anal. Comput. Syst."],"published-print":{"date-parts":[[2022,5,26]]},"abstract":"<jats:p>The high efficiency of domain-specific hardware accelerators for machine learning (ML) has come fromspecialization, with the trade-off of less configurability\/ flexibility. There is growing interest in developingflexible ML accelerators to make them future-proof to the rapid evolution of Deep Neural Networks (DNNs). However, the notion of accelerator flexibility has always been used in an informal manner, restricting computer architects from conducting systematic apples-to-apples design-space exploration (DSE) across trillions of choices. In this work, we formally define accelerator flexibility and show how it can be integrated for DSE. % flows. Specifically, we capture DNN accelerator flexibility across four axes: %the map-space of DNN accelerator along four flexibility axes: tiling, ordering, parallelization, and array shape. We categorize existing accelerators into 16 classes based on their axes of flexibility support, and define a precise quantification of the degree of flexibility of an accelerator across each axis. We leverage these to develop a novel flexibility-aware DSE framework. %It respects the difference of accelerator flexibility classes and degree of flexibility support in different accelerators, creating unique map-spaces. %and forms a unique map space for exploration. % We demonstrate how this can be used to perform first-of-their-kind evaluations, including an isolation study to identify the individual impact of the flexibility axes. We demonstrate that adding flexibility features to a hypothetical DNN accelerator designed in 2014 improves runtime on future (i.e., present-day) DNNs by 11.8x geomean.<\/jats:p>","DOI":"10.1145\/3530907","type":"journal-article","created":{"date-parts":[[2022,6,6]],"date-time":"2022-06-06T17:16:18Z","timestamp":1654535778000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["A Formalism of DNN Accelerator Flexibility"],"prefix":"10.1145","volume":"6","author":[{"given":"Sheng-Chun","family":"Kao","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyoukjun","family":"Kwon","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Pellauer","sequence":"additional","affiliation":[{"name":"Nvidia, Westford, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angshuman","family":"Parashar","sequence":"additional","affiliation":[{"name":"Nvidia, Westford, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tushar","family":"Krishna","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, GA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,6,6]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"https:\/\/github.com\/maestro-project\/gamma.  https:\/\/github.com\/maestro-project\/gamma."},{"key":"e_1_2_1_2_1","unstructured":"https:\/\/https:\/\/github.com\/georgia-tech-synergy-lab\/maeri.  https:\/\/https:\/\/github.com\/georgia-tech-synergy-lab\/maeri."},{"key":"e_1_2_1_3_1","volume-title":"https:\/\/www.eda.ncsu.edu\/wiki\/FreePDK15:Contents","year":"2019","unstructured":"Freepdk15:contents. https:\/\/www.eda.ncsu.edu\/wiki\/FreePDK15:Contents , 2019 . Freepdk15:contents. https:\/\/www.eda.ncsu.edu\/wiki\/FreePDK15:Contents, 2019."},{"key":"e_1_2_1_4_1","unstructured":"Cuda c++ programming guide. 2021.  Cuda c++ programming guide. 2021."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/3195638.3195664"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA45697.2020.00081"},{"key":"e_1_2_1_7_1","volume-title":"Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures","author":"Bergstra James","year":"2013","unstructured":"James Bergstra , Daniel Yamins , and David Daniel Cox . Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures . 2013 . James Bergstra, Daniel Yamins, and David Daniel Cox. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. 2013."},{"key":"e_1_2_1_8_1","volume-title":"Marvel: A data-centric compiler for dnn operators on spatial accelerators. arXiv preprint arXiv:2002.07752","author":"Chatarasi Prasanth","year":"2020","unstructured":"Prasanth Chatarasi , Hyoukjun Kwon , Natesh Raina , Saurabh Malik , Vaisakh Haridas , Angshuman Parashar , Michael Pellauer , Tushar Krishna , and Vivek Sarkar . Marvel: A data-centric compiler for dnn operators on spatial accelerators. arXiv preprint arXiv:2002.07752 , 2020 . Prasanth Chatarasi, Hyoukjun Kwon, Natesh Raina, Saurabh Malik, Vaisakh Haridas, Angshuman Parashar, Michael Pellauer, Tushar Krishna, and Vivek Sarkar. Marvel: A data-centric compiler for dnn operators on spatial accelerators. arXiv preprint arXiv:2002.07752, 2020."},{"key":"e_1_2_1_9_1","first-page":"1691","volume-title":"International Conference on Machine Learning","author":"Chen Mark","year":"2020","unstructured":"Mark Chen , Alec Radford , Rewon Child , Jeffrey Wu , Heewoo Jun , David Luan , and Ilya Sutskever . Generative pretraining from pixels . In International Conference on Machine Learning , pages 1691 -- 1703 . PMLR, 2020 . Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International Conference on Machine Learning, pages 1691--1703. PMLR, 2020."},{"key":"e_1_2_1_10_1","first-page":"3389","volume-title":"Advances in Neural Information Processing Systems","author":"Chen Tianqi","year":"2018","unstructured":"Tianqi Chen , Lianmin Zheng , Eddie Yan , Ziheng Jiang , Thierry Moreau , Luis Ceze , Carlos Guestrin , and Arvind Krishnamurthy . Learning to optimize tensor programs . In Advances in Neural Information Processing Systems , pages 3389 -- 3400 , 2018 . Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. Learning to optimize tensor programs. In Advances in Neural Information Processing Systems, pages 3389--3400, 2018."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001177"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2016.2616357"},{"key":"e_1_2_1_13_1","volume-title":"Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices","author":"Chen Yu-Hsin","year":"2019","unstructured":"Yu-Hsin Chen , Tien-Ju Yang , Joel Emer , and Vivienne Sze . Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices . IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 9(2):292--308, 2019 . Yu-Hsin Chen, Tien-Ju Yang, Joel Emer, and Vivienne Sze. Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(2):292--308, 2019."},{"key":"e_1_2_1_14_1","volume-title":"Dmazerunner: Executing perfectly nested loops on dataflow accelerators. ACM Transactions on Embedded Computing Systems (TECS), 18(5s):1--27","author":"Dave Shail","year":"2019","unstructured":"Shail Dave , Youngbin Kim , Sasikanth Avancha , Kyoungwoo Lee , and Aviral Shrivastava . Dmazerunner: Executing perfectly nested loops on dataflow accelerators. ACM Transactions on Embedded Computing Systems (TECS), 18(5s):1--27 , 2019 . Shail Dave, Youngbin Kim, Sasikanth Avancha, Kyoungwoo Lee, and Aviral Shrivastava. Dmazerunner: Executing perfectly nested loops on dataflow accelerators. ACM Transactions on Embedded Computing Systems (TECS), 18(5s):1--27, 2019."},{"key":"e_1_2_1_15_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 2018 . Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2749469.2750389"},{"key":"e_1_2_1_17_1","volume-title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961","author":"Fedus William","year":"2021","unstructured":"William Fedus , Barret Zoph , and Noam Shazeer . Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961 , 2021 . William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304014"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO50266.2020.00062"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_2_1_22_1","unstructured":"Kartik Hegde Po-An Tsai Sitao Huang Vikas Chandra Angshuman Parashar and Christopher W Fletcher. Mind mappings: Enabling efficient algorithm-accelerator mapping space search extended abstract.  Kartik Hegde Po-An Tsai Sitao Huang Vikas Chandra Angshuman Parashar and Christopher W Fletcher. Mind mappings: Enabling efficient algorithm-accelerator mapping space search extended abstract."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2018.00062"},{"key":"e_1_2_1_24_1","volume-title":"ACM Meas. Anal. Comput. Syst.","volume":"6","author":"Huang Qijing","unstructured":"Qijing Huang , Minwoo Kang , Grace Dinh , Thomas Norell , Aravind Kalaiah , James Demmel , John Wawrzynek , and Yakun Sophia Shao . Cosa : Scheduling by constrained optimization for spatial accelerators. arXiv preprint Proc . ACM Meas. Anal. Comput. Syst. , Vol. 6 , No. 2, Article 41. Publication date: June 2022. 41: 22 Sheng-Chun Kao, Hyoukjun Kwon, Michael Pellauer, Angshuman Parashar, and Tushar Krishna arXiv:2105.01898, 2021. Qijing Huang, Minwoo Kang, Grace Dinh, Thomas Norell, Aravind Kalaiah, James Demmel, John Wawrzynek, and Yakun Sophia Shao. Cosa: Scheduling by constrained optimization for spatial accelerators. arXiv preprint Proc. ACM Meas. Anal. Comput. Syst., Vol. 6, No. 2, Article 41. Publication date: June 2022. 41:22 Sheng-Chun Kao, Hyoukjun Kwon, Michael Pellauer, Angshuman Parashar, and Tushar Krishna arXiv:2105.01898, 2021."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360307"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3400302.3415639"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_2_1_29_1","volume-title":"Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097--1105","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E Hinton . Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097--1105 , 2012 . Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097--1105, 2012."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2020.2985963"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3296957.3173176"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2017.29"},{"key":"e_1_2_1_33_1","unstructured":"Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G. Azzolini Dmytro Dzhulgakov Andrey Mallevich Ilia Cherniavskii Yinghai Lu Raghuraman Krishnamoorthi Ansha Yu Volodymyr Kondratenko Stephanie Pereira Xianjie Chen Wenlin Chen Vijay Rao Bill Jia Liang Xiong and Misha Smelyanskiy. Deep learning recommendation model for personalization and recommendation systems. CoRR abs\/1906.00091 2019.  Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G. Azzolini Dmytro Dzhulgakov Andrey Mallevich Ilia Cherniavskii Yinghai Lu Raghuraman Krishnamoorthi Ansha Yu Volodymyr Kondratenko Stephanie Pereira Xianjie Chen Wenlin Chen Vijay Rao Bill Jia Liang Xiong and Misha Smelyanskiy. Deep learning recommendation model for personalization and recommendation systems. CoRR abs\/1906.00091 2019."},{"key":"e_1_2_1_34_1","unstructured":"Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G Azzolini et al. Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091 2019.  Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G Azzolini et al. Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091 2019."},{"key":"e_1_2_1_35_1","unstructured":"NVIDIA. Nvdla deep learning accelerator. http:\/\/nvdla.org 2017.  NVIDIA. Nvdla deep learning accelerator. http:\/\/nvdla.org 2017."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2019.00042"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297858.3304025"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3195970.3196033"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA47549.2020.00015"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3352460.3358302"},{"key":"e_1_2_1_42_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014 . Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00293"},{"key":"e_1_2_1_44_1","volume-title":"Proc. ACM Meas. Anal. Comput. Syst.","volume":"6","author":"Winograd Shmuel","unstructured":"Shmuel Winograd . Arithmetic Complexity of Computations, chapter 7, pages 71--92 . Proc. ACM Meas. Anal. Comput. Syst. , Vol. 6 , No. 2, Article 41. Publication date: June 2022. A Formalism of DNN Accelerator Flexibility 41: 23 Shmuel Winograd. Arithmetic Complexity of Computations, chapter 7, pages 71--92. Proc. ACM Meas. Anal. Comput. Syst., Vol. 6, No. 2, Article 41. Publication date: June 2022. A Formalism of DNN Accelerator Flexibility 41:23"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3373376.3378514"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA52012.2021.00085"}],"container-title":["Proceedings of the ACM on Measurement and Analysis of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530907","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3530907","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3530907","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:27Z","timestamp":1750183767000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530907"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,26]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,5,26]]}},"alternative-id":["10.1145\/3530907"],"URL":"https:\/\/doi.org\/10.1145\/3530907","relation":{},"ISSN":["2476-1249"],"issn-type":[{"value":"2476-1249","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,26]]},"assertion":[{"value":"2022-06-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}