{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:31:46Z","timestamp":1750221106971,"version":"3.41.0"},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,12,31]],"date-time":"2018-12-31T00:00:00Z","timestamp":1546214400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Hessian LOEWE initiative within the Software - Factory 4.0 project"},{"name":"Graduate School of Excellence Computational Engineering"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2018,12,31]]},"abstract":"<jats:p>Training a deep neural network (DNN) involves selecting a set of hyperparameters that define the network topology and influence the accuracy of the resulting network. Often, the goal is to maximize prediction accuracy on a given dataset. However, non-functional requirements of the trained network -- such as inference speed, size, and energy consumption -- can be very important as well. In this article, we aim to automate the process of selecting an appropriate DNN topology that fulfills both functional and non-functional requirements of the application. Specifically, we focus on tuning two important hyperparameters, depth and width, which together define the shape of the resulting network and directly affect its accuracy, speed, size, and energy consumption. To reduce the time needed to search the design space, we train a fraction of DNNs and build a model to predict the performances of the remaining ones. We are able to produce tuned ResNets, which are up to 4.22 times faster than original depth-scaled ResNets on a batch of 128 images while matching their accuracy.<\/jats:p>","DOI":"10.1145\/3291053","type":"journal-article","created":{"date-parts":[[2019,1,8]],"date-time":"2019-01-08T15:53:12Z","timestamp":1546962792000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["The Art of Getting Deep Neural Networks in Shape"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1832-3492","authenticated-orcid":false,"given":"Rahim","family":"Mammadli","sequence":"first","affiliation":[{"name":"Technische Universit\u00e4t Darmstadt, Graduate School of Excellence Computational Engineering, Hessen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felix","family":"Wolf","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Darmstadt, Department of Computer Science, Hessen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali","family":"Jannesari","sequence":"additional","affiliation":[{"name":"Iowa State University, Department of Computer Science, Ames, IA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,1,8]]},"reference":[{"volume-title":"Do deep nets really need to be deep? In Advances in Neural Information Processing Systems 27","author":"Ba Jimmy","key":"e_1_2_1_1_1"},{"key":"e_1_2_1_2_1","unstructured":"Bowen Baker Otkrist Gupta Nikhil Naik and Ramesh Raskar. 2016. Designing neural network architectures using reinforcement learning. (2016). arxiv:1611.02167  Bowen Baker Otkrist Gupta Nikhil Naik and Ramesh Raskar. 2016. Designing neural network architectures using reinforcement learning. (2016). arxiv:1611.02167"},{"key":"e_1_2_1_3_1","first-page":"281","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","journal-title":"Journal of Machine Learning Research 13"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588768.2576783"},{"key":"e_1_2_1_5_1","unstructured":"Patryk Chrabaszcz Ilya Loshchilov and Frank Hutter. 2017. A downsampled variant of ImageNet as an alternative to the CIFAR datasets. (2017). arxiv:1707.08819  Patryk Chrabaszcz Ilya Loshchilov and Frank Hutter. 2017. A downsampled variant of ImageNet as an alternative to the CIFAR datasets. (2017). arxiv:1707.08819"},{"volume-title":"Advances in Neural Information Processing Systems 28","author":"Courbariaux Matthieu","key":"e_1_2_1_6_1"},{"volume-title":"Dally","year":"2015","author":"Han Song","key":"e_1_2_1_7_1"},{"key":"e_1_2_1_8_1","unstructured":"Sayed Hadi Hashemi Shadi A. Noghabi and William Gropp. 2016. Performance modeling of distributed deep neural networks. (2016). arxiv:1612.00521  Sayed Hadi Hashemi Shadi A. Noghabi and William Gropp. 2016. Performance modeling of distributed deep neural networks. (2016). arxiv:1612.00521"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38"},{"volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE. 2261--2269","author":"Huang Gao","key":"e_1_2_1_11_1"},{"key":"e_1_2_1_12_1","unstructured":"Forrest N. Iandola Song Han Matthew W. Moskewicz Khalid Ashraf William J. Dally and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and &lt;0.5MB model size. (2016) 1--13. arxiv:1602.07360  Forrest N. Iandola Song Han Matthew W. Moskewicz Khalid Ashraf William J. Dally and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and &lt;0.5MB model size. (2016) 1--13. arxiv:1602.07360"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"volume-title":"Learning multiple layers of features from tiny images. Science Department","year":"2009","author":"Krizhevsky Alex","key":"e_1_2_1_14_1"},{"volume":"1","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems -","author":"Krizhevsky Alex","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40047-6_80"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273556"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"volume-title":"Retrieved","year":"2018","author":"LeCun Yann","key":"e_1_2_1_19_1"},{"volume-title":"Elliot Meyerson, Aditya Rawal, Daniel Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat.","year":"2017","author":"Miikkulainen Risto","key":"e_1_2_1_20_1"},{"volume-title":"Deeparchitect: Automatically designing and training deep architectures.","year":"2017","author":"Negrinho Renato","key":"e_1_2_1_21_1"},{"volume-title":"Proceedings of the International Conference on Learning Representations.","year":"2017","author":"Qi Hang","key":"e_1_2_1_22_1"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_32"},{"volume-title":"Antoine Chassang, Carlo Gatta, and Yoshua Bengio.","year":"2014","author":"Romero Adriana","key":"e_1_2_1_24_1"},{"volume-title":"Pytorch: Tensors and dynamic neural networks in python with strong GPU acceleration.","year":"2017","author":"Team Pytorch Core","key":"e_1_2_1_25_1"},{"volume-title":"Ozlem Aslan, Shengjie Wang, Rich Caruana, Abdelrahman Mohamed, Matthai Philipose, and Matt Richardson.","year":"2016","author":"Urban Gregor","key":"e_1_2_1_26_1"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Tien-Ju Yang Yu-Hsin Chen and Vivienne Sze. 2017. Designing energy-efficient convolutional neural networks using energy-aware pruning. (2017). arxiv:1611.05128  Tien-Ju Yang Yu-Hsin Chen and Vivienne Sze. 2017. Designing energy-efficient convolutional neural networks using energy-aware pruning. (2017). arxiv:1611.05128","DOI":"10.1109\/CVPR.2017.643"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. (2016). arxiv:1605.07146  Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. (2016). arxiv:1605.07146","DOI":"10.5244\/C.30.87"},{"key":"e_1_2_1_29_1","unstructured":"Shuchang Zhou Yuxin Wu Zekun Ni Xinyu Zhou He Wen and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. (2016). arxiv:1606.06160  Shuchang Zhou Yuxin Wu Zekun Ni Xinyu Zhou He Wen and Yuheng Zou. 2016. DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients. (2016). arxiv:1606.06160"},{"volume-title":"Le","year":"2016","author":"Zoph Barret","key":"e_1_2_1_30_1"},{"volume-title":"Le","year":"2017","author":"Zoph Barret","key":"e_1_2_1_31_1"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3291053","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3291053","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:01:52Z","timestamp":1750208512000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3291053"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,31]]},"references-count":31,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,12,31]]}},"alternative-id":["10.1145\/3291053"],"URL":"https:\/\/doi.org\/10.1145\/3291053","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"type":"print","value":"1544-3566"},{"type":"electronic","value":"1544-3973"}],"subject":[],"published":{"date-parts":[[2018,12,31]]},"assertion":[{"value":"2018-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}