{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T16:32:57Z","timestamp":1782232377677,"version":"3.54.5"},"reference-count":31,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,23]],"date-time":"2021-08-23T00:00:00Z","timestamp":1629676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002261","name":"Russian Foundation for Basic Research","doi-asserted-by":"publisher","award":["19-29-09085"],"award-info":[{"award-number":["19-29-09085"]}],"id":[{"id":"10.13039\/501100002261","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002261","name":"Russian Foundation for Basic Research","doi-asserted-by":"publisher","award":["19-31-90172"],"award-info":[{"award-number":["19-31-90172"]}],"id":[{"id":"10.13039\/501100002261","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>A lot of deep learning applications are desired to be run on mobile devices. Both accuracy and inference time are meaningful for a lot of them. While the number of FLOPs is usually used as a proxy for neural network latency, it may not be the best choice. In order to obtain a better approximation of latency, the research community uses lookup tables of all possible layers for the calculation of the inference on a mobile CPU. It requires only a small number of experiments. Unfortunately, on a mobile GPU, this method is not applicable in a straightforward way and shows low precision. In this work, we consider latency approximation on a mobile GPU as a data- and hardware-specific problem. Our main goal is to construct a convenient Latency Estimation Tool for Investigation (LETI) of neural network inference and building robust and accurate latency prediction models for each specific task. To achieve this goal, we make tools that provide a convenient way to conduct massive experiments on different target devices focusing on a mobile GPU. After evaluation of the dataset, one can train the regression model on experimental data and use it for future latency prediction and analysis. We experimentally demonstrate the applicability of such an approach on a subset of the popular NAS-Benchmark 101 dataset for two different mobile GPU.<\/jats:p>","DOI":"10.3390\/computers10080104","type":"journal-article","created":{"date-parts":[[2021,8,23]],"date-time":"2021-08-23T04:22:20Z","timestamp":1629692540000},"page":"104","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Latency Estimation Tool and Investigation of Neural Networks Inference on Mobile GPU"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2817-8420","authenticated-orcid":false,"given":"Evgeny","family":"Ponomarev","sequence":"first","affiliation":[{"name":"Skolkovo Institute of Science and Technology, 143026 Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8000-8595","authenticated-orcid":false,"given":"Sergey","family":"Matveev","sequence":"additional","affiliation":[{"name":"Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, 119991 Moscow, Russia"},{"name":"Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences (RAS), 119333 Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2071-2163","authenticated-orcid":false,"given":"Ivan","family":"Oseledets","sequence":"additional","affiliation":[{"name":"Skolkovo Institute of Science and Technology, 143026 Moscow, Russia"},{"name":"Marchuk Institute of Numerical Mathematics, Russian Academy of Sciences (RAS), 119333 Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Valery","family":"Glukhov","sequence":"additional","affiliation":[{"name":"Noah\u2019s Ark Lab., Huawei Technologies, 121614 Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1007\/978-3-319-46493-0_38","article-title":"Identity mappings in deep residual networks","volume":"9908","author":"He","year":"2016","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"Microsoft COCO: Common objects in context","volume":"8693","author":"Lin","year":"2014","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wani, M. (2020). Advances in Deep Learning, Springer.","DOI":"10.1007\/978-981-13-6794-6"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18\u201323). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA.","DOI":"10.1109\/CVPR.2018.00716"},{"key":"ref_6","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_7","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). TensorFlow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI, Savannah, GA, USA."},{"key":"ref_8","unstructured":"Lee, J., Chirkov, N., Ignasheva, E., Pisarchyk, Y., Shieh, M., Riccardi, F., Sarokin, R., Kulik, A., and Grundmann, M. (2019). On-device neural net inference with mobile gpus. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18\u201323). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_10","unstructured":"Ying, C., Klein, A., Christiansen, E., Real, E., Murphy, K., and Hutter, F. (2019, January 9\u201315). NAS-bench-101: Towards reproducible neural architecture search. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_11","unstructured":"Dong, X., and Yang, Y. (May, January 26). Nas-bench-201: Extending the scope of reproducible neural architecture search. Proceedings of the International Conference on Learning Representations, Online."},{"key":"ref_12","unstructured":"Krizhevsky, A., Nair, V., and Hinton, G. (2021, August 19). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_13","unstructured":"Liu, H., Simonyan, K., and Yang, Y. (2019, January 6\u20139). DARTS: Differentiable architecture search. Proceedings of the International Conference on Learning Representations, New Orleans, LO, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, X., Zhao, K., and Chu, X. (2021). AutoML: A Survey of the State-of-the-Art. Knowledge-Based Systems, Elsevier.","DOI":"10.1016\/j.knosys.2020.106622"},{"key":"ref_15","unstructured":"(2021, August 19). Google AI Blog: EfficientNet-EdgeTPU: Creating Accelerator-Optimized Neural Networks with AutoML. Available online: https:\/\/ai.googleblog.com\/2019\/08\/efficientnet-edgetpu-creating.html."},{"key":"ref_16","unstructured":"Cai, H., Zhu, L., and Han, S. (2019, January 6\u20139). ProxylessNAS: Direct neural architecture search on target task and hardware. Proceedings of the International Conference on Learning Representations, New Orleans, LO, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dai, X., Zhang, P., Wu, B., Yin, H., Sun, F., Wang, Y., Dukhan, M., Hu, Y., Wu, Y., and Jia, Y. (2019, January 16\u201320). Chamnet: Towards efficient network design through platform-aware model adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01166"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., and Keutzer, K. (2019, January 16\u201320). FBnet: Hardware-aware efficient convnet design via differentiable neural architecture search. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01099"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chu, X., Zhang, B., and Xu, R. (2020, January 4\u20138). MOGA: Searching beyond mobilenet v3. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054428"},{"key":"ref_20","unstructured":"Dudziak, \u0141., Chau, T., Abdelfattah, M.S., Lee, R., Kim, H., and Lane, N.D. (2020, January 6\u201312). BRP-NAS: Prediction-based NAS using GCNs. Proceedings of the Conference on Neural Information Processing Systems, Online."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ignatov, A., Timofte, R., Chou, W., Wang, K., Wu, M., Hartley, T., and Van Gool, L. (2019). AI Benchmark: Running deep neural networks on android smartphones. Lect. Notes Comput. Sci.","DOI":"10.1007\/978-3-030-11021-5_19"},{"key":"ref_22","unstructured":"Rapin, J., and Teytaud, O. (2021, August 19). Nevergrad\u2014A Gradient-Free Optimization Platform. Available online: https:\/\/github.com\/facebookresearch\/nevergrad."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Martin","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A scalable tree boosting system. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_25","unstructured":"Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2017). CatBoost: Unbiased boosting with categorical features. arXiv."},{"key":"ref_26","unstructured":"Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. (2021, August 19). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Technical Report, Available online: https:\/\/dl.acm.org\/doi\/pdf\/10.5555\/3294996.3295074."},{"key":"ref_27","unstructured":"(2021, August 19). Caffe2. Available online: https:\/\/caffe2.ai\/."},{"key":"ref_28","unstructured":"(2021, August 19). Pytorch Mobile. Available online: https:\/\/pytorch.org\/mobile\/home\/."},{"key":"ref_29","unstructured":"Luo, C., He, X., Zhan, J., Wang, L., Gao, W., and Dai, J. (2020). Comparison and Benchmarking of AI Models and Frameworks on Mobile Devices. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_31","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-Supervised Classification with Graph Convolutional Networks. Proceedings of the International Conference on Learning Representations, Toulon, France."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/8\/104\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:49:27Z","timestamp":1760165367000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/8\/104"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,23]]},"references-count":31,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["computers10080104"],"URL":"https:\/\/doi.org\/10.3390\/computers10080104","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,23]]}}}