{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T03:23:11Z","timestamp":1776309791933,"version":"3.50.1"},"reference-count":52,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,2,20]],"date-time":"2020-02-20T00:00:00Z","timestamp":1582156800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ICON RADIANCE","award":["HBC.2017.0629"],"award-info":[{"award-number":["HBC.2017.0629"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The application of artificial intelligence enhances the ability of sensor and networking technologies to realize smart systems that sense, monitor and automatically control our everyday environments. Intelligent systems and applications often automate decisions based on the outcome of certain machine learning models. They collaborate at an ever increasing scale, ranging from smart homes and smart factories to smart cities. The best performing machine learning model, its architecture and parameters for a given task are ideally automatically determined through a hyperparameter tuning process. At the same time, edge computing is an emerging distributed computing paradigm that aims to bring computation and data storage closer to the location where they are needed to save network bandwidth or reduce the latency of requests. The challenge we address in this work is that hyperparameter tuning does not take into consideration resource trade-offs when selecting the best model for deployment in smart environments. The most accurate model might be prohibitively expensive to computationally evaluate on a resource constrained node at the edge of the network. We propose a multi-objective optimization solution to find acceptable trade-offs between model accuracy and resource consumption to enable the deployment of machine learning models in resource constrained smart environments. We demonstrate the feasibility of our approach by means of an anomaly detection use case. Additionally, we evaluate the extent that transfer learning techniques can be applied to reduce the amount of training required by reusing previous models, parameters and trade-off points from similar settings.<\/jats:p>","DOI":"10.3390\/s20041176","type":"journal-article","created":{"date-parts":[[2020,2,21]],"date-time":"2020-02-21T10:49:16Z","timestamp":1582282156000},"page":"1176","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["Resource Usage and Performance Trade-offs for Machine Learning Models in Smart Environments"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6279-4430","authenticated-orcid":false,"given":"Davy","family":"Preuveneers","sequence":"first","affiliation":[{"name":"imec\u2013DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7714-5238","authenticated-orcid":false,"given":"Ilias","family":"Tsingenopoulos","sequence":"additional","affiliation":[{"name":"imec\u2013DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium"}]},{"given":"Wouter","family":"Joosen","sequence":"additional","affiliation":[{"name":"imec\u2013DistriNet, KU Leuven, Celestijnenlaan 200A, B-3001 Heverlee, Belgium"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Borelli, E., Paolini, G., Antoniazzi, F., Barbiroli, M., Benassi, F., Chesani, F., Chiari, L., Fantini, M., Fuschini, F., and Galassi, A. (2019). HABITAT: An IoT Solution for Independent Elderly. Sensors, 19.","DOI":"10.3390\/s19051258"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"128325","DOI":"10.1109\/ACCESS.2019.2925082","article-title":"Comparative Analysis of Machine Learning Techniques for Predicting Air Quality in Smart Cities","volume":"7","author":"Ameer","year":"2019","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mauldin, T.R., Canby, M.E., Metsis, V., Ngu, A.H.H., and Rivera, C.C. (2018). SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning. Sensors, 18.","DOI":"10.3390\/s18103363"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Park, D., Kim, S., An, Y., and Jung, J. (2018). LiReD: A Light-Weight Real-Time Fault Detection System for Edge Computing Using LSTM Recurrent Neural Networks. Sensors, 18.","DOI":"10.3390\/s18072110"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MIE.2016.2615575","article-title":"Intelligent buildings of the future: Cyberaware, deep learning powered, and human interacting","volume":"10","author":"Manic","year":"2016","journal-title":"IEEE Ind. Electron. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2923","DOI":"10.1109\/COMST.2018.2844341","article-title":"Deep learning for IoT big data and streaming analytics: A survey","volume":"20","author":"Mohammadi","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","unstructured":"Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., and Cheng-Yue, R. (2015). An empirical evaluation of deep learning on highway driving. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pham, T., Tran, T., Phung, D., and Venkatesh, S. (2016). Deepcare: A deep dynamic memory model for predictive medicine. Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer.","DOI":"10.1007\/978-3-319-31750-2_3"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2018.02.010","article-title":"Deep learning for sensor-based activity recognition: A survey","volume":"119","author":"Wang","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ribeiro, M., Grolinger, K., and Capretz, M.A. (2015, January 9\u201311). Mlaas: Machine learning as a service. Proceedings of the 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA.","DOI":"10.1109\/ICMLA.2015.152"},{"key":"ref_11","unstructured":"Li, L.E., Chen, E., Hermann, J., Zhang, P., and Wang, L. (2017, January 24\u201325). Scaling machine learning as a service. Proceedings of the International Conference on Predictive Applications and APIs, Boston, MA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1162\/089976600300015187","article-title":"Gradient-based optimization of hyperparameters","volume":"12","author":"Bengio","year":"2000","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.neucom.2004.11.022","article-title":"Evolutionary tuning of multiple SVM parameters","volume":"64","author":"Friedrichs","year":"2005","journal-title":"Neurocomputing"},{"key":"ref_14","unstructured":"Snoek, J., Larochelle, H., and Adams, R.P. (2012). Practical bayesian optimization of machine learning algorithms. Proceedings of the 25th International Conference on Neural Information Processing Systems\u2014Volume 2, Curran Associates Inc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/JIOT.2016.2579198","article-title":"Edge computing: Vision and challenges","volume":"3","author":"Shi","year":"2016","journal-title":"IEEE Internet Things J."},{"key":"ref_16","unstructured":"Kone\u010dn\u1ef3, J., McMahan, H.B., Yu, F.X., Richt\u00e1rik, P., Suresh, A.T., and Bacon, D. (2016). Federated learning: Strategies for improving communication efficiency. arXiv."},{"key":"ref_17","unstructured":"Kone\u010dn\u1ef3, J., McMahan, H.B., Ramage, D., and Richt\u00e1rik, P. (2016). Federated optimization: Distributed machine learning for on-device intelligence. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1145\/3298981","article-title":"Federated machine learning: Concept and applications","volume":"10","author":"Yang","year":"2019","journal-title":"TIST"},{"key":"ref_19","unstructured":"Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., and Ramage, D. (2018). Federated learning for mobile keyboard prediction. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hitaj, B., Ateniese, G., and P\u00e9rez-Cruz, F. (November, January 30). Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS \u201917), Dallas, TX, USA.","DOI":"10.1145\/3133956.3134012"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nasr, M., Shokri, R., and Houmansadr, A. (2019, January 20\u201322). Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP.2019.00065"},{"key":"ref_22","unstructured":"Ngatchou, P., Zarei, A., and El-Sharkawi, A. (2005, January 6\u201310). Pareto multi objective optimization. Proceedings of the 13th International Conference on, Intelligent Systems Application to Power Systems, Arlington, VA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"15:1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly Detection: A Survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s40537-016-0043-6","article-title":"A survey of transfer learning","volume":"3","author":"Weiss","year":"2016","journal-title":"J. Big Data"},{"key":"ref_26","unstructured":"Yogatama, D., and Mann, G. (2014, January 22\u201325). Efficient transfer learning method for automatic hyperparameter tuning. Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, Reykjavik, Iceland."},{"key":"ref_27","unstructured":"Perrone, V., Jenatton, R., Seeger, M., and Archambeau, C. (2018). Scalable Hyperparameter Transfer Learning. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 3\u20138 December, Curran Associates Inc."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kotthoff, L., Thornton, C., Hoos, H.H., Hutter, F., and Leyton-Brown, K. (2019). Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA. Automated Machine Learning: Methods, Systems, Challenges, Springer International Publishing.","DOI":"10.1007\/978-3-030-05318-5_4"},{"key":"ref_29","unstructured":"Frank, E., Hall, M.A., and Witten, I.H. (2016). The WEKA Workbench, Morgan Kaufmann."},{"key":"ref_30","unstructured":"Hutter, F., Hoos, H.H., and Leyton-Brown, K. (2011, January 17\u201321). Sequential Model-Based Optimization for General Algorithm Configuration. Proceedings of the 5th International Conference on Learning and Intelligent Optimization, Rome, Italy. LION\u201905."},{"key":"ref_31","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). Efficient and Robust Automated Machine Learning. Advances in Neural Information Processing Systems 28, Curran Associates, Inc."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jin, H., Song, Q., and Hu, X. (2019, January 4\u20138). Auto-Keras: An Efficient Neural Architecture Search System. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330648"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., and Sculley, D. (2017). Google Vizier: A Service for Black-Box Optimization. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, 13\u201317 August 2017, Association for Computing Machinery. KDD \u201917.","DOI":"10.1145\/3097983.3098043"},{"key":"ref_34","unstructured":"Hsu, C.H., Chang, S.H., Liang, J.H., Chou, H.P., Liu, C.H., Chang, S.C., Pan, J.Y., Chen, Y.T., Wei, W., and Juan, D.C. (2018). Monas: Multi-objective neural architecture search using reinforcement learning. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dong, J., Cheng, A., Juan, D., Wei, W., and Sun, M. (2018, January 8\u201314). DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures. Proceedings of the 2018 European Conference on Computer Vision\u2014Part XI, Munich, Germany. ECCV 2018.","DOI":"10.1007\/978-3-030-01252-6_32"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Cheng, A.C., Dong, J.D., Hsu, C.H., Chang, S.H., Sun, M., Chang, S.C., Pan, J.Y., Chen, Y.T., Wei, W., and Juan, D.C. Searching toward Pareto-Optimal Device-Aware Neural Architectures. Proceedings of the International Conference on Computer-Aided Design, Marrakech, Morocco, 19\u201321 March 2018, Association for Computing Machinery. ICCAD \u201918.","DOI":"10.1145\/3240765.3243494"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tan, M., Chen, B., Pang, R., Vasudevan, V., and Le, Q.V. (2018). MnasNet: Platform-Aware Neural Architecture Search for Mobile. arXiv.","DOI":"10.1109\/CVPR.2019.00293"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1657","DOI":"10.1109\/COMST.2017.2705720","article-title":"On multi-access edge computing: A survey of the emerging 5G network edge cloud architecture and orchestration","volume":"19","author":"Taleb","year":"2017","journal-title":"IEEE Commun. Surv. Tutor. Tutor."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zanzi, L., Giust, F., and Sciancalepore, V. (2018, January 15\u201318). M2EC: A multi-tenant resource orchestration in multi-access edge computing systems. Proceedings of the 2018 IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain.","DOI":"10.1109\/WCNC.2018.8377292"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Baresi, L., Mendon\u00e7a, D.F., and Quattrocchi, G. (2019, January 28\u201331). PAPS: A Framework for Decentralized Self-management at the Edge. Proceedings of the Service-Oriented Computing\u201317th International Conference, ICSOC 2019, Toulouse, France.","DOI":"10.1007\/978-3-030-33702-5_39"},{"key":"ref_41","first-page":"826","article-title":"Auto-WEKA 2.0: Automatic Model Selection and Hyperparameter Optimization in WEKA","volume":"18","author":"Kotthoff","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","unstructured":"Laptev, N., Amizadeh, S., and Billawala, Y. (2020, January 17). A Benchmark Dataset for Time Series Anomaly Detection. Available online: https:\/\/research.yahoo.com\/news\/announcing-benchmark-dataset-time-series-anomaly-detection."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Huch, F., Golagha, M., Petrovska, A., and Krauss, A. (2018, January 20). Machine learning-based run-time anomaly detection in software systems: An industrial evaluation. Proceedings of the 2018 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation, MaLTeSQuE@SANER 2018, Campobasso, Italy.","DOI":"10.1109\/MALTESQUE.2018.8368453"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Sharafaldin, I., Lashkari, A.H., and Ghorbani, A.A. (2018, January 22\u201324). Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. Proceedings of the 4th International Conference on Information Systems Security and Privacy, ICISSP 2018, Funchal, Madeira, Portugal.","DOI":"10.5220\/0006639801080116"},{"key":"ref_45","unstructured":"Laptev, N. (2018, August 20). AnoGen: Deep Anomaly Generator. Available online: https:\/\/research.fb.com\/publications\/anogen-deep-anomaly-generator\/."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Kittler, J., and Roli, F. (2001). Combining One-Class Classifiers. Multiple Classifier Systems, Springer.","DOI":"10.1007\/3-540-48219-9"},{"key":"ref_47","unstructured":"Thomas, A., Gramfort, A., and Cl\u00e9men\u00e7on, S. (2016, January 24). Learning Hyperparameters for Unsupervised Anomaly Detection. Proceedings of the Anomaly Detection Workshop, ICML 2016, New York, NY, USA."},{"key":"ref_48","unstructured":"Baldi, P. (2011, January 2). Autoencoders, Unsupervised Learning and Deep Architectures. Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning Workshop\u2014Volume 27, Bellevue, WA, USA. UTLW\u201911."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zhou, C., and Paffenroth, R.C. (2017, January 13\u201317). Anomaly detection with robust deep autoencoders. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098052"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.jspi.2014.12.004","article-title":"On using the hypervolume indicator to compare Pareto fronts: Applications to multi-criteria optimal experimental design","volume":"160","author":"Cao","year":"2015","journal-title":"J. Stat. Plann. Inference"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"11441","DOI":"10.1073\/pnas.1604850113","article-title":"Convolutional networks for fast, energy-efficient neuromorphic computing","volume":"113","author":"Esser","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3289185","article-title":"[DL] A Survey of FPGA-Based Neural Network Inference Accelerators","volume":"12","author":"Guo","year":"2019","journal-title":"ACM Trans. Reconfigurable Technol. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1176\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:59:29Z","timestamp":1760173169000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1176"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,20]]},"references-count":52,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20041176"],"URL":"https:\/\/doi.org\/10.3390\/s20041176","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,20]]}}}