{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T15:32:11Z","timestamp":1773588731354,"version":"3.50.1"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"European Social Fund via IT Academy program"},{"name":"SERB, India","award":["CRG\/2021\/003888"],"award-info":[{"award-number":["CRG\/2021\/003888"]}]},{"name":"UoH-IoE by MHRD, India","award":["F11\/9\/2019-U3(A)"],"award-info":[{"award-number":["F11\/9\/2019-U3(A)"]}]},{"name":"Telia, Estonia for financing the hardware setup"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2026,3,31]]},"abstract":"<jats:p>The rise of Internet of Things (IoT) applications has led to massive data generation. However, dealing with such massive data is challenging. Nowadays, data pipelines are popular mechanisms used to properly deal with data operations at scale in the IoT continuum. Serverless Data Pipelines (SDP) is one such approach to performing event-driven data analysis on data streams. Data pipelines are composed of many components, and scaling the entire pipeline without leaving any bottlenecks is challenging. This study aims to assess the performance of scaling mechanisms in handling stochastic workloads efficiently and understanding critical resource utilization in fog environments. We applied workload-based techniques (Request per Second, Queue Length, Message Rate) and resource-based scaling (CPU) on SDP components of two IoT applications: Aeneas (long-running functions) and PuhatuMonitoring (short-running functions). Using Azure serverless workload patterns, we compared scaling approaches in real-time fog environments, evaluating QoS metrics like processing time and CPU utilization. Our analysis of suitability, using the weighted average scoring method on two QoS metrics, revealed that for compute-intensive tasks, the resource-based scaling approach works effectively for jump, steady, spike, and fluctuation workloads. For short execution time tasks, workload-based scaling suits all four workloads.<\/jats:p>","DOI":"10.1145\/3747186","type":"journal-article","created":{"date-parts":[[2025,7,3]],"date-time":"2025-07-03T11:45:46Z","timestamp":1751543146000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Scaling Approaches for Serverless Data Pipelines in Edge and Fog Computing Environments: A Performance Evaluation"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7294-2484","authenticated-orcid":false,"given":"Shivananda","family":"Poojara","sequence":"first","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Tartu, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9452-8230","authenticated-orcid":false,"given":"Pelle","family":"Jakovits","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Tartu, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9754-6496","authenticated-orcid":false,"given":"Rajkumar","family":"Buyya","sequence":"additional","affiliation":[{"name":"The Cloud Computing and Distributed Systems (CLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7600-7124","authenticated-orcid":false,"given":"Satish Narayana","family":"Srirama","sequence":"additional","affiliation":[{"name":"School of Computer and Information Sciences, University of Hyderabad, Hyderabad, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,10]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/PerCom45495.2020.9127385"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA51224.2020.00014"},{"key":"e_1_3_2_4_2","first-page":"11","volume-title":"Journal of Innovation in Information Technology","author":"Chatti S.","year":"2019","unstructured":"S. Chatti and Spark Using. 2019. Kafka and NIFI for future generation of ETL in IT industry. Journal of Innovation in Information Technology 3 (2019), 11\u201314."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-69035-3_28"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICII.2018.00020"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-10-5026-8_1"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2021.12.012"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCC.2019.00018"},{"key":"e_1_3_2_10_2","first-page":"107","volume-title":"Predictive Analytics in Cloud, Fog, and Edge Computing: Perspectives and Practices of Blockchain, IoT, and 5G","author":"Poojara S.","year":"2022","unstructured":"S. Poojara, C. Dehury, P. Jakovits, and S. Srirama. 2022. Serverless data pipelines for IoT data analytics: A cloud vendors perspective and solutions. In Predictive Analytics in Cloud, Fog, and Edge Computing: Perspectives and Practices of Blockchain, IoT, and 5G. Hiren Kumar Thakkar, Chinmaya Kumar Dehury, Prasan Kumar Sahoo, and Bharadwaj Veeravalli (Eds.), Springer, 107\u2013132."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3153471"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2021.103604"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.measen.2023.100699"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2946426"},{"key":"e_1_3_2_15_2","unstructured":"J. Scheuner S. Eismann S. Talluri E. Van Eyk C. Abad P. Leitner and A. Iosup. 2022. Let\u2019s trace it: Fine-grained serverless benchmarking using synchronous and asynchronous orchestrated applications. arXiv:2205.07696. Retrieved from https:\/\/arxiv.org\/abs\/2205.07696"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2021.06.004"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/PDSW49588.2019.00005"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.3243"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid54584.2022.00027"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/BlackSeaCom54372.2022.9858271"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/PerComWorkshops53856.2022.9767437"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2021.102461"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/SERVICES.2019.00057"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366623.3368139"},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"Z. Zhou C. Zhang L. Ma J. Gu H. Qian Q. Wen L. Sun P. Li and Z. Tang. 2023. AHPA: Adaptive horizontal pod autoscaling systems on Alibaba cloud container service for Kubernetes. arXiv:2303.03640. Retrieved from https:\/\/arxiv.org\/abs\/2303.03640","DOI":"10.1609\/aaai.v37i13.26852"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Q. Trieu B. Javadi J. Basilakis and A. Toosi. 2022. Performance evaluation of serverless edge computing for machine learning applications. arXiv:2210.10331. Retrieved from https:\/\/arxiv.org\/abs\/2210.10331","DOI":"10.1109\/UCC56403.2022.00025"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3184407.3184430"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCC.2022.3169619"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510412"},{"key":"e_1_3_2_30_2","first-page":"3995","volume-title":"IEEE Transactions on Network and Service Management","volume":"19","author":"Xu M.","year":"2022","unstructured":"M. Xu, C. Song, S. Ilager, S. Gill, J. Zhao, K. Ye, and C. Xu. 2022. CoScal: Multi-faceted scaling of microservices with reinforcement learning. IEEE Transactions on Network and Service Management 19, 4 (2022), 3995\u20134009."},{"key":"e_1_3_2_31_2","first-page":"2968","article-title":"Microservice deployment in edge computing based on deep Q learning","volume":"33","author":"Lv W.","year":"2022","unstructured":"W. Lv, Q. Wang, P. Yang, Y. Ding, B. Yi, Z. Wang, and C. Lin. 2022. Microservice deployment in edge computing based on deep Q learning. IEEE Transactions on Parallel and Distributed Systems 33 (2011), 2968\u20132978.","journal-title":"IEEE Transactions on Parallel and Distributed Systems"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2964405"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid51090.2021.00098"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318216.3363299"},{"key":"e_1_3_2_35_2","first-page":"21","volume-title":"A Quantitative Measure of Fairness and Discrimination","author":"Jain R.","year":"1984","unstructured":"R. Jain, D. Chiu, and W. Hawe. 1984. A Quantitative Measure of Fairness and Discrimination. Eastern Research Laboratory, Digital Equipment Corporation, Hudson, MA, 21."},{"key":"e_1_3_2_36_2","unstructured":"M. Shahrad R. Fonseca I. Goiri G. Chaudhry P. Batum J. Cooke E. Laureano C. Tresness M. Russinovich and R. Bianchini. 2020. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. arXiv:2003.03423. Retrieved from https:\/\/arxiv.org\/abs\/2003.03423"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM41043.2020.9155363"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3464298.3493398"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2021.3123959"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-024-04870-0"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2024.100650"},{"key":"e_1_3_2_42_2","unstructured":"P. Benedetti M. Femminella and G. Reali. Management of autoscaling serverless functions in edge computing via Q-learning. SSRN 5116838. Retrieved from https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=5116838"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3686253"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3736726"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3747186","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T14:10:47Z","timestamp":1773583847000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3747186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,10]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,3,31]]}},"alternative-id":["10.1145\/3747186"],"URL":"https:\/\/doi.org\/10.1145\/3747186","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"value":"1556-4665","type":"print"},{"value":"1556-4703","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,10]]},"assertion":[{"value":"2024-04-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}