{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T04:59:06Z","timestamp":1770353946325,"version":"3.49.0"},"reference-count":33,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T00:00:00Z","timestamp":1770076800000},"content-version":"vor","delay-in-days":2,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T00:00:00Z","timestamp":1769904000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100004901","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de Minas Gerais","doi-asserted-by":"publisher","award":["APQ\u201001400\u201014"],"award-info":[{"award-number":["APQ\u201001400\u201014"]}],"id":[{"id":"10.13039\/501100004901","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004901","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de Minas Gerais","doi-asserted-by":"publisher","award":["APQ\u201000202\u201024"],"award-info":[{"award-number":["APQ\u201000202\u201024"]}],"id":[{"id":"10.13039\/501100004901","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["573871\/2008\u20106"],"award-info":[{"award-number":["573871\/2008\u20106"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Concurrency and Computation"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Although energy has become a major concern in data processing systems, it is usually hard to get a deep understanding of how performance and energy consumption relate to each other when planning how to configure a computing environment to execute a specific data\u2010oriented workload. In this paper, we propose a multi\u2010layered methodology to analyze the energy consumption of big data workloads executed using Apache Spark in virtualized cloud environments. The approach is structured into three layers: Resource provisioning, system\u2010level resource utilization, and application\u2010level resource utilization. Using direct energy measurements using a Power Distribution Unit (PDU) and detailed system monitoring, the study investigates how infrastructure choices and workload characteristics influence energy consumption. Results show that optimal virtual machine configurations depend on workload type and input size; while provisioning decisions affect energy consumption, system\u2010level metrics such as CPU utilization and disk I\/O offer a deeper understanding of the final performance versus energy consumption results. By applying our methodology, our results reveal the impact of task distribution and resource under\u2010utilization on overall energy efficiency. The findings demonstrate that energy optimization in big data environments requires a comprehensive understanding of factors across infrastructure, system, and application layers. The proposed methodology serves as a practical guide for energy\u2010aware design and decision\u2010making in cloud\u2010based data processing systems.<\/jats:p>","DOI":"10.1002\/cpe.70565","type":"journal-article","created":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T06:29:51Z","timestamp":1770186591000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Multi\u2010Layered Analysis of Energy Consumption in Spark"],"prefix":"10.1002","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-2378-1401","authenticated-orcid":false,"given":"Nestor D. O.","family":"Volpini","sequence":"first","affiliation":[{"name":"Departamento de Eletroelet\u00f4nica e Computa\u00e7\u00e3o CEFET\u2010MG  Belo Horizonte Brazil"},{"name":"Departamento de Ci\u00eancia da Computa\u00e7\u00e3o UFMG  Belo Horizonte Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8324-8487","authenticated-orcid":false,"given":"Vin\u00edcius","family":"Dias","sequence":"additional","affiliation":[{"name":"Departamento de Ci\u00eancia da Computa\u00e7\u00e3o UFLA  Lavras Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dorgival","family":"Guedes","sequence":"additional","affiliation":[{"name":"Departamento de Ci\u00eancia da Computa\u00e7\u00e3o UFMG  Belo Horizonte Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2026,2,3]]},"reference":[{"key":"e_1_2_14_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1721654.1721672"},{"key":"e_1_2_14_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2014.08.003"},{"key":"e_1_2_14_4_1","unstructured":"J.WhitneyandP.Delforge \u201cScaling up Energy Efficiency Across the Data Center Industry: Evaluating Key Drivers and Barriers \u201d(2014) https:\/\/www.nrdc.org\/energy\/files\/data\u2010center\u2010efficiency\u2010assessment\u2010IP.pdf."},{"key":"e_1_2_14_5_1","unstructured":"IEA \u201cData Centres and Data Transmission Networks \u201d(2022) https:\/\/www.iea.org\/reports\/data\u2010centres\u2010and\u2010data\u2010transmission\u2010networks."},{"key":"e_1_2_14_6_1","article-title":"Cloud Computing's Coming Energy Crisis\u2010the Cloud's Electricity Needs Are Growing Unsustainably","author":"Pesce M.","year":"2021","journal-title":"IEEE Spectrum"},{"key":"e_1_2_14_7_1","volume-title":"HotCarbon 2022: 1st Workshop on Sustainable Computer Systems Design and Implementation","author":"Anand V.","year":"2022"},{"issue":"2","key":"e_1_2_14_8_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3177754","article-title":"Rapl in Action: Experiences in Using Rapl for Power Measurements","volume":"3","author":"Khan K. N.","year":"2018","journal-title":"ACM Transactions on Modeling and Performance Evaluation of Computing Systems"},{"key":"e_1_2_14_9_1","unstructured":"Foundation T.A.S \u201cApache Spark Lightning\u2010Fast Custer Computing \u201d(2015) http:\/\/spark.apache.org\/."},{"key":"e_1_2_14_10_1","doi-asserted-by":"crossref","DOI":"10.7717\/peerj-cs.321","article-title":"Big Data Clustering Techniques Based on Spark: A Literature Review","volume":"6","author":"Saeed M. M.","year":"2020","journal-title":"PeerJ Computer Science"},{"key":"e_1_2_14_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1740390.1740405"},{"key":"e_1_2_14_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2014.2358556"},{"key":"e_1_2_14_13_1","first-page":"51","volume-title":"Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems. ASPLOS \u201813","author":"Goiri I. N.","year":"2013"},{"key":"e_1_2_14_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-019-02947-9"},{"key":"e_1_2_14_15_1","doi-asserted-by":"crossref","first-page":"229","DOI":"10.5753\/wscad.2023.235799","volume-title":"Anais do XXIV Simp\u00f3sio em Sistemas Computacionais de Alto Desempenho","author":"Gon\u00e7alves T. D. S.","year":"2023"},{"key":"e_1_2_14_16_1","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1145\/2377978.2377984","volume-title":"Proceedings of the 1st Workshop on Architectures and Systems for Big Data","author":"Gu X.","year":"2011"},{"key":"e_1_2_14_17_1","first-page":"449","volume-title":"Proceedings of the 2014 IEEE 34th International Conference on Distributed Computing Systems (ICDCS)","author":"Berral J. L.","year":"2014"},{"key":"e_1_2_14_18_1","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/ISCC.2018.8538673","volume-title":"Proceedings of the 2018 IEEE Symposium on Computers and Communications (ISCC)","author":"Forte C. H.","year":"2018"},{"key":"e_1_2_14_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1816038.1816004"},{"key":"e_1_2_14_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.adhoc.2015.06.008"},{"key":"e_1_2_14_21_1","first-page":"1:1","volume-title":"Proceedings of the 7th International Workshop on Middleware for Grids, Clouds and e\u2010Science. MGC \u201809","author":"Kim K. H.","year":"2009"},{"key":"e_1_2_14_22_1","first-page":"1132","volume-title":"Proceedings of the 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)","author":"Zacheilas N.","year":"2017"},{"key":"e_1_2_14_23_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.5753\/wscad.2021.18512","volume-title":"Simp\u00f3sio em Sistemas Computacionais de Alto Desempenho (SSCAD)","author":"Bernardo F.","year":"2021"},{"issue":"7","key":"e_1_2_14_24_1","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/3213770","article-title":"Always Measure One Level Deeper","volume":"61","author":"Ousterhout J.","year":"2018","journal-title":"Communications of the ACM"},{"key":"e_1_2_14_25_1","unstructured":"T.Wuttge \u201cBenchframe: A Framework for Benchmarking Power Monitoring Tools \u201d (Ph.D. Thesis Vrije Universiteit Amsterdam)(2025)."},{"key":"e_1_2_14_26_1","doi-asserted-by":"publisher","DOI":"10.3390\/en12112204"},{"key":"e_1_2_14_27_1","first-page":"207","volume-title":"Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. ASPLOS \u201824","author":"Patel P.","year":"2024"},{"key":"e_1_2_14_28_1","doi-asserted-by":"crossref","first-page":"324","DOI":"10.5753\/sscad.2024.244769","volume-title":"Simp\u00f3sio em Sistemas Computacionais de Alto Desempenho (SSCAD)","author":"Volpini N. D. O.","year":"2024"},{"key":"e_1_2_14_29_1","unstructured":"E.Higgs \u201cEhiggs\/Spark\u2010Terasort \u201d(2018) https:\/\/github.com\/ehiggs\/spark\u2010terasort."},{"issue":"3","key":"e_1_2_14_30_1","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1007\/s10586-016-0723-1","article-title":"Sparkbench: A Spark Benchmarking Suite Characterizing Large\u2010Scale In\u2010Memory Data Analytics","volume":"20","author":"Li M.","year":"2017","journal-title":"Cluster Computing"},{"key":"e_1_2_14_31_1","first-page":"53","volume-title":"Proceedings of the 12th ACM International Conference on Computing Frontiers","author":"Li M.","year":"2015"},{"key":"e_1_2_14_32_1","unstructured":"A.Spark \u201cTuning Spark \u201d(2015) https:\/\/spark.apache.org\/docs\/2.2.0\/tuning.html."},{"key":"e_1_2_14_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2018.01.015"},{"key":"e_1_2_14_34_1","volume-title":"Anais do XVII Workshop em Desempenho de Sistemas Computacionais e de Comunica\u00e7 ao, Natal\u2010RN","author":"Concei\u00e7\u00e3o V. S.","year":"2018"}],"container-title":["Concurrency and Computation: Practice and Experience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpe.70565","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/cpe.70565","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpe.70565","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T14:32:28Z","timestamp":1770301948000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cpe.70565"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1002\/cpe.70565"],"URL":"https:\/\/doi.org\/10.1002\/cpe.70565","archive":["Portico"],"relation":{},"ISSN":["1532-0626","1532-0634"],"issn-type":[{"value":"1532-0626","type":"print"},{"value":"1532-0634","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2]]},"assertion":[{"value":"2025-08-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70565"}}