{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T03:20:17Z","timestamp":1776482417071,"version":"3.51.2"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,5,10]],"date-time":"2017-05-10T00:00:00Z","timestamp":1494374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMETRICS Perform. Eval. Rev."],"published-print":{"date-parts":[[2017,5,10]]},"abstract":"<jats:p>Big Data applications allow to successfully analyze large amounts of data not necessarily structured, though at the same time they present new challenges. For example, predicting the performance of frameworks such as Hadoop and Spark can be a costly task, hence the necessity to provide models that can be a valuable support for designers and developers. Big Data systems are becoming a central force in society and the use of models can also enable the development of intelligent systems providing Quality of Service (QoS) guarantees to their users through runtime system reconfiguration. This paper provides a new contribution in studying a novel modeling approach based on fluid Petri nets to predict MapReduce and Spark applications execution time which is suitable for runtime performance prediction. Models have been validated by an extensive experimental campaign performed at CINECA, the Italian supercomputing center, and on the Microsoft Azure HDInsight data platform. Results have shown that the achieved accuracy is around 9.5% for Map Reduce and about 10% for Spark of the actual measurements on average.<\/jats:p>","DOI":"10.1145\/3092819.3092824","type":"journal-article","created":{"date-parts":[[2017,5,10]],"date-time":"2017-05-10T18:08:53Z","timestamp":1494439733000},"page":"23-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Fluid Petri Nets for the Performance Evaluation of MapReduce and Spark Applications"],"prefix":"10.1145","volume":"44","author":[{"given":"Eugenio","family":"Gianniti","sequence":"first","affiliation":[{"name":"Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessandro Maria","family":"Rizzi","sequence":"additional","affiliation":[{"name":"Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Enrico","family":"Barbierato","sequence":"additional","affiliation":[{"name":"Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marco","family":"Gribaudo","sequence":"additional","affiliation":[{"name":"Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Danilo","family":"Ardagna","sequence":"additional","affiliation":[{"name":"Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Milano, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,5,10]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41366-7_1"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1007\/978-3-319-49583-5_47","volume-title":"Algorithms and Architectures for Parallel Processing - 16th International Conference, ICA3PP","author":"Ardagna D.","year":"2016","unstructured":"D. Ardagna , S. Bernardi , E. Gianniti , S. K. Aliabadi , D. Perez-Palacin , and J. I. Requeno . Modeling performance of hadoop applications: A journey from queueing networks to stochastic well formed nets . In Algorithms and Architectures for Parallel Processing - 16th International Conference, ICA3PP 2016 , Granada, Spain, December 14-16, 2016, Proceedings , pages 599 -- 613 , 2016. D. Ardagna, S. Bernardi, E. Gianniti, S. K. Aliabadi, D. Perez-Palacin, and J. I. Requeno. Modeling performance of hadoop applications: A journey from queueing networks to stochastic well formed nets. In Algorithms and Architectures for Parallel Processing - 16th International Conference, ICA3PP 2016, Granada, Spain, December 14-16, 2016, Proceedings, pages 599--613, 2016."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87879-7_1"},{"key":"e_1_2_1_4_1","volume-title":"MICAS-SYNASC 2016 Workshops Proceedings. To appear.","author":"Ataie E.","unstructured":"E. Ataie , E. Gianniti , D. Ardagna , and A. Movaghar . A combined analytical modeling machine learning approach for performance prediction of mapreduce jobs in hadoop clusters . In MICAS-SYNASC 2016 Workshops Proceedings. To appear. E. Ataie, E. Gianniti, D. Ardagna, and A. Movaghar. A combined analytical modeling machine learning approach for performance prediction of mapreduce jobs in hadoop clusters. In MICAS-SYNASC 2016 Workshops Proceedings. To appear."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.11.1042"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.valuetools.2013.254398"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 8th International Workshop on Practical Applications of Stochastic Modelling.","author":"Barbierato E.","unstructured":"E. Barbierato , M. Gribaudo , and M. Iacono . Modeling hybrid systems in SIMTHESys . In Proceedings of the 8th International Workshop on Practical Applications of Stochastic Modelling. to appear, 2016. E. Barbierato, M. Gribaudo, and M. Iacono. Modeling hybrid systems in SIMTHESys. In Proceedings of the 8th International Workshop on Practical Applications of Stochastic Modelling. to appear, 2016."},{"key":"e_1_2_1_8_1","first-page":"60","volume-title":"Fluid Approximation of Pool Depletion Systems","author":"Barbierato E.","year":"2016","unstructured":"E. Barbierato , M. Gribaudo , and D. Manini . Fluid Approximation of Pool Depletion Systems , pages 60 -- 75 . Springer International Publishing , Cham , 2016 . E. Barbierato, M. Gribaudo, and D. Manini. Fluid Approximation of Pool Depletion Systems, pages 60--75. Springer International Publishing, Cham, 2016."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2013.07.016"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.99195"},{"key":"e_1_2_1_11_1","unstructured":"M. Ciavotta and D. Ardagna.  M. Ciavotta and D. Ardagna."},{"key":"e_1_2_1_12_1","volume-title":"The digital universe","author":"Gantz J.","year":"2020","unstructured":"J. Gantz . The digital universe in 2020 . http:\/\/idcdocserv.com\/1414. J. Gantz. The digital universe in 2020. http:\/\/idcdocserv.com\/1414."},{"key":"e_1_2_1_13_1","volume-title":"INFQ 2016 Workshop Proceedings.","author":"Gianniti E.","unstructured":"E. Gianniti , A. M. Rizzi , E. Barbierato , M. Gribaudo , and D. Ardagna . Fluid petri nets for the performance evaluation of mapreduce applications . In INFQ 2016 Workshop Proceedings. E. Gianniti, A. M. Rizzi, E. Barbierato, M. Gribaudo, and D. Ardagna. Fluid petri nets for the performance evaluation of mapreduce applications. In INFQ 2016 Workshop Proceedings."},{"key":"e_1_2_1_14_1","volume-title":"Stage Aware Performance Modeling of DAG Based in Memory Analytic Platforms. In In IEEE Cloud 2016 Proceedings.","author":"Gibilisco G. P.","unstructured":"G. P. Gibilisco , M. Li , L. Zhang , and D. Ardagna . Stage Aware Performance Modeling of DAG Based in Memory Analytic Platforms. In In IEEE Cloud 2016 Proceedings. G. P. Gibilisco, M. Li, L. Zhang, and D. Ardagna. Stage Aware Performance Modeling of DAG Based in Memory Analytic Platforms. In In IEEE Cloud 2016 Proceedings."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLOUD.2016.0034"},{"key":"e_1_2_1_16_1","first-page":"271","volume-title":"Fluid Models in Performance Analysis","author":"Gribaudo M.","year":"2007","unstructured":"M. Gribaudo and M. Telek . Fluid Models in Performance Analysis , pages 271 -- 317 . Springer Berlin Heidelberg, Berlin , Heidelberg , 2007 . M. Gribaudo and M. Telek. Fluid Models in Performance Analysis, pages 271--317. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2611567"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2014.01.003"},{"key":"e_1_2_1_19_1","volume-title":"3D data management: Controlling data volume, velocity, and variety. Technical report","author":"Laney D.","year":"2012","unstructured":"D. Laney . 3D data management: Controlling data volume, velocity, and variety. Technical report , META Group , 2012 . D. Laney. 3D data management: Controlling data volume, velocity, and variety. Technical report, META Group, 2012."},{"key":"e_1_2_1_20_1","volume-title":"Quantitative System Performance","author":"Lazowska E. D.","year":"1984","unstructured":"E. D. Lazowska , J. Zahorjan , G. S. Graham , and K. C. Sevcik . Quantitative System Performance . Prentice-Hall , 1984 . E. D. Lazowska, J. Zahorjan, G. S. Graham, and K. C. Sevcik. Quantitative System Performance. Prentice-Hall, 1984."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/71.852402"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2567529.2567534"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ClusterW.2012.24"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/71.80155"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/12.2213"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.08.036"},{"key":"e_1_2_1_27_1","volume-title":"Scheduling: Theory, Algorithms, and Systems","author":"Pinedo M. L.","year":"2008","unstructured":"M. L. Pinedo . Scheduling: Theory, Algorithms, and Systems . Springer Publishing Company, Inc orporated, 3rd edition, 2008 . M. L. Pinedo. Scheduling: Theory, Algorithms, and Systems. Springer Publishing Company, Incorporated, 3rd edition, 2008."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2012.122112.110163"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742790"},{"key":"e_1_2_1_30_1","unstructured":"C. Shanklin. Benchmarking Apache Hive 13 for enterprise Hadoop. https:\/\/hadoop.apache.org\/docs\/r2.4.1\/hadoopyarn\/hadoop-yarn-site\/CapacityScheduler.html.  C. Shanklin. Benchmarking Apache Hive 13 for enterprise Hadoop. https:\/\/hadoop.apache.org\/docs\/r2.4.1\/hadoopyarn\/hadoop-yarn-site\/CapacityScheduler.html."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1998582.1998637"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10766-012-0227-4"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCC-CSS-ICESS.2015.246"},{"key":"e_1_2_1_34_1","first-page":"15","volume-title":"Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12)","author":"Zaharia M.","year":"2012","unstructured":"M. Zaharia , M. Chowdhury , T. Das , A. Dave , J. Ma , M. McCauly , M. J. Franklin , S. Shenker , and I. Stoica . Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing . In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12) , pages 15 -- 28 , San Jose, CA , 2012 . USENIX. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauly, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12), pages 15--28, San Jose, CA, 2012. USENIX."},{"key":"e_1_2_1_35_1","first-page":"10","volume-title":"Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10","author":"Zaharia M.","year":"2010","unstructured":"M. Zaharia , M. Chowdhury , M. J. Franklin , S. Shenker , and I. Stoica . Spark: Cluster computing with working sets . In Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10 , pages 10 -- 10 , Berkeley, CA, USA , 2010 . USENIX Association. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10, pages 10--10, Berkeley, CA, USA, 2010. USENIX Association."},{"key":"e_1_2_1_36_1","first-page":"10","volume-title":"Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10","author":"Zaharia M.","year":"2010","unstructured":"M. Zaharia , M. Chowdhury , M. J. Franklin , S. Shenker , and I. Stoica . Spark: Cluster computing with working sets . In Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10 , pages 10 -- 10 , Berkeley, CA, USA , 2010 . USENIX Association. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In Proceedings of the 2Nd USENIX Conference on Hot Topics in Cloud Computing, HotCloud'10, pages 10--10, Berkeley, CA, USA, 2010. USENIX Association."}],"container-title":["ACM SIGMETRICS Performance Evaluation Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3092819.3092824","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3092819.3092824","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:37:27Z","timestamp":1750217847000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3092819.3092824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,10]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,5,10]]}},"alternative-id":["10.1145\/3092819.3092824"],"URL":"https:\/\/doi.org\/10.1145\/3092819.3092824","relation":{},"ISSN":["0163-5999"],"issn-type":[{"value":"0163-5999","type":"print"}],"subject":[],"published":{"date-parts":[[2017,5,10]]},"assertion":[{"value":"2017-05-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}