{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T22:10:57Z","timestamp":1773785457508,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":53,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,11,19]],"date-time":"2021-11-19T00:00:00Z","timestamp":1637280000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,11,19]]},"DOI":"10.1145\/3501774.3501775","type":"proceedings-article","created":{"date-parts":[[2022,3,26]],"date-time":"2022-03-26T16:12:37Z","timestamp":1648311157000},"page":"1-5","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Systematizing the Meta-Analytical Process in Software Engineering"],"prefix":"10.1145","author":[{"given":"Letizia","family":"Jaccheri","sequence":"first","affiliation":[{"name":"Department of Computer Science Faculty of Information Technology and Electrical Engineering, Norwegian University of Science and Technology, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zamira","family":"Kholmatova","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science,Lab for Industrializing the Production of Software, Innopolis University, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giancarlo","family":"Succi","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science,Lab for Industrializing the Production of Software, Innopolis University, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,3,26]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"19","volume-title":"Proceedings of the 37th International Conference on Software Engineering -","volume":"1","author":"Siegmund Janet","unstructured":"Janet Siegmund , Norbert Siegmund , and Sven Apel . Views on internal and external validity in empirical software engineering . In Proceedings of the 37th International Conference on Software Engineering - Volume 1 , ICSE \u201915, page 9\u2013 19 . IEEE Press, 2015. Janet Siegmund, Norbert Siegmund, and Sven Apel. Views on internal and external validity in empirical software engineering. In Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE \u201915, page 9\u201319. IEEE Press, 2015."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9173-9"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2008.35"},{"issue":"2","key":"e_1_3_2_1_4_1","first-page":"5","article-title":"Automatic detection of bad smells in code: An experimental assessment","volume":"11","author":"Fontana Francesca Arcelli","year":"2012","unstructured":"Francesca Arcelli Fontana , Pietro Braione , and Marco Zanoni . Automatic detection of bad smells in code: An experimental assessment . J. Object Technol. , 11 ( 2 ): 5 \u2013 1 , 2012 . Francesca Arcelli Fontana, Pietro Braione, and Marco Zanoni. Automatic detection of bad smells in code: An experimental assessment. J. Object Technol., 11(2):5\u20131, 2012.","journal-title":"J. Object Technol."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.50"},{"key":"e_1_3_2_1_6_1","volume-title":"Software reuse: metrics and models. ACM Computing Surveys (CSUR), 28(2):415\u2013435","author":"Frakes William","year":"1996","unstructured":"William Frakes and Carol Terry . Software reuse: metrics and models. ACM Computing Surveys (CSUR), 28(2):415\u2013435 , 1996 . William Frakes and Carol Terry. Software reuse: metrics and models. ACM Computing Surveys (CSUR), 28(2):415\u2013435, 1996."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2013.142"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009793700999"},{"key":"e_1_3_2_1_9_1","first-page":"151","volume-title":"Proceedings Sixth International Software Metrics Symposium (Cat. No. PR00403)","author":"Hayes Will","unstructured":"Will Hayes . Research synthesis in software engineering: a case for meta-analysis . In Proceedings Sixth International Software Metrics Symposium (Cat. No. PR00403) , pages 143\u2013 151 . IEEE, 1999. Will Hayes. Research synthesis in software engineering: a case for meta-analysis. In Proceedings Sixth International Software Metrics Symposium (Cat. No. PR00403), pages 143\u2013151. IEEE, 1999."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(00)00024-8"},{"key":"e_1_3_2_1_11_1","volume-title":"Metaanalysis: Fixed effect vs. random effects. Meta-analysis. com","author":"Borenstein Michael","year":"2007","unstructured":"Michael Borenstein , Larry Hedges , and Hannah Rothstein . Metaanalysis: Fixed effect vs. random effects. Meta-analysis. com , 2007 . Michael Borenstein, Larry Hedges, and Hannah Rothstein. Metaanalysis: Fixed effect vs. random effects. Meta-analysis. com, 2007."},{"key":"e_1_3_2_1_12_1","volume-title":"Hans Dieter Rombach, and Marvin V. Zelkowitz. Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili","author":"Boehm Barry","year":"2010","unstructured":"Barry Boehm , Hans Dieter Rombach, and Marvin V. Zelkowitz. Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili . Springer Publishing Company, Inc orporated, 1st edition, 2010 . Barry Boehm, Hans Dieter Rombach, and Marvin V. Zelkowitz. Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili. Springer Publishing Company, Incorporated, 1st edition, 2010."},{"key":"e_1_3_2_1_13_1","first-page":"281","volume-title":"Proceedings. 26th International Conference on Software Engineering","author":"Kitchenham Barbara A","unstructured":"Barbara A Kitchenham , Tore Dyba , and Magne Jorgensen . Evidencebased software engineering . In Proceedings. 26th International Conference on Software Engineering , pages 273\u2013 281 . IEEE, 2004. Barbara A Kitchenham, Tore Dyba, and Magne Jorgensen. Evidencebased software engineering. In Proceedings. 26th International Conference on Software Engineering, pages 273\u2013281. IEEE, 2004."},{"key":"e_1_3_2_1_14_1","first-page":"1","article-title":"Procedures for performing systematic reviews. Keele, UK","volume":"33","author":"Kitchenham Barbara","year":"2004","unstructured":"Barbara Kitchenham . Procedures for performing systematic reviews. Keele, UK , Keele University , 33 (2004): 1 \u2013 26 , 2004 . Barbara Kitchenham. Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004):1\u201326, 2004.","journal-title":"Keele University"},{"key":"e_1_3_2_1_15_1","volume-title":"Systematic literature reviews in software engineering\u2013a systematic literature review. Information and software technology, 51(1):7\u201315","author":"Kitchenham Barbara","year":"2009","unstructured":"Barbara Kitchenham , O Pearl Brereton , David Budgen , Mark Turner , John Bailey , and Stephen Linkman . Systematic literature reviews in software engineering\u2013a systematic literature review. Information and software technology, 51(1):7\u201315 , 2009 . Barbara Kitchenham, O Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. Systematic literature reviews in software engineering\u2013a systematic literature review. Information and software technology, 51(1):7\u201315, 2009."},{"key":"e_1_3_2_1_16_1","volume-title":"Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80(4):571\u2013583","author":"Brereton Pearl","year":"2007","unstructured":"Pearl Brereton , Barbara A Kitchenham , David Budgen , Mark Turner , and Mohamed Khalil . Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80(4):571\u2013583 , 2007 . Pearl Brereton, Barbara A Kitchenham, David Budgen, Mark Turner, and Mohamed Khalil. Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80(4):571\u2013583, 2007."},{"key":"e_1_3_2_1_17_1","unstructured":"Acm computing surveys. http:\/\/https:\/\/dl.acm.org\/journal\/csur. Accessed: 2021-04-12.  Acm computing surveys. http:\/\/https:\/\/dl.acm.org\/journal\/csur. Accessed: 2021-04-12."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X005010003"},{"key":"e_1_3_2_1_19_1","volume-title":"Cibele Andrucioli de Mattos Pimenta, and Moacyr Roberto Cuce Nobre. The pico strategy for the research question construction and evidence search. Revista latinoamericana de enfermagem, 15(3):508\u2013511","author":"da Costa Santos Cristina Mamedio","year":"2007","unstructured":"Cristina Mamedio da Costa Santos , Cibele Andrucioli de Mattos Pimenta, and Moacyr Roberto Cuce Nobre. The pico strategy for the research question construction and evidence search. Revista latinoamericana de enfermagem, 15(3):508\u2013511 , 2007 . Cristina Mamedio da Costa Santos, Cibele Andrucioli de Mattos Pimenta, and Moacyr Roberto Cuce Nobre. The pico strategy for the research question construction and evidence search. Revista latinoamericana de enfermagem, 15(3):508\u2013511, 2007."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_1_21_1","first-page":"714","volume-title":"Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics","volume":"1","author":"Succi Giancarlo","year":"2000","unstructured":"Giancarlo Succi , Romana Spasojevic , Jason J Hayes , Michael R Smith , and Witold Pedrycz . Application of statistical meta-analysis to software engineering metrics data . In Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics , volume 1 , pages 709\u2013 714 , 2000 . Giancarlo Succi, Romana Spasojevic, Jason J Hayes, Michael R Smith, and Witold Pedrycz. Application of statistical meta-analysis to software engineering metrics data. In Proceedings of the World Multiconference on Systemics, Cybernetics and Informatics, volume 1, pages 709\u2013714, 2000."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.02.001"},{"key":"e_1_3_2_1_23_1","volume-title":"A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education (TOCE), 17(4):1\u201313","author":"Umapathy Karthikeyan","year":"2017","unstructured":"Karthikeyan Umapathy and Albert D Ritzhaupt . A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education (TOCE), 17(4):1\u201313 , 2017 . Karthikeyan Umapathy and Albert D Ritzhaupt. A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education (TOCE), 17(4):1\u201313, 2017."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.28"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3418777"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2770124"},{"key":"e_1_3_2_1_27_1","first-page":"36","volume-title":"2019 IEEE\/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)","author":"Yi Jooyong","unstructured":"Jooyong Yi , Vladimir Ivanov , and Giancarlo Succi . Mining plausible hypotheses from the literature via meta-analysis . In 2019 IEEE\/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) , pages 33\u2013 36 . IEEE, 2019. Jooyong Yi, Vladimir Ivanov, and Giancarlo Succi. Mining plausible hypotheses from the literature via meta-analysis. In 2019 IEEE\/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pages 33\u201336. IEEE, 2019."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/9781119536604"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.315.7121.1533"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.19153\/cleiej.15.1.2"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0028086"},{"issue":"1","key":"e_1_3_2_1_32_1","first-page":"2","article-title":"Effect size estimates: current use, calculations, and interpretation. Journal of experimental psychology","volume":"141","author":"Fritz Catherine O","year":"2012","unstructured":"Catherine O Fritz , Peter E Morris , and Jennifer J Richler . Effect size estimates: current use, calculations, and interpretation. Journal of experimental psychology : General , 141 ( 1 ): 2 , 2012 . Catherine O Fritz, Peter E Morris, and Jennifer J Richler. Effect size estimates: current use, calculations, and interpretation. Journal of experimental psychology: General, 141(1):2, 2012.","journal-title":"General"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164409344534"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.3.4.486"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.322.7300.1479"},{"key":"e_1_3_2_1_36_1","first-page":"292","volume-title":"21st IEEE International Conference on Software Maintenance (ICSM\u201905)","author":"Baldassarre Maria Teresa","unstructured":"Maria Teresa Baldassarre , Alessandro Bianchi , Danilo Caivano , and Giuseppe Visaggio . An industrial case study on reuse-oriented development . In 21st IEEE International Conference on Software Maintenance (ICSM\u201905) , pages 283\u2013 292 . IEEE, 2005. Maria Teresa Baldassarre, Alessandro Bianchi, Danilo Caivano, and Giuseppe Visaggio. An industrial case study on reuse-oriented development. In 21st IEEE International Conference on Software Maintenance (ICSM\u201905), pages 283\u2013292. IEEE, 2005."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(96)00152-5"},{"key":"e_1_3_2_1_38_1","first-page":"199","volume-title":"Proceedings of IEEE 18th International Conference on Software Engineering","author":"Devanbu Prem","unstructured":"Prem Devanbu , Sakke Karstu , Walcelio Melo , and William Thomas . Analytical and empirical evaluation of software reuse metrics . In Proceedings of IEEE 18th International Conference on Software Engineering , pages 189\u2013 199 . IEEE, 1996. Prem Devanbu, Sakke Karstu, Walcelio Melo, and William Thomas. Analytical and empirical evaluation of software reuse metrics. In Proceedings of IEEE 18th International Conference on Software Engineering, pages 189\u2013199. IEEE, 1996."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1033227"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(00)00121-7"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.922717"},{"key":"e_1_3_2_1_42_1","first-page":"523","volume-title":"International Conference on Computational Science and Its Applications","author":"Deniz Berkhan","unstructured":"Berkhan Deniz and Semih Bilgen . An empirical study of software reuse and quality in an industrial setting . In International Conference on Computational Science and Its Applications , pages 508\u2013 523 . Springer, 2014. Berkhan Deniz and Semih Bilgen. An empirical study of software reuse and quality in an industrial setting. In International Conference on Computational Science and Its Applications, pages 508\u2013523. Springer, 2014."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.372147"},{"key":"e_1_3_2_1_44_1","first-page":"291","volume-title":"Proceedings. 26th International Conference on Software Engineering","author":"Mohagheghi Parastoo","unstructured":"Parastoo Mohagheghi , Reidar Conradi , Ole M Killi , and Henrik Schwarz . An empirical study of software reuse vs. defect-density and stability . In Proceedings. 26th International Conference on Software Engineering , pages 282\u2013 291 . IEEE, 2004. Parastoo Mohagheghi, Reidar Conradi, Ole M Killi, and Henrik Schwarz. An empirical study of software reuse vs. defect-density and stability. In Proceedings. 26th International Conference on Software Engineering, pages 282\u2013291. IEEE, 2004."},{"key":"e_1_3_2_1_45_1","first-page":"401","volume-title":"Reuse and productivity in integrated computer-aided software engineering: An empirical study. MIS quarterly","author":"Banker Rajiv D","year":"1991","unstructured":"Rajiv D Banker and Robert J Kauffman . Reuse and productivity in integrated computer-aided software engineering: An empirical study. MIS quarterly , pages 375\u2013 401 , 1991 . Rajiv D Banker and Robert J Kauffman. Reuse and productivity in integrated computer-aided software engineering: An empirical study. MIS quarterly, pages 375\u2013401, 1991."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9081-9"},{"key":"e_1_3_2_1_47_1","first-page":"8","volume-title":"2015 Sixth International Green and Sustainable Computing Conference (IGSC)","author":"Abdulsalam Sarah","unstructured":"Sarah Abdulsalam , Ziliang Zong , Qijun Gu , and Meikang Qiu . Using the greenup, powerup, and speedup metrics to evaluate software energy efficiency . In 2015 Sixth International Green and Sustainable Computing Conference (IGSC) , pages 1\u2013 8 . IEEE, 2015. Sarah Abdulsalam, Ziliang Zong, Qijun Gu, and Meikang Qiu. Using the greenup, powerup, and speedup metrics to evaluate software energy efficiency. In 2015 Sixth International Green and Sustainable Computing Conference (IGSC), pages 1\u20138. IEEE, 2015."},{"key":"e_1_3_2_1_48_1","first-page":"2019","article-title":"Enhancing offloading systems with smart decisions, adaptive monitoring, and mobility support","author":"Rego Paulo AL","year":"2019","unstructured":"Paulo AL Rego , Fernando AM Trinta , Masum Z Hasan , and Jose N de Souza . Enhancing offloading systems with smart decisions, adaptive monitoring, and mobility support . Wireless Communications and Mobile Computing , 2019 , 2019 . Paulo AL Rego, Fernando AM Trinta, Masum Z Hasan, and Jose N de Souza. Enhancing offloading systems with smart decisions, adaptive monitoring, and mobility support. Wireless Communications and Mobile Computing, 2019, 2019.","journal-title":"Wireless Communications and Mobile Computing"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2593743.2593750"},{"key":"e_1_3_2_1_50_1","first-page":"12","volume-title":"2016 5th International Conference on Smart Cities and Green ICT Systems (SMARTGREENS)","author":"Westfield Benjamin","unstructured":"Benjamin Westfield and Anandha Gopalan . Orka : A new technique to profile the energy usage of android applications . In 2016 5th International Conference on Smart Cities and Green ICT Systems (SMARTGREENS) , pages 1\u2013 12 . IEEE, 2016. Benjamin Westfield and Anandha Gopalan. Orka: A new technique to profile the energy usage of android applications. In 2016 5th International Conference on Smart Cities and Green ICT Systems (SMARTGREENS), pages 1\u201312. IEEE, 2016."},{"key":"e_1_3_2_1_51_1","volume-title":"Caching to reduce mobile app energy consumption. ACM Transactions on the Web (TWEB), 12(1):1\u2013 30","author":"Dutta Kaushik","year":"2017","unstructured":"Kaushik Dutta and Debra Vandermeer . Caching to reduce mobile app energy consumption. ACM Transactions on the Web (TWEB), 12(1):1\u2013 30 , 2017 . Kaushik Dutta and Debra Vandermeer. Caching to reduce mobile app energy consumption. ACM Transactions on the Web (TWEB), 12(1):1\u2013 30, 2017."},{"key":"e_1_3_2_1_52_1","volume-title":"Performance evaluation of kotlin and java on android runtime","author":"Schwermer Patrik","year":"2018","unstructured":"Patrik Schwermer . Performance evaluation of kotlin and java on android runtime , 2018 . Patrik Schwermer. Performance evaluation of kotlin and java on android runtime, 2018."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136014.3136031"}],"event":{"name":"ESSE 2021: 2021 2nd European Symposium on Software Engineering","location":"Larissa Greece","acronym":"ESSE 2021"},"container-title":["2021 2nd European Symposium on Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3501774.3501775","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3501774.3501775","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:45Z","timestamp":1750191105000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3501774.3501775"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,19]]},"references-count":53,"alternative-id":["10.1145\/3501774.3501775","10.1145\/3501774"],"URL":"https:\/\/doi.org\/10.1145\/3501774.3501775","relation":{},"subject":[],"published":{"date-parts":[[2021,11,19]]},"assertion":[{"value":"2022-03-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}