{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T01:12:03Z","timestamp":1779930723370,"version":"3.53.1"},"reference-count":144,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,8,3]],"date-time":"2021-08-03T00:00:00Z","timestamp":1627948800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n            <jats:italic>Context:<\/jats:italic>\n            Evaluating software architectures in uncertain environments raises new challenges, which require continuous approaches. We define continuous evaluation as multiple evaluations of the software architecture that begins at the early stages of the development and is periodically and repeatedly performed throughout the lifetime of the software system. Numerous approaches have been developed for continuous evaluation; to handle dynamics and uncertainties at run-time, over the past years, these approaches are still very few, limited, and lack maturity.\n            <jats:italic>Objective:<\/jats:italic>\n            This review surveys efforts on architecture evaluation and provides a unified terminology and perspective on the subject.\n            <jats:italic>Method:<\/jats:italic>\n            We conducted a systematic literature review to identify and analyse architecture evaluation approaches for uncertainty including continuous and non-continuous, covering work published between 1990\u20132020. We examined each approach and provided a classification framework for this field. We present an analysis of the results and provide insights regarding open challenges.\n            <jats:italic>Major results and conclusions:<\/jats:italic>\n            The survey reveals that most of the existing architecture evaluation approaches typically lack an explicit linkage between design-time and run-time. Additionally, there is a general lack of systematic approaches on how continuous architecture evaluation can be realised or conducted. To remedy this lack, we present a set of necessary requirements for continuous evaluation and describe some examples.\n          <\/jats:p>","DOI":"10.1145\/3464305","type":"journal-article","created":{"date-parts":[[2021,8,4]],"date-time":"2021-08-04T04:09:51Z","timestamp":1628050191000},"page":"1-50","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":28,"title":["Evaluation of Software Architectures under Uncertainty"],"prefix":"10.1145","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9709-7731","authenticated-orcid":false,"given":"Dalia","family":"Sobhy","sequence":"first","affiliation":[{"name":"Computer Engineering Department, Arab Academy of Science and Technology and Maritime Transport, Egypt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rami","family":"Bahsoon","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Birmingham and FRSA, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leandro","family":"Minku","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rick","family":"Kazman","sequence":"additional","affiliation":[{"name":"Information Technology Management, University of Hawaii and SEI\/CMU, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,8,3]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Henrik Rexed. 2017. Challenges to performance engineering IoT apps. https:\/\/techbeacon.com\/app-dev-testing\/5-challenges-performance-engineering-iot-apps?amp_=  Henrik Rexed. 2017. Challenges to performance engineering IoT apps. https:\/\/techbeacon.com\/app-dev-testing\/5-challenges-performance-engineering-iot-apps?amp_="},{"key":"e_1_2_1_2_1","first-page":"2099","article-title":"Better software analytics via \u201cDUO\u201d: Data mining algorithms using\/used-by optimizers.Empir","volume":"25","author":"Agrawal Amritanshu","year":"2020","journal-title":"Softw. Eng."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1809100.1809103"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2017.2711009"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062508"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MOMPES.2009.5069138"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.64"},{"key":"e_1_2_1_8_1","volume-title":"et\u00a0al","author":"Amram Martha","year":"1998"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2004.38"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASWEC.2004.1290484"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018431.1021454"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA.2008.45"},{"key":"e_1_2_1_14_1","volume-title":"Clark","author":"Baldwin Carliss Young","year":"2000"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2010.09.003"},{"key":"e_1_2_1_16_1","volume-title":"Software Architecture in Practice","author":"Bass Len"},{"key":"e_1_2_1_17_1","volume-title":"DevOps: A Software Architect\u2019s Perspective","author":"Bass Len"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSR.1998.685756"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(03)00080-3"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the International Workshop on Eternal Systems. Springer, 16\u201329","author":"Bennaceur Amel","year":"2012"},{"key":"e_1_2_1_21_1","volume-title":"et\u00a0al","author":"Berander Patrik","year":"2005"},{"key":"e_1_2_1_22_1","volume-title":"France","author":"Blair Gordon","year":"2009"},{"key":"e_1_2_1_23_1","volume-title":"et\u00a0al","author":"Boehm Barry W.","year":"1981"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the Conference on the Future of Software Engineering. ACM, 319\u2013343","author":"Barry"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24769-2_14"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1075405.1075411"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2011.06.002"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSA-C.2019.00036"},{"key":"e_1_2_1_29_1","volume-title":"Assurances for Self-Adaptive Systems","author":"Calinescu Radu"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.92"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958746.1958823"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070512"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.5555\/2821357.2821384"},{"key":"e_1_2_1_34_1","volume-title":"Cost-effectiveness and cost-benefit analysis. Handb. Pract. Prog. Eval. 4","author":"Cellini Stephanie Riegg","year":"2015"},{"key":"e_1_2_1_35_1","volume-title":"Designing Software Architectures: A Practical Approach","author":"Cervantes Humberto"},{"key":"e_1_2_1_36_1","unstructured":"Franck Chauvel Nicolas Ferry Brice Morin Alessandro Rossini and Arnor Solberg. 2013. Models@ runtime to support the iterative and continuous design of autonomic reasoners. In MoDELS@ Run.time. Springer 26\u201338.  Franck Chauvel Nicolas Ferry Brice Morin Alessandro Rossini and Arnor Solberg. 2013. Models@ runtime to support the iterative and continuous design of autonomic reasoners. In MoDELS@ Run.time. Springer 26\u201338."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSA.2018.00013"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204459"},{"key":"e_1_2_1_39_1","volume-title":"et\u00a0al","author":"Cheng Betty H. C.","year":"2014"},{"key":"e_1_2_1_40_1","volume-title":"Rainbow: Cost-effective software architecture-based self-adaptation. ProQuest.","author":"Cheng Shang-Wen","year":"2008"},{"key":"e_1_2_1_41_1","unstructured":"Shang-Wen Cheng. 2008. Rainbow: Cost-effective Software Architecture-based Self-adaptation. Ph.D. Dissertation. Pittsburgh PA.  Shang-Wen Cheng. 2008. Rainbow: Cost-effective Software Architecture-based Self-adaptation. Ph.D. Dissertation. Pittsburgh PA."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15114-9_11"},{"key":"e_1_2_1_43_1","volume-title":"et\u00a0al","author":"Clements Paul","year":"2003"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2004.833312"},{"key":"e_1_2_1_45_1","first-page":"1061","article-title":"IEEE Standard for a Software Quality Metrics Methodology","author":"Software Engineering Standards Committee","year":"1998","journal-title":"Std."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.36"},{"key":"e_1_2_1_47_1","volume-title":"et\u00a0al","author":"Lemos Rog\u00e9rio De","year":"2013"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-29983-5_12"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2004.1293071"},{"key":"e_1_2_1_50_1","first-page":"105","article-title":"Aleatory or epistemic? Does it matter?Struct","volume":"31","author":"Kiureghian Armen Der","year":"2009","journal-title":"Saf."},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the Conference on Computer Security, Dependability and Assurance. IEEE, 171","author":"Deswarte Yves","year":"1998"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2015.2471196"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1019479"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2554850.2555044"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882291.1882296"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/0377-2217(90)90118-U"},{"key":"e_1_2_1_57_1","volume-title":"Relating System Quality and Software Architecture","author":"Eloranta Veli-Pekka"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070513"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the International Workshop on Reuse Economics \u201cRedirecting Reuse Economics\u201d Tuesday.","author":"Erdogmus Hakan","year":"2002"},{"key":"e_1_2_1_60_1","unstructured":"Hakan Erdogmus and Jennifer Vandergraaf. 1999. Quantitative approaches for assessing the value of COTS-centric development. (1999).  Hakan Erdogmus and Jennifer Vandergraaf. 1999. Quantitative approaches for assessing the value of COTS-centric development. (1999)."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.37"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025147"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2013.6606550"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA.2011.25"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSR.1996.496121"},{"key":"e_1_2_1_66_1","volume-title":"Judg. Dec. Mak.","author":"Fox Craig R.","year":"2011"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-managing Systems. ACM, 17\u201326","author":"Fredericks Erik M."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24769-2_15"},{"key":"e_1_2_1_69_1","volume-title":"Dealing with non-functional requirements for adaptive systems via dynamic software product-lines","author":"Ghezzi Carlo"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134431"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2013.01.010"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2168260.2168270"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(01)00189-6"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379787"},{"key":"e_1_2_1_75_1","volume-title":"Software Engineering for Self-adaptive Systems","author":"Heaven William"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897356.2897359"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSAW.2017.33"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2613863"},{"key":"e_1_2_1_79_1","volume-title":"Proceedings of the 4th Working IEEE\/IFIP Conference on Software Architecture. IEEE, 277\u2013280","author":"Ionita Mugurel T."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/WICSA.2005.61"},{"key":"e_1_2_1_81_1","volume-title":"Lattanze","author":"Jones Lawrence G.","year":"2001"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2001.919103"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.1994.296768"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.05.001"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.21236\/ADA382629"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1109\/SNPD.2007.507"},{"key":"e_1_2_1_87_1","volume-title":"Proceedings of the ICSE Workshop on Software Engineering for Adaptive and Self-managing Systems. IEEE, 76\u201385","author":"Kim Dongsun","year":"2009"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.09.009"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2010.03.006"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/2000259.2000263"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.19"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.469759"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2014.09.009"},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9094-4"},{"key":"e_1_2_1_95_1","volume-title":"Proceedings of the 36th International Conference on Software Engineering. ACM, 883\u2013894","author":"Letier Emmanuel"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87879-7_12"},{"key":"e_1_2_1_97_1","first-page":"89","article-title":"Strategy as a portfolio of real options","volume":"76","author":"Luehrman Timothy A.","year":"1998","journal-title":"Harv. Bus. Rev."},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2017.03.013"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2012.08.010"},{"key":"e_1_2_1_100_1","volume-title":"Cowles foundation monograph no. 16","author":"Markowitz Harry","year":"1981"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00158-003-0368-6"},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEES.2012.6225486"},{"key":"e_1_2_1_103_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.04.056"},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/2000259.2000275"},{"key":"e_1_2_1_105_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-012-0277-5"},{"key":"e_1_2_1_106_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2016.59"},{"key":"e_1_2_1_107_1","volume-title":"Computer 42, 10","author":"Morin Brice","year":"2009"},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.1109\/IoTDI.2015.41"},{"key":"e_1_2_1_109_1","doi-asserted-by":"crossref","unstructured":"Nanjangud Narendra and Prasant Misra. 2016. Research Challenges in the Internet of Mobile Things. Retrieved from https:\/\/iot.ieee.org\/newsletter\/march-2016\/research-challenges-in-the-int ernet-of-mobile-things.html. DOI:https:\/\/doi.org\/10.1145\/3204459  Nanjangud Narendra and Prasant Misra. 2016. Research Challenges in the Internet of Mobile Things. Retrieved from https:\/\/iot.ieee.org\/newsletter\/march-2016\/research-challenges-in-the-int ernet-of-mobile-things.html. DOI:https:\/\/doi.org\/10.1145\/3204459","DOI":"10.1145\/3204459"},{"key":"e_1_2_1_110_1","doi-asserted-by":"publisher","DOI":"10.1057\/ejis.2012.26"},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2004.03.002"},{"key":"e_1_2_1_112_1","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889218"},{"key":"e_1_2_1_113_1","doi-asserted-by":"publisher","DOI":"10.1109\/EDOCW.2013.45"},{"key":"e_1_2_1_114_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESC.2007.8"},{"key":"e_1_2_1_115_1","doi-asserted-by":"publisher","DOI":"10.1145\/141874.141884"},{"key":"e_1_2_1_116_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.03.007"},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1109\/ECBS.2010.16"},{"key":"e_1_2_1_118_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2018.12.015"},{"key":"e_1_2_1_119_1","volume-title":"Redwine Jr and William E. Riddle","author":"Samuel","year":"1985"},{"key":"e_1_2_1_120_1","volume-title":"Modeling and Simulating Software Architectures: The Palladio Approach","author":"Reussner Ralf H."},{"key":"e_1_2_1_121_1","unstructured":"Matthias Rohr Simon Giesecke Marcel Hiel Willem-Jan van den Heuvel Hans Weigand and Wilhelm Hasselbring. 2006. A classification scheme for self-adaptation research. (2006).  Matthias Rohr Simon Giesecke Marcel Hiel Willem-Jan van den Heuvel Hans Weigand and Wilhelm Hasselbring. 2006. A classification scheme for self-adaptation research. (2006)."},{"key":"e_1_2_1_122_1","first-page":"82","article-title":"Methods for evaluating software architecture: A survey","volume":"545","author":"Roy Banani","year":"2008","journal-title":"School Comput. TR"},{"key":"e_1_2_1_123_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJSSCI.2008.017590"},{"key":"e_1_2_1_125_1","volume-title":"Software Architecture: Perspectives on an Emerging Discipline.","author":"Shaw Mary","year":"1996"},{"key":"e_1_2_1_126_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-48992-6_4"},{"key":"e_1_2_1_127_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110428"},{"key":"e_1_2_1_128_1","first-page":"215","article-title":"Software design as an investment activity: A real options perspective","volume":"10","author":"Sullivan Kevin J.","year":"1999","journal-title":"Real Opt. Bus. Strat.: Applic. Decis. Mak."},{"key":"e_1_2_1_129_1","doi-asserted-by":"publisher","DOI":"10.1145\/503271.503224"},{"key":"e_1_2_1_130_1","first-page":"34","article-title":"Vision and challenges for realising the Internet of Things. Cluster of European Research Projects on the Internet of Things","volume":"3","author":"Sundmaeker Harald","year":"2010","journal-title":"European Commision"},{"key":"e_1_2_1_131_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-013-0394-9"},{"key":"e_1_2_1_132_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESC.2007.11"},{"key":"e_1_2_1_133_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2007.21"},{"key":"e_1_2_1_134_1","volume-title":"Real Options: Managerial Flexibility and Strategy in Resource Allocation","author":"Trigeorgis Lenos","year":"1996"},{"key":"e_1_2_1_135_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465478.2465487"},{"key":"e_1_2_1_136_1","doi-asserted-by":"publisher","DOI":"10.5220\/0006815404780490"},{"key":"e_1_2_1_137_1","volume-title":"Rationale Management in Software Engineering","author":"van der Ven Jan Salvador"},{"key":"e_1_2_1_138_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2005.12.011"},{"key":"e_1_2_1_139_1","volume-title":"Proceedings of the 3rd International Workshop on Software and Performance. ACM, 179\u2013189","author":"Lloyd"},{"key":"e_1_2_1_140_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_2_1_141_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2008.06.039"},{"key":"e_1_2_1_142_1","doi-asserted-by":"publisher","DOI":"10.5555\/2227057.2227071"},{"key":"e_1_2_1_143_1","doi-asserted-by":"publisher","DOI":"10.1109\/ADPRL.2011.5967371"},{"key":"e_1_2_1_144_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0377-2217(03)00272-8"},{"key":"e_1_2_1_145_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-005-4251-0"},{"key":"e_1_2_1_146_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3183465"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464305","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3464305","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:16Z","timestamp":1750191136000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464305"}},"subtitle":["A Systematic Literature Review"],"short-title":[],"issued":{"date-parts":[[2021,8,3]]},"references-count":144,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3464305"],"URL":"https:\/\/doi.org\/10.1145\/3464305","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,3]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}