{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T15:31:19Z","timestamp":1783611079057,"version":"3.55.0"},"reference-count":128,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,11,21]],"date-time":"2022-11-21T00:00:00Z","timestamp":1668988800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"FAPESP - S\u00e3o Paulo Research Foundation","award":["2015\/24144-7 and 2017\/06195-9"],"award-info":[{"award-number":["2015\/24144-7 and 2017\/06195-9"]}]},{"name":"CNPq - National Council for Scientific and Technological Development","award":["312634\/2018-8"],"award-info":[{"award-number":["312634\/2018-8"]}]},{"name":"Capes - Coordination for the Improvement of Higher Education Personnel"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"<jats:p>The quality of large and complex<jats:bold>Systems-of-Systems (SoS)<\/jats:bold>that have emerged in critical application domains depends on the quality of their architectures, which are inherently dynamic in terms of reorganization at runtime to comply with domain needs. However, the way to model and evaluate the quality of these architectures is not clear. This article presents the state-of-the-art regarding how SoS architectures have been evaluated. For this, we systematically examined the literature and, as a result, we discovered and summarized relevant architectural evaluation methods and associated modeling techniques and quality attributes, the maturity of these methods, as well as the benefits and costs of adopting them. We also address open issues and research opportunities and recommend that the mindset for SoS architecture evaluation must be changed to ensure the quality of SoS in critical domains.<\/jats:p>","DOI":"10.1145\/3519020","type":"journal-article","created":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T12:29:59Z","timestamp":1646396999000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Evaluation of Systems-of-Systems Software Architectures: State of the Art and Future Perspectives"],"prefix":"10.1145","volume":"55","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4015-1722","authenticated-orcid":false,"given":"Daniel S.","family":"Santos","sequence":"first","affiliation":[{"name":"University of S\u00e3o Paulo - USP, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1992-4196","authenticated-orcid":false,"given":"Brauner R. N.","family":"Oliveira","sequence":"additional","affiliation":[{"name":"University of S\u00e3o Paulo - USP, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rick","family":"Kazman","sequence":"additional","affiliation":[{"name":"University of Hawaii, Hawaii, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisa Y.","family":"Nakagawa","sequence":"additional","affiliation":[{"name":"University of S\u00e3o Paulo - USP, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,21]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134406"},{"key":"e_1_3_2_3_2","first-page":"175","volume-title":"20th International Symposium on Software Reliability Engineering (ISSRE)","author":"Ackermann C.","year":"2009","unstructured":"C. Ackermann, M. Lindvall, and R. Cleaveland. 2009. Towards behavioral reflexion models. In 20th International Symposium on Software Reliability Engineering (ISSRE). 175\u2013184."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.03.005"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2018.8428759"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASWEC.2004.1290484"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2012.6187442"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.03.032"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3043036"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA418428"},{"key":"e_1_3_2_11_2","volume-title":"Software Architecture in Practice","author":"Bass L.","year":"2013","unstructured":"L. Bass, P. Clements, and R. Kazman. 2013. Software Architecture in Practice. Addison-Wesley."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2008.03.066"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSR.1998.685756"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1109\/ISSREW.2019.00064","volume-title":"IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)","author":"Bertolino A.","year":"2019","unstructured":"A. Bertolino, G. De Angelis, and F. Lonetti. 2019. Governing regression testing in systems of systems. In IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). 144\u2013148."},{"key":"e_1_3_2_15_2","first-page":"23","volume-title":"3rd International Workshop on Software Engineering and Systems-of-Systems (SESoS)","author":"Bianchi T.","year":"2015","unstructured":"T. Bianchi, D. Santos, and K. Felizardo. 2015. System-of-systems quality attributes: A systematic literature review. In 3rd International Workshop on Software Engineering and Systems-of-Systems (SESoS). 23\u201330."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2006.1652284"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.082080899"},{"key":"e_1_3_2_18_2","volume-title":"Design and Use of Software Architectures: Adopting and Evolving a Product-Line Approach","author":"Bosch J.","year":"2000","unstructured":"J. Bosch. 2000. Design and Use of Software Architectures: Adopting and Evolving a Product-Line Approach. Addison Wesley. 00036362"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/11921998_16"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.5555\/1951720"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387940.3391474"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2020.102533"},{"key":"e_1_3_2_23_2","first-page":"69","volume-title":"1st International Workshop on Green and Sustainable Software (GREENS)","author":"Chen H.","year":"2012","unstructured":"H. Chen, R. Kazman, and C. Hong-Mei. 2012. Architecting ultra-large-scale green information systems. In 1st International Workshop on Green and Sustainable Software (GREENS). 69\u201375."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2014.6892503"},{"key":"e_1_3_2_25_2","volume-title":"Documenting Software Architectures: Views and Beyond","author":"Clements P.","year":"2010","unstructured":"P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, P. Merson, R. Nord, and J. Stafford. 2010. Documenting Software Architectures: Views and Beyond. Addison-Wesley."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.5555\/599933"},{"key":"e_1_3_2_27_2","volume-title":"Evaluating Software Architectures: Methods and Case Studies","author":"Clements P.","year":"2002","unstructured":"P. Clements, R. Kazman, and M. Klein. 2002. Evaluating Software Architectures: Methods and Case Studies. Addison-Wesley."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSTEMS.2008.4518994"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.9.3.458"},{"key":"e_1_3_2_30_2","volume-title":"Systems of Systems","author":"Dersin P.","year":"2014","unstructured":"P. Dersin. 2014. Systems of Systems. Technical Report. IEEE-Reliability Society."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1019479"},{"key":"e_1_3_2_32_2","volume-title":"Systems Engineering Guide for System of Systems","year":"2008","unstructured":"DoD. 2008. Systems Engineering Guide for System of Systems. Technical Report. US Department of Defense."},{"key":"e_1_3_2_33_2","volume-title":"DoDAF Architecture Framework","year":"2010","unstructured":"DoD. 2010. DoDAF Architecture Framework. Technical Report. US Department of Defense."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2011.5747653"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigDIA51454.2020.00082"},{"key":"e_1_3_2_36_2","first-page":"178","volume-title":"2nd International Symposium on Empirical Software Engineering and Measurement (ESEM)","author":"Dyb\u00e5 T.","year":"2008","unstructured":"T. Dyb\u00e5 and T. Dings\u00f8yr. 2008. Strength of evidence in systematic reviews in software engineering. In 2nd International Symposium on Empirical Software Engineering and Measurement (ESEM). 178\u2013187."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2005.6"},{"key":"e_1_3_2_38_2","first-page":"267","volume-title":"International Symposium of the National Council of System Engineering (NCOSE)","author":"Eisner H.","year":"1993","unstructured":"H. Eisner. 1993. RCASSE: Rapid computer-aided systems of systems engineering. In International Symposium of the National Council of System Engineering (NCOSE). 267\u2013273."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSYST.2010.2045541"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2013.01.029"},{"key":"e_1_3_2_41_2","volume-title":"Ultra-large-scale Systems: The Software Challenge of the Future","author":"Feiler P.","year":"2006","unstructured":"P. Feiler, K. Sullivan, K. Wallnau, R. Gabriel, J. Goodenough, R. Linger, T. Longstaff, R. Kazman, M. Klein, L. Northrop, and D. Schmidt. 2006. Ultra-large-scale Systems: The Software Challenge of the Future. Software Engineering Institute, Carnegie Mellon University."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/568760.568807"},{"issue":"3","key":"e_1_3_2_43_2","first-page":"12","article-title":"A uniform approach for system of systems architecture evaluation","volume":"22","author":"Gagliardi M.","year":"2009","unstructured":"M. Gagliardi, W. Wood, J. Klein, and J. Morley. 2009. A uniform approach for system of systems architecture evaluation. The Journal of Defense Software Engineering 22, 3 (2009), 12\u201315.","journal-title":"The Journal of Defense Software Engineering"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.10.005"},{"issue":"1","key":"e_1_3_2_45_2","first-page":"1","article-title":"On the reliability analysis of systems and SoS: The RAMSAS method and related extensions","volume":"9","author":"Garro A.","year":"2014","unstructured":"A. Garro and A. Tundis. 2014. On the reliability analysis of systems and SoS: The RAMSAS method and related extensions. IEEE Systems Journal 9, 1 (2014), 1\u201310.","journal-title":"IEEE Systems Journal"},{"key":"e_1_3_2_46_2","volume-title":"Introduction to Functional Dependency Network Analysis","author":"Garvey P.","year":"2009","unstructured":"P. Garvey and A. Pinto. 2009. Introduction to Functional Dependency Network Analysis. Technical Report. Massachusetts Institute of Technology (MIT)."},{"key":"e_1_3_2_47_2","first-page":"1","volume-title":"2nd International Symposium on Engineering Systems","author":"Garvey P.","year":"2010","unstructured":"P. Garvey and C. Pinto. 2010. Introduction to functional dependency network analysis. In 2nd International Symposium on Engineering Systems. 1."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1002\/sys.21253"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2017.7994944"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2007.4304218"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2010.5544088"},{"key":"e_1_3_2_52_2","first-page":"265","volume-title":"Procedia Computer Science","author":"Guariniello C.","year":"2013","unstructured":"C. Guariniello and D. DeLaurentis. 2013. Dependency analysis of system-of-systems operational and development networks. In Procedia Computer Science, Vol. 16. 265\u2013274."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2014.03.086"},{"issue":"1","key":"e_1_3_2_54_2","first-page":"741","article-title":"Ark: A constraint-based method for architectural synthesis of smart systems","volume":"19","author":"Guessi M.","year":"2019","unstructured":"M. Guessi, F. Oquendo, and E. Nakagawa. 2019. Ark: A constraint-based method for architectural synthesis of smart systems. Software and Systems Modeling 19, 1 (2019), 741\u2013762.","journal-title":"Software and Systems Modeling"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110484"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2018.8428776"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2016.023"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC48747.2019.00056"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-008-9068-1"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSMC.2012.6377904"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.micpro.2017.06.014"},{"key":"e_1_3_2_62_2","first-page":"137","volume-title":"6th System of Systems Engineering (SoSE)","author":"Iacobucci J.","year":"2011","unstructured":"J. Iacobucci and D. Mavris. 2011. A method for the generation and evaluation of architecture alternatives on the cloud. In 6th System of Systems Engineering (SoSE). 137\u2013142."},{"key":"e_1_3_2_63_2","volume-title":"Systems and Software Engineering - Vocabulary","author":"24765 ISO\/IEC","year":"2010","unstructured":"ISO\/IEC 24765. 2010. Systems and Software Engineering - Vocabulary. Technical Report. International Organization for Standardization and International Electrotechnical Commission."},{"key":"e_1_3_2_64_2","volume-title":"Systems and Software Engineering\u2013Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2013System and Software Quality Models","author":"25010 ISO\/IEC","year":"2011","unstructured":"ISO\/IEC 25010. 2011. Systems and Software Engineering\u2013Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2013System and Software Quality Models. Technical Report. International Organization for Standardization and International Electrotechnical Commission."},{"key":"e_1_3_2_65_2","volume-title":"Systems and Software Engineering\u2013Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2013Evaluation Process","author":"25040 ISO\/IEC","year":"2011","unstructured":"ISO\/IEC 25040. 2011. Systems and Software Engineering\u2013Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2013Evaluation Process. Technical Report. International Organization for Standardization and International Electrotechnical Commission."},{"key":"e_1_3_2_66_2","volume-title":"Systems and Software Engineering\u2013Architecture Description","author":"42010 ISO\/IEC\/IEEE","year":"2011","unstructured":"ISO\/IEC\/IEEE 42010. 2011. Systems and Software Engineering\u2013Architecture Description. Technical Report. International Organization for Standardization and International Electrotechnical Commission."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2011.5966581"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1002\/9780470403501"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781420065893"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-03241-1"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2004.08.026"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2005.103"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA408740"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.1994.296768"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.03.050"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA382629"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.5555\/2994449"},{"key":"e_1_3_2_78_2","volume-title":"Guidelines for Performing Systematic Literature Reviews in Software Engineering","author":"Kitchenham B.","year":"2007","unstructured":"B. Kitchenham and S. Charters. 2007. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Technical Report. Keele University and Durham University."},{"key":"e_1_3_2_79_2","first-page":"1","article-title":"System-of-systems viewpoint for system architecture documentation.","volume":"1801","author":"Klein J.","year":"2018","unstructured":"J. Klein and H. van Vliet. 2018. System-of-systems viewpoint for system architecture documentation.Computing Research Repository 1801.06837 (2018), 1\u201349.","journal-title":"Computing Research Repository"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.5555\/2967390"},{"key":"e_1_3_2_81_2","volume-title":"Systems of Systems as Communicating Structures","author":"Kotov V.","year":"1997","unstructured":"V. Kotov. 1997. Systems of Systems as Communicating Structures. Technical Report. Hewlett-Packard Company."},{"key":"e_1_3_2_82_2","first-page":"322","article-title":"PRISM 2.0: A tool for probabilistic model checking","author":"Kwiatkowska M.","year":"2004","unstructured":"M. Kwiatkowska, G. Norman, and D. Parker. 2004. PRISM 2.0: A tool for probabilistic model checking. 1st International Conference on the Quantitative Evaluation of Systems (QEST). 322\u2013323.","journal-title":"1st International Conference on the Quantitative Evaluation of Systems (QEST)"},{"key":"e_1_3_2_83_2","first-page":"14","volume-title":"What is a System of Systems and Why Should I Care?","author":"Lane J.","year":"2013","unstructured":"J. Lane. 2013. What is a System of Systems and Why Should I Care?Technical Report. Department of Industrial and Systems Engineering - University of Southern California. 14 pages."},{"key":"e_1_3_2_84_2","first-page":"1103","volume-title":"2nd Nordic Software Architecture Workshop (NOSA)","author":"Lassing N.","year":"1999","unstructured":"N. Lassing, D. Rijsenbrij, and H. van Vliet. 1999. On software architecture analysis of flexibility, complexity of changes: Size isn\u2019t everything. In 2nd Nordic Software Architecture Workshop (NOSA). 1103\u20131581."},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIII.2013.6703521"},{"key":"e_1_3_2_86_2","first-page":"15","volume-title":"Conference of the Centre for Advanced Studies on Collaborative Research (CASCON)","author":"Lung C.","year":"1997","unstructured":"C. Lung, S. Bot, K. Kalaichelvan, and R. Kazman. 1997. An approach to software architecture analysis for evolution and reusability. In Conference of the Centre for Advanced Studies on Collaborative Research (CASCON). 15\u201326."},{"key":"e_1_3_2_87_2","volume-title":"Systems of Systems","author":"Luzeaux D.","year":"2010","unstructured":"D. Luzeaux and J. Ruault. 2010. Systems of Systems. Wiley."},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1002\/sys.21272"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1520-6858(1998)1:4<267::AID-SYS3>3.0.CO;2-D"},{"issue":"3","key":"e_1_3_2_90_2","first-page":"305","article-title":"The emerging joint system of systems: A systems engineering challenge and opportunity for APL","volume":"17","author":"Manthorpe W.","year":"1996","unstructured":"W. Manthorpe. 1996. The emerging joint system of systems: A systems engineering challenge and opportunity for APL. The Emerging Joint System of Systems 17, 3 (1996), 305\u2013313.","journal-title":"The Emerging Joint System of Systems"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/bxz028"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3025934"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICACSIS51025.2020.9263119"},{"key":"e_1_3_2_94_2","first-page":"1","volume-title":"Nordic Workshop on Software Architecture","author":"Molter G.","year":"1999","unstructured":"G. Molter. 1999. Integrating SAAM in domain-centric and reuse-based development processes. In Nordic Workshop on Software Architecture. 1\u201310."},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/2794381"},{"key":"e_1_3_2_96_2","volume-title":"A Structured Approach for Reviewing Architecture Documentation","author":"Nord R.","year":"2009","unstructured":"R. Nord, P. Clements, D. Emery, and R. Hilliard. 2009. A Structured Approach for Reviewing Architecture Documentation. Technical Report. Software Engineering Institute (SEI)."},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSTEMS.2010.5482478"},{"key":"e_1_3_2_98_2","first-page":"89","volume-title":"17th Artificial Reality and Telexistence (ICAT)","author":"Oliveira M.","year":"2007","unstructured":"M. Oliveira and J. Pereira. 2007. Extensible virtual environment systems using system of systems engineering approach. In 17th Artificial Reality and Telexistence (ICAT). 89\u201396."},{"key":"e_1_3_2_99_2","unstructured":"OMG. 2012. OMG Systems Modeling Language (OMG SysML) Version 1.3."},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2013.01.026"},{"key":"e_1_3_2_101_2","first-page":"967","volume-title":"9th International Conference on Computing for Sustainable Global Development (INDIACom)","author":"Patidar A.","year":"2015","unstructured":"A. Patidar and U. Suman. 2015. A survey on software architecture evaluation methods. In 9th International Conference on Computing for Sustainable Global Development (INDIACom). 967\u2013972."},{"issue":"5","key":"e_1_3_2_102_2","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1002\/sys.21551","article-title":"A network perspective on assessing system architectures: Robustness to cascading failure","volume":"23","author":"Potts M.","year":"2020","unstructured":"M. Potts, P. Sartor, A. Johnson, and S. Bullock. 2020. A network perspective on assessing system architectures: Robustness to cascading failure. Systems Engineering 23, 5 (2020), 597\u2013616.","journal-title":"Systems Engineering"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.5555\/1400067.1400069"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.09.297"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.09.298"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(02)00080-8"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-137-32803-8"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.5555\/2072649"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.09.116"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.07.021"},{"key":"e_1_3_2_111_2","first-page":"107072","article-title":"Aircraft architecture and fleet assessment framework for urban air mobility using a system of systems approach","author":"Prakasha P. Shiva","year":"2021","unstructured":"P. Shiva Prakasha, N. Naeem, P. Ratei, and B. Nagel. 2021. Aircraft architecture and fleet assessment framework for urban air mobility using a system of systems approach. Aerospace Science and Technology (2021), 107072.","journal-title":"Aerospace Science and Technology"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSCON.2018.8369581"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSTEMS.2009.4815801"},{"key":"e_1_3_2_114_2","first-page":"1","volume-title":"10th International Conference on Software Engineering and Knowledge Engineering (SEKE)","author":"Spitznagel B.","year":"1989","unstructured":"B. Spitznagel and D. Garlan. 1989. Architecture-based performance analysis. In 10th International Conference on Software Engineering and Knowledge Engineering (SEKE). 1\u20136."},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/AERO.2009.4839620"},{"key":"e_1_3_2_116_2","first-page":"581","volume-title":"Toward a Research Agenda for M&S Support of System of Systems Engineering","author":"Tolk A.","year":"2015","unstructured":"A. Tolk and L. Rainey. 2015. Toward a Research Agenda for M&S Support of System of Systems Engineering. Wiley, 581\u2013592."},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2010.5544073"},{"key":"e_1_3_2_118_2","volume-title":"NATO Code of Best Practice for C2 Assessment","author":"S. U.","year":"2002","unstructured":"U. S.2002. NATO Code of Best Practice for C2 Assessment. Technical Report. CCRP Washington, D.C."},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2013.22"},{"key":"e_1_3_2_120_2","first-page":"177","volume-title":"Reliable and Smart Decision Support System for Emergency Management Based on Crowdsourcing Information","author":"Villela K.","year":"2018","unstructured":"K. Villela, C. Nass, R. Novais, P. Sim\u00f5es, A. Traina, J. Rodrigues, J. Menendez, J. Kurano, T. Franke, and A. Poxrucker. 2018. Reliable and Smart Decision Support System for Emergency Management Based on Crowdsourcing Information. Springer International Publishing, Cham, 177\u2013198."},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1109\/SYSOSE.2011.5966566"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2005.09.004"},{"key":"e_1_3_2_123_2","first-page":"1","volume-title":"Realising Network Enabled Capability","author":"Webster D.","year":"2008","unstructured":"D. Webster, N. Looker, D. Russell, L. Liu, and J. Xu. 2008. An ontology for evaluation of network enabled capability. In Realising Network Enabled Capability. 1\u201310."},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_125_2","doi-asserted-by":"crossref","first-page":"55","DOI":"10.21236\/ADA460414","volume-title":"Attribute-driven Design (ADD)","author":"Wojcik R.","year":"2006","unstructured":"R. Wojcik, F. Bachmann, L. Bass, P. Clements, P. Merson, R. Nord, and B. Wood. 2006. Attribute-driven Design (ADD). Technical Report. Software Engineering Institute, Carnegie Mellon University, 55 pages."},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICECCS.2013.18"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-75132-8_8"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.4304\/jsw.7.1.169-175"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-79588-9_11"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3519020","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3519020","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:20Z","timestamp":1750191140000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3519020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,21]]},"references-count":128,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3519020"],"URL":"https:\/\/doi.org\/10.1145\/3519020","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,21]]},"assertion":[{"value":"2021-03-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-02-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}