{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T17:17:03Z","timestamp":1765041423175,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"U.S. Government","award":["W56KGU-18-D-0004","52.216-22"],"award-info":[{"award-number":["W56KGU-18-D-0004","52.216-22"]}]},{"name":"Department of Defense FAR","award":["W56KGU-18-D-0004","52.216-22"],"award-info":[{"award-number":["W56KGU-18-D-0004","52.216-22"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Autonomous and AI-enabled systems present a challenge for integration within the System of Sys-tems (SoS) paradigm. A full system of systems (SoS) testbed is necessary to verify the integrity of a given system and preserve the modularization and accountability of its constituent systems. This integrated system needs to support iterative, continuous testing and development. This need war-rants the development of a virtual environment that provides the ground truth in a simulated sce-nario, interfaces with real-world data, and uses various domain-specific and domain-agnostic simulation systems for development, testing, and evaluation. These required features present a non-trivial challenge wherein constructive models and systems at different levels of fidelity need to interoperate to advance the testing, evaluation, and integration of complex systems. Such a virtual and constructive SoS architecture should be independent of the underlying computational infra-structure but must be cloud-enabled for wider integration of AI-enabled software components. This paper will present a modular Simulation, Experimentation, Analytics, and Test (SEAT) Lay-ered Architecture Framework, a 10-step methodology. This paper will also demonstrate a case study of a hybrid cloud-enabled simulation SoS that allows extensibility, composability, and de-ployability in different target environments.<\/jats:p>","DOI":"10.3390\/info14100528","type":"journal-article","created":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T07:50:26Z","timestamp":1695887426000},"page":"528","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Providing a User Extensible Service-Enabled Multi-Fidelity Hybrid Cloud-Deployable SoS Test and Evaluation (T&amp;E) Infrastructure: Application of Modeling and Simulation (M&amp;S) as a Service (MSaaS)"],"prefix":"10.3390","volume":"14","author":[{"given":"Saurabh","family":"Mittal","sequence":"first","affiliation":[{"name":"The MITRE Corporation, 7515 Colshire Dr, McLean, VA 22102, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert L.","family":"Wittman","sequence":"additional","affiliation":[{"name":"The MITRE Corporation, 7515 Colshire Dr, McLean, VA 22102, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Gibson","sequence":"additional","affiliation":[{"name":"The MITRE Corporation, 7515 Colshire Dr, McLean, VA 22102, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josh","family":"Huffman","sequence":"additional","affiliation":[{"name":"The MITRE Corporation, 7515 Colshire Dr, McLean, VA 22102, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hans","family":"Miller","sequence":"additional","affiliation":[{"name":"The MITRE Corporation, 7515 Colshire Dr, McLean, VA 22102, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,28]]},"reference":[{"key":"ref_1","unstructured":"David, R., and Nielsen, P. (2016). Summer Study on Autonomy."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mittal, S., Durak, U., and Oren, T. (2017). Guide to Simulation-Based Disciplines: Advancing Our Computational Future, Springer.","DOI":"10.1007\/978-3-319-61264-5"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"DeLaurentis, D.A., Moolchandani, K., and Guariniello, C. (2022). System of Systems Modeling and Analysis, CRC Press. [1st ed.].","DOI":"10.1201\/9781003231011"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jamshidi, M. (2008). System of Systems Engineering for 21st Century, Wiley & Sons.","DOI":"10.1002\/9780470403501"},{"key":"ref_5","unstructured":"Muehlen, M., Hamilton, D., and Peak, R. (2023, May 20). Integration of M&S (Modeling and Simulation), Software Design and DoDAF (Department of Defense Architecture Framework RT 24), A013\u2013Final Technical Report SERC-2012-TR-024, Systems Engineering Research Center. Available online: https:\/\/apps.dtic.mil\/sti\/pdfs\/ADA582069.pdf."},{"key":"ref_6","unstructured":"Mittal, S., and Martin, J.L.R. (2017, January 23\u201326). DEVSML 3.0: Rapid Deployment of DEVS Farm in Cloud Environment using Microservices and Containers. Proceedings of the Symposium on Theory of Modeling and Simulation, Spring Simulation Multi-Conference, Virginia Beach, VA, USA."},{"key":"ref_7","unstructured":"Air Force (2023, May 20). Platform One. Available online: https:\/\/software.af.mil\/team\/platformone\/."},{"key":"ref_8","unstructured":"(2023, May 20). Amazon Cloud Services . Available online: https:\/\/aws.amazon.com\/govcloud-us\/?whats-new-ess.sort-by=item.additionalFields.postDateTime&whats-new-ess.sort-order=desc."},{"key":"ref_9","unstructured":"Henninger, A. (2023, May 20). Strategic Implications of Cloud Computing for Modeling and Simulation (Briefing), IDA. Available online: https:\/\/apps.dtic.mil\/sti\/pdfs\/AD1018235.pdf."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mittal, S., Kasdaglis, N., Harrell, L., Wittman, R., Gibson, J., and Rocca, D. (2020, January 14\u201318). Autonomous and Composable M&S System of Systems with the Simulation, Experimentation, Analytics and Testing (SEAT) Framework. Proceedings of the Winter Simulation Conference, Orlando, FL, USA.","DOI":"10.1109\/WSC48552.2020.9384040"},{"key":"ref_11","unstructured":"(2023, May 20). Simulation Interoperability Standards Organization. Available online: http:\/\/www.sisostds.org."},{"key":"ref_12","unstructured":"IEEE Standard for Distributed Interactive Simulation\u2014Application Protocols. Standard No. IEEE Standard 1278.1-2012. Available online: https:\/\/standards.ieee.org\/ieee\/1278.1\/4949\/."},{"key":"ref_13","unstructured":"(2020). Framework and Rules, High Level Architecture Federation Rules. Standard No. IEEE Std 1516-2020."},{"key":"ref_14","unstructured":"(2010). Federate Interface Specification, The Runtime Infrastructure (RTI) Service Specification. Standard No. IEEE Std 1516.1-2010."},{"key":"ref_15","unstructured":"(2010). Object Model Template Specification The HLA Object Model Format Specification. Standard No. IEEE Std 1516.2-2010."},{"key":"ref_16","unstructured":"(2010). Distributed Simulation Engineering and Execution Process. Standard No. IEEE Std 1730."},{"key":"ref_17","unstructured":"Turrel, C., Brown, R., Igarza, J.L., Pixius, K., Renda, F., and Rouget, C. (2004). Federation Development and Execution Process (FEDEP) Tools in Support for NATO Modelling & Simulation (M&S) Programmes, NATO Research and Technology Agency. Available online: https:\/\/apps.dtic.mil\/dtic\/tr\/fulltext\/u2\/a429045.pdf."},{"key":"ref_18","unstructured":"(2020). Standard for Command-and-Control Systems- Simulation Systems Interoperation. Standard No. SISO-STD-019-2020."},{"key":"ref_19","unstructured":"(2023, May 20). Test and Training Enabling Architecture. Available online: http:\/\/tena-sda.org."},{"key":"ref_20","unstructured":"Test Resource Management Center (2023, May 20). Test and Training Enabling Architecture Overview. Available online: https:\/\/www.trmc.osd.mil\/tena-home.html."},{"key":"ref_21","first-page":"115","article-title":"Containerization of high level architecture-based simulations: A Case Study","volume":"14","author":"Siegel","year":"2016","journal-title":"J. Def. Model. Simul."},{"key":"ref_22","unstructured":"Kewley, R., McGroarty, G., McDonnell, J., Gallant, S., and Diemunsch, J. (2023, May 20). Cloud-Based Modeling and Simulation Study Group. Available online: https:\/\/www.sisostds.org\/StandardsActivities\/StudyGroups\/CBMSSG.aspx."},{"key":"ref_23","unstructured":"Hannay, J.E., and van den Berg, T. (2023, May 20). The NATO MSG-136 Reference Architecture for M&S as a Service. STO-MP-MSG-149. Available online: https:\/\/www.sto.nato.int\/publications\/STO%20Meeting%20Proceedings\/STO-MP-MSG-149\/MP-MSG-149-03.pdf."},{"key":"ref_24","unstructured":"Modelling and Simulation as a Service, Volume 2: MSaaS Discover Service and Metadata; TR-MSG-136-Part-V. Standard No. NATO MSG-136. Available online: https:\/\/apps.dtic.mil\/sti\/pdfs\/AD1078382.pdf."},{"key":"ref_25","unstructured":"Bucher, N. (2013, January 28\u201331). \u201cAlways On\u2013On Demand\u201d: Supporting the Development, Test and Training of Operational Networks & Net-Centric Systems. Proceedings of the NDIA 16th Annual Systems Engineering Conference, Hyatt Regency, Crystal City, VA, USA."},{"key":"ref_26","unstructured":"(2023, May 20). US Army Futures and Concepts Center, Battle Lab Collaborative Simulation Environment (BLCSE) Overview. Available online: https:\/\/www.erdc.usace.army.mil\/Media\/News-Stories\/Article\/2782269\/battle-lab-collaborative-simulation-environment-offers-erdc-new-capabilities\/2021."},{"key":"ref_27","unstructured":"(2023, May 20). Open API 3.0. Available online: http:\/\/openapis.org."},{"key":"ref_28","unstructured":"(2023, May 20). Swagger Toolset. Available online: http:\/\/swagger.io."},{"key":"ref_29","unstructured":"(2023, May 20). Async API. Available online: http:\/\/asyncapis.com."},{"key":"ref_30","unstructured":"OUSD (2020). DOD Instructure 5000.87: Operation of the Software Acquisition Pathway, under Secretary of Defense for Acquisition and Sustainment."},{"key":"ref_31","unstructured":"Bond, J., Rocca, D., Foote, A., Gibson, J., and Dalphond, J. (2023, May 20). Symphony Documentation. Available online: https:\/\/symphonydocs.mitre.org\/."},{"key":"ref_32","unstructured":"Symphony Development Team (2023, May 20). Symphony Core. Available online: https:\/\/git.codev.mitre.org\/projects\/SYMPHONYDEV\/repos\/symphony-core\/."},{"key":"ref_33","unstructured":"Symphony Team (2023, May 20). Internal Symphony Website. Available online: https:\/\/symphony.mitre.org\/."},{"key":"ref_34","unstructured":"McKeown, D. (2023, May 20). Memorandum for Senior Pentagon Leadership Defense Agency and DoD Field Activity Directors, Subject: Continuous Authorization to Operate (cATO). Available online: https:\/\/mitre.sharepoint.com\/sites\/straighttalkforgov\/Shared%20Documents\/General\/CONTINUOUS-AUTHORIZATION-TO-OPERATE.PDF."},{"key":"ref_35","unstructured":"Google (2023, May 20). Protocol Buffers. Available online: https:\/\/developers.google.com\/protocol-buffers."},{"key":"ref_36","unstructured":"(2023, May 20). EMQX. Available online: https:\/\/www.emqx.io\/."},{"key":"ref_37","unstructured":"(2023, May 20). MQTT. Available online: http:\/\/mqtt.org."},{"key":"ref_38","unstructured":"(2023, May 20). Airflow. Available online: http:\/\/airflow.apache.org."},{"key":"ref_39","unstructured":"Wilensky, U., and Rand, B. (2015). Introduction to Agent-Based Modeling: Modeling Natural, Social and Engineering Complex Systems with NetLogo, MIT Press."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/10\/528\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:00:55Z","timestamp":1760130055000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/10\/528"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":39,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["info14100528"],"URL":"https:\/\/doi.org\/10.3390\/info14100528","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2023,9,28]]}}}