{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T12:18:14Z","timestamp":1784204294819,"version":"3.55.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T00:00:00Z","timestamp":1707868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>We are increasingly surrounded by systems connecting us with the digital world and facilitating our life by supporting our work, leisure, activities at home, health, and so on. These systems are pressed by two forces. On the one side, they operate in environments that are increasingly challenging due to uncertainty and uncontrollability. On the other side, they need to evolve, often in a continuous fashion, to meet changing needs, to offer new functionalities, or also to fix emerging failures. To make the picture even more complex, these systems rarely work in isolation and often need to collaborate with other systems, as well as humans. All such facets call for moving their validation during operation, as offered by approaches called testing in the field.<\/jats:p>\n          <jats:p>\n            In this article, we observe that even the field-based testing approaches should change over time to follow and adapt to the changes and evolution of collaborating systems or environments or users\u2019 behaviors. We provide a taxonomy of this new category of testing that we call\n            <jats:bold>self-adaptive testing in the field (SATF)<\/jats:bold>\n            , together with a reference architecture for SATF approaches. To achieve this objective, we surveyed the literature and collected feedback and contributions from experts in the domain via a questionnaire and interviews.\n          <\/jats:p>","DOI":"10.1145\/3627163","type":"journal-article","created":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T15:25:20Z","timestamp":1697037920000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Self-Adaptive Testing in the Field"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1124-0347","authenticated-orcid":false,"given":"Samira","family":"Silva","sequence":"first","affiliation":[{"name":"Gran Sasso Science Institute (GSSI), Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5438-2281","authenticated-orcid":false,"given":"Patrizio","family":"Pelliccione","sequence":"additional","affiliation":[{"name":"Gran Sasso Science Institute (GSSI), Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8749-1356","authenticated-orcid":false,"given":"Antonia","family":"Bertolino","sequence":"additional","affiliation":[{"name":"ISTI-CNR, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,2,14]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"ISO\/IEC\/IEEE 42010:2011. 2011. Systems and software engineering\u2014Architecture description 37 pages."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2004.2"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-85347-1_3"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/1882362.1882367"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372785"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.25"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2011.227"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447240"},{"key":"e_1_3_2_10_2","first-page":"86","volume-title":"International Conference on Web Information Systems and Technologies","author":"Bertolino Antonia","year":"2012","unstructured":"Antonia Bertolino, Guglielmo De Angelis, and Andrea Polini. 2012. Governance policies for verification and validation of service choreographies. In International Conference on Web Information Systems and Technologies. Springer, 86\u2013102."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30985-5_5"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-022-01022-z"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(02)00108-8"},{"key":"e_1_3_2_14_2","first-page":"77","volume-title":"Proceedings of the 11th Annual International Workshop on Software Technology and Engineering Practice","author":"Cai Kai-Yuan","year":"2003","unstructured":"Kai-Yuan Cai, Jo\u00e3o W Cangussu, Raymond A DeCarlo, and Aditya P. Mathur. 2003. An overview of software cybernetics. In Proceedings of the 11th Annual International Workshop on Software Technology and Engineering Practice. IEEE, 77\u201386."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527233"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST46399.2020.00037"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02161-9_1"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1207\/s15327809jls1301_2"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2014.03.013"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2014.2312953"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4121205"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106482"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.06.063"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAMS.2015.15"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2593929.2593937"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAMS.2013.6595504"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST53961.2022.00017"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2017.10"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2009.9"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSREW.2019.00070"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICWR51868.2021.9443109"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/VACE.2017.5"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/FAS-W.2016.32"},{"key":"e_1_3_2_34_2","volume-title":"Designing an Effective Survey","author":"Kasunic Mark","year":"2005","unstructured":"Mark Kasunic. 2005. Designing an Effective Survey. Technical Report."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1160055"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2013.07.010"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2014.09.009"},{"key":"e_1_3_2_38_2","first-page":"1","article-title":"A survey on runtime testing of dynamically adaptable and distributed systems","author":"Lahami Mariam","year":"2021","unstructured":"Mariam Lahami and Moez Krichen. 2021. A survey on runtime testing of dynamically adaptable and distributed systems. Software Quality Journal (2021), 1\u201339.","journal-title":"Software Quality Journal"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2016.02.002"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2019.00076"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/SEAMS51251.2021.00017"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2994291.2994302"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/S-Cube.2012.6225511"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-018-0611-x"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2009.18"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2020.102414"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1097\/XEB.0000000000000050"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1725"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503229.3547048"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICECA.2017.8212749"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/1516533.1516538"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/2988336.2988348"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.5555\/1324786"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524844.3528050"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993288.2993294"},{"key":"e_1_3_2_56_2","first-page":"284","volume-title":"Proceedings of the 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)","author":"Siqueira Bento R.","year":"2020","unstructured":"Bento R. Siqueira, Fabiano C. Ferrari, Kathiani E. Souza, Daniel S. M. Santib\u00e1\u00f1ez, and Valter V. Camargo. 2020. Fault types of adaptive and context-aware systems and their relationship with fault-based testing approaches. In Proceedings of the 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 284\u2013293."},{"key":"e_1_3_2_57_2","article-title":"STANDARD 610-1990,","author":"STANDARD I","year":"1990","unstructured":"I STANDARD. 1990. STANDARD 610-1990,. IEEE Standard Computer Dictionary: Compilation of IEEE Standard Computer Glossaries (1990).","journal-title":"IEEE Standard Computer Dictionary: Compilation of IEEE Standard Computer Glossaries"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-29044-2"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3627163","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3627163","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:45:39Z","timestamp":1750178739000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3627163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,14]]},"references-count":58,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3627163"],"URL":"https:\/\/doi.org\/10.1145\/3627163","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"value":"1556-4665","type":"print"},{"value":"1556-4703","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,14]]},"assertion":[{"value":"2022-10-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}