{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T14:02:59Z","timestamp":1762956179679,"version":"3.41.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,3,31]],"date-time":"2019-03-31T00:00:00Z","timestamp":1553990400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"German Research Foundation (DFG) in the context of the project CYPHOC","award":["HA 5480\/3-1,SI 674\/9-1"],"award-info":[{"award-number":["HA 5480\/3-1,SI 674\/9-1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2019,3,31]]},"abstract":"<jats:p>Self-adaptation has been proposed as a mechanism to counter complexity in control problems of technical systems. A major driver behind self-adaptation is the idea to transfer traditional design-time decisions to runtime and into the responsibility of systems themselves. To deal with unforeseen events and conditions, systems need creativity\u2014typically realized by means of machine learning capabilities. Such learning mechanisms are based on different sources of knowledge. Feedback from the environment used for reinforcement purposes is probably the most prominent one within the self-adapting and self-organizing (SASO) systems community. However, the impact of other (sub-)systems on the success of the individual system\u2019s learning performance has mostly been neglected in this context.<\/jats:p>\n          <jats:p>In this article, we propose a novel methodology to identify effects of actions performed by other systems in a shared environment on the utility achievement of an autonomous system. Consider smart cameras (SC) as illustrating example: For goals such as 3D reconstruction of objects, the most promising configuration of one SC in terms of pan\/tilt\/zoom parameters depends largely on the configuration of other SCs in the vicinity. Since such mutual influences cannot be pre-defined for dynamic systems, they have to be learned at runtime. Furthermore, they have to be taken into consideration when self-improving their own configuration decisions based on a feedback loop concept, e.g., known from the SASO domain or the Autonomic and Organic Computing initiatives.<\/jats:p>\n          <jats:p>We define a methodology to detect such influences at runtime, present an approach to consider this information in a reinforcement learning technique, and analyze the behavior in artificial as well as real-world SASO system settings.<\/jats:p>","DOI":"10.1145\/3345319","type":"journal-article","created":{"date-parts":[[2019,9,12]],"date-time":"2019-09-12T14:22:21Z","timestamp":1568298141000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Mutual Influence-aware Runtime Learning of Self-adaptation Behavior"],"prefix":"10.1145","volume":"14","author":[{"given":"Stefan","family":"Rudolph","sequence":"first","affiliation":[{"name":"University of Augsburg, Augsburg, Germany"}]},{"given":"Sven","family":"Tomforde","sequence":"additional","affiliation":[{"name":"University of Kassel, Kassel, Germany"}]},{"given":"J\u00f6rg","family":"H\u00e4hner","sequence":"additional","affiliation":[{"name":"University of Augsburg, Augsburg, Germany"}]}],"member":"320","published-online":{"date-parts":[[2019,9,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1160055"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Christian M\u00fcller-Schloer and Sven Tomforde. 2018. Organic Computing\u2014Technical Systems for Survival in the Real World. Springer International Publishing.  Christian M\u00fcller-Schloer and Sven Tomforde. 2018. Organic Computing\u2014Technical Systems for Survival in the Real World. Springer International Publishing.","DOI":"10.1007\/978-3-319-68477-2"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/332833.332837"},{"key":"e_1_2_1_4_1","unstructured":"Michael Wooldridge. An Introduction to Multiagent Systems. John Wiley 8 Sons.   Michael Wooldridge. An Introduction to Multiagent Systems. John Wiley 8 Sons."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00287-014-0827-z"},{"key":"e_1_2_1_6_1","unstructured":"Serge Kernbach Thomas Schmickl and Jonathan Timmis. Collective adaptive systems: Challenges beyond evolvability. Arxiv abs\/1108.5643 ({n.d.}).  Serge Kernbach Thomas Schmickl and Jonathan Timmis. Collective adaptive systems: Challenges beyond evolvability. Arxiv abs\/1108.5643 ({n.d.})."},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Danny Weyns Bradley Schmerl Vincenzo Grassi Sam Malek Raffaela Mirandola Christian Prehofer Jochen Wuttke Jesper Andersson Holger Giese and Karl M. G\u00f6schka. 2013. On patterns for decentralized control in self-adaptive systems. In Software Engineering for Self-Adaptive Systems II. Springer 76--107.  Danny Weyns Bradley Schmerl Vincenzo Grassi Sam Malek Raffaela Mirandola Christian Prehofer Jochen Wuttke Jesper Andersson Holger Giese and Karl M. G\u00f6schka. 2013. On patterns for decentralized control in self-adaptive systems. In Software Engineering for Self-Adaptive Systems II. Springer 76--107.","DOI":"10.1007\/978-3-642-35813-5_4"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/2593416.2593421"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2014.09.009"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1520-6858(1998)1:4<267::AID-SYS3>3.0.CO;2-D"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2016.16"},{"volume-title":"Proceedings of the 27th International Conference on Architecture of Computing Systems, Workshop (ARCS\u201914)","author":"Tomforde S.","key":"e_1_2_1_12_1"},{"volume-title":"Proceedings of the IEEE International Conference on Autonomic Computing (ICAC\u201916)","author":"Tomforde Sven","key":"e_1_2_1_13_1"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2659021.2659052"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2015.2426575"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2017.9"},{"volume-title":"Proceedings of the IEEE 3rd International Workshops on Foundations and Applications of Self<sup>*<\/sup> Systems (FAS<sup>*<\/sup>W\u201918)","author":"Bellman Kirstie L.","key":"e_1_2_1_17_1"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/FAS-W.2019.00014"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-30695-7_25"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.5220\/0005697001810189"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2015.23"},{"volume-title":"Proceedings of the 7th International Conference on Informatics in Control, Automation, and Robotics (ICINCO\u201910)","author":"Tomforde S.","key":"e_1_2_1_22_1"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02704-8_2"},{"key":"e_1_2_1_24_1","unstructured":"Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning (1st ed.). The MIT Press Cambridge MA.   Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning (1st ed.). The MIT Press Cambridge MA."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/882477.883624"},{"key":"e_1_2_1_26_1","unstructured":"Axel van Lamsweerde. Requirements Engineering: From System Goals to UML Models to Software Specifications (1st ed.). Wiley Publishing.   Axel van Lamsweerde. Requirements Engineering: From System Goals to UML Models to Software Specifications (1st ed.). Wiley Publishing."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2593929.2593937"},{"volume-title":"Chapter: Organic Network Control\u2014Turning Standard Protocols into Evolving Systems, 11--35.","year":"2011","author":"Tomforde S.","key":"e_1_2_1_28_1"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2007.913919"},{"key":"e_1_2_1_30_1","unstructured":"Marco Wiering and Martijn van Otterlo. Reinforcement Learning: State-of-the-Art. Springer Publishing Company Incorporated.   Marco Wiering and Martijn van Otterlo. Reinforcement Learning: State-of-the-Art. Springer Publishing Company Incorporated."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"volume-title":"Proceedings of the 28th International Conference on Architecture of Computing Systems (ARCS'15)","author":"Rudolph S.","key":"e_1_2_1_32_1"},{"volume-title":"Proceedings of the 12th IEEE International Workshops on Enabling Technologies (WETICE\u201903)","author":"Keil David","key":"e_1_2_1_33_1"},{"volume-title":"Proceedings of the IADIS European Conference on Data Mining, Ajith Abraham (Ed.). IADIS, 97--101","author":"Logie Robert","key":"e_1_2_1_34_1"},{"key":"e_1_2_1_35_1","article-title":"Investigating agent influence and nested other-agent behaviour","volume":"2","author":"Logie Robert","year":"2010","journal-title":"Int. J. Adv. Intell. Syst."},{"volume":"3","volume-title":"Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201910)","author":"Broersen Jan M.","key":"e_1_2_1_36_1"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008942012299"},{"volume-title":"Proceedings of the International Conference on Advanced Robotics (ICAR\u201903)","year":"2003","author":"Kok Jelle R.","key":"e_1_2_1_38_1"},{"volume-title":"Proceedings of the IEEE Symposium on Computational Intelligence and Games (CIG\u201905)","year":"2005","author":"Kok Jelle R.","key":"e_1_2_1_39_1"},{"volume-title":"Proceedings of the 20th Belgian-Netherlands Conference on Artificial Intelligence. 83--90","year":"2009","author":"De Hauwere Yann-Micha\u00ebl","key":"e_1_2_1_40_1"},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems","volume":"1","author":"De Hauwere Yann-Michael","year":"2010"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201911)","volume":"3","author":"De Hauwere Yann-Micha\u00ebl","year":"2011"},{"volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems. 4190--4203","year":"2017","author":"Lanctot Marc","key":"e_1_2_1_43_1"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.159"},{"volume-title":"Proceedings of the 22nd Conference on Advances in Neural Information Processing Systems. 441--448","author":"Farahmand Amir Massoud","key":"e_1_2_1_45_1"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553442"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11633-015-0893-y"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2017.08.038"},{"volume-title":"Proceedings of the 30th GI\/ITG International Conference on Architecture of Computing Systems (ARCS\u201917)","author":"Rudolph Stefan","key":"e_1_2_1_49_1"},{"key":"e_1_2_1_50_1","doi-asserted-by":"crossref","unstructured":"Karl Pearson. Note on regression and inheritance in the case of two parents. 58 347--352 ({n.d.}) 240--242.  Karl Pearson. Note on regression and inheritance in the case of two parents. 58 347--352 ({n.d.}) 240--242.","DOI":"10.1098\/rspl.1895.0041"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/30.1-2.81"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1214\/009053607000000505"},{"key":"e_1_2_1_53_1","unstructured":"Claude Shannon and Warren Weaver. The Mathematical Theory of Communication. University of Illinois Press.   Claude Shannon and Warren Weaver. The Mathematical Theory of Communication. University of Illinois Press."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.066138"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1205438"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1177\/0037549705058073"},{"key":"e_1_2_1_57_1","unstructured":"Matthew E. Taylor and Peter Stone. 2009. Transfer learning for reinforcement learning domains: A survey. J. Machine Learn. Res. 10 (July 2009) 1633--1685.   Matthew E. Taylor and Peter Stone. 2009. Transfer learning for reinforcement learning domains: A survey. J. Machine Learn. Res. 10 (July 2009) 1633--1685."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1162\/evco.1995.3.2.149"},{"key":"e_1_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Stewart W. Wilson. 2000. Get real! XCS with continuous-valued inputs. In Learning Classifier Systems Pier Luca Lanzi Wolfgang Stolzmann and Stewart W. Wilson (Eds.). Springer Berlin 209--219.   Stewart W. Wilson. 2000. Get real! XCS with continuous-valued inputs. In Learning Classifier Systems Pier Luca Lanzi Wolfgang Stolzmann and Stewart W. Wilson (Eds.). Springer Berlin 209--219.","DOI":"10.1007\/3-540-45027-0_11"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908812.2908859"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2017.02.004"},{"key":"e_1_2_1_62_1","unstructured":"Wolfgang H\u00e4rdle and L\u00e9opold Simar. 2007. Applied Multivariate Statistical Analysis. Vol. 22007. Springer.  Wolfgang H\u00e4rdle and L\u00e9opold Simar. 2007. Applied Multivariate Statistical Analysis. Vol. 22007. Springer."}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3345319","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3345319","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:19Z","timestamp":1750201999000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3345319"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,31]]},"references-count":62,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,3,31]]}},"alternative-id":["10.1145\/3345319"],"URL":"https:\/\/doi.org\/10.1145\/3345319","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"type":"print","value":"1556-4665"},{"type":"electronic","value":"1556-4703"}],"subject":[],"published":{"date-parts":[[2019,3,31]]},"assertion":[{"value":"2018-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}