{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T13:38:00Z","timestamp":1776519480404,"version":"3.51.2"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>Communication is essential for successful interaction. In Human\u2013Robot Interaction (HRI), implicit communication holds the potential to enhance robots\u2019 understanding of human needs, emotions and intentions. This article introduces a method to foster implicit communication in HRI without explicitly modelling human intentions or relying on pre-existing knowledge. Leveraging Transfer Entropy, we modulate influence between agents in social interactions in scenarios involving either collaboration or competition. By integrating influence into agents\u2019 rewards within a partially observable Markov decision process, we demonstrate that boosting influence enhances collaboration and interaction, while resisting influence promotes social independence and diminishes performance in certain scenarios. Our findings are validated through simulations and real-world experiments with human participants in social navigation and autonomous driving settings.<\/jats:p>","DOI":"10.1145\/3800955","type":"journal-article","created":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T15:21:27Z","timestamp":1772896887000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Influence-Based Reward Modulation for Implicit Communication in Human\u2013Robot Interaction"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-5779-8238","authenticated-orcid":false,"given":"Haoyang","family":"Jiang","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9639-5291","authenticated-orcid":false,"given":"Elizabeth","family":"Croft","sequence":"additional","affiliation":[{"name":"University of Victoria, Victoria, British Columbia, Canada and Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7426-1498","authenticated-orcid":false,"given":"Michael","family":"Burke","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, Australia and The University of Edinburgh, Edinburgh, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,18]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-007-0094-5"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.2307\/2234532"},{"key":"e_1_3_3_4_2","unstructured":"Seung Ki Baek Woo-Sung Jung Okyu Kwon and Hie-Tae Moon. 2005. Transfer entropy analysis of the stock market. arXiv:physics\/0509014. Retrieved from https:\/\/arxiv.org\/abs\/physics\/0509014"},{"key":"e_1_3_3_5_2","volume-title":"Dynamic Programming","author":"Bellman Richard E.","year":"1957","unstructured":"Richard E. Bellman. 1957. Dynamic Programming. Princeton University Press, Princeton, NJ."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/HUMANOIDS.2014.7041459"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-43222-9_4"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2005.1545011"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-017-0400-4"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2020.2964824"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559817"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2013.6483603"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0169734"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511686948"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2018.00065"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.4135\/9781412983419"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.2307\/1912791"},{"key":"e_1_3_3_18_2","first-page":"1861","volume-title":"Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning. Jennifer Dy and Andreas Krause (Eds.), Proceedings of Machine Learning Research, Vol. 80, PMLR, 1861\u20131870."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.physa.2017.04.089"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.51.4282"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","unstructured":"Nicholas J. Hetherington Elizabeth A. Croft and H. F. Machiel Van der Loos. 2021. Hey robot which way are you going? Nonverbal motion legibility cues for human-robot spatial interaction. IEEE Robotics and Automation Letters 6 3 (2021) 5010\u20135015. DOI: 10.1109\/LRA.2021.3068708","DOI":"10.1109\/LRA.2021.3068708"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1063\/5.0149995"},{"key":"e_1_3_3_23_2","first-page":"3040","volume-title":"Proceedings of the 36th International Conference on Machine Learning","volume":"97","author":"Jaques Natasha","year":"2019","unstructured":"Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro Ortega, Dj Strouse, Joel Z. Leibo, and Nando De Freitas. 2019. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning. Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.), Proceedings of Machine Learning Research, Vol. 97, PMLR, 3040\u20133049."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610977.3634933"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11467-017-0689-3"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.3141\/1999-10"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2005.1554676"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.5555\/3091125.3091194"},{"key":"e_1_3_3_29_2","unstructured":"Edouard Leurent. 2018. An Environment for Autonomous Driving Decision-Making. Retrieved from https:\/\/github.com\/eleurent\/highway-env"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1063\/1.5132945"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMA.2014.6885705"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMA.2014.6885705"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2017.2647882"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300325"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2012.6343829"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.2.043250"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","unstructured":"Christoforos Mavrogiannis Francesca Baldini Allan Wang Dapeng Zhao Pete Trautman Aaron Steinfeld and Jean Oh. 2023. Core challenges of social robot navigation: A survey. Journal of Human-Robot Interaction 12 3 Article 36 (Apr. 2023) 39 pages. DOI: 10.1145\/3583741","DOI":"10.1145\/3583741"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_3_39_2","first-page":"2125","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS \u201915)","volume":"2","author":"Mohamed Shakir","year":"2015","unstructured":"Shakir Mohamed and Danilo J. Rezende. 2015. Variational information maximisation for intrinsically motivated reinforcement learning. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS \u201915), Vol. 2. MIT Press, Cambridge, MA, 2125\u20132133."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/12441.001.0001"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.26451\/abc.05.04.03.2018"},{"key":"e_1_3_3_42_2","first-page":"1","volume-title":"Proceedings of the 7th German Conference on Robotics (ROBOTIK \u201912)","author":"Roesmann Christoph","year":"2012","unstructured":"Christoph Roesmann, Wendelin Feiten, Thomas Woesch, Frank Hoffmann, and Torsten Bertram. 2012. Trajectory modification considering dynamic constraints of autonomous robots. In Proceedings of the 7th German Conference on Robotics (ROBOTIK \u201912). VDE-Verlag, Berlin, 1\u20136."},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.029"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","unstructured":"Thomas Schreiber. 2000. Measuring information transfer. Physical Review Letters 85 2 (July 2000) 461\u2013464. DOI: 10.1103\/PhysRevLett.85.461","DOI":"10.1103\/PhysRevLett.85.461"},{"key":"e_1_3_3_45_2","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv:1707.06347. Retrieved from https:\/\/arxiv.org\/abs\/1707.06347"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319502.3374812"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.3390\/e22101176"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"e_1_3_3_49_2","unstructured":"David Silver Thomas Hubert Julian Schrittwieser Ioannis Antonoglou Matthew Lai Arthur Guez Marc Lanctot L. Sifre Dharshan Kumaran Thore Graepel et\u00a0al. 2017. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv:1712.01815. Retrieved from https:\/\/arxiv.org\/abs\/1712.01815"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssci.2008.05.006"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/DEVLRN.2007.4354069"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1998.712192"},{"key":"e_1_3_3_53_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6217"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.62.1805"},{"key":"e_1_3_3_56_2","unstructured":"Han Wang Binbin Chen Zhang Zhang and Baoxiang Wang. 2025. Learning to communicate through implicit communication channels. In Proceedings of the International Conference on Representation Learning. Y. Yue A. Garg N. Peng F. Sha and R. Yu (Eds.) ICLR 55179\u201355195. Retrieved from https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2025\/file\/89b89c04f55ea7c7ca989992bb6a98c0-Paper-Conference.pdf"},{"key":"e_1_3_3_57_2","unstructured":"C. J. C. H. Watkins. 1989. Learning from Delayed Rewards. Ph.D. Dissertation. University of Cambridge Cambridge."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511810534.015"},{"key":"e_1_3_3_59_2","volume-title":"Proceedings of the Workshop on Autonomous Mobile Service Robots (AMSR) and IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Wise Melonee","year":"2016","unstructured":"Melonee Wise, Michael Ferguson, Daniel King, Eric Diehr, and David Dymesich. 2016. Fetch & freight: Standard platforms for service robot applications. In Proceedings of the Workshop on Autonomous Mobile Service Robots (AMSR) and IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:42886148"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2022.3161813"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3800955","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T12:42:45Z","timestamp":1776516165000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3800955"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,18]]},"references-count":59,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3800955"],"URL":"https:\/\/doi.org\/10.1145\/3800955","relation":{},"ISSN":["2573-9522"],"issn-type":[{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,18]]},"assertion":[{"value":"2025-05-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-20","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}