{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T04:31:23Z","timestamp":1776141083134,"version":"3.50.1"},"reference-count":57,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,6,6]],"date-time":"2020-06-06T00:00:00Z","timestamp":1591401600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["ES 434\/8-1"],"award-info":[{"award-number":["ES 434\/8-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001655","name":"Deutscher Akademischer Austauschdienst","doi-asserted-by":"publisher","award":["50018217"],"award-info":[{"award-number":["50018217"]}],"id":[{"id":"10.13039\/501100001655","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>After conversational agents have been made available to the broader public, we speculate that applying them as a mediator for adaptive environments reduces control complexity and increases user experience by providing a more natural interaction. We implemented and tested four agents, each of them differing in their system intelligence and input modality, as personal assistants for Mediated Atmospheres, an adaptive smart office prototype. They were evaluated in a user study ( N = 33 ) to collect subjective and objective measures. Results showed that a smartphone application was the most favorable system, followed by conversational text and voice agents that were perceived as being more engaging and intelligent than a non-conversational voice agent. Significant differences were observed between native and non-native speakers in both subjective and objective measures. Our findings reveal the potential of conversational agents for the interaction with adaptive environments to reduce work and information overload.<\/jats:p>","DOI":"10.3390\/mti4020027","type":"journal-article","created":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T04:19:39Z","timestamp":1591676379000},"page":"27","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Exploring Smart Agents for the Interaction with Multimodal Mediated Environments"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0272-5403","authenticated-orcid":false,"given":"Robert","family":"Richer","sequence":"first","affiliation":[{"name":"Machine Learning and Data Analytics Lab., Department of Computer Science, Friedrich-Alexander-Universtit\u00e4t Erlangen-N\u00fcrnberg (FAU), Carl-Thiersch-Str. 2b, 91052 Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nan","family":"Zhao","sequence":"additional","affiliation":[{"name":"Responsive Environments Group, MIT Media Lab., 75 Amherst St, Cambridge, MA 02139, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0417-0336","authenticated-orcid":false,"given":"Bjoern M.","family":"Eskofier","sequence":"additional","affiliation":[{"name":"Machine Learning and Data Analytics Lab., Department of Computer Science, Friedrich-Alexander-Universtit\u00e4t Erlangen-N\u00fcrnberg (FAU), Carl-Thiersch-Str. 2b, 91052 Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph A.","family":"Paradiso","sequence":"additional","affiliation":[{"name":"Responsive Environments Group, MIT Media Lab., 75 Amherst St, Cambridge, MA 02139, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,6]]},"reference":[{"key":"ref_1","unstructured":"Smith, A. (2020, April 15). U.S. Smartphone Use in 2015. Available online: http:\/\/www.pewinternet.org\/2015\/04\/01\/us-smartphone-use-in-2015."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Andrews, S., Ellis, D.A., Shaw, H., and Piwek, L. (2015). Beyond Self-Report: Tools to Compare Estimated and Real-World Smartphone Use. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0139004"},{"key":"ref_3","unstructured":"Google (2020, April 11). Google Assistant, your Own Personal Google. Available online: https:\/\/assistant.google.com\/."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1145\/2133806.2133812","article-title":"Talking to machines","volume":"55","author":"Geller","year":"2012","journal-title":"Commun. ACM"},{"key":"ref_5","unstructured":"Laurel, B., and Mountford, S.J. (1990). Conversation as Direct Manipulation: An Iconoclastic View. The Art of Human-Computer Interface Design, Addison-Wesley Longman Publishing Co., Inc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/MIS.1999.757623","article-title":"An Intelligent Environment must be adaptive","volume":"14","author":"Mozer","year":"1999","journal-title":"IEEE Intell. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kidd, C.D., Orr, R., Abowd, G.D., Atkeson, C.G., Essa, I.A., MacIntyre, B., Mynatt, E., Starner, T.E., and Newstetter, W. (1999, January 1\u20132). The Aware Home: A Living Laboratory for Ubiquitous Computing Research. Proceedings of the International Workshop on Cooperative Buildings, Pittsburgh, PA, USA.","DOI":"10.1007\/10705432_17"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3132027","article-title":"Mediated Atmospheres: A Multimodal Mediated Work Environment","volume":"1","author":"Zhao","year":"2017","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_9","unstructured":"Gensler Research Institute (2020, April 25). 2013 U.S. Workplace Survey Key Findings. Available online: https:\/\/www.gensler.com\/research-insight\/gensler-research-institute\/the-2013-us-workplace-survey-1."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1145\/176789.176792","article-title":"Agents that reduce work and information overload","volume":"37","author":"Maes","year":"1994","journal-title":"Commun. ACM"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Luger, E., and Sellen, A. (2016, January 7). Like Having a Really Bad PA: The Gulf between User Expectation and Experience of Conversational Agents. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2858036.2858288"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.enbuild.2015.07.059","article-title":"Personal lighting control with occupancy and daylight adaptation","volume":"105","author":"Rossi","year":"2015","journal-title":"Energy Build."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chen, N.H., Nawyn, J., Thompson, M., Gibbs, J., and Larson, K. (2013). Context-aware tunable office lighting application and user response. SPIE Optical Engineering+ Applications, International Society for Optics and Photonics.","DOI":"10.1117\/12.2021750"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Aldrich, M., Zhao, N., and Paradiso, J. (2010). Energy efficient control of polychromatic solid state lighting using a sensor network. SPIE Optical Engineering+ Applications, International Society for Optics and Photonics.","DOI":"10.1117\/12.860755"},{"key":"ref_15","unstructured":"Kalyanam, R., and Hoffmann, S. (2016, January 30). Visual and thermal comfort with electrochromic glass using an adaptive control strategy. Proceedings of the 15th International Radiance Workshop (IRW), Padua, Italy."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Feldmeier, M., and Paradiso, J.A. (December, January 29). Personalized HVAC control system. Proceedings of the 2010 Internet of Things (IOT), Tokyo, Japan.","DOI":"10.1109\/IOT.2010.5678444"},{"key":"ref_17","unstructured":"Kainulainen, A., Turunen, M., Hakulinen, J., Salonen, E.P., and Prusi, P. (2005, January 17\u201319). A Speech-based and Auditory Ubiquitous Office Environment. Proceedings of the 10th International Conference on Speech and Computer, Patras, Greece."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. (1998, January 21\u201325). The office of the future: A unified approach to image-based modeling and spatially immersive displays. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, CA, USA.","DOI":"10.1145\/280814.280861"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tomitsch, M., Grechenig, T., Moere, A.V., and Renan, S. (2008, January 8\u201311). Information Sky: Exploring the Visualization of Information on Architectural Ceilings. Proceedings of the 12th International Conference on Information Visualisation (IV \u201908), London, UK.","DOI":"10.1109\/IV.2008.81"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/MPRV.2002.1012339","article-title":"The Interactive Workspaces project: Experiences with ubiquitous computing rooms","volume":"1","author":"Johanson","year":"2002","journal-title":"IEEE Pervasive Comput."},{"key":"ref_21","unstructured":"Pejsa, T., Kantor, J., Benko, H., Ofek, E., and Wilson, A. (March, January 27). Room2Room: Enabling life-size telepresence in a projected augmented reality environment. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, San Francisco, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Schn\u00e4delbach, H., Glover, K., and Irune, A.A. (2010, January 16\u201320). ExoBuilding: Breathing Life into Architecture. Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, Reykjavik, Iceland.","DOI":"10.1145\/1868914.1868965"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B., and Yarin, P. (1998, January 18\u201323). ambientROOM: Integrating Ambient Media with Architectural Space. Proceedings of the CHI 98 Conference Summary on Human Factors in Computing Systems, Los Angeles, CA, USA.","DOI":"10.1145\/286498.286652"},{"key":"ref_24","unstructured":"Mozer, M.C. (1998, January 23\u201325). The neural network house: An environment that adapts to its inhabitants. Proceedings of the AAAI Spring Symposium on Intelligent Environments, Palo Alto, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Mozer, M.C., Dodier, R., Miller, D., Anderson, M., Anderson, J., Bertini, D., Bronder, M., Colagrosso, M., Cruickshank, R., and Daugherty, B. (2005, January 28). The adaptive house. Proceedings of the IEE Seminar on Intelligent Building Environments, Colchester, UK.","DOI":"10.1049\/ic:20050196"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mennicken, S., Vermeulen, J., and Huang, E.M. (2014, January 13\u201317). From today\u2019s augmented houses to tomorrow\u2019s smart homes: New directions for home automation research. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Seattle, WA, USA.","DOI":"10.1145\/2632048.2636076"},{"key":"ref_27","unstructured":"Cook, D.J., Youngblood, M., Heierman, E.O., Gopalratnam, K., Rao, S., Litvin, A., and Khawaja, F. (2003, January 23\u201326). MavHome: An Agent-Based Smart Home. Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, Fort Worth, TX, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Abras, S., Ploix, S., Pesty, S., and Jacomino, M. (2008). A Multi-agent Home Automation System for Power Management. Informatics in Control Automation and Robotics, Springer.","DOI":"10.3182\/20070709-3-RO-4910.00014"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1145\/2943770","article-title":"Tariff Agent: Interacting with a Future Smart Energy System at Home","volume":"23","author":"Alan","year":"2016","journal-title":"ACM Trans. Comput. Hum. Interact. (TOCHI)"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Danninger, M., and Stiefelhagen, R. (2008, January 27\u201331). A context-aware virtual secretary in a smart office environment. Proceedings of the 16th ACM International Conference on Multimedia, Vancouver, BC, USA.","DOI":"10.1145\/1459359.1459430"},{"key":"ref_31","unstructured":"Bagci, F., Petzold, J., Trumler, W., and Ungerer, T. (2003, January 12). Ubiquitous mobile agent system in a P2P-network. Proceedings of the Fifth Annual Conference on Ubiquitous Computing, Seattle, WA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1007\/s11042-012-1008-1","article-title":"Adaptive interaction support in ambient-aware environments based on quality of context information","volume":"67","author":"Hossain","year":"2013","journal-title":"Multimed. Tools Appl."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Stra\u00dfmann, C., von der P\u00fctten, A.R., Yaghoubzadeh, R., Kaminski, R., and Kr\u00e4mer, N. (2016, January 20\u201323). The Effect of an Intelligent Virtual Agent\u2019s Nonverbal Behavior with Regard to Dominance and Cooperativity. Proceedings of the International Conference on Intelligent Virtual Agents, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-47665-0_2"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mennicken, S., Zihler, O., Juldaschewa, F., Molnar, V., Aggeler, D., and Huang, E.M. (2016, January 12\u201316). It\u2019s like living with a friendly stranger: Perceptions of personality traits in a smart home. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971757"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1093\/mind\/LIX.236.433","article-title":"Computing machinery and intelligence","volume":"59","author":"Turing","year":"1950","journal-title":"Mind"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA\u2014A Computer Program For the Study of Natural Language Communication Between Man And Machine","volume":"9","author":"Weizenbaum","year":"1966","journal-title":"Commun. ACM"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"D\u2019Haro, L.F., Kim, S., Yeo, K.H., Jiang, R., Niculescu, A.I., Banchs, R.E., and Li, H. (2015). CLARA: A Multifunctional Virtual Agent for Conference Support and Touristic Information. Natural Language Dialog Systems and Intelligent Assistants, Springer.","DOI":"10.1007\/978-3-319-19291-8_22"},{"key":"ref_38","unstructured":"Gnewuch, U., Morana, S., and Maedche, A. (2017, January 10\u201313). Towards Designing Cooperative and Social Conversational Agents for Customer Service. Proceedings of the Thirty Eigth International Conference on Information Systems, Seoul, Korea."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1145\/2818717","article-title":"The rise of social bots","volume":"59","author":"Ferrara","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1248","DOI":"10.1093\/jamia\/ocy072","article-title":"Conversational agents in healthcare: A systematic review","volume":"25","author":"Laranjo","year":"2018","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.eswa.2019.03.054","article-title":"Survey of conversational agents in health","volume":"129","author":"Montenegro","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Provoost, S., Lau, H.M., Ruwaard, J., and Riper, H. (2017). Embodied conversational agents in clinical psychology: A scoping review. J. Med. Internet Res., 19.","DOI":"10.2196\/jmir.6553"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chu, Z., Gianvecchio, S., Wang, H., and Jajodia, S. (2010, January 6\u201310). Who is tweeting on Twitter: Human, bot, or cyborg?. Proceedings of the 26th Annual Computer Security Applications Conference, Austin, TX, USA.","DOI":"10.1145\/1920261.1920265"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/MC.2016.183","article-title":"The DARPA Twitter bot challenge","volume":"49","author":"Subrahmanian","year":"2016","journal-title":"Computer"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.ijhcs.2014.11.004","article-title":"Modeling input modality choice in mobile graphical and speech interfaces","volume":"75","author":"Schaffer","year":"2015","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_46","unstructured":"Wei\u00df, B., M\u00f6ller, S., and Schulz, M. (February, January 30). Modality Preferences of Different User Groups. Proceedings of the Fifth International Conference on Advances in Computer-Human Interactions (ACHI 2012), Valencia, Spain."},{"key":"ref_47","unstructured":"Silva, S., Almeida, N., Pereira, C., Martins, A.I., Rosa, A.F., Oliveira e Silva, M., and Teixeira, A. (2015). Design and Development of Multimodal Applications: A Vision on Key Issues and Methods. Universal Access in Human-Computer Interaction, Proceedings of the Today\u2019s Technologies: 9th International Conference, UAHCI 2015, Los Angeles, CA, USA, 2\u20137 August 2015, Springer International Publishing."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"K\u00fchnel, C., Westermann, T., Wei\u00df, B., and M\u00f6ller, S. (2010, January 16\u201320). Evaluating Multimodal Systems: A Comparison of Established Questionnaires and Interaction Parameters. Proceedings of the 6th Nordic Conference on Human-Computer Interaction (NordiCHI 2010), Reykjavik, Iceland.","DOI":"10.1145\/1868914.1868949"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Pyae, A., and Scifleet, P. (2019, January 4\u20139). Investigating the Role of User\u2019s English Language Proficiency in Using a Voice User Interface: A Case of Google Home Smart Speaker. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290607.3313038"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Sivasubramanian, S. (2012, January 20\u201324). Amazon dynamoDB: A seamlessly scalable non-relational database service. Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, Scottsdale, AZ, USA.","DOI":"10.1145\/2213836.2213945"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1111\/j.2044-8295.1984.tb02785.x","article-title":"A multidimensional approach to the relationship between mood and weather","volume":"75","author":"Howarth","year":"1984","journal-title":"Br. J. Psychol."},{"key":"ref_52","unstructured":"Luminous, S. (2020, April 15). The Circadian Rhythm and Color Temperature. Available online: https:\/\/sigmaluminous.com\/the-circadian-rhythm-and-color-temperature."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.jsmc.2009.01.004","article-title":"Effect of light on human circadian physiology","volume":"4","author":"Duffy","year":"2009","journal-title":"Sleep Med. Clin."},{"key":"ref_54","first-page":"4","article-title":"SUS\u2014A quick and dirty usability scale","volume":"189","author":"Brooke","year":"1996","journal-title":"Usability Eval. Ind."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1017\/S1351324900002497","article-title":"Towards a tool for the subjective assessment of speech system interfaces (SASSI)","volume":"6","author":"Hone","year":"2000","journal-title":"Nat. Lang. Eng."},{"key":"ref_56","unstructured":"Mennicken, S., Brillman, R., Thom, J., and Cramer, H. (2018). Challenges and Methods in Design of Domain-Specific Voice Assistants. 2018 AAAI Spring Symposium Series, Association for the Advancement of Artificial Intelligence."},{"key":"ref_57","unstructured":"Zuckerberg, M. (2020, April 10). Building Jarvis. Available online: https:\/\/www.facebook.com\/notes\/mark-zuckerberg\/building-jarvis\/10154361492931634."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/2\/27\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:36:14Z","timestamp":1760175374000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/2\/27"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,6]]},"references-count":57,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["mti4020027"],"URL":"https:\/\/doi.org\/10.3390\/mti4020027","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,6]]}}}