{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T14:02:04Z","timestamp":1770818524031,"version":"3.50.1"},"reference-count":87,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2019,6,6]],"date-time":"2019-06-06T00:00:00Z","timestamp":1559779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008382","name":"Fundo Regional para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["POCI-01-0247-FEDER-007678"],"award-info":[{"award-number":["POCI-01-0247-FEDER-007678"]}],"id":[{"id":"10.13039\/100008382","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008382","name":"Fundo Regional para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["CENTRO-01-0145-FEDER-000010"],"award-info":[{"award-number":["CENTRO-01-0145-FEDER-000010"]}],"id":[{"id":"10.13039\/100008382","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100011264","name":"FP7 People: Marie-Curie Actions","doi-asserted-by":"publisher","award":["FP7-PEOPLE-2013-IAPP"],"award-info":[{"award-number":["FP7-PEOPLE-2013-IAPP"]}],"id":[{"id":"10.13039\/100011264","id-type":"DOI","asserted-by":"publisher"}]},{"name":"IEETA","award":["UID\/CEC\/00127\/2013"],"award-info":[{"award-number":["UID\/CEC\/00127\/2013"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Technologies, such as smart sensors, actuators, and other kinds of devices, are often installed in our environments (e.g., our Homes) and available to integrate our daily lives. Despite their installation being motivated by the pursuit of automation and increased efficiency, making these environments usable, acceptable and enjoyable in a sustainable, energy efficient way is not only a matter of automation. Tackling these goals is a complex task demanding the combination of different perspectives including building and urban Architecture, Ubiquitous Computing and Human-Computer Interaction (HCI) to provide occupants with the means to shape these environments to their needs. Interaction is of paramount relevance in the creation of adequate relations of users with their environments, but it cannot be seen independently from the ubiquitous sensing and computing or the environment\u2019s architecture. In this regard, there are several challenges to HCI, particularly in how to integrate this multidisciplinary effort. Although there are several solutions to address some of these challenges, the complexity and dynamic nature of the smart environments and the diversity of technologies involved still present many challenges, particularly for its development. In general, the development is complex, and it is hard to create a dynamic environment providing versatile and adaptive forms of interaction. To participate in the multidisciplinary effort, the development of interaction must be supported by tools capable of facilitating co-design by multidisciplinary teams. In this article, we address the development of interaction for complex smart environments and propose the AM4I architecture and framework, a novel modular approach to design and develop adaptive multiplatform multilingual multi-device multimodal interactive systems. The potential of the framework is demonstrated by proof-of-concept applications in two different smart environment contexts, non-residential buildings and smart homes.<\/jats:p>","DOI":"10.3390\/s19112587","type":"journal-article","created":{"date-parts":[[2019,6,7]],"date-time":"2019-06-07T03:56:31Z","timestamp":1559879791000},"page":"2587","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["The AM4I Architecture and Framework for Multimodal Interaction and Its Application to Smart Environments"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9742-5587","authenticated-orcid":false,"given":"Nuno","family":"Almeida","sequence":"first","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Institute of Electronics and Informatics Engineering of Aveiro, Universidade de Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7675-1236","authenticated-orcid":false,"given":"Ant\u00f3nio","family":"Teixeira","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Institute of Electronics and Informatics Engineering of Aveiro, Universidade de Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9858-8249","authenticated-orcid":false,"given":"Samuel","family":"Silva","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Institute of Electronics and Informatics Engineering of Aveiro, Universidade de Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Maksym","family":"Ketsmur","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications and Informatics, Universidade de Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Institute of Electronics and Informatics Engineering of Aveiro, Universidade de Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"DiSalvo, C., and Jenkins, T. (2017, January 10\u201314). Fruit Are Heavy: A Prototype Public IoT System to Support Urban Foraging. Proceedings of the 2017 Conference on Designing Interactive Systems, Edinburgh, UK.","DOI":"10.1145\/3064663.3064748"},{"key":"ref_2","unstructured":"Crabtree, A., and Tolmie, P. (March, January 27). A Day in the Life of Things in the Home. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, San Francisco, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hargreaves, T., and Wilson, C. (2017). Smart Homes and Their Users, Springer.","DOI":"10.1007\/978-3-319-68018-7"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.patrec.2013.07.003","article-title":"Multimodal interaction: A review","volume":"36","author":"Turk","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dumas, B., Lalanne, D., and Ingold, R. (2009, January 2\u20134). HephaisTK: A toolkit for rapid prototyping of multimodal interfaces. Proceedings of the International Conference on Multimodal Interfaces\u2014ICMI-MLMI \u201909, Cambridge, MA, USA.","DOI":"10.1145\/1647314.1647360"},{"key":"ref_6","unstructured":"Teixeira, A., Pereira, C., Oliveira e Silva, M., Pacheco, O., Neves, A., and Casimiro, J. (2011, January 5\u20137). AdaptO\u2014Adaptive Multimodal Output. Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems, Vilamoura, Algarve, Portugal."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bouchet, J., and Nigay, L. (2004, January 24\u201329). ICARE: A component-based approach for the design and development of multimodal interfaces. Proceedings of the Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems\u2014CHI \u201904, Vienna, Austria.","DOI":"10.1145\/985921.986055"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Serrano, M., Nigay, L., Lawson, J.Y.L., Ramsay, A., Murray-Smith, R., and Denef, S. (2008, January 5\u201310). The openinterface framework: A tool for multimodal interaction. Proceedings of the Twenty-Sixth Annual CHI Conference Extended Abstracts on Human Factors in Computing Systems\u2014CHI \u201908, Florence, Italy.","DOI":"10.1145\/1358628.1358881"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hoste, L., Dumas, B., and Signer, B. (2011, January 14\u201318). Mudra: A unified multimodal interaction framework. Proceedings of the 13th International Conference on Multimodal Interfaces\u2014ICMI \u201911, Alicante, Spain.","DOI":"10.1145\/2070481.2070500"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sonntag, D., Engel, R., Herzog, G., Pfalzgraf, A., Pfleger, N., Romanelli, M., and Reithinger, N. (2007). SmartWeb Handheld\u2014Multimodal Interaction with Ontological Knowledge Bases and Semantic Web Services. Artifical Intelligence for Human Computing, Springer.","DOI":"10.1007\/978-3-540-72348-6_14"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Niklfeld, G., Finan, R., and Pucher, M. (2001, January 3\u20137). Architecture for adaptive multimodal dialog systems based on voiceXML. Proceedings of the 7th European Conference on Speech Communication and Technology, 2nd INTERSPEECH Event, Aalborg, Denmark.","DOI":"10.21437\/Eurospeech.2001-528"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lawson, J.Y.L., Al-Akkad, A.A., Vanderdonckt, J., and Macq, B. (2009, January 15\u201317). An open source workbench for prototyping multimodal interactions based on off-the-shelf heterogeneous components. Proceedings of the 1st ACM SIGCHI Symposium on Engineering Interactive Computing Systems\u2014EICS \u201909, Pittsburgh, PA, USA.","DOI":"10.1145\/1570433.1570480"},{"key":"ref_13","unstructured":"Bodell, M., Dahl, D.A., Kliche, I., Larson, J., and Porter, B. (2019, March 04). Multimodal Architecture and Interfaces, W3C. Available online: https:\/\/www.w3.org\/TR\/mmi-arch\/."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s12193-013-0120-5","article-title":"The W3C multimodal architecture and interfaces standard","volume":"7","author":"Dahl","year":"2013","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_15","unstructured":"Dahl, D.A. (2017). Applications of the Multimodal Interaction Architecture in Ambient Assisted Living. Multimodal Interaction with W3C Standards: Towards Natural User Interfaces to Everything, Springer."},{"key":"ref_16","unstructured":"Teixeira, A., Almeida, N., Pereira, C., and Oliveira e Silva, M. (2013, January 22\u201323). W3C MMI Architecture as a Basis for Enhanced Interaction for Ambient Assisted Living. Proceedings of the Get Smart: Smart Homes, Cars, Devices and the Web, W3C Workshop on Rich Multimodal Application Development, New York, NY, USA."},{"key":"ref_17","unstructured":"Almeida, N. (2017). Multimodal Interaction\u2014Contributions to Simplify Application Development. [Ph.D. Thesis, Universidade de Aveiro]."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dumas, B., Ingold, R., and Lalanne, D. (2009, January 2\u20134). Benchmarking fusion engines of multimodal interactive systems. Proceedings of the 2009 International Conference on Multimodal Interfaces\u2014ICMI-MLMI \u201909, Cambridge, MA, USA.","DOI":"10.1145\/1647314.1647345"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhou, J., and Salvendy, G. (2016). Interactive, Multi-Device Visualization Supported by a Multimodal Interaction Framework: Proof of Concept. Human Aspects of IT for the Aged Population, Design for Aging, Springer.","DOI":"10.1007\/978-3-319-39943-0"},{"key":"ref_20","unstructured":"Dahl, D.A. (2017). Multimodal Fusion and Fission within the W3C MMI Architectural Pattern. Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything, Springer."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rousseau, C., Bellik, Y., and Vernier, F. (2005, January 4\u20136). Multimodal Output Specification\/Simulation Platform. Proceedings of the 7th International Conference on Multimodal Interfaces, Torento, Italy.","DOI":"10.1145\/1088463.1088480"},{"key":"ref_22","unstructured":"Rousseau, C., Bellik, Y., Vernier, F., and Bazalgette, D. (2004, January 22\u201324). Architecture Framework For Output Multimodal Systems Design. Proceedings of the Supporting Community Interaction: Possibilities and Challenges, OZCHI 2004, Wollongong, Australia."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Rousseau, C., Bellik, Y., and Vernier, F. (2005, January 27\u201330). WWHT: Un Mod\u00e8le Conceptuel Pour La Pr\u00e8sentation Multimodale D\u2019Information. Proceedings of the 17th Conference on L\u2019Interaction Homme-Machine, Toulouse, France.","DOI":"10.1145\/1148550.1148558"},{"key":"ref_24","unstructured":"Coetzee, L., Viviers, I., and Barnard, E. (December, January 30). Model based estimation for multi-modal user interface component selection. Proceedings of the 20th Annual Symposium of the Pattern Recognition Association of South Africa (PRASA 2009), Stellenbosch, South Africa."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/3121348","article-title":"Opportunities and Challenges for Cross-device Interactions in the Wild","volume":"24","author":"Houben","year":"2017","journal-title":"Interactions"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Dong, T., Churchill, E.F., and Nichols, J. (2016, January 4\u20136). Understanding the Challenges of Designing and Developing Multi-Device Experiences. Proceedings of the 2016 ACM Conference on Designing Interactive Systems, Brisbane, Australia.","DOI":"10.1145\/2901790.2901851"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s00779-017-1016-2","article-title":"Cross-device media: A review of second screening and multi-device television","volume":"21","author":"Neate","year":"2017","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_28","unstructured":"Shen, C., Esenther, A., Forlines, C., and Ryall, K. (2006, January 22\u201323). Three modes of multisurface interaction and visualization. Proceedings of the Information Visualization and Interaction Techniques for Collaboration across Multiple Displays Workshop associated with CHI, Montreal, QC, Canada."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wo\u017aniak, P., Lischke, L., Schmidt, B., Zhao, S., and Fjeld, M. (2014, January 26\u201330). Thaddeus: A Dual Device Interaction Space for Exploring Information Visualisation. Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational\u2014NordiCHI \u201914, Helsinki, Finland.","DOI":"10.1145\/2639189.2639237"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/MMUL.2009.75","article-title":"Intelligent Multimedia Presentation in Ubiquitous Multidevice Scenarios","volume":"17","author":"Kernchen","year":"2010","journal-title":"IEEE Multimed."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Houben, S., and Marquardt, N. (2015, January 18\u201323). WATCHCONNECT: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea.","DOI":"10.1145\/2702123.2702215"},{"key":"ref_32","unstructured":"Weibel, N., Satyanarayan, A., Lazar, A., Oda, R., Yamaoka, S., Doerr, K.U., Kuester, F., Griswold, W.G., and Hollan, J.D. (2011, January 21\u201328). Hiperface: A multichannel architecture to explore multimodal interactions with ultra-scale wall displays. Proceedings of the 33rd International Conference on Software Engineering, ICSE\u201911, Honolulu, HI, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Badam, S.K., and Elmqvist, N. (2014, January 16\u201319). PolyChrome: A Cross-Device Framework for Collaborative Web Visualization. Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces\u2014ITS \u201914, Dresden, Germany.","DOI":"10.1145\/2669485.2669518"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Heikkinen, T., Goncalves, J., Kostakos, V., Elhart, I., and Ojala, T. (2014, January 3\u20134). Tandem Browsing Toolkit: Distributed Multi-Display Interfaces with Web Technologies. Proceedings of the International Symposium on Pervasive Displays, Copenhagen, Denmark.","DOI":"10.1145\/2611009.2611026"},{"key":"ref_35","unstructured":"Hamilton, P., and Wigdor, D.J. (May, January 26). Conductor: Enabling and understanding cross-device interaction. Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems\u2014CHI \u201914, Toronto, ON, Canada."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.1007\/s00779-013-0727-2","article-title":"VisPorter: Facilitating information sharing for collaborative sensemaking on multiple displays","volume":"18","author":"Chung","year":"2014","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Blumendorf, M., Roscher, D., and Albayrak, S. (2010, January 8\u201310). Dynamic user interface distribution for flexible multimodal interaction. Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction on\u2014ICMI-MLMI \u201910, Beijing, China.","DOI":"10.1145\/1891903.1891930"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1007\/s12273-017-0355-2","article-title":"Simulation and visualization of energy-related occupant behavior in office buildings","volume":"10","author":"Chen","year":"2017","journal-title":"Build. Simul."},{"key":"ref_39","unstructured":"Ghiani, G., Manca, M., and Patern\u00f2, F. (December, January 30). Authoring context-dependent cross-device user interfaces based on trigger\/action rules. Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, Linz, Austria."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Di Mauro, D., Augusto, J.C., Origlia, A., and Cutugno, F. (2017, January 26\u201328). A framework for distributed interaction in intelligent environments. Proceedings of the European Conference on Ambient Intelligence, Malaga, Spain.","DOI":"10.1007\/978-3-319-56997-0_11"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Nebeling, M., Kubitza, T., Patern\u00f2, F., Dong, T., Li, Y., and Nichols, J. (2016, January 21\u201324). End-user development of cross-device user interfaces. Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Brussels, Belgium.","DOI":"10.1145\/2933242.2948128"},{"key":"ref_42","unstructured":"Dahl, D.A. (2017). Multi-Device Applications Using the Multimodal Architecture. Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything, Springer."},{"key":"ref_43","unstructured":"Seyed, A. (2013). Examining User Experience in Multi-Display Environments. [Ph.D. Thesis, University of Calgary]."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Patern\u00f2, F. (2015, January 18\u201323). Design and Adaptation for Cross-Device, Context-Dependent User Interfaces. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems\u2014CHI EA \u201915, Seoul, Korea.","DOI":"10.1145\/2702613.2706686"},{"key":"ref_45","unstructured":"Rowland, C., Goodman, E., Charlier, M., Light, A., and Lui, A. (2015). Designing Connected Products: UX for the Consumer Internet of Things, O\u2019Reilly Media, Inc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1454","DOI":"10.1016\/j.jclepro.2016.10.006","article-title":"A review of Internet of Things for smart home: Challenges and solutions","volume":"140","author":"Trivodaliev","year":"2017","journal-title":"J. Clean. Prod."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Shahzad, K., and Oelmann, B. (2014, January 26\u201329). A comparative study of in-sensor processing vs. raw data transmission using ZigBee, BLE and Wi-Fi for data intensive monitoring applications. Proceedings of the 2014 11th International Symposium on Wireless Communications Systems (ISWCS), Barcelona, Spain.","DOI":"10.1109\/ISWCS.2014.6933409"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Williams, J.D., Kamal, E., Ashour, M., Amr, H., Miller, J., and Zweig, G. (2015, January 2\u20134). Fast and easy language understanding for dialog systems with Microsoft Language Understanding Intelligent Service (LUIS). Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Prague, Czech Republic.","DOI":"10.18653\/v1\/W15-4622"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Dahl, D.A. (2017). Standard Portals for Intelligent Services. Multimodal Interaction with W3C Standards, Springer.","DOI":"10.1007\/978-3-319-42816-1_11"},{"key":"ref_50","unstructured":"Young, S. (2009). CUED Standard Dialogue Acts, Cambridge University Engineering Depqrtment. Technical Report."},{"key":"ref_51","unstructured":"Bunt, H., Alexandersson, J., Carletta, J., Choe, J.W., Fang, A.C., Hasida, K., Lee, K., Petukhova, V., Popescu-Belis, A., and Romary, L. (2010, January 17\u201323). Towards an ISO Standard for Dialogue Act Annotation. Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC\u201910), European Language Resources Association (ELRA), Valletta, Malta."},{"key":"ref_52","unstructured":"Bui, T. (2006). Multimodal Dialogue Management-State of the Art. (CTIT Technical Report Series; No. 06-01), Centre for Telematics and Information Technology (CTIT), University of Twente."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Dahl, D.A. (2017). Multimodal Interaction with W3C Standards, Springer.","DOI":"10.1007\/978-3-319-42816-1"},{"key":"ref_54","unstructured":"Baggia, P., Burnett, D.C., Carter, J., Dahl, D.A., McCobb, G., and Raggett, D. (2019, March 04). EMMA: Extensible MultiModal Annotation Markup Language. Available online: https:\/\/www.w3.org\/TR\/emma\/."},{"key":"ref_55","unstructured":"Almeida, N., Silva, S., and Teixeira, A. (2014, January 17\u201320). Multimodal Multi-Device Application Supported by an SCXML State Chart Machine. Proceedings of the Workshop on Engineering Interactive Systems with SCXML, The Sixth ACM SIGCHI Symposium on Computing Systems, Rome, Italy."},{"key":"ref_56","unstructured":"Nordby, K., Helmersen, P., Gilmore, D.J., and Arnesen, S.A. (1995). Four Easy Pieces for Assessing the Usability of Multimodal Interaction: The Care Properties. Human\u2013Computer Interaction, Springer. Chapter Four Easy."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Almeida, N., Silva, S., and Teixeira, A. (2014, January 22\u201327). Design and Development of Speech Interaction: A Methodology. Proceedings of Human-Computer Interaction International Conference, HCI International 2014, Crete, Greece.","DOI":"10.1007\/978-3-319-07230-2_36"},{"key":"ref_58","unstructured":"Teixeira, A., Braga, D., Coelho, L., Fonseca, J., Alvarelh\u00e3o, J., Mart\u00edn, I., Alexandra, Q., Rocha, N., Calado, A., and Dias, M. (2009, January 3\u20135). Speech as the Basic Interface for Assistive Technology. Proceedings of the DSAI\u2014Software Development for Enhancing Accessibility and Fighting Info-exclusion, Lisbon, Portugal."},{"key":"ref_59","unstructured":"Teixeira, A., Francisco, P., Almeida, N., Pereira, C., and Silva, S. (2014, January 20\u201324). Services to Support Use and Development of Speech Input for Multilingual Multimodal Applications for Mobile Scenarios. Proceedings of the Ninth International Conference on Internet and Web Applications and Services (ICIW 2014), Track WSSA\u2014Web Services-based Systems and Applications, Paris, France."},{"key":"ref_60","first-page":"1","article-title":"Services to Support Use and Development of Multilingual Speech Input","volume":"8","author":"Teixeira","year":"2015","journal-title":"Int. J. Adv. Internet Technol."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Ward, W. (1991, January 14\u201317). Understanding spontaneous speech: The Phoenix system. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, ICASSP 91, Toronto, ON, Canada.","DOI":"10.1109\/ICASSP.1991.150352"},{"key":"ref_62","unstructured":"Kurosu, M. (2015). Giving Voices to Multimodal Applications. Human Aspects of IT for the Aged Population. Design for Aging, Springer."},{"key":"ref_63","unstructured":"Sole, A.D. (2017). Microsoft Computer Vision APIs Distilled: Getting Started with Cognitive Services, Apress."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Barnett, J. (2017). Introduction to SCXML. Multimodal Interaction with W3C Standards, Springer.","DOI":"10.1007\/978-3-319-42816-1_5"},{"key":"ref_65","unstructured":"Commons, A. (2018, January 07). SCXML-Commons SCXML. Available online: http:\/\/commons.apache.org\/scxml\/."},{"key":"ref_66","unstructured":"Almeida, N., Teixeira, A., Silva, S., and Freitas, J. (2016, January 23\u201325). Fusion of Speech with other modalities in a W3C based Multimodal Interaction Framework. Proceedings of the IberSpeech, Lisbon, Portugal."},{"key":"ref_67","unstructured":"Vieira, D. (2015). Enhanced Multimodal Interaction Framework and Applications. [Master\u2019s Thesis, Universidade de Aveiro]."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/j.procs.2015.09.272","article-title":"Multilingual speech recognition for the elderly: The AALFred personal life assistant","volume":"67","author":"Teixeira","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_69","unstructured":"Saldanha, N., Avelar, J., Dias, M., Teixeira, A., Gon\u00e7alves, D., Bonnet, E., Lan, K., G\u00e9za, N., Csobanka, P., and Kolesinski, A. (2013, January 24\u201326). A Personal Life Assistant for \u201cnatural\u201d interaction: The PaeLife project. Proceedings of the AAL Forum 2013 Forum, Norrk\u00f6ping, Sweden."},{"key":"ref_70","unstructured":"Bartashevich, D., Oliveira, L., Teixeira, A., and Silva, S. (2018, January 3\u20134). Hands Free Presentations: Multimodal Interaction with PowerPoint. Proceedings of the INForum 2018, Coimbra, Portugal."},{"key":"ref_71","unstructured":"Rogers, Y., Sharp, H., and Preece, J. (2011). Interaction Design: Beyond Human-Computer Interaction, John Wiley & Sons."},{"key":"ref_72","first-page":"28","article-title":"Designing Future Ubiquitous Homes with OUI Interiors: Possibilities and Challenges","volume":"32","author":"Nabil","year":"2017","journal-title":"Interact. Des. Archit."},{"key":"ref_73","unstructured":"Cooper, A., Reimann, R., and Cronin, D. (2007). About Face 3.0: The Essentials of Interaction Design, John Wiley & Sons Inc."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.erss.2017.05.037","article-title":"Oh behave! Survey stories and lessons learned from building occupants in high-performance buildings","volume":"31","author":"Day","year":"2017","journal-title":"Energy Res. Soc. Sci."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.erss.2017.06.034","article-title":"Using stories, narratives, and storytelling in energy and climate change research","volume":"31","author":"Moezzi","year":"2017","journal-title":"Energy Res. Soc. Sci."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Leal, A., Teixeira, A., and Silva, S. (2016, January 17\u201322). On the creation of a Persona to support the development of technologies for children with Autism Spectrum Disorder. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Toronto, ON, Canada.","DOI":"10.1007\/978-3-319-40238-3_21"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"2156","DOI":"10.1007\/s10803-019-03898-1","article-title":"Design and Development for Individuals with ASD: Fostering Multidisciplinary Approaches through Personas","volume":"49","author":"Silva","year":"2019","journal-title":"J. Autism Dev. Disorders"},{"key":"ref_78","unstructured":"Zhou, J., and Salvendy, G. (2017). Silent Speech Interaction for Ambient Assisted Living Scenarios. Human Aspects of IT for the Aged Population. Aging, Design and User Experience, Springer."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Freitas, J., Teixeira, A., Dias, M.S., and Silva, S. (2017). An Introduction to Silent Speech Interfaces, Springer.","DOI":"10.1007\/978-3-319-40174-4"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Silva, S., and Teixeira, A. (2017, January 21\u201323). An Anthropomorphic Perspective for Audiovisual Speech Synthesis. Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2017), Porto, Portugal.","DOI":"10.5220\/0006150201630172"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Henriques, T., Silva, S., Br\u00e1s, S., Soares, S.C., Almeida, N., and Teixeira, A. (2018, January 20\u201322). Emotionally-Aware Multimodal Interfaces: Preliminary Work on a Generic Affective Modality. Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion, DSAI 2018, Thessaloniki, Greece.","DOI":"10.1145\/3218585.3218589"},{"key":"ref_82","unstructured":"Dahl, D.A. (2017). Discovery and Registration: Finding and Integrating Components into Dynamic Systems. Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything, Springer."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.erss.2017.06.012","article-title":"Mental models: Exploring how people think about heat flows in the home","volume":"31","author":"Goodhew","year":"2017","journal-title":"Energy Res. Soc. Sci."},{"key":"ref_84","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1016\/j.erss.2017.08.002","article-title":"Synthesizing building physics with social psychology: An interdisciplinary framework for context and occupant behavior in office buildings","volume":"34","author":"Chen","year":"2017","journal-title":"Energy Res. Soc. Sci."},{"key":"ref_85","first-page":"39:1","article-title":"A Unified Model for User Identification on Multi-Touch Surfaces: A Survey and Meta-Analysis","volume":"24","author":"Kharrufa","year":"2018","journal-title":"ACM Trans. Comput. Hum. Interact."},{"key":"ref_86","unstructured":"Pereira, C. (2016). Dynamic Evaluation for Reactive Scenarios. [Ph.D. Thesis, Universidade de Aveiro]."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Pereira, C., Almeida, N., Martins, A.I., Silva, S., Rosa, A.F., Silva, M.O.E., and Teixeira, A. (2015, January 2\u20137). Evaluation of Complex Distributed Multimodal Applications Evaluating a TeleRehabilitation System When It Really Matters. Proceedings of the Human Aspects of IT for the Aged Population, Design for Everyday Life, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-20913-5_14"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2587\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:56:43Z","timestamp":1760187403000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2587"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,6]]},"references-count":87,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["s19112587"],"URL":"https:\/\/doi.org\/10.3390\/s19112587","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,6]]}}}