{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T06:44:20Z","timestamp":1768286660791,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":42,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T00:00:00Z","timestamp":1687824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,27]]},"DOI":"10.1145\/3596454.3597177","type":"proceedings-article","created":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T15:02:27Z","timestamp":1687273347000},"page":"9-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["A Review on Quality in Use Evaluation of Smart Environment Applications: What\u2019s Next?"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1129-5073","authenticated-orcid":false,"given":"Maria Paula Corr\u00eaa","family":"Angeloni","sequence":"first","affiliation":[{"name":"CNRS, UMR 8201 - LAMIH, Universit\u00e9 Polytechnique Hauts-de-France, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8146-5966","authenticated-orcid":false,"given":"K\u00e1thia Mar\u00e7al de","family":"Oliveira","sequence":"additional","affiliation":[{"name":"CNRS, UMR 8201 - LAMIH, Universit\u00e9 Polytechnique Hauts-de-France, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8429-4012","authenticated-orcid":false,"given":"Emmanuelle","family":"Grislin-Le Strugeon","sequence":"additional","affiliation":[{"name":"CNRS, UMR 8201 - LAMIH, Universit\u00e9 Polytechnique Hauts-de-France, France and INSA Hauts-de-France, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8636-3213","authenticated-orcid":false,"given":"Rafael","family":"Duque","sequence":"additional","affiliation":[{"name":"Depart. de Matem\u00e1ticas, Estad\u00edstica y Computaci\u00f3n, Universidad de Cantabria, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7129-2237","authenticated-orcid":false,"given":"Cristina","family":"Tirnauca","sequence":"additional","affiliation":[{"name":"Depart. de Matem\u00e1ticas, Estad\u00edstica y Computaci\u00f3n, Universidad de Cantabria, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,6,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"M.\u00a0B. Abughazala M.\u00a0T. Moghaddam H. Muccini and K. Vaidhyanathan. 2021. Human Behavior-Oriented Architectural Design. In Software Architecture. 134 \u2013 143.  M.\u00a0B. Abughazala M.\u00a0T. Moghaddam H. Muccini and K. Vaidhyanathan. 2021. Human Behavior-Oriented Architectural Design. In Software Architecture. 134 \u2013 143.","DOI":"10.1007\/978-3-030-86044-8_9"},{"key":"e_1_3_2_1_2_1","first-page":"2238","article-title":"A co simulation of photovoltaic power generation and human activity for smart building energy management and energy sharing, Corrado V., Fabrizio E., Gasparella A., and Patuzzi F. (Eds.)","volume":"4","author":"Albouys-Perrois J.","year":"2019","unstructured":"J. Albouys-Perrois , N. Sabouret , Y. Haradji , M. Schumann , and C. Inard . 2019 . A co simulation of photovoltaic power generation and human activity for smart building energy management and energy sharing, Corrado V., Fabrizio E., Gasparella A., and Patuzzi F. (Eds.) . Building Simulation Conf. Proc. 4 (2019), 2238 \u2013 2245 . J. Albouys-Perrois, N. Sabouret, Y. Haradji, M. Schumann, and C. Inard. 2019. A co simulation of photovoltaic power generation and human activity for smart building energy management and energy sharing, Corrado V., Fabrizio E., Gasparella A., and Patuzzi F. (Eds.). Building Simulation Conf. Proc. 4 (2019), 2238 \u2013 2245.","journal-title":"Building Simulation Conf. Proc."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.082080899"},{"key":"e_1_3_2_1_4_1","volume-title":"SUS: A quick and dirty","author":"Brooke J.","year":"1996","unstructured":"J. Brooke . 1996 . SUS: A quick and dirty \u2019 usability scale. Usability Evaluation in Industry ( 1996), 189 \u2013 194. J. Brooke. 1996. SUS: A quick and dirty\u2019 usability scale. Usability Evaluation in Industry (1996), 189 \u2013 194."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1057\/jos.2016.10"},{"key":"e_1_3_2_1_6_1","first-page":"85","article-title":"The ROBOCARE project: The user\u2019s view","volume":"22","author":"Cesta A.","year":"2008","unstructured":"A. Cesta , G. Cortellessa , V. Giuliani , F. Pecora , M. Scopelliti , and L. Tiberio . 2008 . The ROBOCARE project: The user\u2019s view . Assistive Technology Research Series 22 (2008), 85 \u2013 104 . https:\/\/doi.org\/10.3233\/978-1-58603-902-8-85 10.3233\/978-1-58603-902-8-85 A. Cesta, G. Cortellessa, V. Giuliani, F. Pecora, M. Scopelliti, and L. Tiberio. 2008. The ROBOCARE project: The user\u2019s view. Assistive Technology Research Series 22 (2008), 85\u2013104. https:\/\/doi.org\/10.3233\/978-1-58603-902-8-85","journal-title":"Assistive Technology Research Series"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.3233\/JIFS-210564"},{"key":"e_1_3_2_1_8_1","volume-title":"Predictive Simulation within the Process of Building Trust. In 2022 IEEE 19th Int. Conf. on Software Architecture Companion, ICSA-C 2022","author":"Cioroaica E.","year":"2022","unstructured":"E. Cioroaica , B. Buhnova , and T. Kuhn . 2022 . Predictive Simulation within the Process of Building Trust. In 2022 IEEE 19th Int. Conf. on Software Architecture Companion, ICSA-C 2022 . IEEE, Honolulu, HI, USA, 47\u201348. https:\/\/doi.org\/10.1109\/ICSA-C54293. 2022 .00017 10.1109\/ICSA-C54293.2022.00017 E. Cioroaica, B. Buhnova, and T. Kuhn. 2022. Predictive Simulation within the Process of Building Trust. In 2022 IEEE 19th Int. Conf. on Software Architecture Companion, ICSA-C 2022. IEEE, Honolulu, HI, USA, 47\u201348. https:\/\/doi.org\/10.1109\/ICSA-C54293.2022.00017"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.118392"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2004.02.001"},{"key":"e_1_3_2_1_11_1","volume-title":"Proc.-4th IEEE Int. Conf. on AI and VR. 134 \u2013 138","author":"Gramoli L.","year":"2021","unstructured":"L. Gramoli , J. Lacoche , A. Foulonneau , V. Gouranton , and B. Arnaldi . 2021. Needs Model for an Autonomous Agent during Long-term Simulations . In Proc.-4th IEEE Int. Conf. on AI and VR. 134 \u2013 138 . https:\/\/doi.org\/10.1109\/AIVR52153. 2021 .00031 10.1109\/AIVR52153.2021.00031 L. Gramoli, J. Lacoche, A. Foulonneau, V. Gouranton, and B. Arnaldi. 2021. Needs Model for an Autonomous Agent during Long-term Simulations. In Proc.-4th IEEE Int. Conf. on AI and VR. 134 \u2013 138. https:\/\/doi.org\/10.1109\/AIVR52153.2021.00031"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"crossref","unstructured":"M. Hassenzahl M. Burmester and F. Koller. 2003. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualit\u00e4t. Mensch & Computer (2003) 187 \u2013 196.  M. Hassenzahl M. Burmester and F. Koller. 2003. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualit\u00e4t. Mensch & Computer (2003) 187 \u2013 196.","DOI":"10.1007\/978-3-322-80058-9_19"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"K.\u00a0S. Hone and R. Graham. 2000. Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI). Natural Language Engineering (2000) 287 \u2013 303.  K.\u00a0S. Hone and R. Graham. 2000. Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI). Natural Language Engineering (2000) 287 \u2013 303.","DOI":"10.1017\/S1351324900002497"},{"key":"e_1_3_2_1_14_1","unstructured":"ISO. 1998. Ergonomics of human-system interaction \u2014 Part 11: Usability: Definitions and concepts. https:\/\/www.iso.org\/obp\/ui\/#iso:std:iso:9241:-11:ed-2:v1:en  ISO. 1998. Ergonomics of human-system interaction \u2014 Part 11: Usability: Definitions and concepts. https:\/\/www.iso.org\/obp\/ui\/#iso:std:iso:9241:-11:ed-2:v1:en"},{"key":"e_1_3_2_1_15_1","unstructured":"ISO\/IEC. 2011. 25010:2011. Systems and software engineering \u2014 Systems and software Quality Requirements and Evaluation (SQuaRE) \u2014 System and software quality models.https:\/\/www.iso.org\/standard\/35733.html  ISO\/IEC. 2011. 25010:2011. Systems and software engineering \u2014 Systems and software Quality Requirements and Evaluation (SQuaRE) \u2014 System and software quality models.https:\/\/www.iso.org\/standard\/35733.html"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/INDIN.2018.8471951"},{"key":"e_1_3_2_1_17_1","volume-title":"Drivers with limited perception: model and application to traffic simulation. Recherche Transports S\u00e9curit\u00e9 2014 (1)","author":"Ketenci U.","year":"2014","unstructured":"U. Ketenci , R. Br\u00e9mond , J.-M. Auberlet , and E.\u00a0Grislin-Le Strugeon. 2014. Drivers with limited perception: model and application to traffic simulation. Recherche Transports S\u00e9curit\u00e9 2014 (1) ( 2014 ), 49 \u2013 63. U. Ketenci, R. Br\u00e9mond, J.-M. Auberlet, and E.\u00a0Grislin-Le Strugeon. 2014. Drivers with limited perception: model and application to traffic simulation. Recherche Transports S\u00e9curit\u00e9 2014 (1) (2014), 49 \u2013 63."},{"key":"#cr-split#-e_1_3_2_1_18_1.1","doi-asserted-by":"crossref","unstructured":"J. Kirakowski and M. Corbett. 2006. SUMI: the Software Usability Measurement Inventory. British Journal of Educational Technology 24 (10 2006) 210 - 212. https:\/\/doi.org\/10.1111\/j.1467-8535.1993.tb00076.x 10.1111\/j.1467-8535.1993.tb00076.x","DOI":"10.1111\/j.1467-8535.1993.tb00076.x"},{"key":"#cr-split#-e_1_3_2_1_18_1.2","doi-asserted-by":"crossref","unstructured":"J. Kirakowski and M. Corbett. 2006. SUMI: the Software Usability Measurement Inventory. British Journal of Educational Technology 24 (10 2006) 210 - 212. https:\/\/doi.org\/10.1111\/j.1467-8535.1993.tb00076.x","DOI":"10.1111\/j.1467-8535.1993.tb00076.x"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.09.009"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.apenergy.2015.11.033"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2762678"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2022.104290"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.07.001"},{"key":"#cr-split#-e_1_3_2_1_24_1.1","doi-asserted-by":"crossref","unstructured":"S. M\u00f6ller K.-P. Engelbrecht C. K\u00fchnel A. Naumann I. Wechsung and B. Weiss. 2010. Chapter 14 - Evaluation of Multimodal Interfaces for Ambient Intelligence. In Human-Centric Interfaces for Ambient Intelligence. Academic Press Oxford 347-370. https:\/\/doi.org\/10.1016\/B978-0-12-374708-2.00014-0 10.1016\/B978-0-12-374708-2.00014-0","DOI":"10.1016\/B978-0-12-374708-2.00014-0"},{"key":"#cr-split#-e_1_3_2_1_24_1.2","doi-asserted-by":"crossref","unstructured":"S. M\u00f6ller K.-P. Engelbrecht C. K\u00fchnel A. Naumann I. Wechsung and B. Weiss. 2010. Chapter 14 - Evaluation of Multimodal Interfaces for Ambient Intelligence. In Human-Centric Interfaces for Ambient Intelligence. Academic Press Oxford 347-370. https:\/\/doi.org\/10.1016\/B978-0-12-374708-2.00014-0","DOI":"10.1016\/B978-0-12-374708-2.00014-0"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01054-6_94"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/2594693.2594699"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32520-6_26"},{"key":"e_1_3_2_1_28_1","volume-title":"23rd Int. W. on Software Measurement and the 8th Int. Conf. on Software Process and Product Measurement. 81\u201389","author":"Petersen K.","year":"2013","unstructured":"K. Petersen and C. Gencel . 2013. Worldviews, Research Methods, and their Relationship to Validity in Empirical Software Engineering Research . In 23rd Int. W. on Software Measurement and the 8th Int. Conf. on Software Process and Product Measurement. 81\u201389 . https:\/\/doi.org\/10.1109\/IWSM-Mensura. 2013 .22 10.1109\/IWSM-Mensura.2013.22 K. Petersen and C. Gencel. 2013. Worldviews, Research Methods, and their Relationship to Validity in Empirical Software Engineering Research. In 23rd Int. W. on Software Measurement and the 8th Int. Conf. on Software Process and Product Measurement. 81\u201389. https:\/\/doi.org\/10.1109\/IWSM-Mensura.2013.22"},{"key":"#cr-split#-e_1_3_2_1_29_1.1","doi-asserted-by":"crossref","unstructured":"M. Petticrew and H. Roberts. 2006. Systematic reviews in the social sciences: A practical guide. Blackwell Publishing Ltd Oxford UK. https:\/\/doi.org\/10.1016\/B978-0-12-374708-2.00014-0 10.1016\/B978-0-12-374708-2.00014-0","DOI":"10.1016\/B978-0-12-374708-2.00014-0"},{"key":"#cr-split#-e_1_3_2_1_29_1.2","doi-asserted-by":"crossref","unstructured":"M. Petticrew and H. Roberts. 2006. Systematic reviews in the social sciences: A practical guide. Blackwell Publishing Ltd Oxford UK. https:\/\/doi.org\/10.1016\/B978-0-12-374708-2.00014-0","DOI":"10.1002\/9780470754887"},{"key":"e_1_3_2_1_30_1","first-page":"2","article-title":"Evaluating the user experience of a mobile user in a smart city context","volume":"3","author":"Rizopoulos C.","year":"2015","unstructured":"C. Rizopoulos , K. Diamantaki , D. Charitos , V. Tsetsos , P. Kovatsis , and I. Theona . 2015 . Evaluating the user experience of a mobile user in a smart city context . Int. J. of Intelligent Engineering Informatics 3 , 2 - 3 , SI (2015), 120\u2013143. https:\/\/doi.org\/10.1504\/IJIEI.2015.069902 10.1504\/IJIEI.2015.069902 C. Rizopoulos, K. Diamantaki, D. Charitos, V. Tsetsos, P. Kovatsis, and I. Theona. 2015. Evaluating the user experience of a mobile user in a smart city context. Int. J. of Intelligent Engineering Informatics 3, 2-3, SI (2015), 120\u2013143. https:\/\/doi.org\/10.1504\/IJIEI.2015.069902","journal-title":"Int. J. of Intelligent Engineering Informatics"},{"key":"e_1_3_2_1_31_1","volume-title":"Interactive context-aware systems interacting with ambient intelligence. Ambient intelligence","author":"Schmidt A.","year":"2005","unstructured":"A. Schmidt . 2005. Interactive context-aware systems interacting with ambient intelligence. Ambient intelligence ( 2005 ), 159\u2013179. A. Schmidt. 2005. Interactive context-aware systems interacting with ambient intelligence. Ambient intelligence (2005), 159\u2013179."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"crossref","unstructured":"E.\u00a0Grislin-Le Strugeon K.\u00a0M. de Oliveira D. Zekri and M. Thilliez. 2021. Agent mining approaches: An ontological view.The Knowledge Engineering Review 36 (2021) E12. https:\/\/doi.org\/DOI=10.1017\/S0269888921000114  E.\u00a0Grislin-Le Strugeon K.\u00a0M. de Oliveira D. Zekri and M. Thilliez. 2021. Agent mining approaches: An ontological view.The Knowledge Engineering Review 36 (2021) E12. https:\/\/doi.org\/DOI=10.1017\/S0269888921000114","DOI":"10.1017\/S0269888921000114"},{"key":"e_1_3_2_1_33_1","volume-title":"Systems Research, and World\u00a0Health Organization.","author":"Tricco C.","year":"2017","unstructured":"A.\u00a0 C. Tricco , E.\u00a0 V. Langlois , S.\u00a0 E. Straus , Alliance for Health\u00a0Policy , Systems Research, and World\u00a0Health Organization. 2017 . Rapid reviews to strengthen health policy and systems: a practical guide. World Health Organization . xix, 119 p. pages. A.\u00a0C. Tricco, E.\u00a0V. Langlois, S.\u00a0E. Straus, Alliance for Health\u00a0Policy, Systems Research, and World\u00a0Health Organization. 2017. Rapid reviews to strengthen health policy and systems: a practical guide. World Health Organization. xix, 119 p. pages."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3233\/TAD-160453"},{"key":"e_1_3_2_1_35_1","volume-title":"35th Annual Meeting of the A. for Computational Linguistics and 8th Conf. of the European Chapter of the A. for Computational Linguistics","author":"Walker A.","unstructured":"M.\u00a0 A. Walker , D.\u00a0 J. Litman , C.\u00a0 A. Kamm , and A. Abella . 1997. PARADISE: A framework for evaluating spoken dialogue agents . In 35th Annual Meeting of the A. for Computational Linguistics and 8th Conf. of the European Chapter of the A. for Computational Linguistics . Madrid, Spain, 271\u2013280. M.\u00a0A. Walker, D.\u00a0J. Litman, C.\u00a0A. Kamm, and A. Abella. 1997. PARADISE: A framework for evaluating spoken dialogue agents. In 35th Annual Meeting of the A. for Computational Linguistics and 8th Conf. of the European Chapter of the A. for Computational Linguistics. Madrid, Spain, 271\u2013280."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2015.05.001"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/329124.329126"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_1_39_1","volume-title":"An Introduction to MultiAgent Systems","author":"Wooldridge M.","unstructured":"M. Wooldridge . 2009. An Introduction to MultiAgent Systems , 2 nd Edition. Wiley . M. Wooldridge. 2009. An Introduction to MultiAgent Systems, 2nd Edition. Wiley.","edition":"2"}],"event":{"name":"EICS '23: The 15th ACM SIGCHI Symposium on Engineering Interactive Computing Systems","location":"Swansea United Kingdom","acronym":"EICS '23","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Companion Proceedings of the 2023 ACM SIGCHI Symposium on Engineering Interactive Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596454.3597177","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3596454.3597177","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:48:00Z","timestamp":1750178880000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596454.3597177"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,27]]},"references-count":42,"alternative-id":["10.1145\/3596454.3597177","10.1145\/3596454"],"URL":"https:\/\/doi.org\/10.1145\/3596454.3597177","relation":{},"subject":[],"published":{"date-parts":[[2023,6,27]]},"assertion":[{"value":"2023-06-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}