{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T04:38:35Z","timestamp":1778215115006,"version":"3.51.4"},"reference-count":136,"publisher":"Maximum Academic Press","issue":"3","license":[{"start":{"date-parts":[[2008,9,1]],"date-time":"2008-09-01T00:00:00Z","timestamp":1220227200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2008,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This literature review focuses on user-centered evaluation (UCE) studies of adaptive and adaptable systems. Usability, perceived usefulness and appropriateness of adaptation are the three most commonly assessed variables. Questionnaires appeared to be the most popular method, followed by interviews and data log analysis. The quality of most questionnaires was questionable, and the reporting of interviews and think-aloud protocols was found to be shallow. Furthermore, data logs need triangulation in order to be useful. The reports encountered lack empirical value. The article models the iterative design process for adaptive and adaptable systems, linked to the goals of UCE: supporting decisions, detecting problems and verifying quality. This model summarizes the variables to be assessed at each stage of the process and the relevant methods to assess them.<\/jats:p>","DOI":"10.1017\/s0269888908001379","type":"journal-article","created":{"date-parts":[[2008,9,3]],"date-time":"2008-09-03T08:47:15Z","timestamp":1220431635000},"page":"261-281","source":"Crossref","is-referenced-by-count":110,"title":["User-centered evaluation of adaptive and adaptable systems: a literature review"],"prefix":"10.48130","volume":"23","author":[{"given":"LEX","family":"VAN VELSEN","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"THEA","family":"VAN DER GEEST","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"ROB","family":"KLAASSEN","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"MICHA\u00cbL","family":"STEEHOUDER","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"27968","published-online":{"date-parts":[[2008,9,1]]},"reference":[{"key":"S0269888908001379_ref136","volume-title":"Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices and Services","author":"Yue","year":"2005"},{"key":"S0269888908001379_ref134","volume-title":"Evaluation of Adaptive Systems","author":"Weibelzahl","year":"2003"},{"key":"S0269888908001379_ref130","doi-asserted-by":"publisher","DOI":"10.1080\/13614569708914688"},{"key":"S0269888908001379_ref128","doi-asserted-by":"publisher","DOI":"10.1080\/0044929021000077724"},{"key":"S0269888908001379_ref126","doi-asserted-by":"publisher","DOI":"10.1145\/345124.345161"},{"key":"S0269888908001379_ref125","volume-title":"Paper Presented at Designing Interactive Systems: Across the Spectrum","author":"Smith","year":"2004"},{"key":"S0269888908001379_ref118","doi-asserted-by":"publisher","DOI":"10.1007\/11768012_14"},{"key":"S0269888908001379_ref115","volume-title":"Paper Presented at the Fifth Workshop on User-Centred Design and Evaluation of Adaptive Systems, Held in Conjunction with the 4th International Conference on Adaptive Hypermedia and Adaptive Web Based Systems (AH'06)","author":"Kuiper","year":"2006"},{"key":"S0269888908001379_ref109","doi-asserted-by":"publisher","DOI":"10.1147\/sj.424.0686"},{"key":"S0269888908001379_ref106","doi-asserted-by":"publisher","DOI":"10.1109\/TCE.2003.1261192"},{"key":"S0269888908001379_ref104","article-title":"My E-journal: exploring the usefulness of personalized access to scholarly articles and services","volume":"9","author":"Hyldegaard","year":"2004","journal-title":"Information Research"},{"key":"S0269888908001379_ref103","volume-title":"Paper Presented at IUI '97","author":"H\u00f6\u00f6k","year":"1997"},{"key":"S0269888908001379_ref100","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8535.00331"},{"key":"S0269888908001379_ref099","volume-title":"Paper Presented at IUI'05","author":"Graziola","year":"2005"},{"key":"S0269888908001379_ref095","volume-title":"Paper Presented at IUI'02","author":"Gena","year":"2002"},{"key":"S0269888908001379_ref094","doi-asserted-by":"publisher","DOI":"10.1080\/13614569808914697"},{"key":"S0269888908001379_ref093","volume-title":"Paper Presented at the Tenth International Conference World Wide Web","author":"Field","year":"2001"},{"key":"S0269888908001379_ref092","doi-asserted-by":"publisher","DOI":"10.1108\/EUM0000000006924"},{"key":"S0269888908001379_ref086","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-27780-4_35"},{"key":"S0269888908001379_ref046","doi-asserted-by":"publisher","DOI":"10.1006\/ijhc.2001.0503"},{"key":"S0269888908001379_ref091","doi-asserted-by":"publisher","DOI":"10.1108\/14684520110395272"},{"key":"S0269888908001379_ref031","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2006.10.001"},{"key":"S0269888908001379_ref047","volume-title":"Qualitative Data Analysis","author":"Miles","year":"1994"},{"key":"S0269888908001379_ref129","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(02)00014-0"},{"key":"S0269888908001379_ref124","volume-title":"Paper Presented at the Workshop on HCI in Mobile Guides","author":"Schmidt-Belz","year":"2003"},{"key":"S0269888908001379_ref044","doi-asserted-by":"publisher","DOI":"10.1109\/47.649557"},{"key":"S0269888908001379_ref067","volume-title":"Paper Presented at the SIGHI Conference on Human Factors in Computing Systems: Common Ground","author":"Virzi","year":"1996"},{"key":"S0269888908001379_ref117","first-page":"163","article-title":"An evaluation of a multiple interface design solution for bloated software","volume":"4","author":"McGrenere","year":"2002","journal-title":"CHI Letters"},{"key":"S0269888908001379_ref057","doi-asserted-by":"publisher","DOI":"10.1016\/S1389-1286(99)00030-4"},{"key":"S0269888908001379_ref005","doi-asserted-by":"publisher","DOI":"10.1016\/0950-7051(93)90012-I"},{"key":"S0269888908001379_ref013","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310555"},{"key":"S0269888908001379_ref008","doi-asserted-by":"publisher","DOI":"10.1016\/S1386-5056(98)00170-1"},{"key":"S0269888908001379_ref078","first-page":"235","article-title":"Using evaluation in the design of an intelligent tutoring system","volume":"8","author":"Bloom","year":"1997","journal-title":"Journal of Interactive Learning Research"},{"key":"S0269888908001379_ref049","doi-asserted-by":"publisher","DOI":"10.1007\/11768012_14"},{"key":"S0269888908001379_ref116","volume-title":"Paper Presented at the Fifth Workshop on User-Centred Design and Evaluation of Adaptive Systems, Held in Conjunction with the 4th International Conference on Adaptive Hypermedia and Adaptive Web Based Systems (AH'06)","author":"Masthoff","year":"2006"},{"key":"S0269888908001379_ref019","doi-asserted-by":"publisher","DOI":"10.1080\/13614569808914697"},{"key":"S0269888908001379_ref028","doi-asserted-by":"publisher","DOI":"10.1207\/S15327590IJHC1501_14"},{"key":"S0269888908001379_ref127","doi-asserted-by":"publisher","DOI":"10.1016\/S1389-1286(99)00030-4"},{"key":"S0269888908001379_ref111","doi-asserted-by":"publisher","DOI":"10.1046\/j.0266-4909.2003.00037.x"},{"key":"S0269888908001379_ref021","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888905000299"},{"key":"S0269888908001379_ref119","volume-title":"Paper Presented at IUI'04","author":"O\u2019Grady","year":"2004"},{"key":"S0269888908001379_ref102","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(98)00037-X"},{"key":"S0269888908001379_ref063","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2004.07.007"},{"key":"S0269888908001379_ref114","doi-asserted-by":"publisher","DOI":"10.1147\/sj.424.0582"},{"key":"S0269888908001379_ref101","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-005-2304-5"},{"key":"S0269888908001379_ref027","volume-title":"Forward, Back and Home Again. Analyzing User Behaviour on the Web","author":"Herder","year":"2006"},{"key":"S0269888908001379_ref079","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45756-9_13"},{"key":"S0269888908001379_ref120","volume-title":"Proceedings of the 2004 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on IT Research in Developing Countries","author":"Patel","year":"2004"},{"key":"S0269888908001379_ref108","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-002-0214-7"},{"key":"S0269888908001379_ref087","volume-title":"Paper Presented at the Artificial Intelligence in Education Conference","author":"Cox","year":"1999"},{"key":"S0269888908001379_ref001","volume-title":"User interfaces for All","author":"Akoumianakis","year":"2001"},{"key":"S0269888908001379_ref016","volume-title":"Paper Presented at the 20th Annual International Conference on Computer Documentation","author":"Dicks","year":"2002"},{"key":"S0269888908001379_ref098","volume-title":"Paper Presented at the Ninth International Conference on Intelligent User Interface","author":"Goren-Bar","year":"2004"},{"key":"S0269888908001379_ref135","volume-title":"Paper Presented at the Fifth Workshop on User-Centered Design and Evaluation of Adaptive Systems, Held in Conjunction with the 4th International Conference on Adaptive Hypermedia and Adaptive Web Based Systems (Atl '06)","author":"Weibelzahl","year":"2006"},{"key":"S0269888908001379_ref110","volume-title":"Paper Presented at the Proceedings of the Mediterranean Electrotechnical Conference\u2014MELECON","author":"Kavcic","year":"2002"},{"key":"S0269888908001379_ref002","volume-title":"Human Computer Interaction Research in Web Design and Evaluation","author":"Alpert","year":"2007"},{"key":"S0269888908001379_ref050","doi-asserted-by":"publisher","DOI":"10.1300\/J136v03n02_09"},{"key":"S0269888908001379_ref015","doi-asserted-by":"publisher","DOI":"10.1177\/1050651997011004003"},{"key":"S0269888908001379_ref090","doi-asserted-by":"publisher","DOI":"10.1007\/11527886_63"},{"key":"S0269888908001379_ref070","first-page":"17","article-title":"Advantages, opportunities, and limits of empirical evaluations: Evaluating adaptive systems","volume":"3","author":"Weibelzahl","year":"2002","journal-title":"K\u00fcnstliche Intelligenz"},{"key":"S0269888908001379_ref035","volume-title":"Human-Computer Interaction Handbook","author":"Jameson","year":"2006"},{"key":"S0269888908001379_ref020","doi-asserted-by":"publisher","DOI":"10.1080\/08839510490463442"},{"key":"S0269888908001379_ref080","volume-title":"Paper Presented at Webnet 98 World Conference of the WWW, Internet, and Intranet,","author":"Brusilovsky","year":"1998"},{"key":"S0269888908001379_ref088","volume-title":"Museums and the Web 2003: Selected Papers from an International Conference","author":"De Almeida","year":"2003"},{"key":"S0269888908001379_ref107","volume-title":"Hybridmedia as a Tool to Deliver Personalised Product-Specific Information About Food. Report of the TIVIK Project","author":"J\u00e4rvinen","year":"2005"},{"key":"S0269888908001379_ref071","volume-title":"Evaluation of Adaptive Systems. PhD thesis","author":"Weibelzahl","year":"2003"},{"key":"S0269888908001379_ref097","unstructured":"Goren-Bar D. , Graziola I. , Kuflik T. , Pianesi F. , Rocchi C. , Stock O . & Zancanaro M. 2005. I like it: An affective interface for a multimodal museum guide. Retrieved on 20 April 2006 from http:\/\/peach.itc.it\/papers\/gorenbar2005.pdf"},{"key":"S0269888908001379_ref003","doi-asserted-by":"publisher","DOI":"10.1016\/S1386-5056(03)00059-5"},{"key":"S0269888908001379_ref065","volume-title":"Paper Presented at the IEEE Professional Communication Conference","author":"Van Velsen","year":"2007"},{"key":"S0269888908001379_ref004","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-7206(01)00085-4"},{"key":"S0269888908001379_ref006","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45756-9_13"},{"key":"S0269888908001379_ref007","volume-title":"ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals","year":"1998"},{"key":"S0269888908001379_ref009","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008350913145"},{"key":"S0269888908001379_ref010","volume-title":"Paper Presented at the Second International Workshop on Community Networking \u2018Integrated Multimedia Services to the Home\u2019","author":"Chesnais","year":"1995"},{"key":"S0269888908001379_ref073","volume-title":"Paper Presented at the User Centered Design and Evaluation Workshop Held in Conjunction with Adaptive Hypermedia '06","author":"Weibelzahl","year":"2006"},{"key":"S0269888908001379_ref011","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-005-1269-8"},{"key":"S0269888908001379_ref040","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1027796"},{"key":"S0269888908001379_ref017","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/5657.001.0001","volume-title":"Protocol Analysis","author":"Ericsson","year":"1993"},{"key":"S0269888908001379_ref022","volume-title":"The Adaptive Web","author":"Gena","year":"2007"},{"key":"S0269888908001379_ref029","volume-title":"Paper Presented at IUI '97","author":"H\u00f6\u00f6k","year":"1997"},{"key":"S0269888908001379_ref122","doi-asserted-by":"publisher","DOI":"10.1080\/0268051042000224789"},{"key":"S0269888908001379_ref023","unstructured":"Goren-Bar D. , Graziola I. , Kuflik T. , Pianesi F. , Rocchi C. , Stock O . & Zancanaro M. 2005. I like it: An affective interface for a multimodal museum guide. Retrieved on 20 April 2006 from http:\/\/peach.itc.it\/papers\/gorenbar2005.pdf"},{"key":"S0269888908001379_ref025","doi-asserted-by":"publisher","DOI":"10.1207\/S15327590IJHC1501_13"},{"key":"S0269888908001379_ref032","first-page":"paper 181","article-title":"My E-journal: Exploring the usefulness of personalized access to scholarly articles and services","volume":"9","author":"Hyldegaard","year":"2004","journal-title":"Information Research"},{"key":"S0269888908001379_ref072","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-59140-567-2.ch015"},{"key":"S0269888908001379_ref042","volume-title":"Context-Aware Services for Mobile Users (Technology and User Experiences)","author":"Kolari","year":"2004"},{"key":"S0269888908001379_ref096","doi-asserted-by":"publisher","DOI":"10.1080\/08839510490463442"},{"key":"S0269888908001379_ref068","first-page":"471","article-title":"A survey of user-centered design practice","volume":"4","author":"Vredenburg","year":"2002","journal-title":"CHI Letters"},{"key":"S0269888908001379_ref043","doi-asserted-by":"publisher","DOI":"10.1145\/345124.345139"},{"key":"S0269888908001379_ref132","doi-asserted-by":"publisher","DOI":"10.1016\/0950-7051(93)90014-K"},{"key":"S0269888908001379_ref018","volume-title":"Paper Presented at the Tenth International Conference World Wide Web","author":"Field","year":"2001"},{"key":"S0269888908001379_ref026","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(98)00037-X"},{"key":"S0269888908001379_ref105","doi-asserted-by":"publisher","DOI":"10.1109\/TCE.2005.1468017"},{"key":"S0269888908001379_ref060","doi-asserted-by":"publisher","DOI":"10.1080\/13614569708914688"},{"key":"S0269888908001379_ref030","doi-asserted-by":"publisher","DOI":"10.1016\/S0953-5438(99)00006-5"},{"key":"S0269888908001379_ref033","volume-title":"ISO 13407: Human-Centered Design Processes for Interactive Systems","year":"1999"},{"key":"S0269888908001379_ref133","first-page":"53","volume":"44","author":"Virvou","year":"2005","journal-title":"Mobile educational features in authoring tools for personalised tutoring"},{"key":"S0269888908001379_ref034","volume-title":"In Human-Computer Interaction Handbook","author":"Jameson","year":"2003"},{"key":"S0269888908001379_ref112","volume-title":"Paper Presented at CHI 2004","author":"Kiris","year":"2004"},{"key":"S0269888908001379_ref036","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-1699(99)00074-5"},{"key":"S0269888908001379_ref037","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-002-0214-7"},{"key":"S0269888908001379_ref089","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(00)00019-4"},{"key":"S0269888908001379_ref039","doi-asserted-by":"publisher","DOI":"10.1046\/j.0266-4909.2003.00037.x"},{"key":"S0269888908001379_ref131","first-page":"91","article-title":"Evaluating the usability of web-based learning tools","volume":"5","author":"Storey","year":"2002","journal-title":"Educational Technology and Society"},{"key":"S0269888908001379_ref041","doi-asserted-by":"publisher","DOI":"10.1080\/01449290512331319030"},{"key":"S0269888908001379_ref045","volume-title":"Paper Presented at the Second Workshop on Empirical Evaluation of Adaptive Systems Held in Conjunction with User Modelling 2003","author":"Magoulas","year":"2003"},{"key":"S0269888908001379_ref014","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.35.8.982"},{"key":"S0269888908001379_ref048","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.soc.22.1.129"},{"key":"S0269888908001379_ref038","doi-asserted-by":"publisher","DOI":"10.1147\/sj.424.0686"},{"key":"S0269888908001379_ref064","volume-title":"Paper Presented at HCI. 2004: Design for Life","author":"Van der Geest","year":"2004"},{"key":"S0269888908001379_ref051","volume-title":"Undertaking Systematic Reviews of Research on Effectiveness","year":"2001"},{"key":"S0269888908001379_ref024","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8535.00331"},{"key":"S0269888908001379_ref052","doi-asserted-by":"publisher","DOI":"10.1007\/11573036_19"},{"key":"S0269888908001379_ref121","doi-asserted-by":"publisher","DOI":"10.1007\/11573036_19"},{"key":"S0269888908001379_ref113","volume-title":"Context-Aware Services for Mobile Users Technology and User Experiences","author":"Kolari","year":"2004"},{"key":"S0269888908001379_ref054","volume-title":"Paper presented at CHI'96","author":"Savage","year":"1996"},{"key":"S0269888908001379_ref077","volume-title":"Paper Presented at the 5th international symposium on wearable computers","author":"Biemans","year":"2001"},{"key":"S0269888908001379_ref055","volume-title":"Paper Presented at the Workshop on HCI in Mobile Guides","author":"Schmidt-Belz","year":"2003"},{"key":"S0269888908001379_ref059","doi-asserted-by":"publisher","DOI":"10.1080\/0044929021000077724"},{"key":"S0269888908001379_ref056","volume-title":"Paper Presented at Designing Interactive Systems: Across the Spectrum","author":"Smith","year":"2004"},{"key":"S0269888908001379_ref062","doi-asserted-by":"publisher","DOI":"10.1080\/0044929031000"},{"key":"S0269888908001379_ref085","doi-asserted-by":"publisher","DOI":"10.1007\/11768012_10"},{"key":"S0269888908001379_ref058","doi-asserted-by":"publisher","DOI":"10.4135\/9781412986038"},{"key":"S0269888908001379_ref053","volume-title":"Qualitative Research and Evaluation Methods","author":"Patton","year":"2002"},{"key":"S0269888908001379_ref061","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90007-M"},{"key":"S0269888908001379_ref066","doi-asserted-by":"publisher","DOI":"10.2307\/30036540"},{"key":"S0269888908001379_ref069","volume-title":"Paper Presented at the 46th Annual Meeting of the Human Factors and Ergonomics Society","author":"Walker","year":"2002"},{"key":"S0269888908001379_ref074","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026201108015"},{"key":"S0269888908001379_ref123","doi-asserted-by":"publisher","DOI":"10.1007\/11768012_22"},{"key":"S0269888908001379_ref075","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2003.12.003"},{"key":"S0269888908001379_ref076","doi-asserted-by":"publisher","DOI":"10.1023\/B:JARS.0000044872.51237.c9"},{"key":"S0269888908001379_ref082","volume-title":"Paper Presented at the Second International Workshop on Community Networking \u2018Integrated Multimedia Services to the Home\u2019","author":"Chesnais","year":"1995"},{"key":"S0269888908001379_ref081","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008350913145"},{"key":"S0269888908001379_ref083","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-005-1269-8"},{"key":"S0269888908001379_ref012","doi-asserted-by":"publisher","DOI":"10.1023\/A:1011127315884"},{"key":"S0269888908001379_ref084","first-page":"17","article-title":"Developing a context-aware electronic tourist guide: Some issues and experiences","volume":"2","author":"Cheverst","year":"2000","journal-title":"CHI Letters"}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888908001379","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:43:46Z","timestamp":1767624226000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888908001379\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,9]]},"references-count":136,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2008,9]]}},"alternative-id":["S0269888908001379"],"URL":"https:\/\/doi.org\/10.1017\/s0269888908001379","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"value":"0269-8889","type":"print"},{"value":"1469-8005","type":"electronic"}],"subject":[],"published":{"date-parts":[[2008,9]]}}}