{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T18:05:37Z","timestamp":1754157937540,"version":"3.41.2"},"reference-count":22,"publisher":"Emerald","issue":"2","license":[{"start":{"date-parts":[[2010,5,4]],"date-time":"2010-05-04T00:00:00Z","timestamp":1272931200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010,5,4]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The purpose of this paper is to present a methodology combining experimental and inquiry methods used for software usability evaluation. The software product of LeViS project funded by the European Commission (Socrates\/Lingua II) is used as an evaluation paradigm. The aim of the paper is twofold: to present the results of the usability evaluation using this software as an example and to suggest a number of improvements for the next version of the software tool; and to portray the advantages of combining methods from different evaluation approaches and the experiences from their application.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>The evaluation for this experiment combined different usability methods, both experimental and inquiry ones. More specifically, the methods employed were the Thinking Aloud Protocol and the User Logging, which were performed in a usability evaluation laboratory, as well as the inquiry methods of Interview and Focus Group.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>In this study, usability problems regarding the Learning via Subtitling (LvS) educational software were revealed as well as issues regarding the use of Thinking Aloud Protocol method and involving users with a specific profile. The research findings presented in this paper constitute an innovative and effective methodology for software usability evaluation and are useful for laboratories aiming to conduct similar evaluations.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Research limitations\/implications<\/jats:title><jats:p>Although this methodology has been successfully applied for over 20 software products, due to practical purposes related to this paper's extent, only one software is used as an example.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>Through the evaluation process, apart from discovering certain usability problems related to the software, there are a number of important conclusions drawn, regarding the methods used and the methodology followed in software usability evaluation.<\/jats:p><\/jats:sec>","DOI":"10.1108\/13287261011042921","type":"journal-article","created":{"date-parts":[[2010,5,8]],"date-time":"2010-05-08T07:07:10Z","timestamp":1273302430000},"page":"120-139","source":"Crossref","is-referenced-by-count":2,"title":["Combining experimental and inquiry methods in software usability evaluation"],"prefix":"10.1108","volume":"12","author":[{"given":"Nektarios","family":"Kostaras","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitris","family":"Stavrinoudis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stavroula","family":"Sokoli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michalis","family":"Xenos","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022021020305545000_b1","unstructured":"Avouris, N.M. (2001), \u201cAn introduction to software usability. Workshop on software usability\u201d, Proceedings of 8th Panhellenic Conference on Informatics, Nicosia, Livanis Publications, Athens, Vol. 2, pp. 514\u201022."},{"key":"key2022021020305545000_b2","unstructured":"Bevan, N. (2008), \u201cClassifying and selecting UX and usability measures\u201d, Proceedings of Meaningful Measures: Valid Useful User Experience Measurement (VUUM), 5th COST294\u2010MAUSE Open Workshop, Reykjavik, 18 June, pp. 13\u201018."},{"key":"key2022021020305545000_b3","unstructured":"Dix, A., Finlay, J., Abowd, G. and Beale, R. (2004), Human\u2010Computer Interaction, 3rd ed., Prentice Hall, New Jersey, NJ."},{"key":"key2022021020305545000_b4","unstructured":"Dumas, J.S. and Redish, J.C. (1999), A Practical Guide to Usability Testing (Revised Edition), Intellect, Exeter."},{"key":"key2022021020305545000_b5","unstructured":"Hellenic Open University (HOU) (2009), available at: www.eap.gr\/ (accessed 8 December)."},{"key":"key2022021020305545000_b6","unstructured":"ISO 9241\u201011 (1997), Draft International Standard on Ergonomic Requirements for Office Work with Visual Display Terminals (VDT), Part 11: Guidance on Usability, ISO, Geneva."},{"key":"key2022021020305545000_b7","unstructured":"ISO\/IEC 9126 (2001), Software Product Evaluation \u2013 Quality Characteristics and Guidelines for the User, International Organization for Standardization, Geneva."},{"key":"key2022021020305545000_b8","doi-asserted-by":"crossref","unstructured":"Ivory, M. and Hearst, M. (2001), \u201cThe state of the art in automating usability evaluation of user interfaces\u201d, ACM Computing Surveys, Vol. 33 No. 4, pp. 470\u2010516.","DOI":"10.1145\/503112.503114"},{"key":"key2022021020305545000_b9","doi-asserted-by":"crossref","unstructured":"Jeffries, R., Miller, J.R., Wharton, C. and Uueda, K.M. (1991), \u201cUser interface evaluation in the real world: a comparison of four techniques\u201d, Proceedings of the Conference on Human Factors in Computing Systems, New Orleans, LA, April, ACM Press, New York, NY, pp. 119\u201024.","DOI":"10.1145\/108844.108862"},{"key":"key2022021020305545000_b10","doi-asserted-by":"crossref","unstructured":"Jokela, T., Koivumaa, J., Pirkola, J., Salminen, P. and Kantola, N. (2006), \u201cMethods for quantitative usability requirements: a case study on the development of the user interface of a mobile phone\u201d, Personal and Ubiquitous Computing, Vol. 10, pp. 345\u201055.","DOI":"10.1007\/s00779-005-0050-7"},{"key":"key2022021020305545000_b11","doi-asserted-by":"crossref","unstructured":"Kennedy, S. (1989), \u201cUsing video in the BNR usability lab\u201d, SIGCHI Bulletin, Vol. 21 No. 2, pp. 92\u20105.","DOI":"10.1145\/70609.70624"},{"key":"key2022021020305545000_b12","doi-asserted-by":"crossref","unstructured":"Matera, M., Rizzo, F. and Carughi, G.T. (2006), \u201cWeb usability: principles and evaluation methods\u201d, in Mosley, N. and Mendes, E. (Eds), Web Engineering, Springer, New York, NY, pp. 143\u201056.","DOI":"10.1007\/3-540-28218-1_5"},{"key":"key2022021020305545000_b13","doi-asserted-by":"crossref","unstructured":"Molich, R., Thomsen, A.D., Karyukina, B., Schmidt, L., Ede, M., Van Oel, W. and Arcuri, M. (1999), \u201cComparative evaluation of usability tests\u201d, Proceedings of the Conference on Human Factors in Computing Systems, Pittsburgh, PA, May, ACM Press, New York, NY, pp. 83\u20106.","DOI":"10.1145\/632716.632768"},{"key":"key2022021020305545000_b14","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1993), Usability Engineering, Academic Press, Boston, MA.","DOI":"10.1016\/B978-0-08-052029-2.50007-3"},{"key":"key2022021020305545000_b15","doi-asserted-by":"crossref","unstructured":"Nielsen, J. and Landauer, T.K. (1993), \u201cA mathematical model of the finding of usability problems\u201d, Proceedings of ACM INTERCHI'93 Conference, Amsterdam, 24\u201029 April, IOS Press, pp. 206\u201013.","DOI":"10.1145\/169059.169166"},{"key":"key2022021020305545000_b16","unstructured":"Rubin, J. (1994), Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, John Wiley & Sons, New York, NY."},{"key":"key2022021020305545000_b17","unstructured":"Schneiderman, B. (1998), Designing the User Interface, Addison Wesley, Menlo Park, CA."},{"key":"key2022021020305545000_b18","unstructured":"Sharp, H., Rogers, Y. and Preece, J. (2007), Interaction Design: Beyond Human\u2010Computer Interaction, 2nd ed., John Wiley & Sons, New York, NY."},{"key":"key2022021020305545000_b19","unstructured":"Software Quality Research Group (SQRG) (2009), available at: http:\/\/quality.eap.gr\/ (accessed 8 December)."},{"key":"key2022021020305545000_b20","doi-asserted-by":"crossref","unstructured":"Stavrinoudis, D., Xenos, M., Peppas, P. and Christodoulakis, D. (2005), \u201cEarly estimation of users' perception of software quality\u201d, Software Quality Journal, Vol. 13 No. 2, pp. 155\u201075.","DOI":"10.1007\/s11219-005-6215-9"},{"key":"key2022021020305545000_b21","doi-asserted-by":"crossref","unstructured":"Tselios, N., Avouris, N. and Komis, V. (2008), \u201cThe effective combination of hybrid usability methods in evaluating educational applications of ICT: issues and challenges\u201d, Education and Information Technologies Journal, Vol. 13 No. 1, pp. 55\u201076.","DOI":"10.1007\/s10639-007-9045-5"},{"key":"key2022021020305545000_b22","doi-asserted-by":"crossref","unstructured":"Wixon, D. and Wilson, C. (1997), \u201cThe usability engineering framework for product design and evaluation\u201d, in Helander, M.G., Landauer, T.K. and Prabju, P.V. (Eds), Handbook of Human\u2010Computer Interaction, Elsevier, Amsterdam, pp. 653\u201088.","DOI":"10.1016\/B978-044481862-1.50093-5"}],"container-title":["Journal of Systems and Information Technology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/13287261011042921","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/13287261011042921\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/13287261011042921\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:24:18Z","timestamp":1753403058000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jsit\/article\/12\/2\/120-139\/246450"}},"subtitle":["The paradigm of LvS educational software"],"short-title":[],"issued":{"date-parts":[[2010,5,4]]},"references-count":22,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2010,5,4]]}},"alternative-id":["10.1108\/13287261011042921"],"URL":"https:\/\/doi.org\/10.1108\/13287261011042921","relation":{},"ISSN":["1328-7265"],"issn-type":[{"type":"print","value":"1328-7265"}],"subject":[],"published":{"date-parts":[[2010,5,4]]}}}