{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:03:46Z","timestamp":1761581026365},"reference-count":44,"publisher":"IGI Global","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Domain-Specific Languages (DSLs) can be regarded as User Interfaces (UIs) because they bridge the gap between the domain experts and the computation platforms. Usability of DSLs by domain experts is a key factor for their successful adoption. The few reports supporting improvement claims are persuasive, but mostly anecdotal. Systematic literature reviews show that evidences on the effects of the introduction of DSLs are actually very scarce. In particular, the evaluation of usability is often skipped, relaxed, or at least omitted from papers reporting the development of DSLs. The few exceptions mostly take place at the end of the development process, when fixing problems is already too expensive. A systematic approach, based on techniques for the experimental evaluation of UIs, should be used to assess suitability of new DSLs. This chapter presents a general experimental evaluation model, tailored for DSLs\u2019 experimental evaluation, and instantiates it in several DSL\u2019s evaluation examples.<\/jats:p>","DOI":"10.4018\/978-1-4666-2092-6.ch014","type":"book-chapter","created":{"date-parts":[[2012,9,21]],"date-time":"2012-09-21T13:39:19Z","timestamp":1348234759000},"page":"386-407","source":"Crossref","is-referenced-by-count":5,"title":["Evaluating the Usability of Domain-Specific Languages"],"prefix":"10.4018","author":[{"given":"Ankica","family":"Bari\u0161ic","sequence":"first","affiliation":[{"name":"Universidade Nova de Lisboa, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vasco","family":"Amaral","sequence":"additional","affiliation":[{"name":"Universidade Nova de Lisboa, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miguel","family":"Goul\u00e3o","sequence":"additional","affiliation":[{"name":"Universidade Nova de Lisboa, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bruno","family":"Barroca","sequence":"additional","affiliation":[{"name":"Universidade Nova de Lisboa, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"2432","reference":[{"key":"978-1-4666-2092-6.ch014.-1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2003.1231149"},{"key":"978-1-4666-2092-6.ch014.-2","doi-asserted-by":"crossref","unstructured":"Bari\u0161i\u0107, A., Amaral, V., Goul\u00e3o, M., & Barroca, B. (2011). Quality in use of domain specific languages: A case study. In 3rd ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2011) (pp. 65-72). Portland, USA.","DOI":"10.1145\/2089155.2089170"},{"key":"978-1-4666-2092-6.ch014.-3","author":"A.Bari\u0161i\u0107","year":"2011","journal-title":"How to reach a usable DSL? Moving toward a systematic evaluation. Electronic Communications of the EASST"},{"key":"978-1-4666-2092-6.ch014.-4","unstructured":"Bari\u0161i\u0107, A., Amaral, V., Goul\u00e3o, M., & Barroca, B. (2011c). Quality in use of DSLs: Current evaluation methods. In 3rd INForum - Simp\u00f3sio de Inform\u00e1tica (INForum2011), Coimbra, Portugal."},{"key":"978-1-4666-2092-6.ch014.-5","doi-asserted-by":"crossref","unstructured":"Basili, V. R. (1996). The role of experimentation in software engineering: past, current, and future. In 18th International Conference on Software Engineering (ICSE 1996) (pp. 442-449).","DOI":"10.1109\/ICSE.1996.493439"},{"key":"978-1-4666-2092-6.ch014.-6","doi-asserted-by":"crossref","unstructured":"Basili, V. R. (2007). The role of controlled experiments in software engineering research. In V. R. Basili, D. Rombach, K. Schneider, B. Kitchenham, D. Pfahl, & R. Selby (Eds.), Empirical software engineering issues:Critical assessment and future directions (pp. 33-37). Berlin, Germany: Springer.","DOI":"10.1007\/978-3-540-71301-2_10"},{"key":"978-1-4666-2092-6.ch014.-7","doi-asserted-by":"publisher","DOI":"10.1145\/505145.505147"},{"key":"978-1-4666-2092-6.ch014.-8","doi-asserted-by":"crossref","unstructured":"Bellamy, R., John, B., Richards, J., & Thomas, J. (2010). Using CogTool to model programming tasks. In 2nd ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2010), Reno, Nevada, USA.","DOI":"10.1145\/1937117.1937118"},{"key":"978-1-4666-2092-6.ch014.-9","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(99)00070-9"},{"key":"978-1-4666-2092-6.ch014.-10","doi-asserted-by":"publisher","DOI":"10.1016\/B978-012095811-5\/50020-1"},{"key":"978-1-4666-2092-6.ch014.-11","doi-asserted-by":"crossref","unstructured":"Bevan, N. (2009). Extending quality in use to provide a framework for usability measurement. Human Centered Design, 13-22.","DOI":"10.1007\/978-3-642-02806-9_2"},{"key":"978-1-4666-2092-6.ch014.-12","doi-asserted-by":"publisher","DOI":"10.1023\/A:1009793700999"},{"key":"978-1-4666-2092-6.ch014.-13","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4379(00)00015-6"},{"key":"978-1-4666-2092-6.ch014.-14","unstructured":"Gabriel, P., Goul\u00e3o, M., & Amaral, V. (2010). Do software languages engineers evaluate their languages? In XIII Congreso Iberoamericano en \u201cSoftware Engineering\u201d (CIbSE'2010) (pp. 149-162). Cuenca, Ecuador."},{"key":"978-1-4666-2092-6.ch014.-15","unstructured":"Goul\u00e3o, M. (2008). Component-based software engineering: A quantitative approach. PhD Dissertation, Faculdade de Ci\u00eancias e Tecnologia, Universidade Nova de Lisboa, Lisboa, Portugal."},{"key":"978-1-4666-2092-6.ch014.-16","doi-asserted-by":"crossref","unstructured":"Goul\u00e3o, M., & Abreu, F. B. (2007). Modeling the experimental software engineering process. In 6th International Conference on the Quality of Information and Communications Technology (QUATIC'2007) (pp. 77-90). Lisbon, Portugal.","DOI":"10.1109\/QUATIC.2007.18"},{"key":"978-1-4666-2092-6.ch014.-17","doi-asserted-by":"crossref","unstructured":"Hermans, F., Pinzger, M., & Deursen, A. V. (2009). Domain-specific languages in practice: A user study on the success factors. In 12th International Conference on Model Driven Engineering Languages and Systems (pp. 423-437). Denver, Colorado, USA.","DOI":"10.1007\/978-3-642-04425-0_33"},{"key":"978-1-4666-2092-6.ch014.-18","year":"2001","journal-title":"ISO\/IEC 9126: Information technology - Software product evaluation - Software quality characteristics and metrics"},{"key":"978-1-4666-2092-6.ch014.-19","unstructured":"ISO. (2001a). ISO\/IEC 9126-1 Quality model."},{"key":"978-1-4666-2092-6.ch014.-20","unstructured":"ISO. (2001b). ISO\/IEC 9241-11 Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability."},{"key":"978-1-4666-2092-6.ch014.-21","unstructured":"Jedlitschka, A., Ciolkowski, M., & Pfahl, D. (2008). Reporting experiments in software engineering. In F. Shull, J. Singer, & D. I. K. Sj\u00f8berg (Eds.), Guide to advanced empirical software engineering,Vol. 5971. London, UK: Springer-Verlag."},{"key":"978-1-4666-2092-6.ch014.-22","unstructured":"Kelly, S., & Tolvanen, J.-P. (2000). Visual domain-specific modelling: benefits and experiences of using metaCASE tools. In International Workshop on Model Engineering, at ECOOP'2000."},{"key":"978-1-4666-2092-6.ch014.-23","doi-asserted-by":"crossref","unstructured":"Kieburtz, R. B., McKinney, L., Bell, J. M., Hook, J., Kotov, A., & Lewis, J. \u2026 Walton, L. (1996). A software engineering experiment in software component generation. In 18th International Conference on Software Engineering (ICSE'1996) (pp. 542-552). Berlin, Germany.","DOI":"10.1109\/ICSE.1996.493448"},{"key":"978-1-4666-2092-6.ch014.-24","author":"A. G.Kleppe","year":"2009","journal-title":"Software language engineering: Creating domain-specific languages using metamodels"},{"key":"978-1-4666-2092-6.ch014.-25","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9172-x"},{"key":"978-1-4666-2092-6.ch014.-26","doi-asserted-by":"publisher","DOI":"10.2298\/CSIS1002247K"},{"key":"978-1-4666-2092-6.ch014.-27","unstructured":"Marcus, A. (2004). The ROI of usability. In R. G. Bias & D. J. Mayhew (Eds.), Cost-justifying usability. North- Holland: Elsevier."},{"key":"978-1-4666-2092-6.ch014.-28","unstructured":"Maroco, J. (2003). An\u00e1lise Estat\u00edstica - Com Utiliza\u00e7\u00e3o do SPSS (2nd ed.). Lisbon, Portugal: Edi\u00e7\u00f5es S\u00edlabo."},{"key":"978-1-4666-2092-6.ch014.-29","doi-asserted-by":"publisher","DOI":"10.1145\/1118890.1118892"},{"key":"978-1-4666-2092-6.ch014.-30","unstructured":"MetaCase. (2007a). EADS case study. Retrieved fromhttp:\/\/www.metacase.com\/papers\/MetaEditinEADS.pdf."},{"key":"978-1-4666-2092-6.ch014.-31","unstructured":"MetaCase. (2007b). Nokia case study. Retrieved fromhttp:\/\/www.metacase.com\/papers\/MetaEditinNokia.pdf"},{"key":"978-1-4666-2092-6.ch014.-32","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(00)00024-8"},{"key":"978-1-4666-2092-6.ch014.-33","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.67"},{"key":"978-1-4666-2092-6.ch014.-34","doi-asserted-by":"publisher","DOI":"10.1006\/jvlc.1999.0150"},{"key":"978-1-4666-2092-6.ch014.-35","author":"J.Nielsen","year":"1993","journal-title":"Usability engineering"},{"key":"978-1-4666-2092-6.ch014.-36","author":"J.Nielsen","year":"2003","journal-title":"Usability return on investment"},{"key":"978-1-4666-2092-6.ch014.-37","doi-asserted-by":"crossref","unstructured":"Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In SIGCHI Conference on Human Factors in Computing Systems: Empowering People (CHI'90) (pp. 249-256). Seattle, WA, USA.","DOI":"10.1145\/97243.97281"},{"key":"978-1-4666-2092-6.ch014.-38","doi-asserted-by":"publisher","DOI":"10.1201\/9781420064995-c20"},{"key":"978-1-4666-2092-6.ch014.-39","unstructured":"Phang, K. Y., Foster, J. S., Hicks, M., & Sazawal, V. (2009). Triaging checklists: A substitute for a PhD in static analysis. In 1st ACM SIGPLAN Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU 2009)."},{"key":"978-1-4666-2092-6.ch014.-40","doi-asserted-by":"publisher","DOI":"10.1109\/2.876288"},{"key":"978-1-4666-2092-6.ch014.-41","first-page":"257","article-title":"Query languages","author":"P.Reisner","year":"1988","journal-title":"Handbook of human-computer interaction"},{"key":"978-1-4666-2092-6.ch014.-42","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.97"},{"key":"978-1-4666-2092-6.ch014.-43","author":"D. M.Weiss","year":"1999","journal-title":"Software product-line engineering: A family-based software development process"},{"key":"978-1-4666-2092-6.ch014.-44","volume":"Vol. 6","author":"C.Wohlin","year":"1999","journal-title":"Experimentation in software engineering: An introduction"}],"container-title":["Formal and Practical Aspects of Domain-Specific Languages"],"original-title":[],"link":[{"URL":"https:\/\/www.igi-global.com\/viewtitle.aspx?TitleId=71827","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,7,4]],"date-time":"2019-07-04T00:02:08Z","timestamp":1562198528000},"score":1,"resource":{"primary":{"URL":"http:\/\/services.igi-global.com\/resolvedoi\/resolve.aspx?doi=10.4018\/978-1-4666-2092-6.ch014"}},"subtitle":[""],"short-title":[],"issued":{"date-parts":[[null]]},"references-count":44,"URL":"https:\/\/doi.org\/10.4018\/978-1-4666-2092-6.ch014","relation":{},"subject":[]}}