{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,27]],"date-time":"2025-05-27T04:18:56Z","timestamp":1748319536414,"version":"3.41.0"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T00:00:00Z","timestamp":1740355200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T00:00:00Z","timestamp":1740355200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001775","name":"University of Technology Sydney","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001775","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Vis"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Narrative visualization integrates data visualization and narrative techniques to convey a compelling story. Narrative visualization is notoriously difficult to evaluate. One solution is heuristic evaluation, using a domain-specific set of heuristics. This paper validates a set of heuristics proposed specifically for evaluating narrative visualization. We conducted studies with experienced narrative visualization practitioners in both summative and formative settings. We found that the set of heuristics showed promise in a summative setting, where similar responses evidenced that the set of heuristics could provide reliable evaluation metrics. Furthermore, in a formative setting, implementing the set of heuristics was reported to be useful in the design process; however, due to their limited focus, we recommend that it be implemented in conjunction with other evaluation guidelines.<\/jats:p>\n          <jats:p>\n            <jats:bold>Graphical abstract<\/jats:bold>\n          <\/jats:p>","DOI":"10.1007\/s12650-025-01051-y","type":"journal-article","created":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T06:12:34Z","timestamp":1740377554000},"page":"645-659","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Heuristics for evaluating narrative visualization: a validation study"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5913-1740","authenticated-orcid":false,"given":"Nina","family":"Errey","sequence":"first","affiliation":[]},{"given":"Didar","family":"Zowghi","sequence":"additional","affiliation":[]},{"given":"Tuck Wah","family":"Leong","sequence":"additional","affiliation":[]},{"given":"Yadong","family":"Wu","sequence":"additional","affiliation":[]},{"given":"Xiaoru","family":"Yuan","sequence":"additional","affiliation":[]},{"given":"Weidong","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Christy Jie","family":"Liang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,2,24]]},"reference":[{"key":"1051_CR1","doi-asserted-by":"publisher","unstructured":"Amar R, Stasko J (2004) A knowledge task-based framework for design and evaluation of information visualization. In: IEEE Symposium on Information Visualiza- tion. Los Alamitos, USA. https:\/\/doi.org\/10.1109\/INFVIS.2004.10","DOI":"10.1109\/INFVIS.2004.10"},{"key":"1051_CR2","doi-asserted-by":"publisher","unstructured":"Amini F, Henry Riche N, Lee B, Hurter C, Irani P (2015) Understanding data videos: looking at narrative visualization through the cinematography lens. CHI \u201915. Seoul, Republic of Korea. https:\/\/doi.org\/10.1145\/2702123.270243","DOI":"10.1145\/2702123.270243"},{"key":"1051_CR3","doi-asserted-by":"publisher","unstructured":"Andrews K (2006) Evaluating Information Visualisations. BELIV\u201906. Venice, Austria. https:\/\/doi.org\/10.1145\/1168149.1168151","DOI":"10.1145\/1168149.1168151"},{"issue":"3","key":"1051_CR4","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1109\/MCG.2017.33","volume":"37","author":"B Bach","year":"2017","unstructured":"Bach B, Henry Riche N, Carpendale S, Pfister H (2017) The emerging genre of data comics. IEEE Comput Graphics Appl 37(3):6\u201313. https:\/\/doi.org\/10.1109\/MCG.2017.33","journal-title":"IEEE Comput Graphics Appl"},{"key":"1051_CR5","doi-asserted-by":"publisher","unstructured":"Brath R, Banissi E (2016) Evaluation of visualization by critiques. BELIV \u201916. Baltimore, USA, pp 19\u201326. https:\/\/doi.org\/10.1145\/2993901.2993904","DOI":"10.1145\/2993901.2993904"},{"issue":"2","key":"1051_CR6","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1191\/1478088706qp063oa","volume":"3","author":"V Braun","year":"2006","unstructured":"Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77\u2013101. https:\/\/doi.org\/10.1191\/1478088706qp063oa","journal-title":"Qual Res Psychol"},{"key":"1051_CR7","first-page":"237","volume-title":"Effective group discussion","author":"JK Brilhart","year":"1998","unstructured":"Brilhart JK, Galanes GJ (1998) Decision making in small groups. Effective group discussion. The McGraw-Hill Companies, USA, pp 237\u2013259"},{"key":"1051_CR8","doi-asserted-by":"publisher","unstructured":"Carpendale S (2008) Evaluating information visualizations. In Kerren A, Stasko J, Fekete, North C, Information Visualization Lecture Notes in Computer Science, (Vol Lecture Notes in Computer Science, pp 19-45). Berlin: Springer. https:\/\/doi.org\/10.1007\/978-3-540-70956-5_2","DOI":"10.1007\/978-3-540-70956-5_2"},{"issue":"1","key":"1051_CR9","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1109\/TVCG.2019.2934283","volume":"26","author":"E Dimara","year":"2020","unstructured":"Dimara E, Perin C (2020) What is interaction for data visualization? IEEE Trans Visual Comput Graphics 26(1):119\u2013129. https:\/\/doi.org\/10.1109\/TVCG.2019.2934283","journal-title":"IEEE Trans Visual Comput Graphics"},{"key":"1051_CR10","doi-asserted-by":"publisher","unstructured":"Ellis G, Dix A (2006) An explorative analysis of user evaluation studies in information visualisation. BELIV\u201906. Venice, Austria. https:\/\/doi.org\/10.1145\/1168149.11681","DOI":"10.1145\/1168149.11681"},{"key":"1051_CR11","doi-asserted-by":"publisher","DOI":"10.1007\/s41060-023-00394-9","author":"N Errey","year":"2023","unstructured":"Errey N, Liang J, Leong TW, Zowghi D (2023) Evaluating narrative visualization: a survey of practitioners. Int J Data Sci Anal. https:\/\/doi.org\/10.1007\/s41060-023-00394-9","journal-title":"Int J Data Sci Anal"},{"key":"1051_CR12","doi-asserted-by":"publisher","unstructured":"Figueiras A (2014) How to tell stories using visualization. IV\u201914, (pp 18\u201326). Lis- born, Portugal. https:\/\/doi.org\/10.1109\/IV.2014.78","DOI":"10.1109\/IV.2014.78"},{"key":"1051_CR13","doi-asserted-by":"publisher","unstructured":"Forsell C, Johanson J (2010) An heuristic set for evaluation in information visualization. AVI \u201910. New York, USA. https:\/\/doi.org\/10.1145\/1842993.18430","DOI":"10.1145\/1842993.18430"},{"key":"1051_CR14","doi-asserted-by":"publisher","unstructured":"Hearst MA, Laskowski P, Silva L (2016) Evaluating information visualization via the interplay of heuristic evaluation and question-based scoring. CHI\u201916. San Jose, USA. https:\/\/doi.org\/10.1145\/2858036.2858280","DOI":"10.1145\/2858036.2858280"},{"key":"1051_CR15","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.apergo.2015.11.016","volume":"56","author":"S Hermawati","year":"2016","unstructured":"Hermawati S, Lawson G (2016) Establishing usability heuristics for heuristics evaluation in a specific domain: Is there a consensus? Appl Ergon 56:34\u201351. https:\/\/doi.org\/10.1016\/j.apergo.2015.11.016","journal-title":"Appl Ergon"},{"issue":"12","key":"1051_CR16","doi-asserted-by":"publisher","first-page":"2231","DOI":"10.1109\/TVCG.2011.255","volume":"17","author":"J Hullman","year":"2011","unstructured":"Hullman J, Diakopoulos N (2011) Visualization rhetoric: framing effects in narrative visualization. IEEE Trans Visual Comput Graph 17(12):2231\u20132240. https:\/\/doi.org\/10.1109\/TVCG.2011.255","journal-title":"IEEE Trans Visual Comput Graph"},{"issue":"12","key":"1051_CR17","doi-asserted-by":"publisher","first-page":"2406","DOI":"10.1109\/TVCG.2013.119","volume":"19","author":"J Hullman","year":"2013","unstructured":"Hullman J, Drucker S, Henry Riche N, Lee B, Fisher D, Adar E (2013) A deeper understanding of sequence in narrative visualization. IEEE Trans Visual Comput Graphics 19(12):2406\u20132415. https:\/\/doi.org\/10.1109\/TVCG.2013.119","journal-title":"IEEE Trans Visual Comput Graphics"},{"key":"1051_CR18","unstructured":"Information is Beautiful Awards. Challenges 2014. Retrieved Oct 1, 2024, from https:\/\/www.informationisbeautifulawards.com\/showcase?type=challenges"},{"key":"1051_CR19","unstructured":"iStock Photo. Retrieved October 2, 2024, from https:\/\/www.istockphoto.com"},{"key":"1051_CR20","unstructured":"Janicki J, Daigle K, Kiyada S, Rao A (2022, December 23) On the Brink. (Reuters) Retrieved October 1, 2024, from https:\/\/www.reuters.com\/graphics\/GLOBAL-ENVIRONMENT\/EXTINCT\/lbvgggdgevq\/"},{"key":"1051_CR21","doi-asserted-by":"publisher","unstructured":"Jeffries R, Miller J, Wharton C, Uyeda K (1991) User interface evaluation in the real world: A comparison of four techniques. CHI \u201991. New Orleans, USA. https:\/\/doi.org\/10.1145\/108844.108862","DOI":"10.1145\/108844.108862"},{"issue":"10","key":"1051_CR22","doi-asserted-by":"publisher","first-page":"1099","DOI":"10.1080\/10810730.2013.878004","volume":"19","author":"A King","year":"2014","unstructured":"King A, Jensen J, Davis L, Carcioppolo N (2014) Perceived visual informativeness (PVI): construct and scale development to assess visual information in printed materials. J Health Commun 19(10):1099\u20131115. https:\/\/doi.org\/10.1080\/10810730.2013.878004","journal-title":"J Health Commun"},{"issue":"2","key":"1051_CR23","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","volume":"15","author":"T Koo","year":"2016","unstructured":"Koo T, Li M (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155\u2013163. https:\/\/doi.org\/10.1016\/j.jcm.2016.02.012","journal-title":"J Chiropr Med"},{"key":"1051_CR24","doi-asserted-by":"publisher","unstructured":"Kosara R (2007) Visualization Criticism - The Missing Link Between Information Visualization and Art. IV\u201907. Zurich, Switzerland. https:\/\/doi.org\/10.1109\/IV.2007.130","DOI":"10.1109\/IV.2007.130"},{"issue":"5","key":"1051_CR25","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1109\/MC.2013.36","volume":"46","author":"R Kosara","year":"2013","unstructured":"Kosara R, Mackinlay J (2013) Storytelling: the next step for visualization. Computer 46(5):44\u201350. https:\/\/doi.org\/10.1109\/MC.2013.36","journal-title":"Computer"},{"key":"1051_CR26","unstructured":"Lee K, Wong D, Duhalde M (2023) America\u2019s debt: what US$ 31 trillion looks like. (South China Morning Post) Retrieved October 1, 2024, from https:\/\/multimedia.scmp.com\/infographics\/news\/world\/article\/3222029\/us-debt\/index.html"},{"key":"1051_CR27","doi-asserted-by":"publisher","unstructured":"McKenna S, Riche N, Lee B, Boy J, Meye M (2017) Visual Narrative Flow: Exploring Factors Shaping Data Visualization Story Reading Experiences. Eurographics. Barcelona, Spain. https:\/\/doi.org\/10.1111\/cgf.13195","DOI":"10.1111\/cgf.13195"},{"key":"1051_CR28","doi-asserted-by":"publisher","unstructured":"Mer\u010dun T (2014) Evaluation of information visualization techniques: Analysing user experience with reaction cards. BELIV \u201914. Paris, France. https:\/\/doi.org\/10.1145\/2669557.2669565","DOI":"10.1145\/2669557.2669565"},{"key":"1051_CR29","first-page":"25","volume-title":"Usability inspection methods","author":"J Nielsen","year":"1994","unstructured":"Nielsen J (1994) Heuristic evaluation. In: Nielsen J, Mack R (eds) Usability inspection methods. John Wiley and Sons, New York, pp 25\u201316"},{"key":"1051_CR30","doi-asserted-by":"publisher","unstructured":"Nowak S, Bartram L, Schiphorst T (2018) A Micro-Phenomenological Lens for Evaluating Narrative Visualization. BELIV \u201918. Berlin, Germany. https:\/\/doi.org\/10.1109\/BELIV.2018.8634072","DOI":"10.1109\/BELIV.2018.8634072"},{"key":"1051_CR31","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1002\/9781119055259.ch8","volume-title":"Storytelling with data: a data visualization guide for business professionals","author":"C Nussbaumer Knaflic","year":"2015","unstructured":"Nussbaumer Knaflic C (2015) Pulling it all together. Storytelling with data: a data visualization guide for business professionals. Wiley, Hoboken, USA, pp 187\u2013205"},{"key":"1051_CR32","unstructured":"Qualtrics. Retrieved October 1, 2024, from https:\/\/www.qualtrics.com"},{"key":"1051_CR33","first-page":"337","volume-title":"Narrative across media: the languages of storytelling","author":"M Ryan","year":"2004","unstructured":"Ryan M (2004) Will new media produce new narratives? In: Ruppert J, Bernet J (eds) Narrative across media: the languages of storytelling. University of Nebraska Press, Lincoln, Nebraska, pp 337\u2013358"},{"key":"1051_CR34","doi-asserted-by":"publisher","unstructured":"Sadowski C, Kurniawan S (2011) Heuristic evaluation of programming language features: two parallel programming case studies. PLATEAU \u201911. Portland, USA. https:\/\/doi.org\/10.1145\/2089155.208916","DOI":"10.1145\/2089155.208916"},{"issue":"6","key":"1051_CR35","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1007\/978-3-319-20901-2_24","volume":"67","author":"B Santos","year":"2015","unstructured":"Santos B, Ferreira B, Dias P (2015) Heuristic evaluation in information visualization using three sets of heuristics: an exploratory study. Human-Comput Interact 67(6):259\u2013270. https:\/\/doi.org\/10.1007\/978-3-319-20901-2_24","journal-title":"Human-Comput Interact"},{"issue":"1","key":"1051_CR36","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/MCG.2016.7","volume":"36","author":"B Santos","year":"2016","unstructured":"Santos B, Ferreira B, Dias P (2016) Using heuristic evaluation to foster visualization analysis and design skills. IEEE Comput Graphics Appl 36(1):86\u201390. https:\/\/doi.org\/10.1109\/MCG.2016.7","journal-title":"IEEE Comput Graphics Appl"},{"key":"1051_CR37","unstructured":"Scriven M (1967) The methodology of evaluation. In Tyler R, Gagne R, Scriven M (Eds), Perspectives of curriculum evaluation (pp 39\u201383). Chicago, USA: Rand McNally"},{"issue":"6","key":"1051_CR38","doi-asserted-by":"publisher","first-page":"1139","DOI":"10.1109\/TVCG.2010.179","volume":"16","author":"E Segel","year":"2010","unstructured":"Segel E, Heer J (2010) Narrative visualization: telling stories with data. IEEE Trans Visual Comput Graphics 16(6):1139\u20131148. https:\/\/doi.org\/10.1109\/TVCG.2010.179","journal-title":"IEEE Trans Visual Comput Graphics"},{"key":"1051_CR39","doi-asserted-by":"publisher","unstructured":"Seyser D, Zeiller M (2018) Scrollytelling \u2013 An Analysis of Visual Storytelling in Online Journalism. IV\u201918. Fisciano, Italy. https:\/\/doi.org\/10.1109\/iV.2018.00075https:\/\/doi.org\/10.1109\/iV.2018.00075","DOI":"10.1109\/iV.2018.00075"},{"key":"1051_CR40","doi-asserted-by":"publisher","unstructured":"Tarrell A, Forsell C, Fruhling A, Grinstein G, Borgo R, Scholtz J (2014) Toward visualization-specific heuristic evaluation. BELIV \u201914. Paris, France. https:\/\/doi.org\/10.1145\/2669557.2669580","DOI":"10.1145\/2669557.2669580"},{"issue":"5","key":"1051_CR41","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1109\/MCG.2005.102","volume":"25","author":"M Tory","year":"2005","unstructured":"Tory M, M\u00f6ller T (2005) Evaluating visualizations: Do expert reviews work? IEEE Comput Graphics Appl 25(5):8\u201311. https:\/\/doi.org\/10.1109\/MCG.2005.102","journal-title":"IEEE Comput Graphics Appl"},{"key":"1051_CR42","doi-asserted-by":"publisher","unstructured":"V\u00e4\u00e4t\u00e4j\u00e4 H, Varsaluoma J, Heimonen T, Tiitinen K, Hakulinen J, Turunen M, Ihantola P (2016) Information visualization heuristics in practical expert evaluation. BELIV\u201916. Baltimore, USA. https:\/\/doi.org\/10.1145\/2993901.2993918","DOI":"10.1145\/2993901.2993918"},{"issue":"1","key":"1051_CR43","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1109\/TVCG.2018.2865146","volume":"25","author":"E Wall","year":"2019","unstructured":"Wall E, Agnihotri M, Matzen L, Divis K, Haass M, Endert A, Stasko J (2019) A heuristic approach to value-driven evaluation of visualizations. IEEE Trans Visual Comput Graph 25(1):491\u2013500. https:\/\/doi.org\/10.1109\/TVCG.2018.2865146","journal-title":"IEEE Trans Visual Comput Graph"},{"key":"1051_CR44","unstructured":"Yau N (2019) A Day in the Life: Women and Men. (Flowing Data) Retrieved October 1, 2024, from https:\/\/flowingdata.com\/2019\/03\/06\/women-men-timeuse\/"},{"key":"1051_CR45","unstructured":"Zapponi C (2014) Has the cost of human life changed in history? Retrieved October 1, 2024, from https:\/\/humancost.carlo.im"},{"key":"1051_CR46","doi-asserted-by":"publisher","unstructured":"Zuk, T, Schlesier, L, Neumann, P, Hancock, M, Carpendale, S (2006) Heuristics for information visualization evaluation. BELIV \u201906. New York, USA. https:\/\/doi.org\/10.1145\/1168149.1168162","DOI":"10.1145\/1168149.1168162"}],"container-title":["Journal of Visualization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12650-025-01051-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12650-025-01051-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12650-025-01051-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,27]],"date-time":"2025-05-27T02:35:46Z","timestamp":1748313346000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12650-025-01051-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,24]]},"references-count":46,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["1051"],"URL":"https:\/\/doi.org\/10.1007\/s12650-025-01051-y","relation":{},"ISSN":["1343-8875","1875-8975"],"issn-type":[{"type":"print","value":"1343-8875"},{"type":"electronic","value":"1875-8975"}],"subject":[],"published":{"date-parts":[[2025,2,24]]},"assertion":[{"value":"15 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 January 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All practitioners agreed to consent form before taking part in research.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Participation"}},{"value":"Ethics approval was provided by the University of Technology Sydney (UTS)","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}