{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T03:52:30Z","timestamp":1774237950147,"version":"3.50.1"},"reference-count":62,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,11,26]],"date-time":"2020-11-26T00:00:00Z","timestamp":1606348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2019R1C1C1005508"],"award-info":[{"award-number":["2019R1C1C1005508"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>The use of data visualization is increasing; however, there is little empirical explanation for how it supports users. Our goal in this paper is to deepen our understanding of the role of interactive visualizations in a particular context of decision making. Specifically, we attempt to understand the role of the working memory system, which is a concept to understand the mechanism of the processing and temporary storage of information in variety of cognitive tasks. We compared two interfaces, SimulSort and its non-visual counterpart Typical Sorting, with a multi-attribute decision-making problem. Because decision outcomes are known to be affected by the limitations of a person\u2019s working memory, we conducted a crowdsourcing-based user study using SimulSort to understand how working memory, especially the phonological loop, can benefit from the using visualizations. We examined the impact on working memory with a well known dual-task methodology by designing a concurrent task to tap into the main decision-making task. The experiment was conducted with a total of 137 participants and an ordered logistic regression using a proportional odds model was applied to analyze the decision quality. The results supported the hypothesis that when using SimulSort, participants required less working memory than they required with Typical Sorting to accomplish the multi-attribute decision-making task even though SimulSort outperformed Typical Sorting in terms of decision quality. We also provide methodologies to conduct working memory studies by implementing an articulatory suppression task on crowdsourcing platforms in which experimenters have less control over the participants.<\/jats:p>","DOI":"10.3390\/informatics7040053","type":"journal-article","created":{"date-parts":[[2020,11,26]],"date-time":"2020-11-26T09:04:15Z","timestamp":1606381455000},"page":"53","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Understanding the Role of Visualizations on Decision Making: A Study on Working Memory"],"prefix":"10.3390","volume":"7","author":[{"given":"Sung-Hee","family":"Kim","sequence":"first","affiliation":[{"name":"Department of ICT Industrial Engineering, Dong-Eui University, Busan 47340, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1503\/cmaj.75262","article-title":"Open access epidemiologic data and an interactive dashboard to monitor the COVID-19 outbreak in Canada","volume":"192","author":"Berry","year":"2020","journal-title":"CMAJ"},{"key":"ref_2","unstructured":"Eberhardt, A., and Silveira, M.S. (June, January 30). Show me the data! A systematic mapping on open government data visualization. Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age, Delft, The Netherlands."},{"key":"ref_3","first-page":"101284","article-title":"Data science empowering the public: Data-driven dashboards for transparent and accountable decision-making in smart cities","volume":"2018","author":"Matheus","year":"2018","journal-title":"Gov. Inf. Q."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cabitza, F., Locoro, A., Fogli, D., and Giacomin, M. (2016, January 7\u201310). Valuable visualization of healthcare information: From the quantified self data to conversations. Proceedings of the International Working Conference on Advanced Visual Interfaces, Bari, Italy.","DOI":"10.1145\/2909132.2927474"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijhcs.2016.05.006","article-title":"Personal informatics for everyday life: How users without prior self-tracking experience engage with personal data","volume":"94","author":"Rapp","year":"2016","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_6","unstructured":"Lacefield, W.E., and Applegate, E.B. (2018, January 13\u201317). Data Visualization in Public Education: Longitudinal Student-, Intervention-, School-, and District-Level Performance Modeling. Proceedings of the Annual Meeting of the American Educational Research Association, New York, NY, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1080\/00933104.2017.1400483","article-title":"Data literacy for social studies: Examining the role of data visualizations in K\u201312 textbooks","volume":"46","author":"Shreiner","year":"2018","journal-title":"Theory Res. Soc. Educ."},{"key":"ref_8","unstructured":"Card, S.K., Mackinlay, J.D., and Shneiderman, B. (1999). Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Fekete, J.D., Van Wijk, J.J., Stasko, J.T., and North, C. (2008). The value of information visualization. Information Visualization, Springer.","DOI":"10.1007\/978-3-540-70956-5_1"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1080\/01621459.1984.10478080","article-title":"Graphical perception: Theory, experimentation, and application to the development of graphical methods","volume":"79","author":"Cleveland","year":"1984","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_11","unstructured":"Fuchs, J., Fischer, F., Mansmann, F., Bertini, E., and Isenberg, P. (May, January 27). Evaluation of alternative glyph designs for time series data in a small multiple setting. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Heer, J., and Bostock, M. (2010, January 10\u201315). Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. Proceedings of the 28th International Conference on Human Factors in Computing Systems, Atlanta, GA, USA.","DOI":"10.1145\/1753326.1753357"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Heer, J., Kong, N., and Agrawala, M. (2009, January 4\u20139). Sizing the horizon: The effects of chart size and layering on the graphical perception of time series visualizations. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA.","DOI":"10.1145\/1518701.1518897"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1109\/TVCG.2018.2864884","article-title":"Face to face: Evaluating visual comparison","volume":"25","author":"Ondov","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"0058","DOI":"10.1038\/s41562-017-0058","article-title":"Five factors that guide attention in visual search","volume":"1","author":"Wolfe","year":"2017","journal-title":"Nat. Hum. Behav."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Plaisant, C. (2004, January 25\u201328). The challenge of information visualization evaluation. Proceedings of the Working Conference on Advanced Visual Interfaces, Gallipoli, Italy.","DOI":"10.1145\/989863.989880"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1109\/TVCG.2015.2467732","article-title":"Beyond memorability: Visualization recognition and recall","volume":"22","author":"Borkin","year":"2015","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Haroz, S., Kosara, R., and Franconeri, S.L. (2015, January 18\u201323). Isotype visualization: Working memory, performance, and engagement with pictographs. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Korea.","DOI":"10.1145\/2702123.2702275"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1109\/TVCG.2010.177","article-title":"Mental models, visual reasoning and interaction in information visualization: A top-down perspective","volume":"16","author":"Liu","year":"2010","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1109\/TVCG.2019.2934286","article-title":"Toward Objective Evaluation of Working Memory in Visualizations: A Case Study Using Pupillometry and a Dual-Task Paradigm","volume":"26","author":"Padilla","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.3758\/s13423-016-1191-6","article-title":"The many faces of working memory and short-term storage","volume":"24","author":"Cowan","year":"2017","journal-title":"Psychon. Bull. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/S0079-7421(08)60452-1","article-title":"Working memory","volume":"8","author":"Baddeley","year":"1974","journal-title":"Psychol. Learn. Motiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1080\/09541440244000328","article-title":"The role of working memory in mental arithmetic","volume":"16","author":"DeStefano","year":"2004","journal-title":"Eur. J. Cogn. Psychol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1086\/380283","article-title":"Decision Making in Information-Rich Environments: The Role of Information Structure","volume":"30","author":"Lurie","year":"2004","journal-title":"J. Consum. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1509\/jmkg.71.1.160","article-title":"Visual representation: Implications for decision making","volume":"71","author":"Lurie","year":"2007","journal-title":"J. Mark."},{"key":"ref_26","first-page":"684","article-title":"SimulSort: Multivariate Data Exploration Through An Enhanced Sorting Technique","volume":"5611","author":"Jacko","year":"2009","journal-title":"Human-Computer Interaction. Novel Interaction Methods and Techniques"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1080\/10447318.2012.713802","article-title":"A Comparative Study of Three Sorting Techniques in Performing Cognitive Tasks on a Tabular Representation","volume":"29","author":"Hur","year":"2013","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2421","DOI":"10.1109\/TVCG.2012.215","article-title":"Does an eye tracker tell the truth about visualizations? Findings while investigating visualizations for decision making","volume":"18","author":"Kim","year":"2012","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.jebo.2016.06.004","article-title":"An experimental study of the decision process with interactive technology","volume":"130","author":"Samek","year":"2016","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hwang, C., and Yoon, K. (1981). Multiple Attribute Decision Making: Methods And Applications: A State-of-the-Art Survey, Springer.","DOI":"10.1007\/978-3-642-48318-9"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yoon, K., and Hwang, C. (1995). Multiple Attribute Decision Making: An Introduction, Sage.","DOI":"10.4135\/9781412985161"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"99","DOI":"10.2307\/1884852","article-title":"A behavioral model of rational choice","volume":"69","author":"Simon","year":"1955","journal-title":"Q. J. Econ."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Payne, J.W., Bettman, J.R., and Johnson, E.J. (1993). The Adaptive Decision Maker, Cambridge University Press.","DOI":"10.1017\/CBO9781139173933"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1037\/0033-2909.134.2.207","article-title":"Heuristics Made Easy: An Effort-reduction Framework","volume":"134","author":"Shah","year":"2008","journal-title":"Psychol. Bull."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1086\/208629","article-title":"Issues in Designing Consumer Information Environments","volume":"2","author":"Bettman","year":"1975","journal-title":"J. Consum. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Rao, R., and Card, S.K. (1994, January 24\u201328). The Table Lens: Merging Graphical and Symbolic Representations in An Interactive Focus+ Context Visualization for Tabular Information. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Celebrating Interdependence, Boston, MA, USA.","DOI":"10.1145\/191666.191776"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2277","DOI":"10.1109\/TVCG.2013.173","article-title":"Lineup: Visual analysis of multi-attribute rankings","volume":"19","author":"Gratzl","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1080\/01621459.1990.10474926","article-title":"Hyperdimensional data analysis using parallel coordinates","volume":"85","author":"Wegman","year":"1990","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_39","unstructured":"Zhang, P., and Whinston, A.B. (1995, January 12\u201314). Business Information Visualization for Decision-Making Support-A Research Strategy. Proceedings of the First Americas Conference on Information Systems, San Francisco, CA, USA."},{"key":"ref_40","first-page":"4","article-title":"Business information visualization","volume":"1","author":"Tegarden","year":"1999","journal-title":"Commun. AIS"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/S0734-189X(85)80004-9","article-title":"Preattentive processing in vision","volume":"31","author":"Treisman","year":"1985","journal-title":"Comput. Vis. Graph. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1006\/obhd.1996.0077","article-title":"The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives","volume":"67","author":"Hsee","year":"1996","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1006\/obhd.1994.1018","article-title":"Information displays and choice processes: Differential effects of organization, form, and sequence","volume":"57","author":"Schkade","year":"1994","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1037\/0033-2909.119.1.3","article-title":"The empirical case for two systems of reasoning","volume":"119","author":"Sloman","year":"1996","journal-title":"Psychol. Bull."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"395","DOI":"10.3758\/BF03200866","article-title":"Counting on working memory in arithmetic problem solving","volume":"22","author":"Logie","year":"1994","journal-title":"Mem. Cogn."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1037\/0278-7393.15.5.898","article-title":"A componential model for mental addition","volume":"15","author":"Widaman","year":"1989","journal-title":"J. Exp. Psychol. Learn. Mem. Cogn."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2306","DOI":"10.1109\/TVCG.2013.234","article-title":"What makes a visualization memorable?","volume":"19","author":"Borkin","year":"2013","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1080\/014492997119707","article-title":"The role of working memory on graphical information processing","volume":"16","author":"Lohse","year":"1997","journal-title":"Behav. Inf. Technol."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1518\/001872095779049408","article-title":"The proximity compatibility principle: Its psychological foundation and relevance to display design","volume":"37","author":"Wickens","year":"1995","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_50","unstructured":"Wickens, C.D. (2000). Engineering Psychology and Human Performance, Prentice-Hall."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1287\/mnsc.31.4.395","article-title":"Effort and Accuracy in Choice","volume":"31","author":"Johnson","year":"1985","journal-title":"Manag. Sci."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"738","DOI":"10.3758\/BF03196112","article-title":"Phonological and visual working memory in mental addition","volume":"31","author":"Trbovich","year":"2003","journal-title":"Mem. Cogn."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1177\/1470593107073842","article-title":"Escaping the Tyranny of Choice: When Fewer Attributes Make Choice Easier","volume":"7","author":"Fasolo","year":"2007","journal-title":"Mark. Theory"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1007\/978-3-662-03750-8_25","article-title":"Testing Decision Rules for Multiattribute Decision Making","volume":"8","author":"Seidl","year":"1999","journal-title":"Current Trends in Economics"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Kim, S.H., Yun, H., and Yi, J.S. (2012, January 14\u201315). How to filter out random clickers in a crowdsourcing-based study?. Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, Seattle, WA, USA.","DOI":"10.1145\/2442576.2442591"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/S0160-2896(01)00096-4","article-title":"A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence","volume":"30","author":"Conway","year":"2002","journal-title":"Intelligence"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1037\/0096-3445.128.3.309","article-title":"Working memory, short-term memory, and general fluid intelligence: A latent-variable approach","volume":"128","author":"Engle","year":"1999","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"498","DOI":"10.3758\/BF03192720","article-title":"An automated version of the operation span task","volume":"37","author":"Unsworth","year":"2005","journal-title":"Behav. Res. Methods"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"86","DOI":"10.3758\/s13414-010-0020-7","article-title":"Individual working memory capacity is uniquely correlated with feature-based attention when combined with spatial attention","volume":"73","author":"Bengson","year":"2011","journal-title":"Atten. Percept. Psychophys."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1037\/0278-7393.33.4.645","article-title":"Attentional control and the relatedness proportion effect in semantic priming","volume":"33","author":"Hutchison","year":"2007","journal-title":"J. Exp. Psychol. Learn. Mem. Cogn."},{"key":"ref_61","unstructured":"Norman, D.A. (1993). Things That Make Us Smart: Defending Human Attributes in the Age of the Machine, Basic Books."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1037\/a0033753","article-title":"Components of working memory and visual selective attention","volume":"40","author":"Burnham","year":"2014","journal-title":"J. Exp. Psychol. Hum. Percept. Perform."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/4\/53\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:37:45Z","timestamp":1760179065000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/7\/4\/53"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,26]]},"references-count":62,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["informatics7040053"],"URL":"https:\/\/doi.org\/10.3390\/informatics7040053","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,26]]}}}