{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T16:52:49Z","timestamp":1776099169430,"version":"3.50.1"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:p>\n            The growing importance of data visualization in business intelligence and data science emphasizes the need for tools that can efficiently generate meaningful visualizations from large datasets. Existing tools fall into two main categories: human-powered tools (\n            <jats:italic>e.g.<\/jats:italic>\n            , Tableau and PowerBI), which require intensive expert involvement, and AI-powered automated tools (\n            <jats:italic>e.g.<\/jats:italic>\n            , Draco and Table2Charts), which often fall short of\n            <jats:italic>guessing<\/jats:italic>\n            specific user needs.\n          <\/jats:p>\n          <jats:p>\n            In this paper, we aim to achieve the best of both worlds. Our key idea is to initially auto-generate a set of high-quality visualizations to minimize manual effort, then refine this process iteratively with user feedback to more closely align with their needs. To this end, we present HAIChart, a reinforcement learning-based framework designed to iteratively recommend good visualizations for a given dataset by incorporating user feedback. Specifically, we propose a Monte Carlo Graph Search-based visualization generation algorithm paired with a composite reward function to efficiently explore the visualization space and automatically generate good visualizations. We devise a visualization hints mechanism to actively incorporate user feedback, thus progressively refining the visualization generation module. We further prove that the top-\n            <jats:italic>k<\/jats:italic>\n            visualization hints selection problem is NP-hard and design an efficient algorithm. We conduct both quantitative evaluations and user studies, showing that HAIChart significantly outperforms state-of-the-art human-powered tools (21% better at Recall and 1.8\u00d7 faster) and AI-powered automatic tools (25.1% and 14.9% better in terms of Hit@3 and R10@30, respectively).\n          <\/jats:p>","DOI":"10.14778\/3681954.3681992","type":"journal-article","created":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T16:23:36Z","timestamp":1725035016000},"page":"3178-3191","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["HAIChart: Human and AI Paired Visualization System"],"prefix":"10.14778","volume":"17","author":[{"given":"Yupeng","family":"Xie","sequence":"first","affiliation":[{"name":"HKUST (GZ)"}]},{"given":"Yuyu","family":"Luo","sequence":"additional","affiliation":[{"name":"HKUST (GZ) \/ HKUST"}]},{"given":"Guoliang","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University"}]},{"given":"Nan","family":"Tang","sequence":"additional","affiliation":[{"name":"HKUST (GZ) \/ HKUST"}]}],"member":"320","published-online":{"date-parts":[[2024,8,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"July 15 2024. Excel. https:\/\/www.microsoft.com\/en-us\/microsoft-365\/excel"},{"key":"e_1_2_1_2_1","unstructured":"July 15 2024. Tableau. https:\/\/www.tableau.com\/"},{"key":"e_1_2_1_3_1","volume-title":"Likert scales and data analyses. Quality progress 40, 7","author":"Elaine Allen I","year":"2007","unstructured":"I Elaine Allen and Christopher A Seaman. 2007. Likert scales and data analyses. Quality progress 40, 7 (2007), 64--65."},{"key":"e_1_2_1_4_1","volume-title":"Finite-time analysis of the multiarmed bandit problem. Machine learning 47","author":"Auer Peter","year":"2002","unstructured":"Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Machine learning 47 (2002), 235--256."},{"key":"e_1_2_1_5_1","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019","author":"Bai Xueying","year":"2019","unstructured":"Xueying Bai, Jian Guan, and Hongning Wang. 2019. A Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8--14, 2019, Vancouver, BC, Canada. 10734--10745."},{"key":"e_1_2_1_6_1","volume-title":"A Markovian decision process. Journal of mathematics and mechanics","author":"Bellman Richard","year":"1957","unstructured":"Richard Bellman. 1957. A Markovian decision process. Journal of mathematics and mechanics (1957), 679--684."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3523210.3523223"},{"key":"e_1_2_1_8_1","volume-title":"Demystifying Artificial Intelligence for Data Preparation. In SIGMOD Conference Companion. ACM, 13--20","author":"Chai Chengliang","year":"2023","unstructured":"Chengliang Chai, Nan Tang, Ju Fan, and Yuyu Luo. 2023. Demystifying Artificial Intelligence for Data Preparation. In SIGMOD Conference Companion. ACM, 13--20."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3148237"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526166"},{"key":"e_1_2_1_11_1","volume-title":"Is GPT-4 a Good Data Analyst? arXiv preprint arXiv:2305.15038","author":"Cheng Liying","year":"2023","unstructured":"Liying Cheng, Xingxuan Li, and Lidong Bing. 2023. Is GPT-4 a Good Data Analyst? arXiv preprint arXiv:2305.15038 (2023)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1609\/icaps.v31i1.15952"},{"key":"e_1_2_1_13_1","first-page":"690","article-title":"Dashbot: Insight-driven dashboard generation based on deep reinforcement learning","volume":"29","author":"Deng Dazhen","year":"2022","unstructured":"Dazhen Deng, Aoyu Wu, Huamin Qu, and Yingcai Wu. 2022. Dashbot: Insight-driven dashboard generation based on deep reinforcement learning. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 690--700.","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-demo.11"},{"key":"e_1_2_1_15_1","volume-title":"Data2vis: Automatic generation of data visualizations using sequence-to-sequence recurrent neural networks","author":"Dibia Victor","year":"2019","unstructured":"Victor Dibia and \u00c7a\u011fatay Demiralp. 2019. Data2vis: Automatic generation of data visualizations using sequence-to-sequence recurrent neural networks. IEEE computer graphics and applications 39, 5 (2019), 33--46."},{"key":"e_1_2_1_16_1","volume-title":"DASFAA 2021, Taipei, Taiwan, April 11--14, 2021, Proceedings, Part III (Lecture Notes in Computer Science","volume":"261","author":"Gao Qianfeng","unstructured":"Qianfeng Gao, Zhenying He, Yinan Jing, Kai Zhang, and X. Sean Wang. 2021. VizGRank: A Context-Aware Visualization Recommendation Method Based on Inherent Relations Between Visualizations. In Database Systems for Advanced Applications - 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11--14, 2021, Proceedings, Part III (Lecture Notes in Computer Science, Vol. 12683). Springer, 244--261."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3422622"},{"key":"e_1_2_1_18_1","volume-title":"Leveraging Large Models for Crafting Narrative Visualization: A Survey. arXiv preprint arXiv:2401.14010","author":"He Yi","year":"2024","unstructured":"Yi He, Shixiong Cao, Yang Shi, Qing Chen, Ke Xu, and Nan Cao. 2024. Leveraging Large Models for Crafting Narrative Visualization: A Survey. arXiv preprint arXiv:2401.14010 (2024)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300358"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-0190(99)00031-9"},{"key":"e_1_2_1_21_1","volume-title":"How Good is ChatGPT in Giving Advice on Your Visualization Design? arXiv preprint arXiv:2310.09617","author":"Kim Nam Wook","year":"2023","unstructured":"Nam Wook Kim, Grace Myers, and Benjamin Bach. 2023. How Good is ChatGPT in Giving Advice on Your Visualization Design? arXiv preprint arXiv:2310.09617 (2023)."},{"key":"e_1_2_1_22_1","unstructured":"Doris Jung-Lin Lee Dixin Tang Kunal Agarwal Thyne Boonmark Caitlyn Chen Jake Kang Ujjaini Mukhopadhyay Jerry Song Micah Yong Marti A Hearst et al. 2021. Lux: always-on visualization recommendations for exploratory dataframe workflows. arXiv preprint arXiv:2105.00121 (2021)."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of The 12th Asian Conference on Machine Learning, ACML 2020, 18--20","volume":"592","author":"Leurent Edouard","year":"2020","unstructured":"Edouard Leurent and Odalric-Ambrym Maillard. 2020. Monte-Carlo Graph Search: the Value of Merging Similar States. In Proceedings of The 12th Asian Conference on Machine Learning, ACML 2020, 18--20 November 2020, Bangkok, Thailand (Proceedings of Machine Learning Research, Vol. 129). PMLR, 577--592."},{"key":"e_1_2_1_24_1","volume-title":"Proc. VLDB Endow. 17","author":"Li Boyan","year":"2024","unstructured":"Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. 2024. The Dawn of Natural Language to SQL: Are We Fully Ready? Proc. VLDB Endow. 17 (2024)."},{"key":"e_1_2_1_25_1","volume-title":"CoInsight: Visual Storytelling for Hierarchical Tables with Connected Insights","author":"Li Guozheng","year":"2024","unstructured":"Guozheng Li, Runfei Li, Yunshan Feng, Yu Zhang, Yuyu Luo, and Chi Harold Liu. 2024. CoInsight: Visual Storytelling for Hierarchical Tables with Connected Insights. IEEE Transactions on Visualization and Computer Graphics (2024)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114863"},{"key":"e_1_2_1_27_1","volume-title":"Feature Augmentation with Reinforcement Learning","author":"Liu Jiabin","unstructured":"Jiabin Liu, Chengliang Chai, Yuyu Luo, Yin Lou, Jianhua Feng, and Nan Tang. 2022. Feature Augmentation with Reinforcement Learning. In ICDE. IEEE, 3360--3372."},{"key":"e_1_2_1_28_1","volume-title":"Interactive Cleaning for Progressive Visualization through Composite Questions","author":"Luo Yuyu","unstructured":"Yuyu Luo, Chengliang Chai, Xuedi Qin, Nan Tang, and Guoliang Li. 2020. Interactive Cleaning for Progressive Visualization through Composite Questions. In ICDE. IEEE, 733--744."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415484"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.2981464"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3193545"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3193545"},{"key":"e_1_2_1_33_1","volume-title":"IEEE Visualization Conference Companion abs\/2112","author":"Luo Yuyu","year":"2021","unstructured":"Yuyu Luo, Jiawei Tang, and Guoliang Li. 2021. nvBench: A Large-Scale Synthesized Dataset for Cross-Domain Natural Language to Visualization Task. NLVIZ, IEEE Visualization Conference Companion abs\/2112.12926 (2021)."},{"key":"e_1_2_1_34_1","volume-title":"IEEE Visualization Conference","author":"Luo Yuyu","year":"2021","unstructured":"Yuyu Luo, Jiawei Tang, Guoliang Li, and Chengliang Chai. 2021. Empowering natural language to visualization neural translation using synthesized benchmarks. IEEE Visualization Conference (2021)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457261"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114848"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588942"},{"key":"e_1_2_1_38_1","volume-title":"Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco","author":"Moritz Dominik","year":"2019","unstructured":"Dominik Moritz, Chenglong Wang, Gregory Nelson, Halden Lin, Adam M. Smith, Bill Howe, and Jeffrey Heer. 2019. Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2019)."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3538703"},{"key":"e_1_2_1_40_1","volume-title":"Deepeye: An automatic big data visualization framework. Big data mining and analytics 1, 1","author":"Qin Xuedi","year":"2018","unstructured":"Xuedi Qin, Yuyu Luo, Nan Tang, and Guoliang Li. 2018. Deepeye: An automatic big data visualization framework. Big data mining and analytics 1, 1 (2018), 75--82."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00588-3"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-3071-5"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2011.11.014"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12391"},{"key":"e_1_2_1_45_1","volume-title":"chess and shogi by planning with a learned model. Nat. 588, 7839","author":"Schrittwieser Julian","year":"2020","unstructured":"Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy P. Lillicrap, and David Silver. 2020. Mastering Atari, Go, chess and shogi by planning with a learned model. Nat. 588, 7839 (2020), 604--609."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2022.3148007"},{"key":"e_1_2_1_47_1","volume-title":"GALVIS: Visualization Construction through Example-Powered Declarative Programming. In CIKM. ACM, 4975--4979.","author":"Shen Leixian","year":"2022","unstructured":"Leixian Shen, Enya Shen, Zhiwei Tai, Yun Wang, Yuyu Luo, and Jianmin Wang. 2022. GALVIS: Visualization Construction through Example-Powered Declarative Programming. In CIKM. ACM, 4975--4979."},{"key":"e_1_2_1_48_1","volume-title":"M-walk: Learning to walk over graphs using monte carlo tree search. Advances in Neural Information Processing Systems 31","author":"Shen Yelong","year":"2018","unstructured":"Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, and Jianfeng Gao. 2018. M-walk: Learning to walk over graphs using monte carlo tree search. Advances in Neural Information Processing Systems 31 (2018)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030403"},{"key":"e_1_2_1_50_1","volume-title":"Mastering the game of Go with deep neural networks and tree search. Nat. 529, 7587","author":"Silver David","year":"2016","unstructured":"David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nat. 529, 7587 (2016), 484--489."},{"key":"e_1_2_1_51_1","volume-title":"Mastering the game of Go without human knowledge. Nat. 550, 7676","author":"Silver David","year":"2017","unstructured":"David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy P. Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Mastering the game of Go without human knowledge. Nat. 550, 7676 (2017), 354--359."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.981851"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3520150"},{"key":"e_1_2_1_54_1","volume-title":"Halevy","author":"Tang Nan","year":"2024","unstructured":"Nan Tang, Chenyu Yang, Ju Fan, Lei Cao, Yuyu Luo, and Alon Y. Halevy. 2024. VerifAI: Verified Generative AI. In CIDR. www.cidrdb.org."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3520153"},{"key":"e_1_2_1_56_1","volume-title":"Chartgpt: Leveraging llms to generate charts from abstract natural language. arXiv preprint arXiv:2311.01920","author":"Tian Yuan","year":"2023","unstructured":"Yuan Tian, Weiwei Cui, Dazhen Deng, Xinjing Yi, Yurun Yang, Haidong Zhang, and Yingcai Wu. 2023. Chartgpt: Leveraging llms to generate charts from abstract natural language. arXiv preprint arXiv:2311.01920 (2023)."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CoG51982.2022.9893579"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i5.16569"},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: EMNLP 2023 - Industry Track","author":"Wang Lei","year":"2023","unstructured":"Lei Wang, Songheng Zhang, Yun Wang, Ee-Peng Lim, and Yong Wang. 2023. LLM4Vis: Explainable Visualization Recommendation using ChatGPT. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: EMNLP 2023 - Industry Track, Singapore, December 6--10, 2023. Association for Computational Linguistics, 675--692."},{"key":"e_1_2_1_60_1","volume-title":"Interactive data analysis with next-step natural language query recommendation. arXiv preprint arXiv:2201.04868","author":"Wang Xingbo","year":"2022","unstructured":"Xingbo Wang, Furui Cheng, Yong Wang, Ke Xu, Jiang Long, Hong Lu, and Huamin Qu. 2022. Interactive data analysis with next-step natural language query recommendation. arXiv preprint arXiv:2201.04868 (2022)."},{"key":"e_1_2_1_61_1","volume-title":"Interactive data visualization: foundations, techniques, and applications","author":"Ward Matthew O","unstructured":"Matthew O Ward, Georges Grinstein, and Daniel Keim. 2010. Interactive data visualization: foundations, techniques, and applications. CRC press."},{"key":"e_1_2_1_62_1","volume-title":"Voyager: Exploratory analysis via faceted browsing of visualization recommendations","author":"Wongsuphasawat Kanit","year":"2015","unstructured":"Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2015. Voyager: Exploratory analysis via faceted browsing of visualization recommendations. IEEE transactions on visualization and computer graphics 22, 1 (2015), 649--658."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025768"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030423"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114826"},{"key":"e_1_2_1_66_1","unstructured":"Eugene Wu Fotis Psallidas Zhengjie Miao Haoci Zhang and Laura Rettig. 2017. Combining Design and Performance in a Data Visualization Management System. In CIDR. www.cidrdb.org."},{"key":"e_1_2_1_67_1","volume-title":"Krysta M Svore, and Jianfeng Gao.","author":"Wu Qiang","year":"2008","unstructured":"Qiang Wu, Chris JC Burges, Krysta M Svore, and Jianfeng Gao. 2008. Ranking, boosting, and model adaptation. Technical Report. Citeseer."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.visinf.2024.04.003"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467279"},{"key":"e_1_2_1_70_1","volume-title":"Are Large Language Models Good Statisticians? CoRR abs\/2406.07815","author":"Zhu Yizhang","year":"2024","unstructured":"Yizhang Zhu, Shiyin Du, Boyan Li, Yuyu Luo, and Nan Tang. 2024. Are Large Language Models Good Statisticians? CoRR abs\/2406.07815 (2024)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3681954.3681992","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T18:39:17Z","timestamp":1725475157000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3681954.3681992"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7]]},"references-count":70,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["10.14778\/3681954.3681992"],"URL":"https:\/\/doi.org\/10.14778\/3681954.3681992","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,7]]},"assertion":[{"value":"2024-08-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}