{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T03:02:35Z","timestamp":1762052555452,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T00:00:00Z","timestamp":1652313600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Chart data extraction is a crucial research field in recovering information from chart images. With the recent rise in image processing and computer vision algorithms, researchers presented various approaches to tackle this problem. Nevertheless, most of them use different datasets, often not publicly available to the research community. Therefore, the main focus of this research was to create a chart data extraction algorithm for circular-shaped and grid-like chart types, which will accelerate research in this field and allow uniform result comparison. A large-scale dataset is provided containing 120,000 chart images organized into 20 categories, with corresponding ground truth for each image. Through the undertaken extensive research and to the best of our knowledge, no other author reports the chart data extraction of the sunburst diagrams, heatmaps, and waffle charts. In this research, a new, fully automatic low-level algorithm is also presented that uses a raster image as input and generates an object-oriented structure of the chart of that image. The main novelty of the proposed approach is in chart processing on binary images instead of commonly used pixel counting techniques. The experiments were performed with a synthetic dataset and with real-world chart images. The obtained results demonstrate two things: First, a low-level bottom-up approach can be shared among different chart types. Second, the proposed algorithm achieves superior results on a synthetic dataset. The achieved average data extraction accuracy on the synthetic dataset can be considered state-of-the-art within multiple error rate groups.<\/jats:p>","DOI":"10.3390\/jimaging8050136","type":"journal-article","created":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T21:46:53Z","timestamp":1652392013000},"page":"136","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Data Extraction of Circular-Shaped and Grid-like Chart Images"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5925-9760","authenticated-orcid":false,"given":"Filip","family":"Baji\u0107","sequence":"first","affiliation":[{"name":"University Computing Centre, University of Zagreb, 10000 Zagreb, Croatia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josip","family":"Job","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, 31000 Osijek, Croatia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, C., H\u00e4rdle, W., Unwin, A., and Friendly, M. (2008). A brief history of data visualization. Handbook of Data Visualization, Springer.","DOI":"10.1007\/978-3-540-33037-0"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1111\/cgf.13193","article-title":"Reverse-engineering visualizations: Recovering visual encodings from chart images","volume":"36","author":"Poco","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_3","first-page":"43","article-title":"Data visualization classification using simple convolutional neural network model","volume":"11","author":"Job","year":"2020","journal-title":"Int. J. Electr. Comput. Eng. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"52926","DOI":"10.1109\/ACCESS.2021.3069205","article-title":"Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data from Chart Images","volume":"9","author":"Shahira","year":"2021","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chester, D., and Elzer, S. (2005). Getting Computers to See Information Graphics So Users Do Not Have to, Springer.","DOI":"10.1007\/11425274_68"},{"key":"ref_6","unstructured":"Huang, W., Tan, C.L., and Leow, W.K. (September, January 31). Associating text and graphics for scientific chart understanding. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, Seoul, Korea."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Gao, J., Zhou, Y., and Barner, K.E. (October, January 30). View: Visual information extraction widget for improving chart images accessibility. Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA.","DOI":"10.1109\/ICIP.2012.6467497"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nair, R.R., Sankaran, N., Nwogu, I., and Govindaraju, V. (2015, January 23\u201326). Automated analysis of line plots in documents. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, Tunis, Tunisia.","DOI":"10.1109\/ICDAR.2015.7333871"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mishchenko, A., and Vassilieva, N. (2011, January 26\u201328). Chart image understanding and numerical data extraction. Proceedings of the 6th International Conference on Digital Information Management, ICDIM, Melbourne, Australia.","DOI":"10.1109\/ICDIM.2011.6093320"},{"key":"ref_10","first-page":"76","article-title":"Model-Based Recognition and Extraction of Information from Chart Images","volume":"2","author":"Mishchenko","year":"2011","journal-title":"J. Multim. Process. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mishchenko, A., and Vassilieva, N. (2011). Model-Based Chart Image Classification, Springer. No. PART 2.","DOI":"10.1007\/978-3-642-24031-7_48"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shi, Y., Wei, Y., Wu, T., and Liu, Q. (2017, January 22\u201325). Statistical graph classification in intelligent mathematics problem solving system for high school student. Proceedings of the ICCSE 2017 12th International Conference on Computer Science and Education, Houston, TX, USA.","DOI":"10.1109\/ICCSE.2017.8085572"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10417","DOI":"10.1007\/s11042-020-10186-z","article-title":"ChartFuse: A novel fusion method for chart classification using heterogeneous microstructures","volume":"80","author":"Mishra","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1111\/cgf.13686","article-title":"Visualizing for the non-visual: Enabling the visually impaired to use visualization","volume":"38","author":"Choi","year":"2019","journal-title":"Comput. Graph. Forum"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Jobin, K.V., Mondal, A., and Jawahar, C.V. (2019, January 22\u201325). DocFigure: A Dataset for Scientific Document Figure Classification. Proceedings of the 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), Sydney, Australia.","DOI":"10.1109\/ICDARW.2019.00018"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kaur, P., and Kiesel, D. (2020, January 27\u201329). Combining image and caption analysis for classifying charts in biodiversity texts. Proceedings of the VISIGRAPP 2020 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Valletta, Malta.","DOI":"10.5220\/0008946701570168"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Baji\u0107, F., and Job, J. (2021). Chart classification using siamese CNN. J. Imaging, 7.","DOI":"10.3390\/jimaging7110220"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jung, D., Kim, W., Song, H., Hwang, J., Lee, B., Kim, B., and Seo, J. (2017, January 6\u201311). ChartSense: Interactive data extraction from chart images. Proceedings of the Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025957"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yang, L., Huang, W., and Tan, C.L. (2006). Semi-Automatic Ground Truth Generation for Chart Image Recognition, Springer.","DOI":"10.1007\/11669487_29"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cliche, M., Rosenberg, D., Madeka, D., and Yee, C. (2017). Scatteract: Automated Extraction of Data from Scatter Plots, Springer.","DOI":"10.1007\/978-3-319-71249-9_9"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chen, L., and Zhao, K. (2021). An Approach for Chart Description Generation in Cyber\u2013Physical\u2013Social System. Symmetry, 13.","DOI":"10.3390\/sym13091552"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Savva, M., Kong, N., Chhajta, A., Li, F.F., Agrawala, M., and Heer, J. (2011, January 16\u201319). ReVision: Automated classification, analysis and redesign of chart images. Proceedings of the UIST\u201911 24th Annual ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, USA.","DOI":"10.1145\/2047196.2047247"},{"key":"ref_23","unstructured":"Balaji, A., Ramanathan, T., and Sonathi, V. (2018). Chart-Text: A Fully Automated Chart Image Descriptor. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"De, P. (2018, January 14\u201315). Automatic Data Extraction from 2D and 3D Pie Chart Images. Proceedings of the 8th International Advance Computing Conference, IACC, Greater Noida, India.","DOI":"10.1109\/IADCC.2018.8692104"},{"key":"ref_25","unstructured":"Liu, X., Klabjan, D., and NBless, P. (2019). Data Extraction from Charts via Single Deep Neural Network. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Al-Zaidy, R.A., and Giles, C.L. (2015, January 7\u201310). Automatic extraction of data from bar charts. Proceedings of the 8th International Conference on Knowledge Capture, K-CAP 2015, Palisades, NY, USA.","DOI":"10.1145\/2815833.2816956"},{"key":"ref_27","unstructured":"Al-Zaidy, R.A., Choudhury, S.R., and Giles, C.L. (2021, September 26). Automatic Summary Generation for Scientific Data Charts, Workshops at the Thirtieth AAAI Conference on Artificial Intelligence. Available online: https:\/\/www.aaai.org\/ocs\/index.php\/WS\/AAAIW16\/paper\/viewPaper\/12661."},{"key":"ref_28","unstructured":"Al-Zaidy, R.A., and Giles, C.L. (2017, January 4\u20139). A machine learning approach for semantic structuring of scientific charts in scholarly documents. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI\u201917), San Francisco, CA, USA. Available online: https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3297863.3297868."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.jvlc.2018.08.005","article-title":"Chart decoder: Generating textual and numeric information from chart images automatically","volume":"48","author":"Dai","year":"2018","journal-title":"J. Vis. Lang. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rane, C., Subramanya, S., Endluri, D., Wu, J., and Giles, C.L. (2021, January 10\u201312). ChartReader: Automatic Parsing of Bar-Plots. Proceedings of the 2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI), Las Vegas, NV, USA.","DOI":"10.1109\/IRI51335.2021.00050"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1007\/s12650-020-00702-6","article-title":"Reverse-engineering bar charts using neural networks","volume":"24","author":"Zhou","year":"2020","journal-title":"J. Vis."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, X., Tang, B., Wang, Z., Xu, X., Shiliang, P., Dapeng, T., and Mingli, S. (2015, January 23\u201326). Chart classification by combining deep convolutional networks and deep belief networks. Proceedings of the International Conference on Document Analysis and Recognition, ICDAR, Tunis, Tunisia.","DOI":"10.1109\/ICDAR.2015.7333872"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Siegel, N., Horvitz, Z., Levin, R., Divvala, S., and Farhadi, A. (2016). FigureSeer: Parsing Result-Figures in Research Papers, Springer.","DOI":"10.1007\/978-3-319-46478-7_41"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kafle, K., Price, B., Cohen, S., and Kanan, C. (2018). DVQA: Understanding Data Visualizations via Question Answering. arXiv.","DOI":"10.1109\/CVPR.2018.00592"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Davila, K., Kota, B.U., Setlur, S., Govindaraju, V., Tenesmeyer, C., Shekhar, S., and Chaudhry, R. (2019, January 20\u201325). CDAR 2019 Competition on Harvesting Raw Tables from Infographics (CHART-Infographics). Proceedings of the 2019 International Conference on Document Analysis and Recognition (ICDAR), Sydney, Australia.","DOI":"10.1109\/ICDAR.2019.00203"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Davila, K., Tensmeyer, C., Shekhar, S., Singh, H., Setlur, S., and Govindaraju, V. (2019). ICPR 2020 Competition on Harvesting Raw Tables from Infographics, Springer.","DOI":"10.1109\/ICDAR.2019.00203"},{"key":"ref_37","unstructured":"Plotly Technologies Inc (2022, March 20). Collaborative Data Science, Plotly Technologies Inc. Available online: https:\/\/plot.ly."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Spagnolo, F., Perri, S., and Corsonello, P. (2019). An Efficient Hardware-Oriented Single-Pass Approach for Connected Component Analysis. Sensors, 19.","DOI":"10.3390\/s19143055"},{"key":"ref_39","unstructured":"Gonzalez, R.C., and Woods, R.E. (2018). Digital Image Processing, Pearson. [4th ed.]."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/5\/136\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:10:03Z","timestamp":1760137803000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/5\/136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,12]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["jimaging8050136"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8050136","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2022,5,12]]}}}