{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T14:16:31Z","timestamp":1783001791982,"version":"3.54.5"},"reference-count":33,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T00:00:00Z","timestamp":1634774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>In recovering information from the chart image, the first step should be chart type classification. Throughout history, many approaches have been used, and some of them achieve results better than others. The latest articles are using a Support Vector Machine (SVM) in combination with a Convolutional Neural Network (CNN), which achieve almost perfect results with the datasets of few thousand images per class. The datasets containing chart images are primarily synthetic and lack real-world examples. To overcome the problem of small datasets, to our knowledge, this is the first report of using Siamese CNN architecture for chart type classification. Multiple network architectures are tested, and the results of different dataset sizes are compared. The network verification is conducted using Few-shot learning (FSL). Many of described advantages of Siamese CNNs are shown in examples. In the end, we show that the Siamese CNN can work with one image per class, and a 100% average classification accuracy is achieved with 50 images per class, where the CNN achieves only average classification accuracy of 43% for the same dataset.<\/jats:p>","DOI":"10.3390\/jimaging7110220","type":"journal-article","created":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:22:02Z","timestamp":1634862122000},"page":"220","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Chart Classification Using Siamese CNN"],"prefix":"10.3390","volume":"7","author":[{"given":"Filip","family":"Baji\u0107","sequence":"first","affiliation":[{"name":"University of Zagreb University Computing Centre, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6998-5907","authenticated-orcid":false,"given":"Josip","family":"Job","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Computer Science and Information Technology Osijek, 31000 Osijek, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, C., H\u00e4rdle, W., Unwin, A., and Friendly, M. (2008). A brief history of data visualization. Handbook of Data Visualization, Springer.","DOI":"10.1007\/978-3-540-33037-0"},{"key":"ref_2","unstructured":"Zhou, Y.P., and Tan, C.L. (2020, May 01). Hough Technique for Bar Charts Detection and Recognition in Document Images. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/899506\/."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhou, Y.P., and Tan, C.L. (2000). Bar charts recognition using hough based syntactic segmentation. Theory and Application of Diagrams, Springer.","DOI":"10.1007\/3-540-44590-0_45"},{"key":"ref_4","unstructured":"Redeke, I. (2020, May 01). Image & Graphic Reader. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/959168\/."},{"key":"ref_5","unstructured":"Gao, J., Zhou, Y., and Barner, K.E. (2020, May 01). View: Visual Information Extraction Widget for Improving Chart Images Accessibility. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/6467497\/."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Battle, L., Duan, P., Miranda, Z., Mukusheva, D., Chang, R., and Stonebraker, M. (2018, January 21\u201326). Beagle: Automated extraction and interpretation of visualizations from the web. Proceedings of the Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3174168"},{"key":"ref_7","unstructured":"Mishchenko, A., and Vassilieva, N. (2020, May 01). Model-Based Recognition and Extraction of Information from Chart Images. Available online: https:\/\/pdfs.semanticscholar.org\/33c3\/2c036fa74d2fe812759e9c3d443767e3fb5b.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mishchenko, A., and Vassilieva, N. (2011). Model-Based Chart Image Classification, Springer.","DOI":"10.1007\/978-3-642-24031-7_48"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1111\/cgf.13193","article-title":"Reverse-engineering visualizations: Recovering visual encodings from chart images","volume":"36","author":"Poco","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Savva, M., Kong, N., Chhajta, A., Li, F.F., Agrawala, M., and Heer, J. (2011, January 16\u201319). ReVision: Automated Classification, Analysis and Redesign of Chart Images. Proceedings of the UIST\u201911 24th Annual ACM Symposium on User Interface Software and Technology, Santa Barbara, CA, USA.","DOI":"10.1145\/2047196.2047247"},{"key":"ref_11","unstructured":"Lin, A., Ford, J., Adar, E., and Hecht, B. (2020, May 01). VizByWiki: Mining Data Visualizations from the Web to Enrich News Articles. Available online: https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3178876.3186135."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1111\/cgf.13686","article-title":"Visualizing for the Non-visual: Enabling the Visually Impaired to Use Visualization","volume":"38","author":"Choi","year":"2019","journal-title":"Comput. Graph. Forum"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Jobin, K.V., Mondal, A., and Jawahar, C. (2020, May 01). DocFigure: A Dataset for Scientific Document Figure Classification. Available online: https:\/\/researchweb.iiit.ac.in\/.","DOI":"10.1109\/ICDARW.2019.00018"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kaur, P., and Kiesel, D. (2020, January 27\u201329). Combining image and caption analysis for classifying charts in biodiversity. Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications\u2014IVAPP, Valletta, Malta.","DOI":"10.5220\/0008946701570168"},{"key":"ref_15","first-page":"43","article-title":"Data visualization classification using simple convolutional neural network model","volume":"11","author":"Job","year":"2020","journal-title":"Int. J. Electr. Comput. Eng. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1250","DOI":"10.1007\/s42452-020-3055-y","article-title":"Multi-label classification of line chart images using convolutional neural networks","volume":"2","author":"Kosemen","year":"2020","journal-title":"SN Appl. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ishihara, T., Morita, K., Shirai, N.C., Wakabayashi, T., and Ohyama, W. (2019). Chart-type classification using convolutional neural network for scholarly figures. Asian Conference on Pattern Recognition, Springer.","DOI":"10.1007\/978-3-030-41299-9_20"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dadhich, K., Daggubati, S.C., and Sreevalsan-Nair, J. (2021, January 28\u201330). BarChartAnalyzer: Digitizing images of bar charts. Proceedings of the International Conference on Image Processing and Vision Engineering\u2014IMPROVE, Online.","DOI":"10.5220\/0010408300170028"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1007\/s12650-020-00702-6","article-title":"Reverse-engineering bar charts using neural networks","volume":"24","author":"Zhou","year":"2021","journal-title":"J. Vis."},{"key":"ref_20","unstructured":"Chagas, P., Freitas, A., Daisuke, R., Miranda, B., Araujo, T., Santos, C., Meiguins, B., and Morais, J. (2020, May 01). Architecture Proposal for Data Extraction of Chart Images Using Convolutional Neural Network. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/8107990\/."},{"key":"ref_21","unstructured":"Huang, S. (2020, August 30). An Image Classification Tool of Wikimedia Commons. Available online: https:\/\/edoc.hu-berlin.de\/handle\/18452\/22325."},{"key":"ref_22","unstructured":"Dai, W., Wang, M., Niu, Z., and Zhang, J. (2020, May 01). Chart decoder: Generating Textual and Numeric Information from Chart Images Automatically. Available online: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1045926X18301162."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Thiyam, J., Singh, S.R., and Bora, P.K. (2021, January 24\u201327). Challenges in chart image classification: A comparative study of different deep learning methods. Proceedings of the 21st ACM Symposium on Document Engineering, Limerick, Ireland.","DOI":"10.1145\/3469096.3474931"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1142\/S0218001493000339","article-title":"Signature verification using a siamese time delay neural network","volume":"7","author":"Bromley","year":"1993","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_25","unstructured":"Song, L., Gong, D., Li, Z., Liu, C., and Liu, W. (November, January 27). Occlusion robust face recognition based on mask learning with pairwise differential siamese network. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_26","unstructured":"Dey, S., Dutta, A., Toledo, J.I., Ghosh, S.K., Llad\u00f3s, J., and Pal, U. (2017). Signet: Convolutional Siamese network for writer independent offline signature verification. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Langford, Z., Eisenbeiser, L., and Vondal, M. (2019, January 15\u201317). Robust signal classification using siamese networks. Proceedings of the ACM Workshop on Wireless Security and Machine Learning, Miami, FL, USA.","DOI":"10.1145\/3324921.3328781"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lian, Z., Li, Y., Tao, J., and Huang, J. (2018, January 26). Speech emotion recognition via contrastive loss under siamese networks. Proceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and First Multi-Modal Affective Computing of Large-Scale Multimedia Data, Seoul, Korea.","DOI":"10.1145\/3267935.3267946"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baji\u0107, F., Job, J., and Nenadi\u0107, K. (2019, January 5\u20137). Chart classification using simplified VGG model. Proceedings of the 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), Osijeka, Croatia.","DOI":"10.1109\/IWSSIP.2019.8787299"},{"key":"ref_30","unstructured":"Tensmeyer, C. (2020, November 15). Competition on Harvesting Raw Tables (CHART) 2019-Synthetic (CHART2019-S), 1, ID: CHART2019-S_1. Available online: http:\/\/tc11.cvc.uab.es\/datasets\/CHART2019-S_1."},{"key":"ref_31","unstructured":"AT&T Laboratories Cambridge (2020, November 17). The Database of Faces 2002. Available online: http:\/\/www.cl.cam.ac.uk\/research\/dtg\/attarchive\/facedatabase.html."},{"key":"ref_32","unstructured":"Koch, G., Zemel, R., and Salakhutdinov, R. (2015, January 10\u201311). Siamese neural networks for one-shot image recognition. Proceedings of the ICML Deep Learning Workshop, Lille, France."},{"key":"ref_33","unstructured":"Lake, B.M., Salakhutdinov, R., Gross, J., and Tenenbaum, J.B. (2011, January 20\u201323). One shot learning of simple visual concepts. Proceedings of the 33rd Annual Conference of the Cognitive Science Society, Boston, MA, USA."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/11\/220\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:20:15Z","timestamp":1760167215000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/11\/220"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,21]]},"references-count":33,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["jimaging7110220"],"URL":"https:\/\/doi.org\/10.3390\/jimaging7110220","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,21]]}}}