{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T09:56:30Z","timestamp":1772963790173,"version":"3.50.1"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,12,5]],"date-time":"2018-12-05T00:00:00Z","timestamp":1543968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Hasler Foundation through the DCrowdLens project"},{"name":"Swiss National Science Foundation through the MAAYA project"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Comput. Cult. Herit."],"published-print":{"date-parts":[[2018,12,29]]},"abstract":"<jats:p>Thanks to the digital preservation of cultural heritage materials, multimedia tools (e.g., based on automatic visual processing) considerably ease the work of scholars in the humanities and help them to perform quantitative analysis of their data. In this context, this article assesses three different Convolutional Neural Network (CNN) architectures along with three learning approaches to train them for hieroglyph classification, which is a very challenging task due to the limited availability of segmented ancient Maya glyphs. More precisely, the first approach, the baseline, relies on pretrained networks as feature extractor. The second one investigates a transfer learning method by fine-tuning a pretrained network for our glyph classification task. The third approach considers directly training networks from scratch with our glyph data. The merits of three different network architectures are compared: a generic sequential model (i.e., LeNet), a sketch-specific sequential network (i.e., Sketch-a-Net), and the recent Residual Networks. The sketch-specific model trained from scratch outperforms other models and training strategies. Even for a challenging 150-class classification task, this model achieves 70.3% average accuracy and proves itself promising in case of a small amount of cultural heritage shape data. Furthermore, we visualize the discriminative parts of glyphs with the recent Grad-CAM method, and demonstrate that the discriminative parts learned by the model agree, in general, with the expert annotation of the glyph specificity (diagnostic features). Finally, as a step toward systematic evaluation of these visualizations, we conduct a perceptual crowdsourcing study. Specifically, we analyze the interpretability of the representations from Sketch-a-Net and ResNet-50. Overall, our article takes two important steps toward providing tools to scholars in the digital humanities: increased performance for automation and improved interpretability of algorithms.<\/jats:p>","DOI":"10.1145\/3230670","type":"journal-article","created":{"date-parts":[[2018,12,5]],"date-time":"2018-12-05T13:04:59Z","timestamp":1544015099000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["How to Tell Ancient Signs Apart? Recognizing and Visualizing Maya Glyphs with CNNs"],"prefix":"10.1145","volume":"11","author":[{"given":"G\u00fclcan","family":"Can","sequence":"first","affiliation":[{"name":"Idiap Research Institute and \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean-Marc","family":"Odobez","sequence":"additional","affiliation":[{"name":"Idiap Research Institute and \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Gatica-Perez","sequence":"additional","affiliation":[{"name":"Idiap Research Institute and \u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne (EPFL), Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,12,5]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2477680"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2905369"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2755985"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3095713.3095746"},{"key":"e_1_2_1_5_1","volume-title":"An analysis of deep neural network models for practical applications. arXiv Preprint arXiv:1605.07678","author":"Canziani Alfredo","year":"2016","unstructured":"Alfredo Canziani , Adam Paszke , and Eugenio Culurciello . 2016. An analysis of deep neural network models for practical applications. arXiv Preprint arXiv:1605.07678 ( 2016 ). Alfredo Canziani, Adam Paszke, and Eugenio Culurciello. 2016. An analysis of deep neural network models for practical applications. arXiv Preprint arXiv:1605.07678 (2016)."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2012.6252544"},{"key":"e_1_2_1_7_1","volume-title":"Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition","author":"Deng Jia","year":"2009","unstructured":"Jia Deng , Wei Dong , Richard Socher , Li-Jia Li , Kai Li , and Li Fei-Fei . 2009 . Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition . IEEE , 248--255. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition. IEEE, 248--255."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2012.119"},{"key":"e_1_2_1_9_1","first-page":"647","article-title":"DeCAF: A deep convolutional activation feature for generic visual recognition","volume":"32","author":"Donahue Jeff","year":"2014","unstructured":"Jeff Donahue , Yangqing Jia , Oriol Vinyals , Judy Hoffman , Ning Zhang , Eric Tzeng , and Trevor Darrell . 2014 . DeCAF: A deep convolutional activation feature for generic visual recognition . In ICML , Vol. 32. 647 -- 655 . Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. 2014. DeCAF: A deep convolutional activation feature for generic visual recognition. In ICML, Vol. 32. 647--655.","journal-title":"ICML"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185540"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2502081.2502199"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics. 249--256","author":"Glorot Xavier","year":"2010","unstructured":"Xavier Glorot and Yoshua Bengio . 2010 . Understanding the difficulty of training deep feedforward neural networks . In Proceedings of the International Conference on Artificial Intelligence and Statistics. 249--256 . Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics. 249--256."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_14_1","volume-title":"The Impact of Imbalanced Training Data for Convolutional Neural Networks","author":"Hensman Paulina","unstructured":"Paulina Hensman and David Masko . 2015. The Impact of Imbalanced Training Data for Convolutional Neural Networks . Technical Report. KTH, Stockholm, Sweden . Degree Project, in Computer Science, First Level . Paulina Hensman and David Masko. 2015. The Impact of Imbalanced Training Data for Convolutional Neural Networks. Technical Report. KTH, Stockholm, Sweden. Degree Project, in Computer Science, First Level."},{"key":"e_1_2_1_15_1","volume-title":"Salakhutdinov","author":"Hinton Geoffrey E.","year":"2012","unstructured":"Geoffrey E. Hinton , Nitish Srivastava , Alex Krizhevsky , Ilya Sutskever , and Ruslan R . Salakhutdinov . 2012 . Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint arXiv:1207.0580 (2012). Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint arXiv:1207.0580 (2012)."},{"key":"e_1_2_1_16_1","volume-title":"One-shot adaptation of supervised deep convolutional models. arXiv Preprint arXiv:1312.6204","author":"Hoffman Judy","year":"2013","unstructured":"Judy Hoffman , Eric Tzeng , Jeff Donahue , Yangqing Jia , Kate Saenko , and Trevor Darrell . 2013. One-shot adaptation of supervised deep convolutional models. arXiv Preprint arXiv:1312.6204 ( 2013 ). Judy Hoffman, Eric Tzeng, Jeff Donahue, Yangqing Jia, Kate Saenko, and Trevor Darrell. 2013. One-shot adaptation of supervised deep convolutional models. arXiv Preprint arXiv:1312.6204 (2013)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1086\/300142"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2015.2411291"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the International Conference on Machine Learning. 448--456","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015 . Batch normalization: Accelerating deep network training by reducing internal covariate shift . In Proceedings of the International Conference on Machine Learning. 448--456 . Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning. 448--456."},{"key":"e_1_2_1_20_1","volume-title":"Adam: A method for stochastic optimization. arXiv Preprint arXiv:1412.6980","author":"Kingma Diederik","year":"2014","unstructured":"Diederik Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv Preprint arXiv:1412.6980 (2014). Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv Preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_21_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems . 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105."},{"key":"e_1_2_1_22_1","volume-title":"Deep learning. Nature 521, 7553","author":"LeCun Yann","year":"2015","unstructured":"Yann LeCun , Yoshua Bengio , and Geoffrey Hinton . 2015. Deep learning. Nature 521, 7553 ( 2015 ), 436--444. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436--444."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/850924.851523"},{"key":"e_1_2_1_25_1","volume-title":"Macri and Matthew George Looper","author":"Martha","year":"2003","unstructured":"Martha J. Macri and Matthew George Looper . 2003 . The New Catalog of Maya Hieroglyphs, Vol . 1: The Classic Period Inscriptions. University of Oklahoma Press . Martha J. Macri and Matthew George Looper. 2003. The New Catalog of Maya Hieroglyphs, Vol. 1: The Classic Period Inscriptions. University of Oklahoma Press."},{"key":"e_1_2_1_26_1","volume-title":"Macri and Gabrielle Vail","author":"Martha","year":"2008","unstructured":"Martha J. Macri and Gabrielle Vail . 2008 . The New Catalog of Maya Hieroglyphs, Vol . 2: The Codical Texts. University of Oklahoma Press . Martha J. Macri and Gabrielle Vail. 2008. The New Catalog of Maya Hieroglyphs, Vol. 2: The Codical Texts. University of Oklahoma Press."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1951.10500769"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3095713.3095719"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46604-0_58"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0387-x"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2425333.2425399"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.daach.2015.03.001"},{"key":"e_1_2_1_33_1","volume-title":"Graph-Based Shape Similarity of Petroglyphs","author":"Seidl Markus","unstructured":"Markus Seidl , Ewald Wieser , Matthias Zeppelzauer , Axel Pinz , and Christian Breiteneder . 2015. Graph-Based Shape Similarity of Petroglyphs . Springer International Publishing , Cham , 133--148. Markus Seidl, Ewald Wieser, Matthias Zeppelzauer, Axel Pinz, and Christian Breiteneder. 2015. Graph-Based Shape Similarity of Petroglyphs. Springer International Publishing, Cham, 133--148."},{"key":"e_1_2_1_34_1","volume-title":"Grad-CAM: Why did you say that?arXiv Preprint arXiv:1611.07450","author":"Selvaraju Ramprasaath R.","year":"2016","unstructured":"Ramprasaath R. Selvaraju , Abhishek Das , Ramakrishna Vedantam , Michael Cogswell , Devi Parikh , and Dhruv Batra . 2016. Grad-CAM: Why did you say that?arXiv Preprint arXiv:1611.07450 ( 2016 ). Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. 2016. Grad-CAM: Why did you say that?arXiv Preprint arXiv:1611.07450 (2016)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2014.131"},{"key":"e_1_2_1_36_1","volume-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv Preprint arXiv:1312.6034","author":"Simonyan Karen","year":"2013","unstructured":"Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv Preprint arXiv:1312.6034 ( 2013 ). Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv Preprint arXiv:1312.6034 (2013)."},{"key":"e_1_2_1_37_1","unstructured":"K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs\/1409.1556 (2014).  K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs\/1409.1556 (2014)."},{"key":"e_1_2_1_38_1","volume-title":"inception-resnet and the impact of residual connections on learning. arXiv Preprint arXiv:1602.07261","author":"Szegedy Christian","year":"2016","unstructured":"Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke , and Alex Alemi . 2016. Inception-v4 , inception-resnet and the impact of residual connections on learning. arXiv Preprint arXiv:1602.07261 ( 2016 ). Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. 2016. Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv Preprint arXiv:1602.07261 (2016)."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2535302"},{"key":"e_1_2_1_41_1","volume-title":"Stuart","author":"Sidney Thompson John Eric","year":"1962","unstructured":"John Eric Sidney Thompson and George E . Stuart . 1962 . A Catalog of Maya Hieroglyphs. University of Oklahoma Press . John Eric Sidney Thompson and George E. Stuart. 1962. A Catalog of Maya Hieroglyphs. University of Oklahoma Press."},{"key":"e_1_2_1_42_1","unstructured":"Jason Yosinski Jeff Clune Yoshua Bengio and Hod Lipson. 2014. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems. 3320--3328.   Jason Yosinski Jeff Clune Yoshua Bengio and Hod Lipson. 2014. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems. 3320--3328."},{"key":"e_1_2_1_43_1","volume-title":"Sketch-a-net that beats humans. arXiv Preprint arXiv:1501.07873","author":"Yu Qian","year":"2015","unstructured":"Qian Yu , Yongxin Yang , Yi-Zhe Song , Tao Xiang , and Timothy Hospedales . 2015. Sketch-a-net that beats humans. arXiv Preprint arXiv:1501.07873 ( 2015 ). Qian Yu, Yongxin Yang, Yi-Zhe Song, Tao Xiang, and Timothy Hospedales. 2015. Sketch-a-net that beats humans. arXiv Preprint arXiv:1501.07873 (2015)."},{"key":"e_1_2_1_44_1","volume-title":"Zeiler and Rob Fergus","author":"Matthew","year":"2014","unstructured":"Matthew D. Zeiler and Rob Fergus . 2014 . Visualizing and understanding convolutional networks. In European Conference on Computer Vision. Springer , 818--833. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In European Conference on Computer Vision. Springer, 818--833."},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"B. Zhou A. Khosla Lapedriza. A. A. Oliva and A. Torralba. 2016. Learning deep features for discriminative localization. CVPR (2016).  B. Zhou A. Khosla Lapedriza. A. A. Oliva and A. Torralba. 2016. Learning deep features for discriminative localization. CVPR (2016).","DOI":"10.1109\/CVPR.2016.319"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-010-0200-z"}],"container-title":["Journal on Computing and Cultural Heritage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3230670","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3230670","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:47Z","timestamp":1750210787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3230670"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,5]]},"references-count":46,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,12,29]]}},"alternative-id":["10.1145\/3230670"],"URL":"https:\/\/doi.org\/10.1145\/3230670","relation":{},"ISSN":["1556-4673","1556-4711"],"issn-type":[{"value":"1556-4673","type":"print"},{"value":"1556-4711","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,5]]},"assertion":[{"value":"2017-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-12-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}