{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T15:07:15Z","timestamp":1764688035430,"version":"3.40.3"},"publisher-location":"Cham","reference-count":29,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031264375"},{"type":"electronic","value":"9783031264382"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T00:00:00Z","timestamp":1677110400000},"content-version":"vor","delay-in-days":53,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The Transformer architecture is shown to provide a powerful framework as an end-to-end model for building expression trees from online handwritten gestures corresponding to glyph strokes. In particular, the attention mechanism was successfully used to encode, learn and enforce the underlying syntax of expressions creating latent representations that are correctly decoded to the exact mathematical expression tree, providing robustness to ablated inputs and unseen glyphs. For the first time, the encoder is fed with spatio-temporal data tokens potentially forming an infinitely large vocabulary, which finds applications beyond that of online gesture recognition. A new supervised dataset of online handwriting gestures is provided for training models on generic handwriting recognition tasks and a new metric is proposed for the evaluation of the syntactic correctness of the output expression trees. A small Transformer model suitable for edge inference was successfully trained to an average normalised Levenshtein accuracy of 94%, resulting in valid postfix RPN tree representation for 94% of predictions.<\/jats:p>","DOI":"10.1007\/978-3-031-26438-2_5","type":"book-chapter","created":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T06:32:56Z","timestamp":1677047576000},"page":"55-67","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Transformer Architecture for\u00a0Online Gesture Recognition of\u00a0Mathematical Expressions"],"prefix":"10.1007","author":[{"given":"Mirco","family":"Ramo","sequence":"first","affiliation":[]},{"given":"Gu\u00e9nol\u00e9 C. M.","family":"Silvestre","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,2,23]]},"reference":[{"key":"5_CR1","volume-title":"Compiler Construction for Digital Computers","author":"D Gries","year":"1971","unstructured":"Gries, D.: Compiler Construction for Digital Computers. Wiley, New York (1971)"},{"issue":"1","key":"5_CR2","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1109\/34.824821","volume":"22","author":"R Plamondon","year":"2000","unstructured":"Plamondon, R., Srihari, S.: Online and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63\u201384 (2000)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"1","key":"5_CR3","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1007\/s10032-021-00365-5","volume":"24","author":"D Sinwar","year":"2021","unstructured":"Sinwar, D., Dhaka, V.S., Pradhan, N., et al.: Offline script recognition from handwritten and printed multilingual documents: a survey. Int. J. Doc. Anal. Recogn. (IJDAR) 24(1), 97\u2013121 (2021)","journal-title":"Int. J. Doc. Anal. Recogn. (IJDAR)"},{"key":"5_CR4","unstructured":"Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: International Conference on Learning Representations (ICLR) (2015)"},{"issue":"16","key":"5_CR5","doi-asserted-by":"publisher","first-page":"10563","DOI":"10.1007\/s00521-021-05813-1","volume":"33","author":"J Poulos","year":"2021","unstructured":"Poulos, J., Valle, R.: Character-based handwritten text transcription with attention networks. Neural Comput. Appl. 33(16), 10563\u201310573 (2021). https:\/\/doi.org\/10.1007\/s00521-021-05813-1","journal-title":"Neural Comput. Appl."},{"key":"5_CR6","doi-asserted-by":"publisher","first-page":"887","DOI":"10.1007\/978-0-85729-859-1_29","volume-title":"Handbook of Document Image Processing and Recognition","author":"JH Kim","year":"2014","unstructured":"Kim, J.H., Sin, B.-K.: Online handwriting recognition. In: Doermann, D., Tombre, K. (eds.) Handbook of Document Image Processing and Recognition, pp. 887\u2013915. Springer, London (2014). https:\/\/doi.org\/10.1007\/978-0-85729-859-1_29"},{"key":"5_CR7","doi-asserted-by":"crossref","unstructured":"Barakat, B., Droby, A., Kassis, M., El-Sana, J.: Text line segmentation for challenging handwritten document images using fully convolutional network. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 374\u2013379 (2018)","DOI":"10.1109\/ICFHR-2018.2018.00072"},{"issue":"6","key":"5_CR8","doi-asserted-by":"publisher","first-page":"1180","DOI":"10.1109\/TPAMI.2016.2572693","volume":"39","author":"D Keysers","year":"2017","unstructured":"Keysers, D., Deselaers, T., Rowley, H.A., et al.: Multi-language online handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1180\u20131194 (2017)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"5_CR9","doi-asserted-by":"publisher","unstructured":"Graves, A.: Generating sequences with recurrent neural networks. arXiv (2013). https:\/\/doi.org\/10.48550\/arXiv.1308.0850","DOI":"10.48550\/arXiv.1308.0850"},{"key":"5_CR10","doi-asserted-by":"crossref","unstructured":"Shrivastava, A., Jaggi, I., Gupta, S., et al.: Handwritten digit recognition using machine learning: a review. In: 2019 2nd International Conference on Power Energy, Environment and Intelligent Control (PEEIC), pp. 322\u2013326 (2019)","DOI":"10.1109\/PEEIC47157.2019.8976601"},{"key":"5_CR11","doi-asserted-by":"publisher","unstructured":"Corr, P.J., Silvestre, G.C., Bleakley, C.J.: Open source dataset and deep learning models for online digit gesture recognition on touchscreens. In: 2017 Irish Machine Vision and Image Processing Conference (IMVIP) (2017). https:\/\/doi.org\/10.48550\/arXiv.1709.06871","DOI":"10.48550\/arXiv.1709.06871"},{"key":"5_CR12","doi-asserted-by":"crossref","unstructured":"Li, Z., Jin, L., Lai, S., et al.: Improving attention-based handwritten mathematical expression recognition with scale augmentation and drop attention. In: 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 175\u2013180 (2020)","DOI":"10.1109\/ICFHR2020.2020.00041"},{"key":"5_CR13","doi-asserted-by":"crossref","unstructured":"Wang, J., Du, J., Zhang, J., et al.: Multi-modal attention network for handwritten mathematical expression recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1181\u20131186 (2019)","DOI":"10.1109\/ICDAR.2019.00191"},{"key":"5_CR14","doi-asserted-by":"publisher","first-page":"2471","DOI":"10.1109\/TMM.2020.3011316","volume":"23","author":"J Zhang","year":"2021","unstructured":"Zhang, J., Du, J., Yang, Y., et al.: SRD: a tree structure based decoder for online handwritten mathematical expression recognition. IEEE Trans. Multimed. 23, 2471\u20132480 (2021)","journal-title":"IEEE Trans. Multimed."},{"key":"5_CR15","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30, pp. 6000\u20136010 (2017)"},{"key":"5_CR16","unstructured":"Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4171\u20134186. Association for Computational Linguistics (2019)"},{"key":"5_CR17","unstructured":"Brown, T., Mann, B., Ryder, N., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877\u20131901 (2020)"},{"key":"5_CR18","doi-asserted-by":"publisher","unstructured":"Wolf, T., Debut, L., Sanh, V., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38\u201345 (2020). https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-demos.6","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"5_CR19","unstructured":"Parmar, N., Vaswani, A., et al.: Image transformer. In: International Conference on Machine Learning, pp. 4055\u20134064. PMLR (2018)"},{"key":"5_CR20","unstructured":"Huang, C.Z.A., Vaswani, A., et al.: Music transformer: generating music with long-term structure. In: International Conference on Learning Representations (ICLR) (2019)"},{"key":"5_CR21","doi-asserted-by":"crossref","unstructured":"Zhao, H., Jiang, L., Jia, J., et al.: Point transformer. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 16259\u201316268 (2021)","DOI":"10.1109\/ICCV48922.2021.01595"},{"key":"5_CR22","doi-asserted-by":"crossref","unstructured":"Kozlov, A., Andronov, V., Gritsenko, Y.: Lightweight network architecture for real-time action recognition. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 2074\u20132080 (2020)","DOI":"10.1145\/3341105.3373906"},{"key":"5_CR23","doi-asserted-by":"crossref","unstructured":"D\u2019Eusanio, A., Simoni, A., Pini, S., et al.: A transformer-based network for dynamic hand gesture recognition. In: 2020 International Conference on 3D Vision (3DV), pp. 623\u2013632 (2020)","DOI":"10.1109\/3DV50981.2020.00072"},{"key":"5_CR24","doi-asserted-by":"publisher","unstructured":"Wang, S., Li, B.Z., Khabsa, M., et al.: Linformer: self-attention with linear complexity. arXiv (2020). https:\/\/doi.org\/10.48550\/arXiv.2006.04768","DOI":"10.48550\/arXiv.2006.04768"},{"key":"5_CR25","unstructured":"Kitaev, N., Kaiser, L., Levskaya, A.: Reformer: the efficient transformer. In: International Conference on Learning Representations (2020)"},{"key":"5_CR26","unstructured":"Choromanski, K., Likhosherstov, V., Dohan, D., et al.: Rethinking attention with performers. In: International Conference on Learning Representations (2021)"},{"key":"5_CR27","doi-asserted-by":"crossref","unstructured":"Rao, R.M., Liu, J., Verkuil, R., et al.: MSA transformer. In: International Conference on Machine Learning, pp. 8844\u20138856. PMLR (2021)","DOI":"10.1101\/2021.02.12.430858"},{"key":"5_CR28","unstructured":"Akinremi, O., Balado, F., Silvestre, G.C.: A machine translation model for online glyph recognition. UCD Internal Research Report (2021, to be published)"},{"issue":"6","key":"5_CR29","doi-asserted-by":"publisher","first-page":"1091","DOI":"10.1109\/TPAMI.2007.1078","volume":"29","author":"L Yujian","year":"2007","unstructured":"Yujian, L., Bo, L.: A normalized levenshtein distance metric. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1091\u20131095 (2007)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Communications in Computer and Information Science","Artificial Intelligence and Cognitive Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-26438-2_5","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,22]],"date-time":"2023-02-22T06:34:45Z","timestamp":1677047685000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-26438-2_5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031264375","9783031264382"],"references-count":29,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-26438-2_5","relation":{},"ISSN":["1865-0929","1865-0937"],"issn-type":[{"type":"print","value":"1865-0929"},{"type":"electronic","value":"1865-0937"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"23 February 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"AICS","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Irish Conference on Artificial Intelligence and Cognitive Science","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Munster","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Ireland","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2022","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"8 December 2022","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"9 December 2022","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"30","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"aics2022","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/aics2022.mtu.ie\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Single-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"EasyChair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"102","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"41","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"40% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"No","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}