{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T19:19:10Z","timestamp":1793128750533,"version":"4.5.0"},"reference-count":100,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T00:00:00Z","timestamp":1729468800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T00:00:00Z","timestamp":1729468800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"University of Innsbruck and Medical University of Innsbruck"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper presents a comprehensive survey of over 100 research works on the topic of form understanding in the context of scanned documents. We delve into recent advancements and breakthroughs in the field, with particular focus on transformer-based models, which have been shown to improve performance in form understanding tasks by up to 25% in accuracy compared to traditional methods. Our research methodology involves an in-depth analysis of popular documents and trends over the last decade, including 15 state-of-the-art models and 10 benchmark datasets. By examining these works, we offer novel insights into the evolution of this domain. Specifically, we highlight how transformers have revolutionized form-understanding techniques by enhancing the ability to process noisy scanned documents with significant improvements in OCR accuracy. Furthermore, we present an overview of the most relevant datasets, such as FUNSD, CORD, and SROIE, which serve as benchmarks for evaluating the performance of the models. By comparing the capabilities of these models and reporting an average improvement of 10\u201315% in key form extraction tasks, we aim to provide researchers and practitioners with useful guidance in selecting the most suitable solutions for their form understanding applications.<\/jats:p>","DOI":"10.1007\/s10462-024-11000-0","type":"journal-article","created":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T05:02:23Z","timestamp":1729486943000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["A survey of recent approaches to form understanding in scanned documents"],"prefix":"10.1007","volume":"57","author":[{"given":"Abdelrahman","family":"Abdallah","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Eberharter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zoe","family":"Pfister","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Jatowt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,10,21]]},"reference":[{"key":"11000_CR1","unstructured":"Abdallah A, Abdalla M, Elkasaby M, Elbendary Y, Jatowt A (2023) Amurd: annotated multilingual receipts dataset for cross-lingual key information extraction and classification, arXiv preprint arXiv:2309.09800"},{"key":"11000_CR2","unstructured":"Abdallah A, Abdalla M, Kasem M. S, Mahmoud M, Abdelhalim I, Elkasaby M, ElBendary Y, Jatowt A (2024) Coru: comprehensive post-ocr parsing and receipt understanding dataset, arXiv preprint arXiv:2406.04493"},{"key":"11000_CR3","doi-asserted-by":"publisher","unstructured":"Appalaraju S, Jasani B, Kota BU, Xie Y, Manmatha R (2021) Docformer: end-to-end transformer for document understanding. In: 2021 IEEE\/CVF international conference on computer vision, ICCV 2021, Montreal, October 10\u201317, 2021, IEEE, pp 973\u2013983. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00103","DOI":"10.1109\/ICCV48922.2021.00103"},{"key":"11000_CR4","doi-asserted-by":"crossref","unstructured":"Appalaraju S, Tang P, Dong Q, Sankaran N, Zhou Y, Manmatha R (2023) Docformerv2: local features for document understanding, arXiv preprint arXiv:2306.01733","DOI":"10.1609\/aaai.v38i2.27828"},{"key":"11000_CR5","doi-asserted-by":"crossref","unstructured":"Appalaraju S, Tang P, Dong Q, Sankaran N, Zhou Y, Manmatha R (2024) Docformerv2: local features for document understanding. In: Proceedings of the AAAI conference on artificial intelligence, vol 38, pp 709\u2013718","DOI":"10.1609\/aaai.v38i2.27828"},{"key":"11000_CR6","first-page":"32897","volume":"35","author":"H Bao","year":"2022","unstructured":"Bao H, Wang W, Dong L, Liu Q, Mohammed OK, Aggarwal K, Som S, Piao S, Wei F (2022) Vlmo: unified vision-language pre-training with mixture-of-modality-experts. Adv Neural Inform Process Syst 35:32897\u201332912","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR7","unstructured":"Bao H, Dong L, Piao S, Wei F (2021) Beit: Bert pre-training of image transformers, arXiv preprint arXiv:2106.08254"},{"key":"11000_CR8","unstructured":"Bao H, Dong L, Wei F, Wang W, Yang N, Liu X, Wang Y, Gao J, Piao S, Zhou M et al (2020) Unilmv2: pseudo-masked language models for unified language model pre-training. In: International conference on machine learning, PMLR, pp 642\u2013652"},{"issue":"6","key":"11000_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3355610","volume":"52","author":"GM Binmakhashen","year":"2019","unstructured":"Binmakhashen GM, Mahmoud SA (2019) Document layout analysis: a comprehensive survey. ACM Comput Surv (CSUR) 52(6):1\u201336","journal-title":"ACM Comput Surv (CSUR)"},{"key":"11000_CR10","doi-asserted-by":"crossref","unstructured":"Biten AF, Tito R, Mafla A, Gomez L, Rusi\u00f1ol M, Valveny E, Jawahar CV, Karatzas D (2019) Scene text visual question answering. arXiv:1905.13648","DOI":"10.1109\/ICCV.2019.00439"},{"key":"11000_CR11","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.1109\/TMM.2020.3037496","volume":"23","author":"H Chen","year":"2020","unstructured":"Chen H, Jiang D, Sahli H (2020) Transformer encoder with multi-modal multi-head attention for continuous affect recognition. IEEE Trans Multimed 23:4171\u20134183","journal-title":"IEEE Trans Multimed"},{"key":"11000_CR12","doi-asserted-by":"crossref","unstructured":"Chen X, Ding M, Wang X, Xin Y, Mo S, Wang Y,Han S, Luo P, Zeng G, Wang J (2022) Context autoencoder for self-supervised representation learning, arXiv preprint arXiv:2202.03026","DOI":"10.1007\/s11263-023-01852-4"},{"key":"11000_CR13","unstructured":"Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning, PMLR, pp 1597\u20131607"},{"key":"11000_CR14","unstructured":"Dalmia S, Mohamed A, Lewis M, Metze F, Zettlemoyer L (2019) Enforcing encoder-decoder modularity in sequence-to-sequence models. arXiv:1911.03782"},{"key":"11000_CR15","doi-asserted-by":"crossref","unstructured":"Davis B, Morse B, Cohen S, Price B, Tensmeyer C (2019) Deep visual template-free form parsing. In: 2019 International conference on document analysis and recognition (ICDAR), IEEE, pp 134\u2013141","DOI":"10.1109\/ICDAR.2019.00030"},{"key":"11000_CR16","unstructured":"Davis B, Morse B, Cohen S, Price B, Tensmeyer C (2019) National archives forms dataset, https:\/\/github.com\/herobd\/NAF_dataset"},{"key":"11000_CR17","unstructured":"Devlin J, Chang M.-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805"},{"key":"11000_CR18","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: J Burstein, C Doran, T Solorio (eds.), Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, NAACL-HLT 2019, Minneapolis, June 2\u20137, volume 1 (long and short papers), association for computational linguistics, 2019, pp 4171\u20134186. https:\/\/doi.org\/10.18653\/v1\/n19-1423","DOI":"10.18653\/v1\/n19-1423"},{"key":"11000_CR19","doi-asserted-by":"crossref","unstructured":"Ding Y, Long S, Huang J, Ren K, Luo X, Chung H, Han SC (2023) Form-nlu: dataset for the form language understanding, arXiv preprint arXiv:2304.01577","DOI":"10.1145\/3539618.3591886"},{"key":"11000_CR20","doi-asserted-by":"publisher","unstructured":"Ethayarajh K (2019) How contextual are contextualized word representations? Comparing the geometry of bert, elmo, and GPT-2 embedding. In: K Inui, J Jiang, V Ng, X Wan (eds.), Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, November 3\u20137, 2019, association for computational linguistics, pp 55\u201365. https:\/\/doi.org\/10.18653\/v1\/D19-1006","DOI":"10.18653\/v1\/D19-1006"},{"key":"11000_CR21","doi-asserted-by":"crossref","unstructured":"Farina M, Pappadopulo D, Gupta A, Huang L, \u0130rsoy O, Solorio T (2023) Distillation of encoder-decoder transformers for sequence labelling. arXiv:2302.05454","DOI":"10.18653\/v1\/2023.findings-eacl.192"},{"key":"11000_CR22","unstructured":"Feng C-M, Yan Y, Chen G, Xu Y, Shao L, Fu H (2022) Multi-modal transformer for accelerated mr imaging. arXiv:2106.14248"},{"key":"11000_CR23","unstructured":"Feng S, Zhan T, Jie Z, Luong T. Q, Jin X (2023) Sequence-to-sequence pre-training with unified modality masking for visual document understanding, arXiv preprint arXiv:2305.10448"},{"key":"11000_CR24","doi-asserted-by":"publisher","unstructured":"Fu T-J, Li P-H, Ma W-Y (2019) GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th annual meeting of the association for computational linguistics, association for computational linguistics, Florence pp 1409\u20131418. https:\/\/doi.org\/10.18653\/v1\/P19-1136. https:\/\/aclanthology.org\/P19-1136","DOI":"10.18653\/v1\/P19-1136"},{"key":"11000_CR25","unstructured":"Fu Z, Lam W, Yu Q, So A. M.-C, Hu S, Liu Z, Collier N (2023) Decoder-only or encoder-decoder? Interpreting language model as a regularized encoder-decoder. arXiv:2304.04052"},{"key":"11000_CR26","doi-asserted-by":"crossref","unstructured":"Gao T, Yen H, Yu J, Chen D (2023) Enabling large language models to generate text with citations. arXiv:2305.14627","DOI":"10.18653\/v1\/2023.emnlp-main.398"},{"key":"11000_CR27","first-page":"1841","volume":"33","author":"J Gu","year":"2020","unstructured":"Gu J, Kuen J, Joty S, Cai J, Morariu V, Zhao H, Sun T (2020) Self-supervised relationship probing. Adv Neural Inform Process Syst 33:1841\u20131853","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR28","unstructured":"Gu J, Nenkova AN, Barmpalios N, Morariu VI, Sun T, Jain RB, Kuen JWY, Zhao H (2023) Unified pretraining framework for document understanding, uS Patent App. 17\/528,061 (May\u00a018)"},{"issue":"1","key":"11000_CR29","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TPAMI.2022.3152247","volume":"45","author":"K Han","year":"2023","unstructured":"Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, Tang Y, Xiao A, Xu C, Xu Y, Yang Z, Zhang Y, Tao D (2023) A survey on vision transformer. IEEE Trans Pattern Anal Mach Intell 45(1):87\u2013110. https:\/\/doi.org\/10.1109\/TPAMI.2022.3152247","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"11000_CR30","unstructured":"Harley AW, Ufkes A, Derpanis KG Evaluation of deep convolutional nets for document image classification and retrieval. In: International conference on document analysis and recognition (ICDAR)"},{"key":"11000_CR31","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961\u20132969","DOI":"10.1109\/ICCV.2017.322"},{"key":"11000_CR32","doi-asserted-by":"publisher","unstructured":"Hewlett D, Lacoste A, Jones L, Polosukhin I, Fandrianto A, Han J, Kelcey M, Berthelot D (2016) Wikireading: a novel large-scale language understanding task over Wikipedia. In: Proceedings of the 54th annual meeting of the association for computational linguistics, ACL 2016, August 7\u201312, Berlin volume 1: long papers, the association for computer linguistics, 2016. https:\/\/doi.org\/10.18653\/v1\/p16-1145","DOI":"10.18653\/v1\/p16-1145"},{"key":"11000_CR33","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, June 27\u201330, 2016, IEEE Computer Society, pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"11000_CR34","doi-asserted-by":"publisher","unstructured":"Hong T, Kim D, Ji M, Hwang W, Nam D, Park S (2022) BROS: a pre-trained language model focusing on text and layout for better key information extraction from documents. In: Thirty-sixth AAAI conference on artificial intelligence, AAAI 2022, thirty-fourth conference on innovative applications of artificial intelligence, IAAI 2022, the twelveth symposium on educational advances in artificial intelligence, EAAI 2022 virtual event, February 22-March 1, 2022, AAAI Press, pp 10767\u201310775. https:\/\/doi.org\/10.1609\/aaai.v36i10.21322","DOI":"10.1609\/aaai.v36i10.21322"},{"key":"11000_CR35","unstructured":"Hsiao C-W, Sun C, Sun M, Chen H-T (2019) Flat2layout: flat representation for estimating layout of general room types. arXiv:1905.12571"},{"key":"11000_CR36","doi-asserted-by":"publisher","unstructured":"Huang Z, Chen K, He J, Bai X, Karatzas D, Lu S, Jawahar C. V (2019) Icdar2019 competition on scanned receipt ocr and information extraction, 2019 international conference on document analysis and recognition (ICDAR). https:\/\/doi.org\/10.1109\/ICDAR.2019.00244","DOI":"10.1109\/ICDAR.2019.00244"},{"key":"11000_CR37","doi-asserted-by":"crossref","unstructured":"Huang Y, Lv T, Cui L, Lu Y, Wei F (2022) Layoutlmv3: Pre-training for document ai with unified text and image masking. In: Proceedings of the 30th ACM international conference on multimedia, pp 4083\u20134091","DOI":"10.1145\/3503161.3548112"},{"issue":"4","key":"11000_CR38","first-page":"567","volume":"46","author":"AE Igorevna","year":"2022","unstructured":"Igorevna AE, Bulatovich BK, Petrovich ND, Olegovna PO, Igorevich SB, Anatolevich AnatolSO (2022) Document image analysis and recognition: a survey, computer optics. Comput Opt 46(4):567\u2013589","journal-title":"Comput Opt"},{"key":"11000_CR39","doi-asserted-by":"crossref","unstructured":"Irie K, Zeyer A, Schl\u00fcter R, Ney H (2019) Language modeling with deep transformers, arXiv preprint arXiv:1905.04226","DOI":"10.21437\/Interspeech.2019-2225"},{"key":"11000_CR40","doi-asserted-by":"publisher","unstructured":"Jaume G, Ekenel HK, Thiran J (2019) FUNSD: a dataset for form understanding in noisy scanned documents. In: 2nd international workshop on open services and tools for document analysis, OST@ICDAR 2019, Sydney, September 22\u201325, IEEE, 2019, pp. 1\u20136. https:\/\/doi.org\/10.1109\/ICDARW.2019.10029","DOI":"10.1109\/ICDARW.2019.10029"},{"key":"11000_CR41","doi-asserted-by":"crossref","unstructured":"Kim G, Hong T, Yim M, Nam J, Park J, Yim J, Hwang W, Yun S, Han D, Park S (2022) Ocr-free document understanding transformer. arXiv:2111.15664","DOI":"10.1007\/978-3-031-19815-1_29"},{"key":"11000_CR42","doi-asserted-by":"crossref","unstructured":"Lee C.-Y, Li C.-L, Dozat T, Perot V, Su G, Hua N, Ainslie J, Wang R, Fujii Y, Pfister T (2022) Formnet: structural encoding beyond sequential modeling in form document information extraction. arXiv:2203.08411","DOI":"10.18653\/v1\/2022.acl-long.260"},{"key":"11000_CR43","unstructured":"Lewis DD, Agam G, Argamon SE, Frieder O, Grossman DA, Heard J (2011) Cdip dataset, https:\/\/ir.nist.gov\/cdip\/"},{"key":"11000_CR44","unstructured":"Lewis DD, Agam G, Argamon SE, Frieder O, Grossman DA, Heard J (2021) The iit cdip test collection, https:\/\/ir.nist.gov\/cdip\/README.txt"},{"key":"11000_CR45","doi-asserted-by":"publisher","unstructured":"Lewis DD, Agam G, Argamon S, Frieder O, Grossman D. A, Heard J (2006) Building a test collection for complex document information processing. In: EN Efthimiadis, ST Dumais, D Hawking, K J\u00e4rvelin (eds.), SIGIR 2006: proceedings of the 29th annual international ACM Sigir conference on research and development in information retrieval, Seattle, August 6\u201311, 2006, ACM, pp 665\u2013666. https:\/\/doi.org\/10.1145\/1148170.1148307","DOI":"10.1145\/1148170.1148307"},{"key":"11000_CR46","first-page":"308","volume-title":"PRICAI 2021: trends in artificial intelligence","author":"S Li","year":"2021","unstructured":"Li S, Ma X, Pan S, Hu J, Shi L, Wang Q (2021) Vtlayout: fusion of visual and text features for document layout analysis. In: Pham DN, Theeramunkong T, Governatori G, Liu F (eds) PRICAI 2021: trends in artificial intelligence. Springer International Publishing, Cham, pp 308\u2013322"},{"key":"11000_CR47","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1007\/s10032-004-0138-z","volume":"7","author":"J Liang","year":"2005","unstructured":"Liang J, Doermann D, Li H (2005) Camera-based analysis of text and documents: a survey. Int J Doc Anal Recogn (IJDAR) 7:84\u2013104","journal-title":"Int J Doc Anal Recogn (IJDAR)"},{"key":"11000_CR48","doi-asserted-by":"publisher","unstructured":"Li C, Bi B, Yan M, Wang W, Huang S, Huang F, Si L (2021) Structural LM: structural pre-training for form understanding. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), association for computational linguistics, pp 6309\u20136318. https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.493. https:\/\/aclanthology.org\/2021.acl-long.493","DOI":"10.18653\/v1\/2021.acl-long.493"},{"key":"11000_CR49","doi-asserted-by":"crossref","unstructured":"Li P, Gu J, Kuen J, Morariu V. I, Zhao H, Jain R, Manjunatha V, Liu H (2021) Selfdoc: self-supervised document representation learning. arXiv:2106.03331","DOI":"10.1109\/CVPR46437.2021.00560"},{"key":"11000_CR50","unstructured":"Li Y, Liang F, Zhao L, Cui Y, Ouyang W, Shao J, Yu F, Yan J (2021) Supervision exists everywhere: a data efficient contrastive language-image pre-training paradigm, arXiv preprint arXiv:2110.05208"},{"key":"11000_CR51","doi-asserted-by":"crossref","unstructured":"Li Q, Li Z, Cai X, Du B, Zhao H (2023) Enhancing visually-rich document understanding via layout structure modeling. In: Proceedings of the 31st ACM international conference on multimedia, pp 4513\u20134523","DOI":"10.1145\/3581783.3612327"},{"key":"11000_CR52","doi-asserted-by":"publisher","unstructured":"Lin T, Doll\u00e1r P, Girshick RB, He K, Hariharan B, Belongie SJ (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, 21\u201326, 2017, IEEE Computer Society, pp 936\u2013944. https:\/\/doi.org\/10.1109\/CVPR.2017.106","DOI":"10.1109\/CVPR.2017.106"},{"key":"11000_CR53","doi-asserted-by":"crossref","unstructured":"Li Y, Qian Y, Yu Y, Qin X, Zhang C, Liu Y, Yao K, Han J, Liu J, Ding E (2021) Structext: structured text understanding with multi-modal transformers. arXiv:2108.02923","DOI":"10.1145\/3474085.3475345"},{"key":"11000_CR54","unstructured":"Liu Y, Yang B, Liu Q, Li Z, Ma Z, Zhang S, Bai X (2024) Textmonkey: an ocr-free large multimodal model for understanding document, arXiv preprint arXiv:2403.04473"},{"key":"11000_CR55","doi-asserted-by":"crossref","unstructured":"Li B, Wang J, Zhao M, Zhou S (2022) Two-stage multimodality fusion for high-performance text-based visual question answering. In: Proceedings of the Asian conference on computer vision, pp 4143\u20134159","DOI":"10.1007\/978-3-031-26316-3_39"},{"key":"11000_CR56","first-page":"89","volume":"32","author":"J Lu","year":"2019","unstructured":"Lu J, Batra D, Parikh D, Lee S (2019) Vilbert: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Adv Neural Inform Process Syst 32:89","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR57","doi-asserted-by":"crossref","unstructured":"Luo C, Shen Y, Zhu Z, Zheng Q, Yu Z, Yao C (2024) Layoutllm: layout instruction tuning with large language models for document understanding. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 15630\u201315640","DOI":"10.1109\/CVPR52733.2024.01480"},{"key":"11000_CR58","doi-asserted-by":"crossref","unstructured":"Mathew M, Karatzas D, Jawahar C (2021) Docvqa: a dataset for vqa on document images, https:\/\/www.docvqa.org\/datasets","DOI":"10.1109\/WACV48630.2021.00225"},{"key":"11000_CR59","doi-asserted-by":"crossref","unstructured":"Mathew M, Karatzas D, Jawahar C (2021) Docvqa: a dataset for vqa on document images. In: Proceedings of the IEEE\/CVF winter conference on applications of computer vision, pp 2200\u20132209","DOI":"10.1109\/WACV48630.2021.00225"},{"key":"11000_CR60","doi-asserted-by":"crossref","unstructured":"Michael J, Labahn R, Gr\u00fcning T, Z\u00f6llner J (2019) Evaluating sequence-to-sequence models for handwritten text recognition. arXiv:1903.07377","DOI":"10.1109\/ICDAR.2019.00208"},{"key":"11000_CR61","unstructured":"Microsoft, What is optical character recognition?, https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/computer-vision\/overview-ocr#read-api (2022)"},{"key":"11000_CR62","doi-asserted-by":"publisher","first-page":"3001","DOI":"10.1007\/s11192-020-03718-9","volume":"125","author":"R Nogueira","year":"2020","unstructured":"Nogueira R, Jiang Z, Cho K, Lin J (2020) Navigation-based candidate expansion and pretrained language models for citation recommendation. Scientometrics 125:3001\u20133016","journal-title":"Scientometrics"},{"issue":"3","key":"11000_CR63","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1007\/s10032-022-00399-3","volume":"25","author":"B Oral","year":"2022","unstructured":"Oral B, Eryi\u011fit G (2022) Fusion of visual representations for multimodal information extraction from unstructured transactional documents. Int J Doc Anal Recognit (IJDAR) 25(3):187\u2013205","journal-title":"Int J Doc Anal Recognit (IJDAR)"},{"key":"11000_CR64","unstructured":"Park S, Shin S, Lee B, Lee J, Surh J, Seo M, Lee H (2019) Cord: A consolidated receipt dataset for post-ocr parsing. In: Workshop on document intelligence at NeurIPS"},{"key":"11000_CR65","unstructured":"Park S, Shin S, Lee B, Lee J, Surh J, Seo M, Lee H (2019) Cord: a consolidated receipt dataset for post-ocr parsing. In: Workshop on document intelligence at NeurIPS"},{"key":"11000_CR66","doi-asserted-by":"crossref","unstructured":"Peng Q, Pan Y, Wang W, Luo B, Zhang Z, Huang Z, Hu T, Yin W, Chen Y, Zhang Y, et al (2022) Ernie-layout: Layout knowledge enhanced pre-training for visually-rich document understanding, arXiv preprint arXiv:2210.06155","DOI":"10.18653\/v1\/2022.findings-emnlp.274"},{"issue":"6","key":"11000_CR67","doi-asserted-by":"publisher","first-page":"4893","DOI":"10.1007\/s10462-022-10265-7","volume":"56","author":"P Pham","year":"2023","unstructured":"Pham P, Nguyen LT, Pedrycz W, Vo B (2023) Deep learning, graph-based text representation and classification: a survey, perspectives and challenges. Artif Intell Rev 56(6):4893\u20134927","journal-title":"Artif Intell Rev"},{"key":"11000_CR68","doi-asserted-by":"crossref","unstructured":"Powalski R, Borchmann \u0141, Jurkiewicz D, Dwojak T, Pietruszka M, Pa\u0142ka G (2021) Going full-tilt boogie on document understanding with text-image-layout transformer. In: Document analysis and recognition\u2013ICDAR 2021: 16th international conference, Lausanne, September 5\u201310, 2021, Proceedings, Part II 16, Springer, pp 732\u2013747","DOI":"10.1007\/978-3-030-86331-9_47"},{"issue":"1","key":"11000_CR69","first-page":"5485","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485\u20135551","journal-title":"J Mach Learn Res"},{"key":"11000_CR70","first-page":"9","volume":"28","author":"S Ren","year":"2015","unstructured":"Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inform Process Syst 28:9","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR71","doi-asserted-by":"publisher","unstructured":"Schmidt H, Butter K, Rider C (2002) Building digital tobacco industry document libraries at the university of California, San Francisco library\/center for knowledge management, D Lib Mag. 8\u00a0(9) . https:\/\/doi.org\/10.1045\/september2002-schmidt","DOI":"10.1045\/september2002-schmidt"},{"issue":"1","key":"11000_CR72","volume":"13","author":"A Shigarov","year":"2023","unstructured":"Shigarov A (2023) Table understanding: problem overview. Wiley Interdiscip Rev 13(1):e1482","journal-title":"Wiley Interdiscip Rev"},{"key":"11000_CR73","doi-asserted-by":"crossref","unstructured":"Shi Y, Kim M, Chae Y (2023) Multi-scale cell-based layout representation for document understanding. In: Proceedings of the IEEE\/CVF winter conference on applications of computer vision (WACV), pp 3670\u20133679","DOI":"10.1109\/WACV56688.2023.00366"},{"key":"11000_CR74","doi-asserted-by":"crossref","unstructured":"Shi D, Liu S, Du J, Zhu H (2023) Layoutgcn: a lightweight architecture for visually rich document understanding. In: International conference on document analysis and recognition, Springer, pp 149\u2013165","DOI":"10.1007\/978-3-031-41682-8_10"},{"key":"11000_CR75","unstructured":"Shinyama Y, et al (2019) Pdfminer, https:\/\/github.com\/euske\/pdfminer"},{"key":"11000_CR76","unstructured":"Subramani N, Matton A, Greaves M, Lam A (2020) A survey of deep learning approaches for ocr and document understanding, arXiv preprint arXiv:2011.13534"},{"key":"11000_CR77","doi-asserted-by":"crossref","unstructured":"Tanaka R, Iki T, Nishida K, Saito K, Suzuki J (2024) Instructor: a dataset for zero-shot generalization of visual document understanding with instructions. In: Proceedings of the AAAI conference on artificial intelligence, vol 38, pp 19071\u201319079","DOI":"10.1609\/aaai.v38i17.29874"},{"key":"11000_CR78","doi-asserted-by":"crossref","unstructured":"Tang Z, Yang Z, Wang G, Fang Y, Liu Y, Zhu C, Zeng M, Zhang C, Bansal M (2023) Unifying vision, text, and layout for universal document processing. arXiv:2212.02623","DOI":"10.1109\/CVPR52729.2023.01845"},{"key":"11000_CR79","unstructured":"UCSF library, industry documents library, https:\/\/www.industrydocuments.ucsf.edu\/"},{"key":"11000_CR80","first-page":"5998","volume":"30","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141 (2017) Attention is all you need. Adv Neural Inform Process Syst 30:5998\u20136008","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR81","doi-asserted-by":"crossref","unstructured":"Voutharoja BP, Qu L, Shiri F (2023) Language independent neuro-symbolic semantic parsing for form understanding. arXiv:2305.04460","DOI":"10.1007\/978-3-031-41679-8_8"},{"key":"11000_CR82","unstructured":"Vu HM, Nguyen DT (2020) Revising FUNSD dataset for key-value detection in document images, CoRR abs\/2010.05322. arXiv:2010.05322"},{"key":"11000_CR83","first-page":"7423","volume":"35","author":"J Wang","year":"2022","unstructured":"Wang J, Gou C, Wu Q, Feng H, Han J, Ding E, Wang J (2022) Rtformer: efficient design for real-time semantic segmentation with transformer. Adv Neural Inform Process Syst 35:7423\u20137436","journal-title":"Adv Neural Inform Process Syst"},{"key":"11000_CR84","doi-asserted-by":"crossref","unstructured":"Wang J, Jin L, Ding K (2022) Lilt: a simple yet effective language-independent layout transformer for structured document understanding. arXiv:2202.13669","DOI":"10.18653\/v1\/2022.acl-long.534"},{"key":"11000_CR85","doi-asserted-by":"crossref","unstructured":"Wang Y, Lipka N, Rossi RA, Siu A, Zhang R, Derr T (2024) Knowledge graph prompting for multi-document question answering. In: Proceedings of the AAAI conference on artificial intelligence, vol 38, pp 19206\u201319214","DOI":"10.1609\/aaai.v38i17.29889"},{"key":"11000_CR86","unstructured":"Wang C, Li M, Smola A. J (2019) Language models with transformers. arXiv:1904.09408"},{"key":"11000_CR87","doi-asserted-by":"crossref","unstructured":"Wang X, Wang X, Jiang B, Tang J, Luo B (2023) Mutualformer: multi-modality representation learning via cross-diffusion attention arXiv:2112.01177","DOI":"10.1007\/s11263-024-02067-x"},{"key":"11000_CR88","unstructured":"Wu Y, Schuster M, Chen Z, Le Q. V, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, Klingner J, Shah A, Johnson M, Liu X, Kaiser \u0141ukasz, Gouws S, Kato Y, Kudo T, Kazawa H, Stevens K, Kurian G, Patil N, Wang W, Young C, Smith J, Riesa J, Rudnick A, Vinyals O, Corrado G, Hughes M, Dean J (2016) Google\u2019s neural machine translation system: bridging the gap between human and machine translation. arXiv:1609.08144"},{"key":"11000_CR89","doi-asserted-by":"crossref","unstructured":"Wu H, Xiao B, Codella N, Liu M, Dai X, Yuan L, Zhang L (2021) Cvt: introducing convolutions to vision transformers. arXiv:2103.15808","DOI":"10.1109\/ICCV48922.2021.00009"},{"key":"11000_CR90","doi-asserted-by":"publisher","unstructured":"Xie S, Girshick R. B, Doll\u00e1r P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, July 21\u201326, 2017, IEEE Computer Society, pp 5987\u20135995. https:\/\/doi.org\/10.1109\/CVPR.2017.634","DOI":"10.1109\/CVPR.2017.634"},{"key":"11000_CR91","doi-asserted-by":"publisher","unstructured":"Xu Y, Li M, Cui L, Huang S, Wei F, Zhou M (2020) Layoutlm: pre-training of text and layout for document image understanding. In: R Gupta, Y Liu, J Tang, BA Prakash (eds.), KDD \u201920: the 26th ACM SIGKDD conference on knowledge discovery and data mining, virtual event, August 23\u201327, ACM, 2020, pp 1192\u20131200. https:\/\/doi.org\/10.1145\/3394486.3403172","DOI":"10.1145\/3394486.3403172"},{"key":"11000_CR92","doi-asserted-by":"crossref","unstructured":"Xu Y, Lv T, Cui L, Wang G, Lu Y, Florencio D, Zhang C, Wei F (2021) Layoutxlm: Multimodal pre-training for multilingual visually-rich document understanding. arXiv:2104.08836","DOI":"10.18653\/v1\/2021.acl-long.201"},{"key":"11000_CR93","doi-asserted-by":"crossref","unstructured":"Xu Y, Xu Y, Lv T, Cui L, Wei F, Wang G, Lu Y, Flor\u00eancio DAF, Zhang C, Che W, Zhang M, Zhou L (2021) Layoutlmv2: multi-modal pre-training for visually-rich document understanding. In: ACL\/IJCNLP","DOI":"10.18653\/v1\/2021.acl-long.201"},{"key":"11000_CR94","doi-asserted-by":"crossref","unstructured":"Xu P, Zhu X, Clifton DA (2023) Multimodal learning with transformers: a survey. arXiv:2206.06488","DOI":"10.1109\/TPAMI.2023.3275156"},{"key":"11000_CR95","unstructured":"Yu Y, Li Y, Zhang C, Zhang X, Guo Z, Qin X, Yao K, Han J, Ding E, Wang J (2023) Structextv2: masked visual-textual prediction for document image pre-training, arXiv preprint arXiv:2303.00289"},{"key":"11000_CR96","doi-asserted-by":"publisher","unstructured":"Yu W, Lu N, Qi X, Gong P, Xiao R (2021) PICK: processing key information extraction from documents using improved graph learning-convolutional networks. In: 25th international conference on pattern recognition, ICPR 2020, Virtual Event\/Milan, January 10\u201315, IEEE, 2020, pp 4363\u20134370. https:\/\/doi.org\/10.1109\/ICPR48806.2021.9412927","DOI":"10.1109\/ICPR48806.2021.9412927"},{"key":"11000_CR97","doi-asserted-by":"crossref","unstructured":"Zhai M, Li Y, Qin X, Yi C, Xie Q, Zhang C, Yao K, Wu Y, Jia Y (2023) Fast-structext: An efficient hourglass transformer with modality-guided dynamic token merge for document understanding, arXiv preprint arXiv:2305.11392","DOI":"10.24963\/ijcai.2023\/585"},{"key":"11000_CR98","doi-asserted-by":"publisher","unstructured":"Zhang P, Xu Y, Cheng Z, Pu S, Lu J, Qiao L, Niu Y, Wu F (2020) TRIE: end-to-end text reading and information extraction for document understanding. In: CW Chen, R Cucchiara, X Hua, G Qi, E Ricci, Z Zhang, R Zimmermann (eds.), MM \u201920: The 28th ACM international conference on multimedia, virtual event\/Seattle, October 12\u201316, ACM, 2020, pp 1413\u20131422. https:\/\/doi.org\/10.1145\/3394171.3413900","DOI":"10.1145\/3394171.3413900"},{"key":"11000_CR99","doi-asserted-by":"crossref","unstructured":"Zheng G, Zhou X, Li X, Qi Z, Shan Y, Li X (2023) Layoutdiffusion: controllable diffusion model for layout-to-image generation. arXiv:2303.17189","DOI":"10.1109\/CVPR52729.2023.02154"},{"key":"11000_CR100","doi-asserted-by":"publisher","unstructured":"Zhong X, Tang J, Jimeno-Yepes A (2019) Publaynet: largest dataset ever for document layout analysis. In: 2019 international conference on document analysis and recognition, ICDAR 2019, Sydney, September 20\u201325, 2019, IEEE, pp. 1015\u20131022. https:\/\/doi.org\/10.1109\/ICDAR.2019.00166","DOI":"10.1109\/ICDAR.2019.00166"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-11000-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-024-11000-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-11000-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T10:21:40Z","timestamp":1731493300000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-024-11000-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,21]]},"references-count":100,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["11000"],"URL":"https:\/\/doi.org\/10.1007\/s10462-024-11000-0","relation":{},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,21]]},"assertion":[{"value":"4 October 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 October 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"342"}}