{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T16:07:35Z","timestamp":1775837255744,"version":"3.50.1"},"reference-count":64,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T00:00:00Z","timestamp":1718668800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This study presents an integrated approach for automatically extracting and structuring information from medical reports, captured as scanned documents or photographs, through a combination of image recognition and natural language processing (NLP) techniques like named entity recognition (NER). The primary aim was to develop an adaptive model for efficient text extraction from medical report images. This involved utilizing a genetic algorithm (GA) to fine-tune optical character recognition (OCR) hyperparameters, ensuring maximal text extraction length, followed by NER processing to categorize the extracted information into required entities, adjusting parameters if entities were not correctly extracted based on manual annotations. Despite the diverse formats of medical report images in the dataset, all in Russian, this serves as a conceptual example of information extraction (IE) that can be easily extended to other languages.<\/jats:p>","DOI":"10.3390\/make6020064","type":"journal-article","created":{"date-parts":[[2024,6,21]],"date-time":"2024-06-21T12:27:33Z","timestamp":1718972853000},"page":"1361-1377","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Image Text Extraction and Natural Language Processing of Unstructured Data from Medical Reports"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-8986-402X","authenticated-orcid":false,"given":"Ivan","family":"Malashin","sequence":"first","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3997-342X","authenticated-orcid":false,"given":"Igor","family":"Masich","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3959-2969","authenticated-orcid":false,"given":"Vadim","family":"Tynchenko","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"}]},{"given":"Andrei","family":"Gantimurov","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4263-2367","authenticated-orcid":false,"given":"Vladimir","family":"Nelyub","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"},{"name":"Scientific Department, Far Eastern Federal University, 690922 Vladivostok, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9648-2395","authenticated-orcid":false,"given":"Aleksei","family":"Borodulin","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Technology Scientific and Education Center, Bauman Moscow State Technical University, 105005 Moscow, Russia"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"520","DOI":"10.3390\/forecast3030033","article-title":"Attention-based CNN-RNN Arabic text recognition from natural scene images","volume":"3","author":"Butt","year":"2021","journal-title":"Forecasting"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bose, P., Srinivasan, S., Sleeman IV, W.C., Palta, J., Kapoor, R., and Ghosh, P. (2021). A survey on recent named entity recognition and relationship extraction techniques on clinical texts. Appl. Sci., 11.","DOI":"10.3390\/app11188319"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chu, Q., Chen, C.P., Hu, H., Wu, X., and Han, B. (2024). iHand: Hand Recognition-Based Text Input Method for Wearable Devices. Computers, 13.","DOI":"10.3390\/computers13030080"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1016\/j.patcog.2003.10.012","article-title":"Text information extraction in images and video: A survey","volume":"37","author":"Jung","year":"2004","journal-title":"Pattern Recognit."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.neucom.2013.05.037","article-title":"Text extraction from natural scene image: A survey","volume":"122","author":"Zhang","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1480","DOI":"10.1109\/TPAMI.2014.2366765","article-title":"Text detection and recognition in imagery: A survey","volume":"37","author":"Ye","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2752","DOI":"10.1109\/TIP.2016.2554321","article-title":"Text detection, tracking and recognition in video: A comprehensive survey","volume":"25","author":"Yin","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1590","DOI":"10.1016\/j.imavis.2010.04.002","article-title":"Text detection in images using sparse representation with discriminative dictionaries","volume":"28","author":"Zhao","year":"2010","journal-title":"Image Vis. Comput."},{"key":"ref_9","unstructured":"Neumann, L., and Matas, J. (2010, January 8\u201312). A method for text localization and recognition in real-world images. Proceedings of the Computer Vision\u2013ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand. Revised Selected Papers, Part III 10."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.cviu.2012.11.002","article-title":"Text extraction from scene images by character appearance and structure modeling","volume":"117","author":"Yi","year":"2013","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_11","unstructured":"Gupta, N., and Banga, V. (2012, January 28\u201329). Image Segmentation for Text Extraction. Proceedings of the 2nd International Conference on Electrical, Electronics and Civil Engineering (ICEECE\u20192012), Singapore."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dang, N.C., Moreno-Garc\u00eda, M.N., and De la Prieta, F. (2020). Sentiment analysis based on deep learning: A comparative study. Electronics, 9.","DOI":"10.3390\/electronics9030483"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"100379","DOI":"10.1016\/j.cosrev.2021.100379","article-title":"A survey on deep learning and its applications","volume":"40","author":"Dong","year":"2021","journal-title":"Comput. Sci. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lu, H., Ma, L., Fu, X., Liu, C., Wang, Z., Tang, M., and Li, N. (2020). Landslides information extraction using object-oriented image analysis paradigm based on deep learning and transfer learning. Remote Sens., 12.","DOI":"10.3390\/rs12050752"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, Y., Wu, Z., Yang, Y., Lian, S., Guo, F., and Wang, Z. (2022). A survey of information extraction based on deep learning. Appl. Sci., 12.","DOI":"10.3390\/app12199691"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yu, W., Lu, N., Qi, X., Gong, P., and Xiao, R. (2021, January 10\u201315). PICK: Processing key information extraction from documents using improved graph learning-convolutional networks. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9412927"},{"key":"ref_17","unstructured":"Revathi, A., and Modi, N.A. (2021, January 17\u201319). Comparative analysis of text extraction from color images using tesseract and opencv. Proceedings of the 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Montejo-R\u00e1ez, A., and Jim\u00e9nez-Zafra, S.M. (2022). Current approaches and applications in natural language processing. Appl. Sci., 12.","DOI":"10.3390\/books978-3-0365-4440-3"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"46335","DOI":"10.1109\/ACCESS.2020.2974101","article-title":"Feature extraction and analysis of natural language processing for deep learning English language","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"041317","DOI":"10.1063\/5.0021106","article-title":"Data-driven materials research enabled by natural language processing and information extraction","volume":"7","author":"Olivetti","year":"2020","journal-title":"Appl. Phys. Rev."},{"key":"ref_21","first-page":"765","article-title":"Integration of computer vision and natural language processing in multimedia robotics application","volume":"7","author":"Shahin","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1016\/j.gie.2020.08.038","article-title":"Application of optical character recognition with natural language processing for large-scale quality metric data extraction in colonoscopy reports","volume":"93","author":"Laique","year":"2021","journal-title":"Gastrointest. Endosc."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.jbi.2017.11.011","article-title":"Clinical information extraction applications: A literature review","volume":"77","author":"Wang","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.ijmedinf.2017.12.013","article-title":"Information extraction from Italian medical reports: An ontology-driven approach","volume":"111","author":"Viani","year":"2018","journal-title":"Int. J. Med. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1055\/s-0040-1702001","article-title":"Medical information extraction in the age of deep learning","volume":"29","author":"Hahn","year":"2020","journal-title":"Yearb. Med. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Dash, G., Sharma, C., and Sharma, S. (2023). Sustainable marketing and the role of social media: An experimental study using natural language processing (NLP). Sustainability, 15.","DOI":"10.3390\/su15065443"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.eng.2019.12.014","article-title":"Progress in neural NLP: Modeling, learning, and reasoning","volume":"6","author":"Zhou","year":"2020","journal-title":"Engineering"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"McMillan-Major, A., Osei, S., Rodriguez, J.D., Ammanamanchi, P.S., Gehrmann, S., and Jernite, Y. (2021). Reusable templates and guides for documenting datasets and models for natural language processing and generation: A case study of the HuggingFace and GEM data and model cards. arXiv.","DOI":"10.18653\/v1\/2021.gem-1.11"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zolotarev, O., Solomentsev, Y., Khakimova, A., and Charnine, M. (2019, January 23\u201326). Identification of semantic patterns in full-text documents using neural network methods. Proceedings of the 29th International Conference on Computer Graphics and Vision, Graphicon-2019, Bryansk, Russia.","DOI":"10.30987\/graphicon-2019-2-276-279"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zolotarev, O. (2020, January 9\u201313). Research and development of linguo-statistical methods for forming a portrait of a subject area. Proceedings of the International Conference \u201cComputing for Physics and Technology\u2014CPT2020\u201d, Moscow, Russia.","DOI":"10.30987\/conferencearticle_5fce2773c5a7b3.23896517"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kanev, A.I., Savchenko, G.A., Grishin, I.A., Vasiliev, D.A., and Duma, E.M. (2022, January 25\u201328). Sentiment analysis of multilingual texts using machine learning methods. Proceedings of the 2022 Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), St. Petersburg, Russia.","DOI":"10.1109\/ElConRus54750.2022.9755568"},{"key":"ref_32","unstructured":"Atnashev, T., Ganeeva, V., Kazakov, R., Matyash, D., Sonkin, M., Voloshina, E., Serikov, O., and Artemova, E. (2022). Razmecheno: Named Entity Recognition from Digital Archive of Diaries Prozhito. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"da Silva, A.R., and Savi\u0107, D. (2021). Linguistic patterns and linguistic styles for requirements specification: Focus on data entities. Appl. Sci., 11.","DOI":"10.3390\/app11094119"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"113305","DOI":"10.1016\/j.eswa.2020.113305","article-title":"Enhanced deep learning algorithm development to detect pain intensity from facial expression images","volume":"149","author":"Bargshady","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1146\/annurev-biodatasci-030421-030931","article-title":"Modern clinical text mining: A guide and review","volume":"4","author":"Percha","year":"2021","journal-title":"Annu. Rev. Biomed. Data Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1080\/00031305.2023.2186952","article-title":"Improved approximation and visualization of the correlation matrix","volume":"77","author":"Graffelman","year":"2023","journal-title":"Am. Stat."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.visres.2020.04.012","article-title":"Mechanisms underlying simultaneous brightness contrast: Early and innate","volume":"173","author":"Sinha","year":"2020","journal-title":"Vis. Res."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Qiao, L., Li, Z., Cheng, Z., Zhang, P., Pu, S., Niu, Y., Ren, W., Tan, W., and Wu, F. (2021, January 5\u201310). LGPMA: Complicated table structure recognition with local and global pyramid mask alignment. Proceedings of the International Conference on Document Analysis and Recognition, Lausanne, Switzerland.","DOI":"10.1007\/978-3-030-86549-8_7"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Singla, S. (2020). AI and IoT in healthcare. Internet of Things Use Cases for the Healthcare Industry, Springer.","DOI":"10.1007\/978-3-030-37526-3_1"},{"key":"ref_40","first-page":"64","article-title":"Ai in finance: Challenges, techniques, and opportunities","volume":"55","author":"Cao","year":"2022","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8812542","DOI":"10.1155\/2021\/8812542","article-title":"A Review of Artificial Intelligence (AI) in Education from 2010 to 2020","volume":"2021","author":"Zhai","year":"2021","journal-title":"Complexity"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ranaldi, L., and Pucci, G. (2023). Knowing knowledge: Epistemological study of knowledge in transformers. Appl. Sci., 13.","DOI":"10.3390\/app13020677"},{"key":"ref_43","unstructured":"Ber\u010di\u0107, B., Golubovi\u0107, A., and Trobok, M. (2022). A Mid-blue Logic. HUMAN RATIONALITY Festschrift for Nenad Smokrovi\u0107, University of Rijeka, Faculty of Humanities and Social Sciences."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Paa\u00df, G., and Konya, I. (2012). Machine learning for document structure recognition. Modeling, Learning, and Processing of Text Technological Data Structures, Springer.","DOI":"10.1007\/978-3-642-22613-7_12"},{"key":"ref_45","first-page":"4563","article-title":"Convolutional neural network based intelligent handwritten document recognition","volume":"70","author":"Ghazal","year":"2022","journal-title":"Comput. Mater. Contin."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"42689","DOI":"10.1109\/ACCESS.2020.2976744","article-title":"Document-level text classification using single-layer multisize filters convolutional neural network","volume":"8","author":"Akhter","year":"2020","journal-title":"IEEE Access"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lozi\u0107, E., and \u0160tular, B. (2021). Documentation of archaeology-specific workflow for airborne LiDAR data processing. Geosciences, 11.","DOI":"10.3390\/geosciences11010026"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"114647","DOI":"10.1016\/j.eswa.2021.114647","article-title":"GA-based approach to optimize an equivalent electric circuit model of a Li-ion battery-pack","volume":"172","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1877","DOI":"10.3390\/make5040090","article-title":"Bayesian Network Structural Learning Using Adaptive Genetic Algorithm with Varying Population Size","volume":"5","year":"2023","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rezaeian, N., Gurina, R., Saltykova, O.A., Hezla, L., Nohurov, M., and Reza Kashyzadeh, K. (2024). Novel GA-Based DNN Architecture for Identifying the Failure Mode with High Accuracy and Analyzing Its Effects on the System. Appl. Sci., 14.","DOI":"10.3390\/app14083354"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Alblooshi, E., Alblooshi, A., Poon, K., and Ouali, A. (2022, January 9\u201313). A GA based approach for solving ring design telecommunication network. Proceedings of the Genetic and Evolutionary Computation Conference Companion, Boston, MA, USA.","DOI":"10.1145\/3520304.3528985"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"109998","DOI":"10.1016\/j.asoc.2023.109998","article-title":"Genetic algorithm based probabilistic model for agile project success in global software development","volume":"135","author":"Shameem","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"157","DOI":"10.3390\/make1010010","article-title":"Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives","volume":"1","author":"Sengupta","year":"2018","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"107404","DOI":"10.1016\/j.asoc.2021.107404","article-title":"A genetic simulated annealing algorithm for parallel partial disassembly line balancing problem","volume":"107","author":"Wang","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_55","first-page":"464","article-title":"A Comparative Study of Simulated Annealing and Ant Colony Optimization for Optimizing MRI-Based Alzheimer\u2019s Disease Classification","volume":"12","author":"Cherian","year":"2024","journal-title":"Int. J. Intell. Syst. Appl. Eng."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Sangaiah, A.K., Hosseinabadi, A.A.R., Shareh, M.B., Bozorgi Rad, S.Y., Zolfagharian, A., and Chilamkurti, N. (2020). IoT resource allocation and optimization based on heuristic algorithm. Sensors, 20.","DOI":"10.3390\/s20020539"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"e4612","DOI":"10.2196\/jmir.4612","article-title":"Automatically detecting failures in natural language processing tools for online community text","volume":"17","author":"Park","year":"2015","journal-title":"J. Med. Internet Res."},{"key":"ref_58","unstructured":"Kim, S.H. (2024). Comparisons in End-to-End Pipeline Designs for Customized Document Information Extraction. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"753","DOI":"10.3390\/make4030036","article-title":"Live Fish Species Classification in Underwater Images by Using Convolutional Neural Networks Based on Incremental Learning with Knowledge Distillation Loss","volume":"4","author":"Benzinou","year":"2022","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1132","DOI":"10.3390\/make5030058","article-title":"Cyberattack Detection in Social Network Messages Based on Convolutional Neural Networks and NLP Techniques","volume":"5","author":"Sidorov","year":"2023","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"699","DOI":"10.3390\/make6010033","article-title":"Medical Image Classifications Using Convolutional Neural Networks: A Survey of Current Methods and Statistical Modeling of the Literature","volume":"6","author":"Mohammed","year":"2024","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"205","DOI":"10.3390\/make3010010","article-title":"Property Checking with Interpretable Error Characterization for Recurrent Neural Networks","volume":"3","author":"Mayr","year":"2021","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"107482","DOI":"10.1016\/j.patcog.2020.107482","article-title":"Accurate, data-efficient, unconstrained text recognition with convolutional neural networks","volume":"108","author":"Yousef","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"106548","DOI":"10.1016\/j.knosys.2020.106548","article-title":"ASRNN: A recurrent neural network with an attention model for sequence labeling","volume":"212","author":"Lin","year":"2021","journal-title":"Knowl.-Based Syst."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/64\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:00:47Z","timestamp":1760108447000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/2\/64"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,18]]},"references-count":64,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["make6020064"],"URL":"https:\/\/doi.org\/10.3390\/make6020064","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,18]]}}}