{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T15:07:54Z","timestamp":1762182474637,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T00:00:00Z","timestamp":1761782400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Stomach ulcers, a common type of gastrointestinal (GI) disease, pose serious health risks if not diagnosed and treated at an early stage. Therefore, in this research, a method is proposed based on two deep learning models for classification and segmentation. The classification model is based on Convolutional Neural Networks (CNN) and incorporates Q-learning to achieve learning stability and decision accuracy through reinforcement-based feedback. In this model, input images are passed through a custom CNN model comprising seven layers, including convolutional, ReLU, max pooling, flattening, and fully connected layers, for feature extraction. Furthermore, the agent selects an action (class) for each input and receives a +1 reward for a correct prediction and \u22121 for an incorrect one. The Q-table stores a mapping between image features (states) and class predictions (actions), and is updated at each step based on the reward using the Q-learning update rule. This process runs over 1000 episodes and utilizes Q-learning parameters (\u03b1 = 0.1, \u03b3 = 0.6, \u03f5 = 0.1) to help the agent learn an optimal classification strategy. After training, the agent is evaluated on the test data using only its learned policy. The classified ulcer images are passed to the proposed attention-based U-Net model to segment the lesion regions. The model contains an encoder, a decoder, and attention layers. The encoder block extracts features through pooling and convolution layers, while the decoder block up-samples the features and reconstructs the segmentation map. Similarly, the attention block is used to highlight the important features obtained from the encoder block before passing them to the decoder block, helping the model focus on relevant spatial information. The model is trained using the selected hyperparameters, including an 8-batch size, the Adam optimizer, and 50 epochs. The performance of the models is evaluated on Kvasir, Nerthus, CVC-ClinicDB, and a private POF dataset. The classification framework provides 99.08% accuracy on Kvasir and 100% accuracy on Nerthus. In contrast, the segmentation framework yields 98.09% accuracy on Kvasir, 99.77% accuracy on Nerthus, 98.49% accuracy on CVC-ClinicDB, and 99.13% accuracy on the private dataset. The achieved results are superior to those of previous methods published in this domain.<\/jats:p>","DOI":"10.3390\/bioengineering12111184","type":"journal-article","created":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T01:23:59Z","timestamp":1761873839000},"page":"1184","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Deep Q-Learning for Gastrointestinal Disease Detection and Classification"],"prefix":"10.3390","volume":"12","author":[{"given":"Aini","family":"Saba","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Wah, Wah Cantt 47040, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4418-1451","authenticated-orcid":false,"given":"Javaria","family":"Amin","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Wah, Wah Cantt 47040, Pakistan"},{"name":"Department of Computer Science, Rawalpindi Women University, Rawalpindi 43600, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7326-1813","authenticated-orcid":false,"given":"Muhammad Umair","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence and Robotics, Sejong University, Seoul 05006, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1053\/j.gastro.2023.05.050","article-title":"Global burden of digestive diseases: A systematic analysis of the global burden of diseases study, 1990 to 2019","volume":"165","author":"Wang","year":"2023","journal-title":"Gastroenterology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1136\/gutjnl-2024-333227","article-title":"Epidemiology of gastrointestinal cancers: A systematic analysis from the global burden of disease study 2021","volume":"74","author":"Danpanichkul","year":"2025","journal-title":"Gut"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wang, R., Li, Z., Liu, S., and Zhang, D. (2023). Global, regional, and national burden of 10 digestive diseases in 204 countries and territories from 1990 to 2019. Front. Public Health, 11.","DOI":"10.3389\/fpubh.2023.1061453"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1007\/s10620-021-07086-z","article-title":"Artificial intelligence in endoscopy","volume":"67","author":"Okagawa","year":"2022","journal-title":"Dig. Dis. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1016\/j.gtc.2024.08.005","article-title":"Artificial intelligence in gastrointestinal endoscopy","volume":"53","author":"Labaki","year":"2024","journal-title":"Gastroenterol. Clin."},{"key":"ref_6","first-page":"104","article-title":"Colorectal cancer statistics, 2014","volume":"64","author":"Siegel","year":"2014","journal-title":"CA A Cancer J. Clin."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2152","DOI":"10.1053\/j.gastro.2018.02.021","article-title":"Determining risk of colorectal cancer and starting age of screening based on lifestyle, environmental, and genetic factors","volume":"154","author":"Jeon","year":"2018","journal-title":"Gastroenterology"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1007\/s10555-023-10158-3","article-title":"Colorectal cancer: A comprehensive review of carcinogenesis, diagnosis, and novel strategies for classified treatments","volume":"43","author":"Abedizadeh","year":"2024","journal-title":"Cancer Metastasis Rev."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1053\/j.gastro.2023.05.045","article-title":"The benefit of artificial intelligence-based diagnosis in gastroenterology and hepatology is highly variable: A diagnostic need and burden analysis","volume":"165","author":"Chinnasamy","year":"2023","journal-title":"Gastroenterology"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Parkash, O., Siddiqui, A.T.S., Jiwani, U., Rind, F., Padhani, Z.A., Rizvi, A., Hoodbhoy, Z., and Das, J.K. (2022). Diagnostic accuracy of artificial intelligence for detecting gastrointestinal luminal pathologies: A systematic review and meta-analysis. Front. Med., 9.","DOI":"10.3389\/fmed.2022.1018937"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hosain, A.S., Islam, M., Mehedi, H.K., Kabir, I.E., and Khan, Z.T. (2022, January 12\u201315). Gastrointestinal disorder detection with a transformer based approach. Proceedings of the 2022 IEEE 13th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada.","DOI":"10.1109\/IEMCON56893.2022.9946531"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lonseko, Z.M., Adjei, P.E., Du, W., Luo, C., Hu, D., Zhu, L., Gan, T., and Rao, N. (2021). Gastrointestinal disease classification in endoscopic images using attention-guided convolutional neural networks. Appl. Sci., 11.","DOI":"10.3390\/app112311136"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Escobar, J., Sanchez, K., Hinojosa, C., Arguello, H., and Castillo, S. (2021, January 15\u201317). Accurate deep learning-based gastrointestinal disease classification via transfer learning strategy. Proceedings of the 2021 XXIII Symposium on Image, Signal Processing and Artificial Vision (STSIVA), Popayan, Colombia.","DOI":"10.1109\/STSIVA53688.2021.9591995"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1007\/s11517-024-03203-y","article-title":"A review of deep learning methods for gastrointestinal diseases classification applied in computer-aided diagnosis system","volume":"63","author":"Jiang","year":"2025","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_15","first-page":"1207","article-title":"Deep convolutional neural networks for accurate classification of gastrointestinal tract syndromes","volume":"78","author":"Khan","year":"2024","journal-title":"CMC-Comput. Mater. Contin."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cambay, V.Y., Barua, P.D., Baig, A.H., Dogan, S., Baygin, M., Tuncer, T., and Acharya, U.R. (2024). Automated detection of gastrointestinal diseases using resnet50*-based explainable deep feature engineering model with endoscopy images. Sensors, 24.","DOI":"10.3390\/s24237710"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1007\/s13755-024-00290-x","article-title":"Spatial-attention convmixer archi- tecture for classification and detection of gastrointestinal diseases using the kvasir dataset","volume":"12","year":"2024","journal-title":"Health Inf. Sci. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Korkmaz, I., and Soygazi, F. (2024, January 10\u201312). Gastrointestinal image classification using deep learning architectures via transfer learning. Proceedings of the 2024 Medical Technologies Congress (TIPTEKNO), Mugla, Turkey.","DOI":"10.1109\/TIPTEKNO63488.2024.10755310"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Regmi, S., Subedi, A., Tomar, N.K., Bagci, U., and Jha, D. (2025, January 17\u201320). Vision transformer for efficient chest x-ray and gastrointestinal image classification. Proceedings of the Medical Imaging 2025: Computer-Aided Diagnosis, San Diego, CA, USA.","DOI":"10.1117\/12.3045810"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kaur, S., Goel, P., and Kapoor, N. (2025, January 10\u201312). Stomach Cancer Prediction and Detection using Deep Learning: A Review. Proceedings of the 2025 International Conference on Machine Learning and Autonomous Systems (ICMLAS), Prawet, Thailand.","DOI":"10.1109\/ICMLAS64557.2025.10968026"},{"key":"ref_21","first-page":"129","article-title":"MNETGIDD: A heuristic-oriented segmentation and deep learning multi-disease detection model for gastrointestinal tracts","volume":"9","author":"Bamini","year":"2025","journal-title":"Int. J. Syst. Innov."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Babu, P.L., and Jana, S. (2025). Gastrointestinal tract disease detection via deep learning based Duo-Feature Optimized Hexa-Classification model. Biomed. Signal Process. Control., 100.","DOI":"10.1016\/j.bspc.2024.106994"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bajhaiya, D., and Unni, S.N. (2024). Deep learning-enabled detection and localization of gastrointestinal diseases using wireless-capsule endoscopic images. Biomed. Signal Process. Control., 93.","DOI":"10.1016\/j.bspc.2024.106125"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1007\/s12559-024-10357-2","article-title":"Crcfusion- aicadx: Integrative cnn-lstm approach for accurate colorectal cancer diagnosis in colonoscopy images","volume":"17","author":"Raju","year":"2025","journal-title":"Cogn. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, J., Xu, Q., He, X., Liu, Z., Zhang, D., Wang, R., Qu, R., and Qiu, G. (2025). Cfformer: Cross cnn-transformer channel attention and spatial feature fusion for improved segmentation of low quality medical images. arXiv.","DOI":"10.2139\/ssrn.5243043"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"10425","DOI":"10.1007\/s11042-024-19197-6","article-title":"Federal learning-based a dual-branch deep learning model for colon polyp segmentation","volume":"84","author":"Cao","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Karthikha, R., Jamal, D.N., and Rafiammal, S.S. (2024). An approach of polyp segmentation from colonoscopy images using dilated-u-net-seg\u2013a deep learning network, Biomedical Signal Processing and Control. Biomed. Signal Process. Control., 93.","DOI":"10.1016\/j.bspc.2024.106197"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Fahad, M., E Mobeen, N., Imran, A.S., Daudpota, S.M., Kastrati, Z., Cheikh, F.A., and Ullah, M. (2025). Deep insights into gastrointestinal health: A compre- hensive analysis of gastrovision dataset using convolutional neural networks and explainable ai. Biomed. Signal Process. Control., 102.","DOI":"10.1016\/j.bspc.2024.107260"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"014505","DOI":"10.1117\/1.JMI.12.1.014505","article-title":"Vision transformer distillation for enhanced gastrointestinal abnormality recognition in wireless capsule endoscopy images","volume":"12","author":"Oukdach","year":"2025","journal-title":"J. Med. Imaging"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"46283","DOI":"10.1007\/s11042-024-18521-4","article-title":"GiensemformerCADx: A hybrid ensemble learning approach for enhanced gastrointestinal cancer recognition","volume":"83","author":"Raju","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e15734056353246","DOI":"10.2174\/0115734056353246241209060804","article-title":"Enhanced detection of colon diseases via a fused deep learning model with an auxiliary fusion layer and residual blocks on endoscopic images","volume":"21","author":"Kumar","year":"2025","journal-title":"Curr. Med. Imaging Former. Curr. Med. Imaging Rev."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Khan, M.A., Shafiq, U., Hamza, A., Mirza, A.M., Baili, J., AlHammadi, D.A., Cho, H.-C., and Chang, B. (2025). A novel network-level fused deep learning architecture with shallow neural network classifier for gastrointestinal cancer classification from wireless capsule endoscopy images, BMC Medical Informatics and Decision Making. BMC Med. Inform. Decis. Mak., 25.","DOI":"10.1186\/s12911-025-02966-0"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9276","DOI":"10.1109\/ACCESS.2025.3526786","article-title":"Detection and classification method for early-stage colorectal cancer using dyadic wavelet packet transform","volume":"13","author":"Takano","year":"2025","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"024004","DOI":"10.1117\/1.JMI.11.2.024004","article-title":"Cmnet: Deep learning model for colon polyp segmentation based on dual-branch structure","volume":"11","author":"Cao","year":"2024","journal-title":"J. Med. Imaging"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pan, X., Mu, Y., Ma, C., and He, Q. (2024). Tfcnet: A texture-aware and fine-grained feature compensated polyp detection network. Comput. Biol. Med., 171.","DOI":"10.1016\/j.compbiomed.2024.108144"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e1919","DOI":"10.7717\/peerj-cs.1919","article-title":"Multi-feature fusion and dandelion optimizer based model for automatically diagnosing the gastrointestinal diseases","volume":"10","author":"Kiziloluk","year":"2024","journal-title":"PeerJ Comput. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.-T., Lux, M., and Schmidt, P.T. (2017, January 20\u201323). Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan.","DOI":"10.1145\/3083187.3083212"},{"key":"ref_38","unstructured":"Pogorelov, K., Randel, K.R., de Lange, T., Eskeland, S.L., Griwodz, C., Johansen, D., Spampinato, C., Taschwer, M., Lux, M., and Schmidt, P.T. (2017, January 20\u201323). Nerthus: A bowel preparation quality video dataset. Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.compmedimag.2015.02.007","article-title":"Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians, Computerized medical imaging and graphics","volume":"43","author":"Bernal","year":"2015","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_40","first-page":"3239","article-title":"Gastrointestinal tract infections classification using deep learning","volume":"69","author":"Ramzan","year":"2021","journal-title":"Comput. Mater. Contin."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ramzan, M., Raza, M., Sharif, M.I., Azam, F., Kim, J., and Kadry, S. (2023). Gastrointestinal tract disorders classification using ensemble of InceptionNet and proposed GITNet based deep feature with ant colony optimization. PLoS ONE, 18.","DOI":"10.1371\/journal.pone.0292601"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mary, X., Raj, A.P.W., Evangeline, C.S., Neebha, T.M., Kumaravelu, V.B., and Manimegalai, P. (2023). Multi-class Classification of Gastrointestinal Diseases using Deep Learning Techniques. Open Biomed. Eng. J., 17.","DOI":"10.2174\/18741207-v17-e230215-2022-HT27-3589-11"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e33655","DOI":"10.1016\/j.heliyon.2024.e33655","article-title":"Efficient colorectal polyp segmentation using wavelet transformation and adaptunet: A hybrid u-net","volume":"10","author":"Rajasekar","year":"2024","journal-title":"Heliyon"},{"key":"ref_44","unstructured":"Chowdhury, M.S., Tapu, S., Sarkar, N.K., Ali, F.B., and Sabrin, L. (2025). Med-2d segnet: A light weight deep neural network for medical 2d image segmentation. arXiv."},{"key":"ref_45","first-page":"1","article-title":"A simple framework for depth-augmented contrastive learning for endoscopic image classification","volume":"73","author":"Weng","year":"2024","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_46","first-page":"1","article-title":"Polyp Detection Using U-Net Deep Learning Method","volume":"12","author":"Sinimoxon","year":"2024","journal-title":"Int. J. Sci. Eng. Technol."},{"key":"ref_47","unstructured":"Demirci, O., and Yilmaz, B. (2025). Tree-net: Enhancing medical image segmentation through efficient low-level feature training. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Jha, D., Ali, S., Emanuelsen, K., Hicks, S.A., Thambawita, V., Garcia-Ceja, E., Riegler, M.A., de Lange, T., Schmidt, P.T., and Johansen, H.D. (2021). Kvasir-instrument: Diagnostic and Therapeutic Tool Segmentation Dataset in Gastrointestinal Endoscopy. MultiMedia Modeling, Proceedings of the 27th International Conference, MMM 2021, Prague, Czech Republic, 22\u201324 June 2021, Springer. Proceedings, Part II 27.","DOI":"10.1007\/978-3-030-67835-7_19"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Dumitru, R.G., Peteleaza, D., and Craciun, C. (2023). Using DUCK-Net for polyp image segmentation. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-36940-5"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/TBME.2022.3216269","article-title":"Li-segpnet: Encoder-decoder mode lightweight segmentation network for colorectal polyps analysis","volume":"70","author":"Sharma","year":"2022","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3041","DOI":"10.1007\/s40747-021-00328-7","article-title":"3D-semantic segmentation and classification of stomach infections using uncertainty aware deep neural networks","volume":"8","author":"Amin","year":"2022","journal-title":"Complex Intell. Syst."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1515\/cdbme-2022-1082","article-title":"Automatic detection of foreign objects and contaminants in colonoscopic video data using deep learning","volume":"Volume 8","author":"Kress","year":"2022","journal-title":"Current Directions in Biomedical Engineering"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Li, Q., Yang, G., Chen, Z., Huang, B., Chen, L., Xu, D., Zhou, X., Zhong, S., Zhang, H., and Wang, T. (2017, January 14\u201316). Colorectal polyp segmentation using a fully convolutional neural network. Proceedings of the 2017 10th International Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), Shanghai, China.","DOI":"10.1109\/CISP-BMEI.2017.8301980"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Jha, D., Riegler, M.A., Johansen, D., Halvorsen, P., and Johansen, H.D. (2020, January 28\u201330). Doubleu-net: A deep convolutional neural network for medical image segmentation. Proceedings of the 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS), Rochester, MN, USA.","DOI":"10.1109\/CBMS49503.2020.00111"}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/12\/11\/1184\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T14:15:55Z","timestamp":1762179355000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/12\/11\/1184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,30]]},"references-count":54,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["bioengineering12111184"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering12111184","relation":{},"ISSN":["2306-5354"],"issn-type":[{"type":"electronic","value":"2306-5354"}],"subject":[],"published":{"date-parts":[[2025,10,30]]}}}