{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T19:06:53Z","timestamp":1771614413623,"version":"3.50.1"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,5,11]],"date-time":"2023-05-11T00:00:00Z","timestamp":1683763200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,11]],"date-time":"2023-05-11T00:00:00Z","timestamp":1683763200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100016047","name":"Science Fund of the Republic of Serbia","doi-asserted-by":"crossref","award":["6526093"],"award-info":[{"award-number":["6526093"]}],"id":[{"id":"10.13039\/501100016047","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Computer vision is one of the artificial intelligence\u2019s most challenging fields, enabling computers to interpret, analyse and derive meaningful information from the visual world. There are various utilizations of computer vision algorithms, and most of them, from simpler to more complicated, have an object and shape recognition in common. Traditional pen and paper tests are designed in a pre-established format and consist of numerous basic shapes, which designate the important parts of the test itself. With that in mind, many computer vision applications regarding pen and paper tests arise as an opportunity. Massive courses and large schooling organizations mostly conduct their exams in paper format and assess them manually, which imposes a significant burden on the teaching staff. Any kind of automatization that will facilitate the grading process is highly desirable. Hence, an automated answer recognition system in assessment was developed to mitigate the problems above. The system uses images of scanned test pages obtained from the test scanning process and performs the necessary image manipulation steps to increase target recognition accuracy. Further, it manages to identify regions of interest containing multiple-choice questions and contours. Finally, the system verifies obtained results using the knowledge of the whereabouts of the test template regions of interest.<\/jats:p>","DOI":"10.1007\/s11042-023-15767-2","type":"journal-article","created":{"date-parts":[[2023,5,11]],"date-time":"2023-05-11T11:11:03Z","timestamp":1683803463000},"page":"2031-2052","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Automated assessment of pen and paper tests using computer vision"],"prefix":"10.1007","volume":"83","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7140-5043","authenticated-orcid":false,"given":"Vladimir","family":"Jocovic","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Milan","family":"Marinkovic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sasa","family":"Stojanovic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1142-9243","authenticated-orcid":false,"given":"Bosko","family":"Nikolic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,5,11]]},"reference":[{"issue":"1","key":"15767_CR1","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.1007\/s11554-021-01106-x","volume":"18","author":"N Bacanin","year":"2021","unstructured":"Bacanin N, Bezdan T, Venkatachalam K, Al-Turjman F (2021) Optimized convolutional neural network by firefly algorithm for magnetic resonance image classification of glioma brain tumor grade. J Real-Time Image Proc 18(1):1085\u20131098. https:\/\/doi.org\/10.1007\/s11554-021-01106-x","journal-title":"J Real-Time Image Proc"},{"issue":"6","key":"15767_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/math8060936","volume":"8","author":"N Bacanin","year":"2020","unstructured":"Bacanin N, Bezdan T, Tuba E, Strumberger I, Tuba M (2020) Monarch butterfly optimization based convolutional neural network design. Mathematics 8(6):1\u201332. https:\/\/doi.org\/10.3390\/math8060936","journal-title":"Mathematics"},{"key":"15767_CR3","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-021-03488-z","author":"M Bansal","year":"2021","unstructured":"Bansal M, Kumar M, Sachdeva M, Mittal A (2021) Transfer learning for image classification using VGG19: Caltech-101 image data set. J Ambient Intell Hum Comput. https:\/\/doi.org\/10.1007\/s12652-021-03488-z","journal-title":"J Ambient Intell Hum Comput"},{"issue":"2","key":"15767_CR4","doi-asserted-by":"publisher","first-page":"331","DOI":"10.5540\/tema.2019.020.02.0331","volume":"20","author":"W Barbosaa","year":"2019","unstructured":"Barbosaa W, Vieira W (2019) On the improvement of multiple circles detection from images using Hough transform. Trends Comput Appl Math (TEMA) 20(2):331\u2013342. https:\/\/doi.org\/10.5540\/tema.2019.020.02.0331","journal-title":"Trends Comput Appl Math (TEMA)"},{"key":"15767_CR5","doi-asserted-by":"publisher","unstructured":"Borade JG, Netak LD (2020) Automated grading of essays: a review. Int Conf Intell Hum Comput Interact 238\u2013249. https:\/\/doi.org\/10.1007\/978-3-030-68449-5_25","DOI":"10.1007\/978-3-030-68449-5_25"},{"key":"15767_CR6","unstructured":"Catalan JA (2017) A framework for automated multiple-choice exam scoring with digital image and assorted processing using readily available software. DLSU Research Congress 2017, De La Salle University, Manila, pp 1\u20135"},{"issue":"1","key":"15767_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.asw.2021.100594","volume":"51","author":"MEG Chamorro","year":"2022","unstructured":"Chamorro MEG (2022) Cognitive validity evidence of computer- and paper-based writing tests and differences in the impact on EFL test-takers in classroom assessment. Assess Writ 51(1):1\u201321. https:\/\/doi.org\/10.1016\/j.asw.2021.100594","journal-title":"Assess Writ"},{"issue":"16","key":"15767_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11042-022-12843-x","volume":"81","author":"P Dansena","year":"2022","unstructured":"Dansena P, Bag S, Pal R (2022) Pen ink discrimination in handwritten documents using statistical and motif texture analysis: a classification based approach. Multimed Tools Appl 81(16):1\u201341. https:\/\/doi.org\/10.1007\/s11042-022-12843-x","journal-title":"Multimed Tools Appl"},{"issue":"1","key":"15767_CR9","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.ijleo.2016.12.064","volume":"133","author":"AO Djekoune","year":"2017","unstructured":"Djekoune AO, Messaoudi K, Amara K (2017) Incremental circle Hough transform: an improved method for circle detection. Optik 133(1):17\u201331. https:\/\/doi.org\/10.1016\/j.ijleo.2016.12.064","journal-title":"Optik"},{"issue":"2","key":"15767_CR10","first-page":"1","volume":"4","author":"D Golekar","year":"2022","unstructured":"Golekar D, Bula R, Hole R, Katare S, Parab S (2022) Sign language recognition using Python and OpenCV. Int Res J Modernization Eng Technol Sci 4(2):1\u20135","journal-title":"Int Res J Modernization Eng Technol Sci"},{"issue":"13","key":"15767_CR11","doi-asserted-by":"publisher","first-page":"2259","DOI":"10.3390\/math10132259","volume":"10","author":"S Gonz\u00e1lez-L\u00f3pez","year":"2022","unstructured":"Gonz\u00e1lez-L\u00f3pez S, Montes-Rosales ZG, L\u00f3pez-Monroy AP, L\u00f3pez-L\u00f3pez A, Garc\u00eda-Gorrostieta JM (2022) Short answer detection for open questions: a sequence labeling approach with deep learning models. Mathematics 10(13):2259. https:\/\/doi.org\/10.3390\/math10132259","journal-title":"Mathematics"},{"key":"15767_CR12","unstructured":"GradeScanner mobile application for grading bubble sheets assessments automatically. (Last access 24.04.2023.) https:\/\/www.gradescanner.net\/"},{"issue":"2","key":"15767_CR13","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1007\/s12046-022-01847-w","volume":"47","author":"M Kumar","year":"2022","unstructured":"Kumar M, Jindal MK, Kumar M (2022) Distortion, rotation and scale invariant recognition of hollow Hindi characters. S\u0101dhan\u0101 47(2):92. https:\/\/doi.org\/10.1007\/s12046-022-01847-w","journal-title":"S\u0101dhan\u0101"},{"key":"15767_CR14","doi-asserted-by":"publisher","first-page":"1107","DOI":"10.1007\/s11831-021-09608-4","volume":"29","author":"M Kumar","year":"2022","unstructured":"Kumar M, Jindal MK, Kumar M (2022) A systematic survey on CAPTCHA recognition: types, creation and breaking techniques. Arch Computat Methods Eng 29:1107\u20131136. https:\/\/doi.org\/10.1007\/s11831-021-09608-4","journal-title":"Arch Computat Methods Eng"},{"issue":"6","key":"15767_CR15","doi-asserted-by":"publisher","first-page":"4957","DOI":"10.1007\/s00521-021-06686-0","volume":"34","author":"M Kumar","year":"2022","unstructured":"Kumar M, Jindal MK, Kumar M (2022) Design of innovative CAPTCHA for hindi language. Neural Comput Appl 34(6):4957\u20134992. https:\/\/doi.org\/10.1007\/s00521-021-06686-0","journal-title":"Neural Comput Appl"},{"key":"15767_CR16","doi-asserted-by":"publisher","first-page":"17069","DOI":"10.1007\/s11042-017-5276-7","volume":"7713","author":"L Lopez-Fuentes","year":"2018","unstructured":"Lopez-Fuentes L, van de Weijer J, Gonz\u00e1lez-Hidalgo M, Skinnemoen H, Bagdanov AD (2018) Review on computer vision techniques in emergency situations. Multimedia Tools Appl 7713:17069\u201317107. https:\/\/doi.org\/10.1007\/s11042-017-5276-7","journal-title":"Multimedia Tools Appl"},{"issue":"5","key":"15767_CR17","doi-asserted-by":"publisher","first-page":"2001","DOI":"10.1007\/s10489-018-1372-2","volume":"49","author":"A Lopez-Martinez","year":"2019","unstructured":"Lopez-Martinez A, Cuevas FJ (2019) Automatic circle detection on images using the teaching learning based optimization algorithm and gradient analysis. Appl Intell 49(5):2001\u20132016. https:\/\/doi.org\/10.1007\/s10489-018-1372-2","journal-title":"Appl Intell"},{"issue":"4","key":"15767_CR18","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1152\/advan.00186.2016","volume":"42","author":"C Loudon","year":"2018","unstructured":"Loudon C, Macias-Mu\u00f1oz A (2018) Item statistics derived from three-option versions of multiple-choice questions are usually as robust as four- or five-option versions:\u00a0implications for exam design. Adv Physiol Educ 42(4):565\u2013575. https:\/\/doi.org\/10.1152\/advan.00186.2016","journal-title":"Adv Physiol Educ"},{"issue":"1","key":"15767_CR19","doi-asserted-by":"publisher","first-page":"1773","DOI":"10.1186\/s13640-016-0149-y","volume":"46","author":"A Manzanera","year":"2016","unstructured":"Manzanera A, Nguyen T, Xu X (2016) Line and circle detection using dense one-to-one Hough transforms on greyscale images. J Image Video Proc 46(1):1773\u20131773. https:\/\/doi.org\/10.1186\/s13640-016-0149-y","journal-title":"J Image Video Proc"},{"issue":"1","key":"15767_CR20","doi-asserted-by":"publisher","first-page":"1075","DOI":"10.1016\/j.patcog.2020.107588","volume":"109","author":"Z Mingyang","year":"2021","unstructured":"Mingyang Z, Xiaohong J, Dong-Ming Y (2021) An occlusion-resistant circle detector using inscribed triangles. Pattern Recogn 109(1):1075\u20131088. https:\/\/doi.org\/10.1016\/j.patcog.2020.107588","journal-title":"Pattern Recogn"},{"issue":"3","key":"15767_CR21","doi-asserted-by":"publisher","first-page":"1495","DOI":"10.1111\/bjet.12644","volume":"50","author":"A Nardi","year":"2019","unstructured":"Nardi A, Ranieri M (2019) Comparing paper-based and electronic multiple-choice examinations with personal devices: impact on students\u2019 performance, self-efficacy and satisfaction. Br J Educ Technol 50(3):1495\u20131506. https:\/\/doi.org\/10.1111\/bjet.12644","journal-title":"Br J Educ Technol"},{"issue":"41","key":"15767_CR22","doi-asserted-by":"publisher","first-page":"30151","DOI":"10.1007\/s11042-020-09481-6","volume":"79","author":"N Odeh","year":"2020","unstructured":"Odeh N, Direkoglu C (2020) Automated shopping system using computer vision. Multimed Tools Appl 79(41):30151\u201330161. https:\/\/doi.org\/10.1007\/s11042-020-09481-6","journal-title":"Multimed Tools Appl"},{"issue":"47","key":"15767_CR23","doi-asserted-by":"publisher","first-page":"34697","DOI":"10.1007\/978-981-16-0507-9","volume":"79","author":"KC Santosh","year":"2020","unstructured":"Santosh KC, Antani SK (2020) Recent trends in image processing and pattern recognition. Multimed Tools Appl 79(47):34697\u201334699. https:\/\/doi.org\/10.1007\/978-981-16-0507-9","journal-title":"Multimed Tools Appl"},{"key":"15767_CR24","doi-asserted-by":"publisher","unstructured":"Shaheed K, Aihua M, Qureshi I, Abbas Q, Kumar M, Xingming Z (2022) Finger-vein presentation attack detection using depthwise separable convolution neural network. Expert Syst Appl 198(1). https:\/\/doi.org\/10.1016\/j.eswa.2022.116786","DOI":"10.1016\/j.eswa.2022.116786"},{"key":"15767_CR25","unstructured":"SmartGrade mobile application for grading multiple-choice answer sheets. (Last access 24.04.2023). https:\/\/smartgrade.net\/"},{"key":"15767_CR26","doi-asserted-by":"publisher","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. Proc IEEE Conf Comput Vis Pattern Recognit (CVPR) 2818\u20132826. https:\/\/doi.org\/10.1109\/CVPR.2016.308","DOI":"10.1109\/CVPR.2016.308"},{"issue":"2","key":"15767_CR27","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1006\/cviu.2001.0923","volume":"83","author":"C Teh-Chuan","year":"2001","unstructured":"Teh-Chuan C, Kuo-Liang C (2001) An efficient randomized algorithm for detecting circles. Comput Vis Image Underst 83(2):172\u2013191. https:\/\/doi.org\/10.1006\/cviu.2001.0923","journal-title":"Comput Vis Image Underst"},{"issue":"5","key":"15767_CR28","first-page":"1","volume":"4","author":"P Tirpude","year":"2022","unstructured":"Tirpude P, Girhepunje P, Sahu S, Zilpe S, Ragite H (2022) Real time object detection using OpenCV-Python. Int Res J Modernization Eng Technol Sci 4(5):1\u20136","journal-title":"Int Res J Modernization Eng Technol Sci"},{"issue":"1","key":"15767_CR29","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1007\/s10459-012-9434-4","volume":"18","author":"R Tractenberg","year":"2013","unstructured":"Tractenberg R, Gushta M, Mulroney S, Weissinger P (2013) Multiple choice questions can be designed or revised to challenge learners\u2019 critical thinking. Adv Health Sci Educ 18(1):945\u2013961. https:\/\/doi.org\/10.1007\/s10459-012-9434-4","journal-title":"Adv Health Sci Educ"},{"key":"15767_CR30","doi-asserted-by":"publisher","unstructured":"Vaidya R, Trivedi D, Satra S, Pimpale PM (2018) Handwritten character recognition using deep-learning. Second Int Conf Inventive Communication Comput Technol (ICICCT) 772\u2013775. https:\/\/doi.org\/10.1109\/ICICCT.2018.8473291","DOI":"10.1109\/ICICCT.2018.8473291"},{"key":"15767_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-20904-9_9","author":"JS Velasco","year":"2020","unstructured":"Velasco JS, Beltran AAV, Alayon JAC, Maranan PEB, Mascardo CMA, Sombrito JMB, Tolentino LKS (2020) Alphanumeric Test Paper Checker Through Intelligent Character Recognition Using OpenCV and Support Vector Machine. World Congress on Engineering and Technology; Innovation and its Sustainability 2018. WCETIS 2018. EAI\/Springer Innovations in Communication and Computing. Springer, Berlin, pp 119\u2013128. https:\/\/doi.org\/10.1007\/978-3-030-20904-9_9"},{"issue":"1","key":"15767_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2018\/7068349","volume":"2018","author":"A Voulodimos","year":"2018","unstructured":"Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018(1):1\u201313. https:\/\/doi.org\/10.1155\/2018\/7068349","journal-title":"Comput Intell Neurosci"},{"issue":"25","key":"15767_CR33","doi-asserted-by":"publisher","first-page":"17245","DOI":"10.1007\/s11042-019-7423-9","volume":"79","author":"S Xiao","year":"2020","unstructured":"Xiao S, Li T, Wang J (2020) Optimization methods of video images processing for mobile object recognition. Multimed Tools Appl 79(25):17245\u201317255. https:\/\/doi.org\/10.1007\/s11042-019-7423-9","journal-title":"Multimed Tools Appl"},{"issue":"1","key":"15767_CR34","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1016\/j.eswa.2015.12.019","volume":"51","author":"Y Zhenjie","year":"2016","unstructured":"Zhenjie Y, Weidong Y (2016) Curvature aided Hough transform for circle detection. Expert Syst Appl 51(1):26\u201333. https:\/\/doi.org\/10.1016\/j.eswa.2015.12.019","journal-title":"Expert Syst Appl"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-15767-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-023-15767-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-15767-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T08:33:37Z","timestamp":1704184417000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-023-15767-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,11]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["15767"],"URL":"https:\/\/doi.org\/10.1007\/s11042-023-15767-2","relation":{"references":[{"id-type":"uri","id":"","asserted-by":"subject"}]},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,11]]},"assertion":[{"value":"21 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 April 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 May 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 May 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare they have no conflicts of interest to report regarding the present study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}