{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T19:37:01Z","timestamp":1788896221362,"version":"build-2803163510"},"reference-count":55,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Sistema Nacional de Investigaci\u00f3n (SNI) of National Secretariat for Science, Technology, and Innovation (SENACYT)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Convolutional neural networks (CNN) have provided great advances for the task of sign language recognition (SLR). However, recurrent neural networks (RNN) in the form of long\u2013short-term memory (LSTM) have become a means for providing solutions to problems involving sequential data. This research proposes the development of a sign language translation system that converts Panamanian Sign Language (PSL) signs into text in Spanish using an LSTM model that, among many things, makes it possible to work with non-static signs (as sequential data). The deep learning model presented focuses on action detection, in this case, the execution of the signs. This involves processing in a precise manner the frames in which a sign language gesture is made. The proposal is a holistic solution that considers, in addition to the seeking of the hands of the speaker, the face and pose determinants. These were added due to the fact that when communicating through sign languages, other visual characteristics matter beyond hand gestures. For the training of this system, a data set of 330 videos (of 30 frames each) for five possible classes (different signs considered) was created. The model was tested having an accuracy of 98.8%, making this a valuable base system for effective communication between PSL users and Spanish speakers. In conclusion, this work provides an improvement of the state of the art for PSL\u2013Spanish translation by using the possibilities of translatable signs via deep learning.<\/jats:p>","DOI":"10.3390\/bdcc8030025","type":"journal-article","created":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T07:51:39Z","timestamp":1708933899000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7811-3687","authenticated-orcid":false,"given":"Alvaro A.","family":"Teran-Quezada","sequence":"first","affiliation":[{"name":"Facultad de Ingenier\u00eda de Sistemas Computacionales, Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9635-2665","authenticated-orcid":false,"given":"Victor","family":"Lopez-Cabrera","sequence":"additional","affiliation":[{"name":"Facultad de Ingenier\u00eda de Sistemas Computacionales, Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1456-2253","authenticated-orcid":false,"given":"Jose Carlos","family":"Rangel","sequence":"additional","affiliation":[{"name":"Facultad de Ingenier\u00eda de Sistemas Computacionales, Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"},{"name":"Centro de Estudios Multidisciplinarios en Ciencias, Ingenier\u00eda y Tecnolog\u00eda (CEMCIT AIP), Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"},{"name":"Sistema Nacional de Investigaci\u00f3n, SENACYT, Edificio 205, Ciudad del Saber, Clayton, Balboa, Panama City P.O. Box 0816-02852, Panama"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8806-7901","authenticated-orcid":false,"given":"Javier E.","family":"Sanchez-Galan","sequence":"additional","affiliation":[{"name":"Facultad de Ingenier\u00eda de Sistemas Computacionales, Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"},{"name":"Centro de Estudios Multidisciplinarios en Ciencias, Ingenier\u00eda y Tecnolog\u00eda (CEMCIT AIP), Universidad Tecnol\u00f3gica de Panam\u00e1 (UTP), El Dorado, Panama City P.O. Box 0819-07289, Panama"},{"name":"Sistema Nacional de Investigaci\u00f3n, SENACYT, Edificio 205, Ciudad del Saber, Clayton, Balboa, Panama City P.O. Box 0816-02852, Panama"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fatmi, R., Rashad, S., and Integlia, R. (2019, January 7\u20139). Comparing ANN, SVM, and HMM based Machine Learning Methods for American Sign Language Recognition using Wearable Motion Sensors. Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2019.8666491"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"26319","DOI":"10.1007\/s11042-021-10768-5","article-title":"ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks","volume":"80","author":"Sharma","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Rahman, M.M., Islam, M.S., Rahman, M.H., Sassi, R., Rivolta, M.W., and Aktaruzzaman, M. (2019, January 24\u201325). A New Benchmark on American Sign Language Recognition using Convolutional Neural Network. Proceedings of the 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI), Dhaka, Bangladesh.","DOI":"10.1109\/STI47673.2019.9067974"},{"key":"ref_4","unstructured":"Jing, L., Vahdani, E., Huenerfauth, M., and Tian, Y. (2019). Recognizing American Sign Language Manual Signs from RGB-D Videos. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kannoth, A., Yang, C., and Guanipa Larice, M.A. (2021, January 5\u20138). Hand Gesture Recognition Using CNN & Publication of World\u2019s Largest ASL Database. Proceedings of the 2021 IEEE Symposium on Computers and Communications (ISCC), Athens, Greece.","DOI":"10.1109\/ISCC53001.2021.9631255"},{"key":"ref_6","first-page":"485","article-title":"Classification of Sign Language Numbers Using the CNN Method","volume":"2","author":"Perdana","year":"2021","journal-title":"JITTER J. Ilm. Teknol. Dan Komput."},{"key":"ref_7","first-page":"93","article-title":"Classification of finger spelling American sign language using convolutional neural network","volume":"10","author":"Marjusalinah","year":"2021","journal-title":"Comput. Eng. Appl. J."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ariesta, M.C., Wiryana, F., and Zahra, A. (2018, January 7\u20138). Sentence level Indonesian sign language recognition using 3D convolutional neural network and bidirectional recurrent neural network. Proceedings of the 2018 Indonesian Association for Pattern Recognition International Conference (INAPR), Jakarta, Indonesia.","DOI":"10.1109\/INAPR.2018.8627016"},{"key":"ref_9","first-page":"1205","article-title":"Advances, challenges and opportunities in continuous sign language recognition","volume":"15","author":"Ibrahim","year":"2020","journal-title":"J. Eng. Appl. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"113794","DOI":"10.1016\/j.eswa.2020.113794","article-title":"Sign language recognition: A deep survey","volume":"164","author":"Rastgoo","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.future.2020.08.005","article-title":"ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis","volume":"115","author":"Basiri","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"107322","DOI":"10.1016\/j.ymssp.2020.107322","article-title":"Analysis of different RNN autoencoder variants for time series classification and machine prognostics","volume":"149","author":"Yu","year":"2021","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_13","unstructured":"Metaxas, D., Dilsizian, M., and Neidle, C. (2018). 8th Workshop on the Representation & Processing of Sign Languages: Involving the Language Community, Language Resources and Evaluation Conference 2018, European Language Resources Association (ELRA)."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2732","DOI":"10.1109\/TAES.2021.3139848","article-title":"Effect of Kinematics and Fluency in Adversarial Synthetic Data Generation for ASL Recognition With RF Sensors","volume":"58","author":"Rahman","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zhou, W., Xie, C., Pu, J., and Li, H. (2016, January 11\u201315). Chinese sign language recognition with adaptive HMM. Proceedings of the 2016 IEEE International Conference on Multimedia and Expo (ICME), Seattle, WA, USA.","DOI":"10.1109\/ICME.2016.7552950"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1504\/IJAPR.2016.079048","article-title":"A survey on manual and non-manual sign language recognition for isolated and continuous sign","volume":"3","author":"Agrawal","year":"2016","journal-title":"Int. J. Appl. Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"100141","DOI":"10.1016\/j.array.2022.100141","article-title":"Indian Sign Language recognition system using SURF with SVM and CNN","volume":"14","author":"Katoch","year":"2022","journal-title":"Array"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Koller, O., Zargaran, O., Ney, H., and Bowden, R. (2016, January 19\u201322). Deep sign: Hybrid CNN-HMM for continuous sign language recognition. Proceedings of the British Machine Vision Conference 2016, York, UK.","DOI":"10.5244\/C.30.136"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1007\/s11263-018-1121-3","article-title":"Deep sign: Enabling robust statistical continuous sign language recognition via hybrid CNN-HMMs","volume":"126","author":"Koller","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Buttar, A.M., Ahmad, U., Gumaei, A.H., Assiri, A., Akbar, M.A., and Alkhamees, B.F. (2023). Deep Learning in Sign Language Recognition: A Hybrid Approach for the Recognition of Static and Dynamic Signs. Mathematics, 11.","DOI":"10.3390\/math11173729"},{"key":"ref_21","first-page":"141","article-title":"Sign language semantic translation system using ontology and deep learning","volume":"11","author":"Elsayed","year":"2020","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_22","unstructured":"Abedin, T., Prottoy, K.S.S., Moshruba, A., and Hakim, S.B. (2021). Bangla sign language recognition using concatenated BdSL network. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Fink, J., Fr\u00e9nay, B., Meurant, L., and Cleve, A. (2021, January 18\u201322). LSFB-CONT and LSFB-ISOL: Two New Datasets for Vision-Based Sign Language Recognition. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.","DOI":"10.1109\/IJCNN52387.2021.9534336"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2021.3109732","article-title":"Toward the Minimum Number of Wearables to Recognize Signer-Independent Italian Sign Language With Machine-Learning Algorithms","volume":"70","author":"Calado","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wei, F., and Chen, Y. (2023, January 1\u20136). Improving continuous sign language recognition with cross-lingual signs. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.02158"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yin, A., Zhao, Z., Jin, W., Zhang, M., Zeng, X., and He, X. (2022, January 18\u201324). Mlslt: Towards multilingual sign language translation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00505"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tornay, S., Razavi, M., and Doss, M.M. (2020, January 4\u20138). Towards multilingual sign language recognition. Proceedings of the ICASSP 2020\u20132020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054631"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1109\/TPAMI.2018.2827052","article-title":"Real-time 3D hand pose estimation with 3D convolutional neural networks","volume":"41","author":"Ge","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Lu, W., Gan, W., and Hou, W. (2021). A contactless method to measure real-time finger motion using depth-based pose estimation. Comput. Biol. Med., 131.","DOI":"10.1016\/j.compbiomed.2021.104282"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Huang, J., Zhou, W., Zhang, Q., Li, H., and Li, W. (2018, January 2\u20137). Video-based sign language recognition without temporal segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11903"},{"key":"ref_31","unstructured":"Pimentel, D., Walker, R., and Fajardo, M. (2018). Lengua de Se\u00f1as Paname\u00f1as, Editora Panam\u00e1 Am\u00e9rica."},{"key":"ref_32","unstructured":"Pimentel Ara\u00faz, M.I. (2018). Sitio Web Para el Aprendizaje de Lengua de Se\u00f1as Paname\u00f1as. [Ph.D. Thesis, Universidad de Panam\u00e1, Vicerrector\u00eda de Investigaci\u00f3n y Postgrado]."},{"key":"ref_33","unstructured":"Flores, A., Gonz\u00e1lez, E., Pan, J.Z., Villarreal, V., and Mu\u00f1oz, L. (2019, January 11\u201313). Sistema de aprendizaje de Lengua de Se\u00f1as Paname\u00f1a (LSP) a trav\u00e9s de un brazo rob\u00f3tico articulado con reconocimiento de gestos. Proceedings of the Memorias de Congresos UTP, Pereira, Colombia."},{"key":"ref_34","unstructured":"(2023, September 15). Gesti\u00f3n Empresarial 3000. IPHE Inclusivo. Available online: https:\/\/play.google.com\/store\/apps\/details?id=ca.costari.apps.ipheinclusivo&pli=1."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1080\/11356405.2022.2058793","article-title":"Presentation and evaluation of a digital tool for sign language (Presentaci\u00f3n y evaluaci\u00f3n de una herramienta digital para la lengua de signos)","volume":"34","year":"2022","journal-title":"Cult. Educ."},{"key":"ref_36","first-page":"663","article-title":"EnSenias: Technological tool to learn, teach, improve and use Panamanian Sign Language","volume":"25","author":"Fuentes","year":"2020","journal-title":"\u00cdkala"},{"key":"ref_37","unstructured":"Ala\u00edn Botacio, L. (2019). Desarrollo y Validaci\u00f3n de una Aplicaci\u00f3n web y cd Educativa Inclusiva Para el Aprendizaje de la Lengua de se\u00f1as Paname\u00f1a. [Ph.D. Thesis, Universidad de Granada]."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bodmer, R., Liu, L., Liu, W., and Rangel, J.C. (2020). Sign language recognition with machine learning for elementary school children. Rev. Iniciaci\u00f3n Cient\u00edfica Edici\u00f3n Espec., 6.","DOI":"10.33412\/rev-ric.v6.0.3161"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Teran-Quezada, A., Lopez-Cabrera, V., Rangel, J.C., and Sanchez-Galan, J.E. (2022, January 9\u201312). Hand Gesture Recognition with ConvNets for School-Aged Children to Learn Basic Arithmetic Operations. Proceedings of the 2022 IEEE 40th Central America and Panama Convention (CONCAPAN), Panama City, Panama.","DOI":"10.1109\/CONCAPAN48024.2022.9997680"},{"key":"ref_40","unstructured":"Bazarevsky, V., and Grishchenko, I. (2023, September 15). MediaPipe Holistic\u2014Simultaneous Face, Hand and Pose Prediction, on Device. Available online: https:\/\/ai.googleblog.com\/2020\/12\/mediapipe-holistic-simultaneous-face.html."},{"key":"ref_41","unstructured":"Bazarevsky, V., and Zhang, F. (2019). On-Device, Real-Time Hand Tracking with MediaPipe. arXiv."},{"key":"ref_42","unstructured":"Ablavatski, A., and Grishchenko, I. (2024, February 20). Real-Time AR Self-Expression with Machine Learning. Available online: https:\/\/blog.research.google\/2019\/03\/real-time-ar-self-expression-with.html."},{"key":"ref_43","unstructured":"Bazarevsky, V., and Grishchenko, I. (2023, September 15). On-Device, Real-Time Body Pose Tracking with MediaPipe BlazePose. Available online: https:\/\/blog.research.google\/2020\/08\/on-device-real-time-body-pose-tracking.html."},{"key":"ref_44","unstructured":"Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2006). MediaPipe Hands: On-device Real-time Hand Tracking. arXiv."},{"key":"ref_45","unstructured":"Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., and Grundmann, M. (2019). BlazeFace: Sub-Millisecond Neural Face Detection on Mobile GPUs. arXiv."},{"key":"ref_46","unstructured":"Bazarevsky, V., Grishchenko, I., Raveendran, K., Zhu, T., Zhang, F., and Grundmann, M. (2020). BlazePose: On-device Real-time Body Pose tracking. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Teran-Quezada, A.A., Lopez-Cabrera, V., Rangel, J.C., and Sanchez-Galan, J.E. (2024, February 20). Panama City Road Traffic Indicidents 2014\u20132022\u2014Social Media Dataset. Mendeley Data, V1. (In Spanish).","DOI":"10.1109\/CONCAPAN48024.2022.9997680"},{"key":"ref_48","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/j.neucom.2019.05.064","article-title":"Improving novelty detection with generative adversarial networks on hand gesture data","volume":"358","author":"Simao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Shen, J., Dudley, J., and Kristensson, P.O. (2021, January 15\u201318). The imaginative generative adversarial network: Automatic data augmentation for dynamic skeleton-based hand gesture and human action recognition. Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), Jodhpur, India.","DOI":"10.1109\/FG52635.2021.9666999"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Liu, Y., De Nadai, M., Zen, G., Sebe, N., and Lepri, B. (2019, January 21\u201325). Gesture-to-gesture translation in the wild via category-independent conditional maps. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3351020"},{"key":"ref_52","first-page":"118993","article-title":"A survey on Sign Language machine translation","volume":"213","author":"Labaka","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"75213","DOI":"10.1109\/ACCESS.2019.2920969","article-title":"Human activity recognition based on motion sensor using u-net","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"126917","DOI":"10.1109\/ACCESS.2021.3110912","article-title":"Deep learning for sign language recognition: Current techniques, benchmarks, and open issues","volume":"9","author":"Khalid","year":"2021","journal-title":"IEEE Access"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Nogales, R.E., and Benalc\u00e1zar, M.E. (2023). Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory. Big Data Cogn. Comput., 7.","DOI":"10.3390\/bdcc7020102"}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/3\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:05:06Z","timestamp":1760105106000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/3\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,26]]},"references-count":55,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["bdcc8030025"],"URL":"https:\/\/doi.org\/10.3390\/bdcc8030025","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,26]]}}}