{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T19:34:23Z","timestamp":1768419263628,"version":"3.49.0"},"reference-count":41,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2019,7,5]],"date-time":"2019-07-05T00:00:00Z","timestamp":1562284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","award":["Finance Code 001"],"award-info":[{"award-number":["Finance Code 001"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["428333\/2016-8; 313423\/2017-2"],"award-info":[{"award-number":["428333\/2016-8; 313423\/2017-2"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multimodal systems are a workaround to enhance the robustness and effectiveness of biometric systems. A proper multimodal dataset is of the utmost importance to build such systems. The literature presents some multimodal datasets, although, to the best of our knowledge, there are no previous studies combining face, iris\/eye, and vital signals such as the Electrocardiogram (ECG). Moreover, there is no methodology to guide the construction and evaluation of a chimeric dataset. Taking that fact into account, we propose to create a chimeric dataset from three modalities in this work: ECG, eye, and face. Based on the Doddington Zoo criteria, we also propose a generic and systematic protocol imposing constraints for the creation of homogeneous chimeric individuals, which allow us to perform a fair and reproducible benchmark. Moreover, we have proposed a multimodal approach for these modalities based on state-of-the-art deep representations built by convolutional neural networks. We conduct the experiments in the open-world verification mode and on two different scenarios (intra-session and inter-session), using three modalities from two datasets: CYBHi (ECG) and FRGC (eye and face). Our multimodal approach achieves impressive decidability of 7.20 \u00b1 0.18, yielding an almost perfect verification system (i.e., Equal Error Rate (EER) of 0.20% \u00b1 0.06) on the intra-session scenario with unknown data. On the inter-session scenario, we achieve a decidability of 7.78 \u00b1 0.78 and an EER of 0.06% \u00b1 0.06. In summary, these figures represent a gain of over 28% in decidability and a reduction over 11% of the EER on the intra-session scenario for unknown data compared to the best-known unimodal approach. Besides, we achieve an improvement greater than 22% in decidability and an EER reduction over 6% in the inter-session scenario.<\/jats:p>","DOI":"10.3390\/s19132968","type":"journal-article","created":{"date-parts":[[2019,7,5]],"date-time":"2019-07-05T11:44:16Z","timestamp":1562327056000},"page":"2968","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Chimerical Dataset Creation Protocol Based on Doddington Zoo: A Biometric Application with Face, Eye, and ECG"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5525-6121","authenticated-orcid":false,"given":"Pedro","family":"Lopes Silva","sequence":"first","affiliation":[{"name":"Computing Department, Federal University of Ouro Preto, Ouro Preto 35400-000, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5249-1559","authenticated-orcid":false,"given":"Eduardo","family":"Luz","sequence":"additional","affiliation":[{"name":"Computing Department, Federal University of Ouro Preto, Ouro Preto 35400-000, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7747-5926","authenticated-orcid":false,"given":"Gladston","family":"Moreira","sequence":"additional","affiliation":[{"name":"Computing Department, Federal University of Ouro Preto, Ouro Preto 35400-000, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3688-3162","authenticated-orcid":false,"given":"Lauro","family":"Moraes","sequence":"additional","affiliation":[{"name":"Computing Department, Federal University of Ouro Preto, Ouro Preto 35400-000, MG, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2430-2030","authenticated-orcid":false,"given":"David","family":"Menotti","sequence":"additional","affiliation":[{"name":"Department of Informatics, Federal University of Paran\u00e1, Curitiba 81531-990, PR, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,5]]},"reference":[{"key":"ref_1","unstructured":"Masek, L. (2003). Recognition of Human Iris Patterns for Biometric Identification, University of Western Australia. Technical Report."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.patrec.2017.12.009","article-title":"Deep periocular representation aiming video surveillance","volume":"114","author":"Luz","year":"2018","journal-title":"Pattern Recognit. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1007\/s10462-016-9474-x","article-title":"Biometric recognition in surveillance scenarios: A survey","volume":"46","author":"Neves","year":"2016","journal-title":"Artif. Intell. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Silva, P.H., Luz, E., Zanlorensi, L.A., Menotti, D., and Moreira, G. (2018, January 8\u201313). Multimodal Feature Level Fusion based on Particle Swarm Optimization with Deep Transfer Learning. Proceedings of the IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, Brazil.","DOI":"10.1109\/CEC.2018.8477817"},{"key":"ref_5","unstructured":"Crihalmeanu, S., Ross, A., Schuckers, S., and Hornak, L. (2007). A Protocol for Multibiometric Data Acquisition, Storage and Dissemination, WVU, Lane Department of Computer Science and Electrical Engineering. Technical Report."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"McCool, C., Marcel, S., Hadid, A., Pietik\u00e4inen, M., Matejka, P., Cernock\u1ef3, J., Poh, N., Kittler, J., Larcher, A., and Levy, C. (2012, January 9\u201313). Bi-modal person recognition on a mobile phone: Using mobile phone data. Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Melbourne, VIC, Australia.","DOI":"10.1109\/ICMEW.2012.116"},{"key":"ref_7","unstructured":"Sequeira, A.F., Monteiro, J.C., Rebelo, A., and Oliveira, H.P. (2014, January 5\u20138). Mobbio: A multimodal database captured with a portable handheld device. Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal."},{"key":"ref_8","unstructured":"Nageshkumar, M., Mahesh, P., and Swamy, M.S. (2009). An efficient secure multimodal biometric fusion using palmprint and face image. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2270","DOI":"10.1016\/j.patcog.2005.01.012","article-title":"Score normalization in multimodal biometric systems","volume":"38","author":"Jain","year":"2005","journal-title":"Pattern Recognit."},{"key":"ref_10","first-page":"1258","article-title":"Learning Deep Off-The-Person Heart Biometrics Representations","volume":"13","author":"Luz","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_11","unstructured":"Wayman, J.L. (2006, January 14\u201319). A path forward for multi-biometrics. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2006), Toulouse, France."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Doddington, G., Liggett, W., Martin, A., Przybocki, M., and Reynolds, D. (1998). Sheep, Goats, Lambs and Wolves: A Statistical Analysis of Speaker Performance in the NIST 1998 Speaker Recognition Evaluation, Technical Report.","DOI":"10.21437\/ICSLP.1998-244"},{"key":"ref_13","unstructured":"Ross, A., Rattani, A., and Tistarelli, M. (2009, January 28\u201330). Exploiting the \u201cdoddington zoo\u201d effect in biometric fusion. Proceedings of the IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, Washington, DC, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1810","DOI":"10.1109\/TIFS.2018.2804890","article-title":"Liveness Detection and Automatic Template Updating using Fusion of ECG and Fingerprint","volume":"13","author":"Komeili","year":"2018","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mura, V., Ghiani, L., Marcialis, G.L., Roli, F., Yambay, D.A., and Schuckers, S.A. (2015, January 8\u201311). Livdet 2015 fingerprint liveness detection competition 2015. Proceedings of the IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS), Arlington, VA, USA.","DOI":"10.1109\/BTAS.2015.7358776"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1932","DOI":"10.1016\/j.patrec.2012.03.010","article-title":"Fusion of electrocardiogram with unobtrusive biometrics: An efficient individual authentication system","volume":"33","author":"Singh","year":"2012","journal-title":"Pattern Recognit. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4835","DOI":"10.1007\/s11042-016-3796-1","article-title":"Fusion of physiological measures for multimodal biometric systems","volume":"76","author":"Barra","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Joshi, S., and Kumar, A. (2016, January 7\u20138). Design of multimodal biometrics system based on feature level fusion. Proceedings of the 10th International Conference on Intelligent Systems and Control (ISCO), Coimbatore, India.","DOI":"10.1109\/ISCO.2016.7727072"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5390","DOI":"10.1016\/j.eswa.2014.02.051","article-title":"Multimodal biometrics: Weighted score level fusion based on non-ideal iris and face images","volume":"41","author":"Sim","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1016\/j.cmpb.2013.11.017","article-title":"Check Your Biosignals Here: A new dataset for off-the-person ECG biometrics","volume":"113","author":"Fred","year":"2014","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wu, H.Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. (2012). Eulerian Video Magnification for Revealing Subtle Changes in The World, MIT Press. Technical Report.","DOI":"10.1145\/2185520.2335416"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1088\/0967-3334\/35\/5\/807","article-title":"Non-contact video-based vital sign monitoring using ambient light and auto-regressive models","volume":"35","author":"Tarassenko","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2002","DOI":"10.1109\/TIFS.2014.2360430","article-title":"On evaluating ECG biometric systems: Session-dependence and body posture","volume":"9","author":"Wahabi","year":"2014","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_24","unstructured":"Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., and Worek, W. (2005, January 20\u201325). Overview of the face recognition grand challenge. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, USA."},{"key":"ref_25","first-page":"888","article-title":"Deep-PRWIS: Periocular Recognition Without the Iris and Sclera Using Deep Learning Frameworks","volume":"13","author":"Neves","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5082","DOI":"10.1109\/TIP.2014.2361285","article-title":"Ocular biometrics by score-level fusion of disparate experts","volume":"23","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lu, Z., Jiang, X., and Kot, A. (2017, January 2\u20134). Enhance deep learning performance in face recognition. Proceedings of the 2nd International Conference on Image, Vision and Computing (ICIVC), Chengdu, China.","DOI":"10.1109\/ICIVC.2017.7984554"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TCSVT.2003.818349","article-title":"An introduction to biometric recognition","volume":"14","author":"Jain","year":"2004","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_29","unstructured":"Daugman, J.G., and Williams, G.O. (1996, January 9\u201312). A proposed standard for biometric decidability. Proceedings of the CardTech\/SecureTech Conference, Atlanta, GA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Anderson, C.J. (2010). Central limit theorem. The Corsini Encyclopedia of Psychology, John Wiley & Son, Inc.","DOI":"10.1002\/9780470479216.corpsy0160"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TBME.1985.325532","article-title":"A real-time QRS detection algorithm","volume":"32","author":"Pan","year":"1985","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., and Pantic, M. (2013, January 2\u20138). 300 faces in-the-wild challenge: The first facial landmark localization challenge. Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, NSW, Australia.","DOI":"10.1109\/ICCVW.2013.59"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kazemi, V., and Sullivan, J. (2014, January 23\u201328). One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.241"},{"key":"ref_34","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_35","unstructured":"Huang, G.B., Mattar, M., Berg, T., and Learned-Miller, E. (2008, January 17\u201318). Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Proceedings of the Workshop on Faces in\u2019Real-Life\u2019Images: Detection, Alignment, and Recognition, Marseille, France."},{"key":"ref_36","unstructured":"Kim, M., Kumar, S., Pavlovic, V., and Rowley, H. (2008, January 23\u201328). Face tracking and recognition with visual constraints in real-world videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Vedaldi, A., and Lenc, K. (2015, January 26\u201330). Matconvnet: Convolutional neural networks for matlab. Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia.","DOI":"10.1145\/2733373.2807412"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Silva, P., Luz, E., Baeta, R., Pedrini, H., Falcao, A.X., and Menotti, D. (2015, January 26\u201329). An approach to iris contact lens detection based on deep image representations. Proceedings of the 28th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Salvador, Brazil.","DOI":"10.1109\/SIBGRAPI.2015.16"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"148658","DOI":"10.1155\/2008\/148658","article-title":"Analysis of human electrocardiogram for biometric recognition","volume":"2008","author":"Wang","year":"2007","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rattani, A., Poh, N., and Ross, A. (2012, January 16\u201321). Analysis of user-specific score characteristics for spoof biometric attacks. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6239226"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1049\/el.2014.4182","article-title":"Mutual dependency of features in multimodal biometric systems","volume":"51","author":"Yang","year":"2015","journal-title":"Electron. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/13\/2968\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:02:53Z","timestamp":1760187773000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/13\/2968"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,5]]},"references-count":41,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["s19132968"],"URL":"https:\/\/doi.org\/10.3390\/s19132968","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,5]]}}}