{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T21:33:38Z","timestamp":1767908018484,"version":"3.49.0"},"reference-count":47,"publisher":"World Scientific Pub Co Pte Ltd","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Neur. Syst."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p> Researchers have shown the limitations of using the single-modal data stream for emotion classification. Multi-modal data streams are therefore deemed necessary to improve the accuracy and performance of online emotion classifiers. An online decision ensemble is a widely used approach to classify emotions in real-time using multi-modal data streams. There is a plethora of online ensemble approaches; these approaches use a fixed parameter ([Formula: see text]) to adjust the weights of each classifier (called penalty) in case of wrong classification and no reward for a good performing classifier. Also, the performance of the ensemble depends on the [Formula: see text], which is set using trial and error. This paper presents a new Reward-Penalty-based Weighted Ensemble (RPWE) for real-time multi-modal emotion classification using multi-modal physiological data streams. The proposed RPWE is thoroughly tested using two prevalent benchmark data sets, DEAP and AMIGOS. The first experiment confirms the impact of the base stream classifier with RPWE for emotion classification in real-time. The RPWE is compared with different popular and widely used online ensemble approaches using multi-modal data streams in the second experiment. The average balanced accuracy, F1-score results showed the usefulness and robustness of RPWE in emotion classification in real-time from the multi-modal data stream. <\/jats:p>","DOI":"10.1142\/s0129065722500496","type":"journal-article","created":{"date-parts":[[2022,7,21]],"date-time":"2022-07-21T17:00:26Z","timestamp":1658422826000},"source":"Crossref","is-referenced-by-count":13,"title":["Reward-Penalty Weighted Ensemble for Emotion State Classification from Multi-Modal Data Streams"],"prefix":"10.1142","volume":"32","author":[{"given":"Arijit","family":"Nandi","sequence":"first","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya (BarcelonaTech), 08034 Barcelona, Spain"},{"name":"Eurecat, Centre Tecnol\u00f2gic de Catalunya, 08005 Barcelona, Spain"}]},{"given":"Fatos","family":"Xhafa","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya (BarcelonaTech), 08034 Barcelona, Spain"}]},{"given":"Laia","family":"Subirats","sequence":"additional","affiliation":[{"name":"Eurecat, Centre Tecnol\u00f2gic de Catalunya, 08005 Barcelona, Spain"},{"name":"ADaS Lab, Universitat Oberta de Catalunya, 08018 Barcelona, Spain"}]},{"given":"Santi","family":"Fort","sequence":"additional","affiliation":[{"name":"Eurecat, Centre Tecnol\u00f2gic de Catalunya, 08005 Barcelona, Spain"}]}],"member":"219","published-online":{"date-parts":[[2022,9,21]]},"reference":[{"key":"S0129065722500496BIB001","doi-asserted-by":"publisher","DOI":"10.1016\/j.bbr.2015.10.036"},{"key":"S0129065722500496BIB002","doi-asserted-by":"crossref","first-page":"1454","DOI":"10.3389\/fpsyg.2017.01454","volume":"8","author":"Tyng C. M.","year":"2017","journal-title":"Front. Psychol."},{"issue":"3","key":"S0129065722500496BIB003","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1111\/mbe.12244","volume":"14","author":"Li L.","year":"2020","journal-title":"Mind Brain Edu."},{"key":"S0129065722500496BIB004","first-page":"1","volume-title":"Int. Conf. Omni-layer Intelligent Systems","volume":"34","author":"Nandi A.","year":"2020"},{"key":"S0129065722500496BIB005","first-page":"1","volume":"34","author":"Houssein E. H.","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"S0129065722500496BIB006","doi-asserted-by":"publisher","DOI":"10.18608\/jla.2016.32.11"},{"issue":"2","key":"S0129065722500496BIB007","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1111\/jcal.12232","volume":"34","author":"Prieto L.","year":"2018","journal-title":"J. Comput. Assist. Learn."},{"key":"S0129065722500496BIB008","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2797176"},{"key":"S0129065722500496BIB009","volume":"50","author":"Gomes H. M.","year":"2017","journal-title":"ACM Comput. Surv."},{"key":"S0129065722500496BIB010","doi-asserted-by":"crossref","unstructured":"S. Tulyakov,  S. Jaeger,  V. Govindaraju and  D. Doermann ,  Review of Classifier Combination Methods, Machine Learning in Document Analysis and Recognition  (Springer,  Berlin, Heidelberg,  2008),  pp. 361\u2013386.","DOI":"10.1007\/978-3-540-76280-5_14"},{"key":"S0129065722500496BIB011","doi-asserted-by":"publisher","DOI":"10.1007\/BF00058655"},{"key":"S0129065722500496BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.15"},{"issue":"2","key":"S0129065722500496BIB013","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1109\/TAFFC.2018.2884461","volume":"12","author":"Miranda-Correa J. A.","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"S0129065722500496BIB014","first-page":"2755","volume":"8","author":"Kolter J. Z.","year":"2007","journal-title":"J. Mach. Learn. Res."},{"key":"S0129065722500496BIB015","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.ins.2013.12.011","volume":"265","author":"Brzezinski D.","year":"2014","journal-title":"Inf. Sci."},{"key":"S0129065722500496BIB016","first-page":"430","volume-title":"2014 11th Int. Computer Conf. Wavelet Actiev Media Technology and Information Processing (ICCWAMTIP)","author":"Gu X.-F.","year":"2014"},{"issue":"5","key":"S0129065722500496BIB017","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.3390\/s21051589","volume":"21","author":"Nandi A.","year":"2021","journal-title":"Sensors"},{"key":"S0129065722500496BIB018","first-page":"423","volume-title":"Proc. 22nd Engineering Applications of Neural Networks Conf.","author":"Nandi A.","year":"2021"},{"key":"S0129065722500496BIB019","first-page":"220","volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition (CVPR) Workshops","author":"Hayes T. L.","year":"2020"},{"key":"S0129065722500496BIB020","first-page":"1601","volume":"11","author":"Bifet A.","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"S0129065722500496BIB021","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1007\/978-3-642-04471-7_11","volume-title":"Smart Sensing and Context","author":"Bannach D.","year":"2009"},{"issue":"4","key":"S0129065722500496BIB022","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MNET.2004.1316761","volume":"18","author":"Sivrikaya F.","year":"2004","journal-title":"IEEE Netw."},{"key":"S0129065722500496BIB023","first-page":"1","volume-title":"GLOBECOM 2020 \u2014 2020 IEEE Global Communications Conf.","author":"Luo Y.","year":"2020"},{"key":"S0129065722500496BIB024","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.inffus.2020.01.011","volume":"59","author":"Zhang J.","year":"2020","journal-title":"Inf. Fusion"},{"key":"S0129065722500496BIB025","doi-asserted-by":"publisher","DOI":"10.1037\/h0054570"},{"issue":"4","key":"S0129065722500496BIB026","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.3758\/s13428-012-0314-x","volume":"45","author":"Warriner A. B.","year":"2013","journal-title":"Behav. Res. Methods"},{"issue":"2","key":"S0129065722500496BIB027","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MSP.2006.1621452","volume":"23","author":"Hanjalic A.","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"S0129065722500496BIB028","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2017.02.004"},{"key":"S0129065722500496BIB029","first-page":"1","volume-title":"Multiple Classifier Systems","author":"Kuncheva L. I.","year":"2004"},{"key":"S0129065722500496BIB030","first-page":"1","volume-title":"Int\u2019l Conf. Electrical, Computer and Communication Engineering","author":"Islam M. R.","year":"2019"},{"key":"S0129065722500496BIB031","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2006.02.005"},{"issue":"17","key":"S0129065722500496BIB032","doi-asserted-by":"crossref","first-page":"4723","DOI":"10.3390\/s20174723","volume":"20","author":"Bota P.","year":"2020","journal-title":"Sensors"},{"key":"S0129065722500496BIB033","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/s40846-019-00505-7","volume":"40","author":"Ayata D.","year":"2020","journal-title":"J. Med. Biol. Eng."},{"key":"S0129065722500496BIB034","first-page":"1","volume-title":"Medical Technologies National Congress","author":"Ayata D.","year":"2016"},{"key":"S0129065722500496BIB035","doi-asserted-by":"publisher","DOI":"10.3389\/fnsys.2020.00043"},{"key":"S0129065722500496BIB036","first-page":"7250","volume-title":"37th Annual Int\u2019l Conf. IEEE Engineering in Medicine and Biology Society","author":"Candra H.","year":"2015"},{"key":"S0129065722500496BIB037","first-page":"464","volume-title":"IEEE Winter Conf. Applications of Computer Vision","author":"Smith L. N.","year":"2017"},{"issue":"4","key":"S0129065722500496BIB038","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.jclinepi.2007.04.021","volume":"61","author":"Saary M. J.","year":"2008","journal-title":"J. Clin. Epidemiol."},{"key":"S0129065722500496BIB039","doi-asserted-by":"crossref","first-page":"101314","DOI":"10.1016\/j.jocs.2021.101314","volume":"51","author":"Grzyb J.","year":"2021","journal-title":"J. Comput. Sci."},{"key":"S0129065722500496BIB041","first-page":"2987","volume-title":"Advances in Neural Information Processing Systems","volume":"31","author":"Tang C.","year":"2018"},{"key":"S0129065722500496BIB042","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2016.12.005"},{"key":"S0129065722500496BIB044","doi-asserted-by":"crossref","first-page":"3265","DOI":"10.1109\/ACCESS.2019.2962085","volume":"8","author":"Huang H.","year":"2020","journal-title":"IEEE Access"},{"key":"S0129065722500496BIB045","doi-asserted-by":"crossref","first-page":"7943","DOI":"10.1109\/ACCESS.2021.3049516","volume":"9","author":"Zhang Y.","year":"2021","journal-title":"IEEE Access"},{"key":"S0129065722500496BIB046","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/ACCESS.2018.2883213","volume":"7","author":"Santamaria-Granados L.","year":"2019","journal-title":"IEEE Access"},{"issue":"3","key":"S0129065722500496BIB047","doi-asserted-by":"crossref","first-page":"102185","DOI":"10.1016\/j.ipm.2019.102185","volume":"57","author":"Li C.","year":"2020","journal-title":"Inf. Process. Manag."},{"key":"S0129065722500496BIB048","first-page":"1184","volume-title":"2019 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP)","author":"Yang H.-C.","year":"2019"},{"key":"S0129065722500496BIB049","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2011.01.032"}],"container-title":["International Journal of Neural Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129065722500496","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,26]],"date-time":"2022-11-26T07:43:01Z","timestamp":1669448581000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0129065722500496"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,21]]},"references-count":47,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1142\/S0129065722500496"],"URL":"https:\/\/doi.org\/10.1142\/s0129065722500496","relation":{},"ISSN":["0129-0657","1793-6462"],"issn-type":[{"value":"0129-0657","type":"print"},{"value":"1793-6462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,21]]},"article-number":"2250049"}}