{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T19:11:08Z","timestamp":1790363468731,"version":"4.1.0"},"reference-count":40,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2018,4,26]],"date-time":"2018-04-26T00:00:00Z","timestamp":1524700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Applying deep learning methods to electroencephalograph (EEG) data for cognitive state assessment has yielded improvements over previous modeling methods. However, research focused on cross-participant cognitive workload modeling using these techniques is underrepresented. We study the problem of cross-participant state estimation in a non-stimulus-locked task environment, where a trained model is used to make workload estimates on a new participant who is not represented in the training set. Using experimental data from the Multi-Attribute Task Battery (MATB) environment, a variety of deep neural network models are evaluated in the trade-space of computational efficiency, model accuracy, variance and temporal specificity yielding three important contributions: (1) The performance of ensembles of individually-trained models is statistically indistinguishable from group-trained methods at most sequence lengths. These ensembles can be trained for a fraction of the computational cost compared to group-trained methods and enable simpler model updates. (2) While increasing temporal sequence length improves mean accuracy, it is not sufficient to overcome distributional dissimilarities between individuals\u2019 EEG data, as it results in statistically significant increases in cross-participant variance. (3) Compared to all other networks evaluated, a novel convolutional-recurrent model using multi-path subnetworks and bi-directional, residual recurrent layers resulted in statistically significant increases in predictive accuracy and decreases in cross-participant variance.<\/jats:p>","DOI":"10.3390\/s18051339","type":"journal-article","created":{"date-parts":[[2018,4,27]],"date-time":"2018-04-27T06:52:23Z","timestamp":1524811943000},"page":"1339","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":49,"title":["Cross-Participant EEG-Based Assessment of Cognitive Workload Using Multi-Path Convolutional Recurrent Neural Networks"],"prefix":"10.3390","volume":"18","author":[{"given":"Ryan","family":"Hefron","sequence":"first","affiliation":[{"name":"Department of Electrical &amp; Computer Engineering, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4982-9859","authenticated-orcid":false,"given":"Brett","family":"Borghetti","sequence":"additional","affiliation":[{"name":"Department of Electrical &amp; Computer Engineering, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine","family":"Schubert Kabban","sequence":"additional","affiliation":[{"name":"Department of Mathematics &amp; Statistics, Air Force Institute of Technology, WPAFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Christensen","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, WPAFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Estepp","sequence":"additional","affiliation":[{"name":"Air Force Research Laboratory, WPAFB, Dayton, OH 45433, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,4,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.neuroimage.2011.07.094","article-title":"Cross-subject workload classification with a hierarchical Bayes model","volume":"59","author":"Wang","year":"2012","journal-title":"NeuroImage"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1109\/THMS.2014.2366914","article-title":"Recognition of mental workload levels under complex human\u2013machine collaboration by using physiological features and adaptive support vector machines","volume":"45","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"19","DOI":"10.3389\/fnbot.2017.00019","article-title":"Cross-subject EEG feature selection for emotion recognition using transfer recursive feature elimination","volume":"11","author":"Yin","year":"2017","journal-title":"Front. Neurorobot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1518\/hfes.45.4.635.27088","article-title":"Real-time assessment of mental workload using psychophysiological measures and artificial neural networks","volume":"45","author":"Wilson","year":"2003","journal-title":"Hum. Factors"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2713","DOI":"10.1016\/j.cor.2004.03.022","article-title":"Improving pilot mental workload classification through feature exploitation and combination: A feasibility study","volume":"32","author":"Noel","year":"2005","journal-title":"Comput. Oper. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1109\/TSMCA.2002.807036","article-title":"Selection of input features across subjects for classifying crewmember workload using artificial neural networks","volume":"32","author":"Laine","year":"2002","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1518\/001872098779480578","article-title":"Monitoring working memory load during computer-based tasks with EEG pattern recognition methods","volume":"40","author":"Gevins","year":"1998","journal-title":"Hum. Factors"},{"key":"ref_8","first-page":"1","article-title":"IMPRINT: The transition and further development of a soldier-system analysis tool","volume":"5","author":"Allender","year":"1997","journal-title":"Manprint Q."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1177\/1541931215591148","article-title":"Improving model cross-applicability for operator workload estimation","volume":"Volume 59","author":"Smith","year":"2015","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","article-title":"Finding structure in time","volume":"14","author":"Elman","year":"1990","journal-title":"Cogn. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1016\/j.eswa.2005.04.011","article-title":"Recurrent neural networks employing Lyapunov exponents for EEG signals classification","volume":"29","year":"2005","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.dsp.2008.07.007","article-title":"Analysis of EEG signals by implementing eigenvector methods\/recurrent neural networks","volume":"19","year":"2009","journal-title":"Digit. Signal Process."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1109\/TNNLS.2015.2496330","article-title":"Brain dynamics in predicting driving fatigue using a recurrent self-evolving fuzzy neural network","volume":"27","author":"Liu","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_14","unstructured":"Lawhern, V.J., Solon, A.J., Waytowich, N.R., Gordon, S.M., Hung, C.P., and Lance, B.J. (arXiv, 2016). EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces, arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/j.image.2016.05.018","article-title":"EEG-based prediction of driver\u2019s cognitive performance by deep convolutional neural network","volume":"47","author":"Hajinoroozi","year":"2016","journal-title":"Signal Process. Image Commun."},{"key":"ref_16","unstructured":"Bashivan, P., Rish, I., Yeasin, M., and Codella, N. (2016, January 2\u20134). Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks. Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3774","DOI":"10.1111\/ejn.12749","article-title":"Spectrotemporal dynamics of the EEG during working memory encoding and maintenance predicts individual behavioral capacity","volume":"40","author":"Bashivan","year":"2014","journal-title":"Eur. J. Neurosci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw."},{"key":"ref_19","first-page":"178","article-title":"Learning robust features using deep learning for automatic seizure detection","volume":"56","author":"Thodoroff","year":"2016","journal-title":"Mach. Learn. Healthc. Conf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1016\/j.neunet.2009.06.003","article-title":"Subject-independent mental state classification in single trials","volume":"22","author":"Fazli","year":"2009","journal-title":"Neural Netw."},{"key":"ref_21","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Caesars Palace, NV, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_23","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Caesars Palace, NV, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Identity mappings in deep residual networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017). Inception-v4, Inception-Resnet and the Impact of Residual Connections on Learning, AAAI.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_26","unstructured":"Lin, M., Chen, Q., and Yan, S. (arXiv, 2013). Network in network, arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: An overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1162\/089976600300015015","article-title":"Learning to forget: Continual prediction with LSTM","volume":"12","author":"Gers","year":"2000","journal-title":"Neural Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.patrec.2017.05.020","article-title":"Deep long short-term memory structures model temporal dependencies improving cognitive workload estimation","volume":"94","author":"Hefron","year":"2017","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Graves, A. (2012). Neural Networks. Supervised Sequence Labelling with Recurrent Neural Networks, Springer.","DOI":"10.1007\/978-3-642-24797-2"},{"key":"ref_32","unstructured":"Comstock, J.R., and Arnegard, R.J. (1992). The Multi-Attribute Task Battery for Human Operator Workload and Strategic Behavior Research, NASA Technical Memorandum 104174."},{"key":"ref_33","first-page":"16","article-title":"The PREP pipeline: Standardized preprocessing for large-scale EEG analysis","volume":"9","author":"Mullen","year":"2015","journal-title":"Front. Neuroinf."},{"key":"ref_34","unstructured":"Chollet, F. (2017, November 15). Keras. Available online: https:\/\/github.com\/fchollet\/keras."},{"key":"ref_35","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2017, November 15). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, 2015. Available online: https:\/\/www.tensorflow.org."},{"key":"ref_36","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_37","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_38","unstructured":"Bashivan, P., Rish, I., and Heisig, S. (arXiv, 2016). Mental State Recognition via Wearable EEG, arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"359","DOI":"10.3389\/fnhum.2017.00359","article-title":"Measuring Mental Workload with EEG+fNIRS","volume":"11","author":"Aghajani","year":"2017","journal-title":"Front. Hum. Neurosci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"045008","DOI":"10.1088\/1741-2560\/9\/4\/045008","article-title":"Estimating workload using EEG spectral power and ERPs in the n-back task","volume":"9","author":"Brouwer","year":"2012","journal-title":"J. Neural Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1339\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:02:15Z","timestamp":1760194935000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1339"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,26]]},"references-count":40,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,5]]}},"alternative-id":["s18051339"],"URL":"https:\/\/doi.org\/10.3390\/s18051339","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4,26]]}}}