{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T14:08:34Z","timestamp":1773842914609,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T00:00:00Z","timestamp":1673395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The cardiopulmonary exercise test (CPET) constitutes a gold standard for the assessment of an individual\u2019s cardiovascular fitness. A trend is emerging for the development of new machine-learning techniques applied to the automatic process of CPET data. Some of these focus on the precise task of detecting the exercise thresholds, which represent important physiological parameters. Three are the major challenges tackled by this contribution: (A) regression (i.e., the process of correctly identifying the exercise intensity domains and their crossing points); (B) generation (i.e., the process of artificially creating a CPET data file ex-novo); and (C) explanation (i.e., proving an interpretable explanation about the output of the machine learning model). The following methods were used for each challenge: (A) a convolutional neural network adapted for multi-variable time series; (B) a conditional generative adversarial neural network; and (C) visual explanations and calculations of model decisions have been conducted using cooperative game theory (Shapley\u2019s values). The results for the regression, generation, and explanatory techniques for AI-assisted CPET interpretation are presented here in a unique framework for the first time: (A) machine learning techniques reported an expert-level accuracy in the classification of exercise intensity domains; (B) experts are not able to substantially differentiate between a real vs an artificially generated CPET; and (C) Shapley\u2019s values can provide an explanation about the choices of the algorithms in terms of ventilatory variables. With the aim to increase their technology-readiness level, all the models discussed in this contribution have been incorporated into a free-to-use Python package called pyoxynet (ver. 12.1). This contribution should therefore be of interest to major players operating in the CPET device market and engineering.<\/jats:p>","DOI":"10.3390\/s23020826","type":"journal-article","created":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T04:59:58Z","timestamp":1673413198000},"page":"826","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Machine Learning Models for the Automatic Detection of Exercise Thresholds in Cardiopulmonary Exercising Tests: From Regression to Generation to Explanation"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1315-5573","authenticated-orcid":false,"given":"Andrea","family":"Zignoli","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering, University of Trento, 38123 Trento, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,11]]},"reference":[{"key":"ref_1","unstructured":"Wasserman, K., Hansen, J.E., Sue, D.Y., Stringer, W.W., and Whipp, B.J. (2005). Principles of Exercise Testing and Interpretation: Including Pathophysiology and Clinical Applications, Lippincott Williams & Wilkins Philadelphia."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"575","DOI":"10.2165\/00007256-200737070-00002","article-title":"Incremental Exercise Test Design and Analysis: Implications for Performance Diagnostics in Endurance Athletes","volume":"37","author":"Bentley","year":"2007","journal-title":"Sport. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1542","DOI":"10.1249\/01.mss.0000177466.01232.7e","article-title":"Oxygen uptake dynamics: From muscle to mouth\u2013an introduction to the symposium","volume":"37","author":"Jones","year":"2005","journal-title":"Med. Sci. Sport. Exerc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s40279-021-01581-z","article-title":"Identification of Non-Invasive Exercise Thresholds: Methods, Strategies, and an Online App","volume":"52","author":"Keir","year":"2022","journal-title":"Sport. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/s13741-017-0082-3","article-title":"Cardiopulmonary exercise testing (CPET) in the United Kingdom\u2014A national survey of the structure, conduct, interpretation and funding","volume":"7","author":"Reeves","year":"2018","journal-title":"Perioper. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1159\/000195180","article-title":"Sources of Error and Variability in the Determination of Anaerobic Threshold in Healthy Humans","volume":"51","author":"Garrard","year":"1987","journal-title":"Respiration"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1080\/02640418408729692","article-title":"Reliability of assessments of ventilatory thresholds","volume":"2","author":"Bouchard","year":"1984","journal-title":"J. Sport. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2082","DOI":"10.1152\/jappl.1985.58.6.2082","article-title":"Gas exchange and lactate anaerobic thresholds: Inter- and intraevaluator agreement","volume":"58","author":"Gladden","year":"1985","journal-title":"J. Appl. Physiol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1080\/02640410801910269","article-title":"Do regression-based computer algorithms for determining the ventilatory threshold agree?","volume":"26","author":"Ekkekakis","year":"2008","journal-title":"J. Sport. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e005193","DOI":"10.1161\/CIRCHEARTFAILURE.118.005193","article-title":"Neural Networks for Prognostication of Patients with Heart Failure: Improving Performance Through the Incorporation of Breath-by-Breath Data From Cardiopulmonary Exercise Testing","volume":"11","author":"Hearn","year":"2018","journal-title":"Circ. Heart Fail."},{"key":"ref_11","unstructured":"Christ, M., Kempa-Liehr, A.W., and Feindt, M. (2016). Distributed and parallel time series feature extraction for industrial big data applications. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5516248","DOI":"10.1155\/2021\/5516248","article-title":"A Machine Learning Approach to the Interpretation of Cardiopulmonary Exercise Tests: Development and Validation","volume":"2021","author":"Inbar","year":"2021","journal-title":"Pulm. Med."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sharma, Y., Coronato, N., and Brown, D.E. (2022). Encoding Cardiopulmonary Exercise Testing Time Series as Images for Classification using Convolutional Neural Network. arXiv.","DOI":"10.1109\/EMBC48229.2022.9871878"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2508037.2508051","article-title":"Early prediction of the highest workload in incremental cardiopulmonary tests","volume":"4","author":"Baralis","year":"2013","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1221","DOI":"10.1080\/17461391.2019.1587523","article-title":"Expert-level classification of ventilatory thresholds from cardiopulmonary exercising test data with recurrent neural networks","volume":"19","author":"Zignoli","year":"2019","journal-title":"Eur. J. Sport Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1080\/17461391.2020.1866081","article-title":"Oxynet: A collective intelligence that detects ventilatory thresholds in cardiopulmonary exercise tests","volume":"22","author":"Zignoli","year":"2021","journal-title":"Eur. J. Sport Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zignoli, A., and Fruet, D. (2022, January 6\u20138). Automatic generation of realistic cardiopulmonary exercise test data with a conditional generative adversarial neural network. Proceedings of the 2022 IEEE International Workshop on Sport, Technology and Research (STAR), Trento-Cavalese, Italy.","DOI":"10.1109\/STAR53492.2022.9859993"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2020","DOI":"10.1152\/jappl.1986.60.6.2020","article-title":"A new method for detecting anaerobic threshold by gas exchange","volume":"60","author":"Beaver","year":"1986","journal-title":"J. Appl. Physiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1249\/00005768-199310000-00015","article-title":"The simplified V-slope method of detecting the gas exchange threshold","volume":"25","author":"Schneider","year":"1993","journal-title":"Med. Sci. Sport. Exerc."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1093\/jcr\/ucz013","article-title":"Resistance to Medical Artificial Intelligence","volume":"46","author":"Longoni","year":"2019","journal-title":"J. Consum. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1038\/s41591-022-01833-z","article-title":"Evaluating and reducing cognitive load should be a priority for machine learning in healthcare","volume":"28","author":"Ehrmann","year":"2022","journal-title":"Nat. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"l886","DOI":"10.1136\/bmj.l886","article-title":"Clinical applications of machine learning algorithms: Beyond the black box","volume":"364","author":"Watson","year":"2019","journal-title":"BMJ"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Commun. ACM"},{"key":"ref_24","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv."},{"key":"ref_25","unstructured":"Smith, K.E., and Smith, A.O. (2020). Conditional GAN for timeseries generation. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1002\/asmb.446","article-title":"Analysis of regression in game theory approach","volume":"17","author":"Lipovetsky","year":"2001","journal-title":"Appl. Stoch. Model. Bus. Ind."},{"key":"ref_27","unstructured":"Shrikumar, A., Greenside, P., and Kundaje, A. (2017). Learning Important Features Through Propagating Activation Differences. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: A review","volume":"33","author":"Forestier","year":"2019","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.patrec.2014.01.008","article-title":"A review of unsupervised feature learning and deep learning for time-series modeling","volume":"42","author":"Karlsson","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat. Biomed. Eng."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1016\/S0140-6736(22)00235-5","article-title":"Bridging the chasm between AI and clinical implementation","volume":"399","author":"Aristidou","year":"2022","journal-title":"Lancet"},{"key":"ref_33","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jablonski, J.A., Angadi, S.S., Sharma, S., and Brown, D.E. (2022, January 10\u201311). Enabling clinically relevant and interpretable deep learning models for cardiopulmonary exercise testing. Proceedings of the 2022 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT), Houston, TX, USA.","DOI":"10.1109\/HI-POCT54491.2022.9744068"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4228","DOI":"10.1109\/JBHI.2022.3163402","article-title":"Using machine learning to identify organ system specific limitations to exercise via cardiopulmonary exercise testing","volume":"26","author":"Portella","year":"2022","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Baralis, E., Cerquitelli, T., Chiusano, S., Giordano, A., Mezzani, A., Susta, D., and Xiao, X. (2015, January 22\u201325). Predicting cardiopulmonary response to incremental exercise test. Proceedings of the 2015 IEEE 28th International Symposium on Computer-Based Medical Systems, Sao Carlos, Brazil.","DOI":"10.1109\/CBMS.2015.60"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/826\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:06:39Z","timestamp":1760119599000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/826"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,11]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020826"],"URL":"https:\/\/doi.org\/10.3390\/s23020826","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,11]]}}}