{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T10:32:27Z","timestamp":1769855547366,"version":"3.49.0"},"reference-count":23,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T00:00:00Z","timestamp":1658534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["MOST 110-2221-E-155-004-MY2"],"award-info":[{"award-number":["MOST 110-2221-E-155-004-MY2"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>There are many surgical operations performed daily in operation rooms worldwide. Adequate anesthesia is needed during an operation. Besides hypnosis, adequate analgesia is critical to prevent autonomic reactions. Clinical experience and vital signs are usually used to adjust the dosage of analgesics. Analgesia nociception index (ANI), which ranges from 0 to 100, is derived from heart rate variability (HRV) via electrocardiogram (ECG) signals, for pain evaluation in a non-invasive manner. It represents parasympathetic activity. In this study, we compared the performance of multilayer perceptron (MLP) and long short-term memory (LSTM) algorithms in predicting expert assessment of pain score (EAPS) based on patient\u2032s HRV during surgery. The objective of this study was to analyze how deep learning models differed from the medical doctors\u2032 predictions of EAPS. As the input and output features of the deep learning models, the opposites of ANI and EAPS were used. This study included 80 patients who underwent operations at National Taiwan University Hospital. Using MLP and LSTM, a holdout method was first applied to 60 training patients, 10 validation patients, and 10 testing patients. As compared to the LSTM model, which had a testing mean absolute error (MAE) of 2.633 \u00b1 0.542, the MLP model had a testing MAE of 2.490 \u00b1 0.522, with a more appropriate shape of its prediction curves. The model based on MLP was selected as the best. Using MLP, a seven-fold cross validation method was then applied. The first fold had the lowest testing MAE of 2.460 \u00b1 0.634, while the overall MAE for the seven-fold cross validation method was 2.848 \u00b1 0.308. In conclusion, HRV analysis using MLP algorithm had a good correlation with EAPS; therefore, it can play role as a continuous monitor to predict intraoperative pain levels, to assist physicians in adjusting analgesic agent dosage. Further studies may consider obtaining more input features, such as photoplethysmography (PPG) and other kinds of continuous variable, to improve the prediction performance.<\/jats:p>","DOI":"10.3390\/s22155496","type":"journal-article","created":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T04:52:47Z","timestamp":1658724767000},"page":"5496","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Comparison of Deep Learning Algorithms in Predicting Expert Assessments of Pain Scores during Surgical Operations Using Analgesia Nociception Index"],"prefix":"10.3390","volume":"22","author":[{"given":"Wei-Horng","family":"Jean","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Yuan Ze University, Taoyuan 320, Taiwan"},{"name":"Department of Anesthesiology, Far Eastern Memorial Hospital, Banqiao District, New Taipei City 220, Taiwan"}]},{"given":"Peter","family":"Sutikno","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Yuan Ze University, Taoyuan 320, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6849-8453","authenticated-orcid":false,"given":"Shou-Zen","family":"Fan","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, College of Medicine, National Taiwan University, Taipei 100, Taiwan"},{"name":"Department of Anesthesiology, En Chu Kong Hospital, New Taipei City 237, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8515-7933","authenticated-orcid":false,"given":"Maysam F.","family":"Abbod","sequence":"additional","affiliation":[{"name":"Department of Electronics and Electrical Engineering, Brunel University London, London UB8 3PH, UK"}]},{"given":"Jiann-Shing","family":"Shieh","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Yuan Ze University, Taoyuan 320, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s10877-012-9354-0","article-title":"Variations of the analgesia nociception index during general anaesthesia for laparoscopic abdominal surgery","volume":"26","author":"Jeanne","year":"2012","journal-title":"Int. J. Clin. Monit. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, J., Abbod, M., and Shieh, J.-S. (2021). Pain and stress detection using wearable sensors and devices\u2014A review. Sensors, 21.","DOI":"10.3390\/s21041030"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1093\/bja\/aet019","article-title":"Influence of nociceptive stimulation on analgesia nociception index (ANI) during propofol\u2013remifentanil anaesthesia","volume":"110","author":"Gruenewald","year":"2013","journal-title":"Br. J. Anaesth."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Logier, R., Jeanne, M., De Jonckheere, J., Dassonneville, A., Delecroix, M., and Tavernier, B. (September, January 31). PhysioDoloris: A Monitoring Device for Analgesia\/Nociception Balance Evaluation Using Heart Rate Variability Analysis. Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina.","DOI":"10.1109\/IEMBS.2010.5625971"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1016\/j.bjan.2017.03.004","article-title":"Analgesia nociception index for perioperative analgesia monitoring in spinal surgery","volume":"67","author":"Turan","year":"2017","journal-title":"Rev. Bras. De Anestesiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1016\/j.bjan.2019.01.003","article-title":"Analgesia Nociception Index: Assessment of acute postoperative pain","volume":"69","author":"Abdullayev","year":"2019","journal-title":"Rev. Bras. De Anestesiol."},{"key":"ref_7","first-page":"e4365","article-title":"Analgesia nociception index: Heart rate variability analysis of emotional status","volume":"11","author":"Abdullayev","year":"2019","journal-title":"Cureus"},{"key":"ref_8","unstructured":"Shapiro, S.C. (2003). Artificial Intelligence (AI) in Encyclopedia of Computer Science, John Wiley and Sons Ltd."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1308\/147870804290","article-title":"Artificial Intelligence in Medicine","volume":"86","author":"Ramesh","year":"2004","journal-title":"Ann. R. Coll. Surg. Engl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"de Silva, I.N., Spatti, D.H., Flauzino, R.A., Liboni, L.H.B., and dos Reis Alves, S.F. (2017). Artificial Neural Networks: A Practical Course, Springer.","DOI":"10.1007\/978-3-319-43162-8"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_12","unstructured":"Brownlee, J. (2017). Long Short-Term Memory Networks with Python: Develop Sequence Prediction Models with Deep Learning, Machine Learning Mastery."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zebin, T., Sperrin, M., Peek, N., and Casson, A.J. (2018, January 18\u201321). Human Activity Recognition from Inertial Sensor Time-Series Using Batch Normalized Deep LSTM Recurrent Networks. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8513115"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1111\/j.1399-6576.2011.02639.x","article-title":"Effect of a fluid challenge on the Surgical Pleth Index during stable propofol-remifentanil anaesthesia","volume":"56","author":"Hans","year":"2012","journal-title":"Acta Anaesthesiol. Scand."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hu, S.B., Wong, D.J.L., Correa, A., Li, N., and Deng, J.C. (2016). Prediction of clinical deterioration in hospitalized adult patients with hematologic malignancies using a neural network model. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0161401"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"536863","DOI":"10.1155\/2015\/536863","article-title":"Computational Depth of Anesthesia via Multiple Vital Signs Based on Artificial Neural Networks","volume":"2015","author":"Sadrawi","year":"2015","journal-title":"BioMed. Res. Int."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e101585","DOI":"10.5812\/amh.101585","article-title":"A novel classification method based on multilayer perceptron-artificial neural network technique for diagnosis of chronic kidney disease","volume":"18","author":"Sharifi","year":"2020","journal-title":"Ann. Mil. Health Sci. Res."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, J., Chae, M., Chang, H.-J., Kim, Y.-A., and Park, E. (2019). Predicting Cardiac Arrest and Respiratory Failure Using Feasible Artificial Intelligence with Simple Trajectories of Patient Data. J. Clin. Med., 8.","DOI":"10.3390\/jcm8091336"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sideris, C., Kalantarian, H., Nemati, E., and Sarrafzadeh, M. (2016, January 18\u201320). Building Continuous Arterial Blood Pressure Prediction Models Using Recurrent Networks. Proceedings of the 2016 IEEE International Conference on Smart Computing (SMARTCOMP), St. Louis, MO, USA.","DOI":"10.1109\/SMARTCOMP.2016.7501681"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"343478","DOI":"10.1155\/2015\/343478","article-title":"Sample entropy analysis of EEG signals via artificial neural networks to model patients\u2019 consciousness level based on anesthesiologists experience","volume":"2015","author":"Jiang","year":"2015","journal-title":"BioMed. Res. Int."},{"key":"ref_21","unstructured":"Logier, R., Jonckheere, J.D., and Dassonneville, A. (2004, January 1\u20135). An Efficient Algorithm for R-R Intervals Series Filtering. Proceedings of the 26th Annual International Conference of the IEEE EMBS, San Francisco, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jeanne, M., Logier, R., Jonckheere, J.D., and Tavernier, B. (2009, January 2\u20136). Validation of A Graphic Measurement of Heart Rate Variability to Assess Analgesia\/Nociception Balance during General Anesthesia. Proceedings of the 31st Annual International Conference of the IEEE EMBS, Minnesota, MN, USA.","DOI":"10.1109\/IEMBS.2009.5332598"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jonckheere, J.D., Rommel, D., Nandrino, J.L., Jeanne, M., and Logier, R. (2012, January 10). Heart Rate Variability Analysis as an Index of Emotion Regulation Processes: Interest of the Analgesia Nociception Index (ANI). Proceedings of the 34th Annual International Conference of the IEEE EMBS, San Diego, CA, USA.","DOI":"10.1109\/EMBC.2012.6346703"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5496\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:55:21Z","timestamp":1760140521000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/15\/5496"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,23]]},"references-count":23,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22155496"],"URL":"https:\/\/doi.org\/10.3390\/s22155496","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,23]]}}}