{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T17:01:31Z","timestamp":1789750891792,"version":"4.0.1"},"reference-count":29,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2023,8,12]],"date-time":"2023-08-12T00:00:00Z","timestamp":1691798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LGF20H040008"],"award-info":[{"award-number":["LGF20H040008"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LQ21F010016"],"award-info":[{"award-number":["LQ21F010016"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["2021C03027"],"award-info":[{"award-number":["2021C03027"]}]},{"name":"Zhejiang Provincial Key Research and Development Program of China","award":["LGF20H040008"],"award-info":[{"award-number":["LGF20H040008"]}]},{"name":"Zhejiang Provincial Key Research and Development Program of China","award":["LQ21F010016"],"award-info":[{"award-number":["LQ21F010016"]}]},{"name":"Zhejiang Provincial Key Research and Development Program of China","award":["2021C03027"],"award-info":[{"award-number":["2021C03027"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Neonatal pain assessment (NPA) represents a huge global problem of essential importance, as a timely and accurate assessment of neonatal pain is indispensable for implementing pain management. Purpose: To investigate the consistency of pain scores derived through video-based NPA (VB-NPA) and on-site NPA (OS-NPA), providing the scientific foundation and feasibility of adopting VB-NPA results in a real-world scenario as the gold standard for neonatal pain in clinical studies and labels for artificial intelligence (AI)-based NPA (AI-NPA) applications. Setting: A total of 598 neonates were recruited from a pediatric hospital in China. Methods: This observational study recorded 598 neonates who underwent one of 10 painful procedures, including arterial blood sampling, heel blood sampling, fingertip blood sampling, intravenous injection, subcutaneous injection, peripheral intravenous cannulation, nasopharyngeal suctioning, retention enema, adhesive removal, and wound dressing. Two experienced nurses performed OS-NPA and VB-NPA at a 10-day interval through double-blind scoring using the Neonatal Infant Pain Scale to evaluate the pain level of the neonates. Intra-rater and inter-rater reliability were calculated and analyzed, and a paired samples t-test was used to explore the bias and consistency of the assessors\u2019 pain scores derived through OS-NPA and VB-NPA. The impact of different label sources was evaluated using three state-of-the-art AI methods trained with labels given by OS-NPA and VB-NPA, respectively. Results: The intra-rater reliability of the same assessor was 0.976\u20130.983 across different times, as measured by the intraclass correlation coefficient. The inter-rater reliability was 0.983 for single measures and 0.992 for average measures. No significant differences were observed between the OS-NPA scores and the assessment of an independent VB-NPA assessor. The different label sources only caused a limited accuracy loss of 0.022\u20130.044 for the three AI methods. Conclusion: VB-NPA in a real-world scenario is an effective way to assess neonatal pain due to its high intra-rater and inter-rater reliability compared to OS-NPA and could be used for the labeling of large-scale NPA video databases for clinical studies and AI training.<\/jats:p>","DOI":"10.3390\/diagnostics13162661","type":"journal-article","created":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T10:28:13Z","timestamp":1692008893000},"page":"2661","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Video-Based versus On-Site Neonatal Pain Assessment in Neonatal Intensive Care Units: The Impact of Video-Based Neonatal Pain Assessment in Real-World Scenario on Pain Diagnosis and Its Artificial Intelligence Application"],"prefix":"10.3390","volume":"13","author":[{"given":"Xiaofei","family":"Chen","sequence":"first","affiliation":[{"name":"Gastroenterology Department, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6918-4088","authenticated-orcid":false,"given":"Huaiyu","family":"Zhu","sequence":"additional","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linli","family":"Mei","sequence":"additional","affiliation":[{"name":"Administration Department of Nosocomial Infection, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Shu","sequence":"additional","affiliation":[{"name":"Administration Department of Nosocomial Infection, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoying","family":"Cheng","sequence":"additional","affiliation":[{"name":"Quality Improvement Office, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feixiang","family":"Luo","sequence":"additional","affiliation":[{"name":"Neonatal Intensive Care Unit, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0763-3658","authenticated-orcid":false,"given":"Yisheng","family":"Zhao","sequence":"additional","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuohui","family":"Chen","sequence":"additional","affiliation":[{"name":"Administration Department of Nosocomial Infection, The Children\u2019s Hospital, Zhejiang University School of Medicine, National Clinical Research Center for Child Health, Hangzhou 310052, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9335-4291","authenticated-orcid":false,"given":"Yun","family":"Pan","sequence":"additional","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1001\/jama.300.1.60","article-title":"Epidemiology and treatment of painful procedures in neonates in intensive care units","volume":"300","author":"Carbajal","year":"2008","journal-title":"JAMA"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.ijnurstu.2016.03.014","article-title":"Epidemiology and neonatal pain management of heelsticks in intensive care units: EPIPPAIN 2, a prospective observational study","volume":"59","author":"Courtois","year":"2016","journal-title":"Int. J. Nurs. Stud."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1002\/ejp.757","article-title":"Epidemiology of painful procedures performed in neonates: A systematic review of observational studies","volume":"20","author":"Cruz","year":"2016","journal-title":"Eur. J. Pain"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"975","DOI":"10.1097\/AJP.0000000000000762","article-title":"Acute pain assessment in prematurely born infants below 29 weeks: A long way to go","volume":"35","author":"Meesters","year":"2019","journal-title":"Clin. J. Pain"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1159\/000013953","article-title":"Clinical importance of pain and stress in preterm neonates","volume":"73","author":"Anand","year":"1998","journal-title":"Neonatology"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1016\/j.jpain.2012.03.010","article-title":"Procedural pain and oxidative stress in premature neonates","volume":"13","author":"Slater","year":"2012","journal-title":"J. Pain"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101005","DOI":"10.1016\/j.siny.2019.04.005","article-title":"Long-term effects of neonatal pain","volume":"24","author":"Walker","year":"2019","journal-title":"Semin. Fetal Neonatal Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/j.cnc.2018.07.013","article-title":"Neonatal pain: Perceptions and current practice","volume":"30","author":"Perry","year":"2018","journal-title":"Crit. Care Nurs. Clin."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e2931","DOI":"10.1590\/1518-8345.1665.2931","article-title":"Assessment and management of pain in newborns hospitalized in a neonatal intensive care unit: A cross-sectional study","volume":"25","author":"Sposito","year":"2017","journal-title":"Rev. Lat.-Am. Enferm."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1155\/2009\/856587","article-title":"Putting pain assessment into practice: Why is it so painful?","volume":"14","author":"Franck","year":"2009","journal-title":"Pain Res. Manag."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1109\/RBME.2017.2777907","article-title":"A review of automated pain assessment in infants: Features, classification tasks, and databases","volume":"11","author":"Zamzmi","year":"2017","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/S0029-6465(22)02475-6","article-title":"Managing pain: The fifth vital sign","volume":"35","author":"Merboth","year":"2000","journal-title":"Nurs. Clin. N. Am."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"746489","DOI":"10.3389\/fped.2021.746489","article-title":"Pain assessment and management in pediatric intensive care units around the world, an international, multicenter study","volume":"9","author":"Grunauer","year":"2021","journal-title":"Front. Pediatr."},{"key":"ref_14","first-page":"59","article-title":"The development of a tool to assess neonatal pain","volume":"12","author":"Lawrence","year":"1993","journal-title":"Neonatal Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.jpainsymman.2015.03.019","article-title":"Neonatal infant pain scale: Cross-cultural adaptation and validation in Brazil","volume":"50","author":"Schardosim","year":"2015","journal-title":"J. Pain Symptom Manag."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Napi\u00f3rkowska-Orkisz, M., Gutysz-Wojnicka, A., Tanajewska, M., and Sadowska-Krawczenko, I. (2022). Evaluation of methods to minimize pain in newborns during capillary blood sampling for screening: A randomized clinical trial. Int. J. Environ. Res. Public Health, 19.","DOI":"10.3390\/ijerph19020870"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1093\/bja\/ael184","article-title":"A comparison of postoperative pain scales in neonates","volume":"97","author":"Suraseranivongse","year":"2006","journal-title":"Br. J. Anaesth."},{"key":"ref_18","first-page":"637","article-title":"Effects of glucose administering approaches on reducing neonatal pain during heel lance procedures","volume":"46","author":"Yao","year":"2011","journal-title":"Chin. J. Nurs."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/TAFFC.2019.2926710","article-title":"A comprehensive and context-sensitive neonatal pain assessment using computer vision","volume":"13","author":"Zamzmi","year":"2019","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_20","first-page":"1159717","article-title":"Video-based infant monitoring using a CNN-LSTM scheme","volume":"11597","author":"Min","year":"2021","journal-title":"Proc. SPIE"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"104150","DOI":"10.1016\/j.compbiomed.2020.104150","article-title":"Multimodal spatio-temporal deep learning approach for neonatal postoperative pain assessment","volume":"129","author":"Salekin","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1016\/j.jsurg.2017.10.005","article-title":"Direct observation vs. video-based assessment in flexible cystoscopy","volume":"75","author":"Mahmood","year":"2018","journal-title":"J. Surg. Educ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1002\/pne2.12060","article-title":"Future roles of artificial intelligence in early pain management of newborns","volume":"3","author":"Salekin","year":"2021","journal-title":"Paediatr. Neonatal Pain"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cheng, X., Zhu, H., Mei, L., Luo, F., Chen, X., Zhao, Y., Chen, S., and Pan, Y. (2022). Artificial intelligence based pain assessment technology in clinical application of real-world neonatal blood sampling. Diagnostics, 12.","DOI":"10.3390\/diagnostics12081831"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"766","DOI":"10.1016\/j.gie.2017.08.020","article-title":"A prospective comparison of live and video-based assessments of colonoscopy performance","volume":"87","author":"Scaffidi","year":"2018","journal-title":"Gastrointest. Endosc."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sun, Y., Kommers, D., Wang, W., Joshi, R., Shan, C., Tan, T., Aarts, R.M., van Pul, C., Andriessen, P., and de With, P.H.N. (2019, January 23\u201327). Automatic and continuous discomfort detection for premature infants in a NICU using video-based motion analysis. Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8857597"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e623","DOI":"10.1016\/S2589-7500(21)00129-1","article-title":"Assessing procedural pain in infants: A feasibility study evaluating a point-of-care mobile solution based on automated facial analysis","volume":"3","author":"Hoti","year":"2021","journal-title":"Lancet Digit. Health"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1016\/j.ijporl.2006.11.009","article-title":"The newborn pain cry: Descriptive acoustic spectrographic analysis","volume":"71","author":"Branco","year":"2007","journal-title":"Int. J. Pediatr. Otorhinolaryngol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3429447","article-title":"Mitigating class-boundary label uncertainty to reduce both model bias and variance","volume":"15","author":"Almeida","year":"2021","journal-title":"ACM Trans. Knowl. Discov. Data"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/16\/2661\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:32:20Z","timestamp":1760128340000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/16\/2661"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,12]]},"references-count":29,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["diagnostics13162661"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics13162661","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,12]]}}}