{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T20:20:17Z","timestamp":1778185217521,"version":"3.51.4"},"reference-count":55,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2022,5,24]],"date-time":"2022-05-24T00:00:00Z","timestamp":1653350400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Indiana University Health and the Indiana Clinical and Translational Sciences Institute","award":["ULI TR002529"],"award-info":[{"award-number":["ULI TR002529"]}]},{"DOI":"10.13039\/100010477","name":"Intuitive Surgical","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010477","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Hum Factors"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>The purpose of this study was to identify objective measures that predict surgeon nontechnical skills (NTS) during surgery.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>NTS are cognitive and social skills that impact operative performance and patient outcomes. Current methods for NTS assessment in surgery rely on observation-based tools to rate intraoperative behavior. These tools are resource intensive (e.g., time for observation or manual labeling) to perform; therefore, more efficient approaches are needed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>Thirty-four robotic-assisted surgeries were observed. Proximity sensors were placed on the surgical team and voice recorders were placed on the surgeon. Surgeon NTS was assessed by trained observers using the NonTechnical Skills for Surgeons (NOTSS) tool. NTS behavior metrics from the sensors included communication, speech, and proximity features. The metrics were used to develop mixed effect models to predict NOTSS score and in machine learning classifiers to distinguish between exemplar NTS scores (highest NOTSS score) and non-exemplar scores.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>NTS metrics were collected from 16 nurses, 12 assistants, 11 anesthesiologists, and four surgeons. Nineteen behavior features and overall NOTSS score were significantly correlated (12 communication features, two speech features, five proximity features). The random forest classifier achieved the highest accuracy of 70% (80% F1 score) to predict exemplar NTS score.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Sensor-based measures of communication, speech, and proximity can potentially predict NOTSS scores of surgeons during robotic-assisted surgery. These sensing-based approaches can be utilized for further reducing resource costs of NTS and team performance assessment in surgical environments.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Application<\/jats:title>\n                    <jats:p>Sensor-based assessment of operative teams\u2019 behaviors can lead to objective, real-time NTS measurement.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/00187208221101292","type":"journal-article","created":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T01:22:55Z","timestamp":1653441775000},"page":"729-743","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":13,"title":["Objective Nontechnical Skills Measurement Using Sensor-based Behavior Metrics in Surgical Teams"],"prefix":"10.1177","volume":"66","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8136-2094","authenticated-orcid":false,"given":"Jackie S.","family":"Cha","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering, Clemson University, SC, USA"},{"name":"School of Industrial Engineering, Purdue University, West Lafayette, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitrios I.","family":"Athanasiadis","sequence":"additional","affiliation":[{"name":"Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuhao","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Industrial Engineering, Purdue University, West Lafayette, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Industrial Engineering, Purdue University, West Lafayette, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicholas E.","family":"Anton","sequence":"additional","affiliation":[{"name":"Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitrios","family":"Stefanidis","sequence":"additional","affiliation":[{"name":"Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5083-1713","authenticated-orcid":false,"given":"Denny","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Industrial Engineering, Purdue University, West Lafayette, In, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2022,5,24]]},"reference":[{"key":"bibr1-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1111\/bju.13426"},{"key":"bibr2-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1080\/00140130050084888"},{"key":"bibr3-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2013.12.002"},{"key":"bibr5-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2009.10.007"},{"key":"bibr6-00187208221101292","volume-title":"lme4: Linear mixed-effects models using S4 classes","author":"Bates D.","year":"2012"},{"key":"bibr4-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.pragma.2014.02.011"},{"key":"bibr7-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1518\/001872098779649265"},{"key":"bibr8-00187208221101292","first-page":"1","volume":"45","author":"Buuren S. v.","year":"2010","journal-title":"Journal of Statistical Software"},{"key":"bibr9-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2018.02.011"},{"key":"bibr10-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-015-4671-2"},{"key":"bibr11-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1177\/0018720821995319"},{"key":"bibr18-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2021.103559"},{"key":"bibr12-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1001\/jamasurg.2019.1738"},{"key":"bibr13-00187208221101292","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.41.2.385"},{"key":"bibr14-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2019.8857691"},{"key":"bibr15-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93846-2_14"},{"key":"bibr16-00187208221101292","volume-title":"Safety at the sharp end: A guide to non-technical skills","author":"Flin R.","year":"2013"},{"key":"bibr17-00187208221101292","volume-title":"Datasheets for datasets","author":"Gebru T.","year":"2020"},{"key":"bibr19-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2013-003525"},{"key":"bibr20-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7_7"},{"key":"bibr21-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2007.02.001"},{"key":"bibr22-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2018-007957"},{"key":"bibr23-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2018.02.021"},{"key":"bibr24-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcm.2016.02.012"},{"key":"bibr25-00187208221101292","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v028.i05"},{"key":"bibr26-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1177\/0261927X03258084"},{"key":"bibr27-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1121\/1.405815"},{"key":"bibr28-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1177\/002383098102400105"},{"key":"bibr29-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2008.03.002"},{"key":"bibr30-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.10.012"},{"key":"bibr31-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/S0892-1997(98)80017-9"},{"key":"bibr32-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2010.04.002"},{"key":"bibr33-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsurg.2019.03.017"},{"key":"bibr34-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1121\/1.418247"},{"key":"bibr35-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-017-5622-x"},{"key":"bibr36-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s10111-016-0368-0"},{"key":"bibr37-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0204819"},{"key":"bibr38-00187208221101292","volume-title":"Automatic detection and classification of prosodic events","author":"Rosenberg A.","year":"2009"},{"key":"bibr39-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/S1553-7250(08)34042-2"},{"key":"bibr40-00187208221101292","first-page":"1","volume":"12","author":"Scherer S.","year":"2012","journal-title":"Proceedings of the 1st International Workshop on Multimodal Learning Analytics - MLA"},{"key":"bibr41-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-012-2287-3"},{"key":"bibr42-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2017-006701"},{"key":"bibr43-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.surge.2014.06.005"},{"key":"bibr44-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1002\/sam.11348"},{"key":"bibr45-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.urology.2018.07.059"},{"key":"bibr46-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.02.034"},{"key":"bibr47-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2018.02.015"},{"key":"bibr48-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsurg.2015.12.009"},{"key":"bibr49-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1016\/j.urology.2017.11.060"},{"key":"bibr50-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0000000000001051"},{"key":"bibr51-00187208221101292","first-page":"459","volume":"27","author":"Yerkes R. M.","year":"1908","journal-title":"Punishment: Issues and Experiments"},{"key":"bibr52-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-016-5047-y"},{"key":"bibr53-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1080\/00140139.2014.884244"},{"key":"bibr54-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00268-007-9320-z"},{"key":"bibr55-00187208221101292","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-006-9090-y"}],"container-title":["Human Factors: The Journal of the Human Factors and Ergonomics Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208221101292","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00187208221101292","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208221101292","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T07:53:19Z","timestamp":1777449199000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00187208221101292"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,24]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10.1177\/00187208221101292"],"URL":"https:\/\/doi.org\/10.1177\/00187208221101292","relation":{},"ISSN":["0018-7208","1547-8181"],"issn-type":[{"value":"0018-7208","type":"print"},{"value":"1547-8181","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,24]]}}}