{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T21:10:02Z","timestamp":1777669802346,"version":"3.51.4"},"reference-count":87,"publisher":"SAGE Publications","issue":"7","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR003015"],"award-info":[{"award-number":["UL1TR003015"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Hum Factors"],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to compare the relative sensitivity between scene-independent and scene-dependent eye metrics in assessing trainees' performance in simulated psychomotor tasks.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Eye metrics have been extensively studied for skill assessment and training in psychomotor tasks, including aviation, driving, and surgery. These metrics can be categorized as scene-independent or scene-dependent, based on whether predefined areas of interest are considered. There is a paucity of direct comparisons between these metric types, particularly in their ability to assess performance during early training.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>Thirteen medical students practiced the peg transfer task in the Fundamentals of Laparoscopic Surgery. Scene-independent and scene-dependent eye metrics, completion time, and tool motion metrics were derived from eye-tracking data and task videos. K-means clustering of nine eye metrics identified three groups of practice trials with similar gaze behaviors, corresponding to three performance levels verified by completion time and tool motion metrics. A random forest model using eye metrics estimated classification accuracy and determined the feature importance of the eye metrics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Scene-dependent eye metrics demonstrated a clearer linear trend with performance levels than scene-independent metrics. The random forest model achieved 88.59% accuracy, identifying the top four predictors of performance as scene-dependent metrics, whereas the two least effective predictors were scene-independent metrics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Scene-dependent eye metrics are overall more sensitive than scene-independent ones for assessing trainee performance in simulated psychomotor tasks.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Application<\/jats:title>\n                    <jats:p>The study\u2019s findings are significant for advancing eye metrics in psychomotor skill assessment and training, enhancing operator competency, and promoting safe operations.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/00187208241302475","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T17:40:42Z","timestamp":1732815642000},"page":"673-694","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["Comparison Between Scene-Independent and Scene-Dependent Eye Metrics in Assessing Psychomotor Skills"],"prefix":"10.1177","volume":"67","author":[{"given":"Shiyu","family":"Deng","sequence":"first","affiliation":[{"name":"Virginia Tech, USA"}]},{"given":"Chaitanya","family":"Kulkarni","sequence":"additional","affiliation":[{"name":"Virginia Tech, USA"}]},{"given":"Jinwoo","family":"Oh","sequence":"additional","affiliation":[{"name":"Virginia Tech, USA"}]},{"given":"Sarah Henrickson","family":"Parker","sequence":"additional","affiliation":[{"name":"Virginia Tech Carilion School of Medicine, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2235-9527","authenticated-orcid":false,"given":"Nathan","family":"Lau","sequence":"additional","affiliation":[{"name":"Virginia Tech, USA"}]}],"member":"179","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"e_1_3_4_2_1","doi-asserted-by":"publisher","DOI":"10.3390\/s23063146"},{"issue":"3","key":"e_1_3_4_3_1","first-page":"295","article-title":"Surgical task and skill classification from eye tracking and tool motion in minimally invasive surgery","volume":"13","author":"Ahmidi N.","year":"2010","unstructured":"Ahmidi N., Hager G. D., Ishii L., Fichtinger G., Gallia G. L., Ishii M. (2010). Surgical task and skill classification from eye tracking and tool motion in minimally invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention, 13(PART 3), 295\u2013302. https:\/\/doi.org\/10.1007\/978-3-642-15711-0_37","journal-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention"},{"key":"e_1_3_4_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/0142159X.2017.1391373"},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.91.2.276"},{"issue":"7","key":"e_1_3_4_6_1","first-page":"569","article-title":"Visual scanning and pilot expertise: The role of attentional flexibility and mental model development","volume":"68","author":"Bellenkes A. H.","year":"1997","unstructured":"Bellenkes A. H., Wickens C. D., Kramer A. F. (1997). Visual scanning and pilot expertise: The role of attentional flexibility and mental model development. Aviation, Space, and Environmental Medicine, 68(7), 569\u2013579. https:\/\/europepmc.org\/article\/med\/9215461","journal-title":"Aviation, Space, and Environmental Medicine"},{"key":"e_1_3_4_7_1","article-title":"Eye tracking and motion data predict endoscopic sinus surgery skill","author":"Berges A. J.","year":"2022","unstructured":"Berges A. J., Vedula S. S., Chara A., Hager G. D., Ishii M., Malpani A. (2022). Eye tracking and motion data predict endoscopic sinus surgery skill. The Laryngoscope. https:\/\/doi.org\/10.1002\/LARY.30121","journal-title":"The Laryngoscope"},{"key":"e_1_3_4_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpsycho.2020.05.010"},{"key":"e_1_3_4_9_1","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00155.2022"},{"key":"e_1_3_4_10_1","first-page":"604","article-title":"Congruency of gaze metrics in action, imagery and action observation","author":"Causer J.","year":"2013","unstructured":"Causer J., Mccormick S. A., Holmes P. S. (2013). Congruency of gaze metrics in action, imagery and action observation. Frontiers in Human Neuroscience, 604. 0(SEP). https:\/\/doi.org\/10.3389\/FNHUM.2013.00604\/BIBTEX","journal-title":"Frontiers in Human Neuroscience"},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/10508414.2017.1366269"},{"key":"e_1_3_4_12_1","doi-asserted-by":"publisher","DOI":"10.1177\/1071181321651263"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/21695067231192642"},{"key":"e_1_3_4_14_1","doi-asserted-by":"publisher","DOI":"10.1177\/1071181322661156"},{"key":"e_1_3_4_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-016-4851-8"},{"key":"e_1_3_4_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(03)00084-1"},{"key":"e_1_3_4_17_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.102.2.211"},{"key":"e_1_3_4_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/bjs.10672"},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.trf.2016.04.007"},{"key":"e_1_3_4_20_1","first-page":"3533","volume-title":"Virtual pointer for gaze guidance in laparoscopic surgery","author":"Feng Y.","year":"2020","unstructured":"Feng Y., Mcgowan H., Semsar A., Zahiri H. R., George I. M., Park A., Kleinsmith A., Mentis H. (2020). Virtual pointer for gaze guidance in laparoscopic surgery. 34, 3533\u20133539. https:\/\/doi.org\/10.1007\/s00464-019-07141-x"},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-018-6513-5"},{"key":"e_1_3_4_22_1","volume-title":"Human performance","author":"Fitts P. M.,","year":"1967","unstructured":"Fitts P. M.,, Posner M. I. (1967). Human performance. https:\/\/psycnet.apa.org\/record\/1967-35040-000"},{"key":"e_1_3_4_23_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.02285"},{"key":"e_1_3_4_24_1","first-page":"1","article-title":"Anatomy of the eye and orbit","author":"Forrester J. V.","year":"2016","unstructured":"Forrester J. V., Dick A. D., McMenamin P. G., Roberts F., Pearlman E. (2016). Anatomy of the eye and orbit. The Eye, 1\u2013102.e2. https:\/\/doi.org\/10.1016\/B978-0-7020-5554-6.00001-0","journal-title":"The Eye"},{"key":"e_1_3_4_25_1","doi-asserted-by":"publisher","DOI":"10.1055\/s-2007-1001366"},{"key":"e_1_3_4_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpsycho.2011.09.023"},{"key":"e_1_3_4_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-011-9174-7"},{"key":"e_1_3_4_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpsycho.2003.12.001"},{"key":"e_1_3_4_29_1","doi-asserted-by":"publisher","DOI":"10.1037\/0278-7393.25.1.172"},{"key":"e_1_3_4_30_1","first-page":"219","article-title":"Variable selection using mean decrease accuracy and mean decrease Gini based on random forest","volume":"0","author":"Han H.","year":"2016","unstructured":"Han H., Guo X., Yu H. (2016). Variable selection using mean decrease accuracy and mean decrease Gini based on random forest. Proceedings of the IEEE International Conference on Software Engineering and Service Sciences, ICSESS, 0, 219\u2013224. https:\/\/doi.org\/10.1109\/ICSESS.2016.7883053","journal-title":"Proceedings of the IEEE International Conference on Software Engineering and Service Sciences, ICSESS"},{"key":"e_1_3_4_31_1","volume-title":"Eye tracking: A comprehensive guide to methods and measures","author":"Holmqvist K.","year":"2011","unstructured":"Holmqvist K., Nystr\u00f6m M., Jarodzka H., Weijer J. V. D., Andersson R., Dewhurst R. (2011). Eye tracking: A comprehensive guide to methods and measures. OUP."},{"key":"e_1_3_4_32_1","volume-title":"Algorithms for clustering data","author":"Jain A. K.,","year":"1988","unstructured":"Jain A. K.,, Dubes R. C. (1988). Algorithms for clustering data. Prentice-Hall, Inc."},{"key":"e_1_3_4_33_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.87.4.329"},{"key":"e_1_3_4_34_1","volume-title":"Proceedings of the 11th international symposium on aviation psychology","author":"Kasarskis P.","year":"2001","unstructured":"Kasarskis P., Stehwien J., Hickox J., Aretz A., Wickens C. D. (2001). Comparison of expert and novice scan behaviors during VFR flight. Proceedings of the 11th international symposium on aviation psychology. https:\/\/www.researchgate.net\/profile\/Christopher-Wickens\/publication\/243787467_Comparison_of_Expert_and_Novice_Scan_Behaviors_During_VFR_Flight\/links\/55082e6f0cf2d7a281275dd9\/Comparison-of-Expert-and-Novice-Scan-Behaviors-During-VFR-Flight.pdf"},{"key":"e_1_3_4_35_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781315812274"},{"key":"e_1_3_4_36_1","doi-asserted-by":"publisher","DOI":"10.1089\/lap.2005.15.575"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-022-09582-3"},{"key":"e_1_3_4_38_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v082.i13"},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","DOI":"10.1037\/bul0000150"},{"key":"e_1_3_4_40_1","volume-title":"IEEE Computer Society","author":"Larose D. T.,","year":"2014","unstructured":"Larose D. T.,, Larose C. D. (2014) Discovering Knowledge IN Data An Introduction to Data Mining Second Edition Wiley Series on Methods and Applications in Data Mining (1). IEEE Computer Society. https:\/\/books.google.com\/books\/about\/Discovering_Knowledge_in_Data.html?id=9hOpAwAAQBAJ"},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/968363.968370"},{"issue":"1","key":"e_1_3_4_42_1","first-page":"1","article-title":"Task engagement and mental workload involved in variation and repetition of a motor skill","volume":"7","author":"Lelis-Torres N.","year":"2017","unstructured":"Lelis-Torres N., Ugrinowitsch H., Apolin\u00e1rio-Souza T., Benda R. N., Lage G. M. (2017). Task engagement and mental workload involved in variation and repetition of a motor skill. Scientific Reports 2017 7(1), 1\u201310. https:\/\/doi.org\/10.1038\/s41598-017-15343-3","journal-title":"Scientific Reports 2017"},{"key":"e_1_3_4_43_1","doi-asserted-by":"crossref","unstructured":"Liu S. Donaldson R. Subramaniam A. Palmer H. Champion C. Cox M. Appelbaum L. G. (2020). Skill acquisition and gaze behavior during laparoscopic surgical simulation. https:\/\/doi.org\/10.1101\/2020.07.17.206763","DOI":"10.1101\/2020.07.17.206763"},{"key":"e_1_3_4_44_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0247061"},{"key":"e_1_3_4_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.promfg.2015.07.783"},{"key":"e_1_3_4_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-012-2631-7"},{"key":"e_1_3_4_47_1","doi-asserted-by":"publisher","DOI":"10.5334\/joc.18"},{"key":"e_1_3_4_48_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-8986.2012.01379.x"},{"key":"e_1_3_4_49_1","first-page":"1608","article-title":"Bimodal assessment of laparoscopic suturing skills Construct and concurrent validity","volume":"18","author":"Moorthy K.","year":"2004","unstructured":"Moorthy K., Mnnz Y., Dosis A., Bello F., Chang A., Darzi A. (2004). Bimodal assessment of laparoscopic suturing skills Construct and concurrent validity. Surg Endosc, 18, 1608\u20131612.","journal-title":"Surg Endosc"},{"key":"e_1_3_4_50_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872087201400405"},{"key":"e_1_3_4_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"e_1_3_4_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/1553350612459808"},{"key":"e_1_3_4_53_1","volume-title":"Statistics in research","author":"Ostle B.","year":"1963","unstructured":"Ostle B. (1963). Statistics in research. In Statistics in research (2nd ed.).","edition":"2"},{"key":"e_1_3_4_54_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193129904300114"},{"key":"e_1_3_4_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1743666.1743701"},{"key":"e_1_3_4_56_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781315776675-16"},{"key":"e_1_3_4_57_1","doi-asserted-by":"publisher","DOI":"10.1080\/24721840.2018.1514978"},{"key":"e_1_3_4_58_1","doi-asserted-by":"publisher","DOI":"10.1080\/17470210902816461"},{"key":"e_1_3_4_59_1","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0b013e3181e464fb"},{"issue":"8","key":"e_1_3_4_60_1","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1109\/TSMC.1978.4310039","article-title":"Picture thresholding using an iterative slection method","volume":"8","author":"Ridler T. W.","year":"1978","unstructured":"Ridler T. W., Calvard S. (1978). Picture thresholding using an iterative slection method. IEEE Transactions on Systems, Man and Cybernetics, SMC-8(8), 630\u2013632. https:\/\/doi.org\/10.1109\/tsmc.1978.4310039","journal-title":"IEEE Transactions on Systems, Man and Cybernetics"},{"key":"e_1_3_4_61_1","volume-title":"Human motor control (505)","author":"Rosenbaum D. A.","year":"2010","unstructured":"Rosenbaum D. A. (2010) Human motor control (505)."},{"key":"e_1_3_4_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-016-1459-3"},{"key":"e_1_3_4_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10649-019-9878-z"},{"key":"e_1_3_4_64_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.1.1"},{"key":"e_1_3_4_65_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781315095080-25"},{"key":"e_1_3_4_66_1","doi-asserted-by":"publisher","DOI":"10.1080\/00031305.1980.10483031"},{"key":"e_1_3_4_67_1","doi-asserted-by":"publisher","DOI":"10.1115\/DETC2017-67607"},{"key":"e_1_3_4_68_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb00917.x"},{"key":"e_1_3_4_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/0001-6918(89)90010-3"},{"key":"e_1_3_4_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neubiorev.2018.12.007"},{"key":"e_1_3_4_71_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.2.127"},{"key":"e_1_3_4_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/s004640080081"},{"key":"e_1_3_4_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2006.11.010"},{"key":"e_1_3_4_74_1","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0b013e318220ef31"},{"key":"e_1_3_4_75_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2014.04.032"},{"key":"e_1_3_4_76_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-014-3683-7"},{"key":"e_1_3_4_77_1","volume-title":"Tobii Pro X3-120 eye tracker product description","author":"Tobii Pro","unstructured":"Tobii Pro. (n.d.). Tobii Pro X3-120 eye tracker product description. www.tobiipro.com"},{"key":"e_1_3_4_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-004-8274-6"},{"key":"e_1_3_4_79_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-017-0909-3"},{"key":"e_1_3_4_80_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2012.02.002"},{"key":"e_1_3_4_81_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511973017.046"},{"key":"e_1_3_4_82_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2021.1874741"},{"key":"e_1_3_4_83_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-010-0986-1"},{"key":"e_1_3_4_84_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-011-1802-2"},{"key":"e_1_3_4_85_1","article-title":"Eye-tracking metrics predict perceived workload in robotic surgical skills training","author":"Wu C.","year":"2019","unstructured":"Wu C., Cha J., Sulek J., Zhou T., Sundaram C. P., Wachs J., Yu D. (2019). Eye-tracking metrics predict perceived workload in robotic surgical skills training. Human Factors. https:\/\/doi.org\/10.1177\/0018720819874544","journal-title":"Human Factors"},{"key":"e_1_3_4_86_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327566IJCE0403_3"},{"key":"e_1_3_4_87_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-009-0522-3"},{"key":"e_1_3_4_88_1","doi-asserted-by":"publisher","DOI":"10.1080\/10508414.2017.1313096"}],"container-title":["Human Factors: The Journal of the Human Factors and Ergonomics Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208241302475","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00187208241302475","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00187208241302475","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T07:54:10Z","timestamp":1777449250000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00187208241302475"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":87,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["10.1177\/00187208241302475"],"URL":"https:\/\/doi.org\/10.1177\/00187208241302475","relation":{},"ISSN":["0018-7208","1547-8181"],"issn-type":[{"value":"0018-7208","type":"print"},{"value":"1547-8181","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,28]]}}}