{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T04:07:11Z","timestamp":1789186031574,"version":"build-2803163510"},"reference-count":47,"publisher":"Wiley","issue":"10","license":[{"start":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T00:00:00Z","timestamp":1779667200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T00:00:00Z","timestamp":1779667200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["The Laryngoscope"],"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To evaluate the construct validity of a commercially available multimodal foundation model (Google Gemini 2.5 Pro) in assessing simulated injection laryngoplasty.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Thirty video recordings of simulated injection laryngoplasty procedures were stratified by operator experience (10 novice, 10 intermediate, and 10 expert participants). Videos were evaluated by Gemini 2.5 Pro using two prompt engineering strategies: zero\u2010shot (rubric\u2010based, no examples) and few\u2010shot (rubric plus examples). Performance was compared against operator training level (ground truth). Model reliability and stability were assessed through 90 repeated inference trials.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Under a zero\u2010shot strategy, the model failed to discriminate between skill levels (Spearman's\n                      <jats:italic>\u03c1<\/jats:italic>\n                      \u2009=\u20090.12,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.52). Conversely, few\u2010shot prompting demonstrated a strong, positive correlation with operator experience (Spearman's\n                      <jats:italic>\u03c1<\/jats:italic>\n                      \u2009=\u20090.66,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.0002) and successfully stratified skill levels (Kruskal\u2013Wallis\n                      <jats:italic>H<\/jats:italic>\n                      \u2009=\u200912.4,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.002). Pairwise analysis confirmed the few\u2010shot model significantly differentiated experts from both novices (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.002) and intermediates (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.026). Additionally, few\u2010shot prompting significantly improved precision, reducing mean absolute error by nearly half (0.74\u20130.41,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.04). Reliability analysis revealed 100% ordinal consistency (75.6% exact match stability), indicating the model varied under identical conditions, but did not commit any gross classification errors.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>General\u2010purpose multimodal models lack the intrinsic surgical judgment necessary to assess procedural skill. However, resource\u2010efficient few\u2010shot prompting successfully calibrates the model to distinguish expert from trainee performance. While promising as a scalable assessment tool, current models exhibit inherent variability that requires mitigation, such as averaging repeat model evaluations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Level of Evidence<\/jats:title>\n                    <jats:p>N\/A<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/lary.70635","type":"journal-article","created":{"date-parts":[[2026,5,26]],"date-time":"2026-05-26T03:44:43Z","timestamp":1779767083000},"page":"4346-4353","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating Injection Laryngoplasty Skills Using a Foundation Model: A Feasibility Study"],"prefix":"10.1002","volume":"136","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4460-9045","authenticated-orcid":false,"given":"Alex T.","family":"Cheng","sequence":"first","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdulla","family":"Elkhadrawy","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5796-4527","authenticated-orcid":false,"given":"Sean A.","family":"Setzen","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandra","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2165-9480","authenticated-orcid":false,"given":"Adrienne","family":"Biskaduros","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julianna C.","family":"Kostas","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1543-2634","authenticated-orcid":false,"given":"Ana\u00efs","family":"Rameau","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology\u2010Head and Neck Surgery Weill Cornell Medicine  New York New York USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,5,25]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00595-012-0313-7"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-023-10335-z"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1503\/cjs.010614"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/lary.30878"},{"key":"e_1_2_10_6_1","article-title":"Competency\u2010Based Postgraduate Medical Education: Past, Present and Future","volume":"34","author":"Ten Cate O.","year":"2017","journal-title":"GMS Journal for Medical Education"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamasurg.2023.6262"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-023-01010-8"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.20786"},{"key":"e_1_2_10_10_1","first-page":"435","article-title":"Task Versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery","volume":"12","author":"Reiley C. E.","year":"2009","journal-title":"International Conference on Medical Image Computing and Computer\u2010Assisted Intervention"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/rcs.1850"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2593957"},{"key":"e_1_2_10_13_1","first-page":"1373","article-title":"Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgery","volume":"2020","author":"Zhang M.","year":"2021","journal-title":"AMIA Annual Symposium Proceedings"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-93202-y"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-022-09487-1"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/lary.32089"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1097\/JS9.0000000000003069"},{"key":"e_1_2_10_18_1","doi-asserted-by":"crossref","DOI":"10.1590\/0100-6991e-20233605-en","article-title":"Innovations in Surgical Training: Exploring the Role of Artificial Intelligence and Large Language Models (LLM)","volume":"50","author":"Varas J.","year":"2023","journal-title":"Revista do Col\u00e9gio Brasileiro de Cirurgi\u00f5es"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.2196\/72153"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03425-5"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/lary.28952"},{"key":"e_1_2_10_22_1","unstructured":"\u201cDeepMind G [Foundation Model] \u201d(2025) Google."},{"key":"e_1_2_10_23_1","unstructured":"A.Holtzman J.Buys M.Forbes andY.Choi \u201cThe Curious Case of Neural Text Degeneration \u201d(2019)."},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.acl-long.996"},{"key":"e_1_2_10_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-019-01995-1"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/jso.26496"},{"key":"e_1_2_10_27_1","unstructured":"A.Rau M.Endo J.Aklilu et\u00a0al. \u201cSystematic Evaluation of Large Vision\u2010Language Models for Surgical Artificial Intelligence \u201d(2025) arXiv Preprint arXiv:250402799."},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01029-4"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.2196\/72644"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae172"},{"key":"e_1_2_10_31_1","first-page":"1877","article-title":"Language Models Are Few\u2010Shot Learners","volume":"33","author":"Brown T.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2025.103873"},{"key":"e_1_2_10_33_1","unstructured":"S.Kondo \u201cZEAL: Surgical Skill Assessment With Zero\u2010Shot Tool Inference Using Unified Foundation Model \u201d(2024)."},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.3109\/0142159X.2010.501190"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.2217\/pme-2022-0014"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/1468-0009.00023"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1001\/journalofethics.2017.19.6.ecas1-1706"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-010-1356-3"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/aet2.10334"},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-018-1735-5"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-024-11208-9"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.1001\/archinternmed.2012.1579"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.1002\/jhm.2540"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0000000000002065"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-025-09575-7"},{"key":"e_1_2_10_46_1","unstructured":"L.Medford\u2010Davis R.Malani C.Snipes andP.Du Plessis \u201cThe Physician Shortage Isn't Going Anywhere \u201d(2026) https:\/\/www.mckinsey.com\/industries\/healthcare\/our\u2010insights\/the\u2010physician\u2010shortage\u2010isnt\u2010going\u2010anywhere."},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0517"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.52202\/068431-1800"}],"container-title":["The Laryngoscope"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/lary.70635","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/lary.70635","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/lary.70635","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T03:20:01Z","timestamp":1789183201000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/lary.70635"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,25]]},"references-count":47,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/lary.70635"],"URL":"https:\/\/doi.org\/10.1002\/lary.70635","archive":["Portico"],"relation":{},"ISSN":["0023-852X","1531-4995"],"issn-type":[{"value":"0023-852X","type":"print"},{"value":"1531-4995","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,25]]},"assertion":[{"value":"2026-01-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-25","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}