{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T23:20:43Z","timestamp":1768519243777,"version":"3.49.0"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T00:00:00Z","timestamp":1652313600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T00:00:00Z","timestamp":1652313600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["ERC-ADG-2015 694581"],"award-info":[{"award-number":["ERC-ADG-2015 694581"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2022,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these <jats:italic>task models<\/jats:italic> are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-022-02642-y","type":"journal-article","created":{"date-parts":[[2022,5,12]],"date-time":"2022-05-12T18:03:41Z","timestamp":1652378621000},"page":"1437-1444","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Task model-specific operator skill assessment in routine fetal ultrasound scanning"],"prefix":"10.1007","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9589-7177","authenticated-orcid":false,"given":"Yipei","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qianye","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lior","family":"Drukker","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aris","family":"Papageorghiou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yipeng","family":"Hu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J. Alison","family":"Noble","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,5,12]]},"reference":[{"issue":"5955","key":"2642_CR1","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1136\/bmj.1.5955.447","volume":"1","author":"RM Harden","year":"1975","unstructured":"Harden RM, Stevenson M, Downie WW, Wilson G (1975) Assessment of clinical competence using objective structured examination. Br Med J 1(5955):447\u2013451","journal-title":"Br Med J"},{"issue":"2","key":"2642_CR2","doi-asserted-by":"publisher","first-page":"57687","DOI":"10.1371\/journal.pone.0057687","volume":"8","author":"MG Tolsgaard","year":"2013","unstructured":"Tolsgaard MG, Todsen T, Sorensen JL, Ringsted C, Lorentzen T, Ottesen B, Tabor A (2013) International multispecialty consensus on how to evaluate ultrasound competence: a Delphi consensus survey. PLoS ONE 8(2):57687","journal-title":"PLoS ONE"},{"issue":"7","key":"2642_CR3","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1111\/acem.12408","volume":"21","author":"R Amini","year":"2014","unstructured":"Amini R, Adhikari S, Fiorello A (2014) Ultrasound competency assessment in emergency medicine residency programs. Acad Emerg Med 21(7):799\u2013801","journal-title":"Acad Emerg Med"},{"key":"2642_CR4","doi-asserted-by":"crossref","unstructured":"Tyrrell RE, Holden MS (2021) Ultrasound video analysis for skill level assessment in fast ultrasound. Comput Methods Biomech Biomed Eng Imaging Vis 9(3):308\u2013312","DOI":"10.1080\/21681163.2020.1835549"},{"issue":"4","key":"2642_CR5","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1097\/TA.0000000000000813","volume":"79","author":"MT Ziesmann","year":"2015","unstructured":"Ziesmann MT, Park J, Unger B, Kirkpatrick AW, Vergis A, Pham C, Kirschner D, Logestty S, Gillman LM (2015) Validation of hand motion analysis as an objective assessment tool for the focused assessment with sonography for trauma examination. J Trauma Acute Care Surg 79(4):631\u2013637","journal-title":"J Trauma Acute Care Surg"},{"key":"2642_CR6","doi-asserted-by":"crossref","unstructured":"Wang Y, Droste R, Jiao J, Sharma H, Drukker L, Papageorghiou AT, Noble JA (2020) Differentiating operator skill during routine fetal ultrasound scanning using probe motion tracking. In: ASMUS, pp 180\u2013188","DOI":"10.1007\/978-3-030-60334-2_18"},{"key":"2642_CR7","doi-asserted-by":"crossref","unstructured":"Sharma H, Drukker L, Papageorghiou AT, Noble JA (2021) Multi-modal learning from video, eye tracking, and pupillometry for operator skill characterization in clinical fetal ultrasound. In: ISBI, pp 1646\u20131649","DOI":"10.1109\/ISBI48211.2021.9433863"},{"issue":"1","key":"2642_CR8","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1097\/SIH.0000000000000465","volume":"16","author":"M Le Lous","year":"2021","unstructured":"Le Lous M, Despinoy F, Klein M, Fustec E, Lavou\u00e9 V, Jannin P (2021) Impact of physician expertise on probe trajectory during obstetric ultrasound: a quantitative approach for skill assessment. Simul Healthc 16(1):67\u201372","journal-title":"Simul Healthc"},{"key":"2642_CR9","doi-asserted-by":"crossref","unstructured":"Saeed SU, Fu Y, Stavrinides V, Baum Z, Yang Q, Rusu M, Fan RE, Sonn GA, Noble JA, Barratt DC, Hu Y (2021) Adaptable image quality assessment using meta-reinforcement learning of task amenability. In: ASMUS, pp. 191\u2013201","DOI":"10.1007\/978-3-030-87583-1_19"},{"key":"2642_CR10","doi-asserted-by":"crossref","unstructured":"Rahmatullah B, Papageorghiou A, Noble JA (2011) Automated selection of standardized planes from ultrasound volume. In: MLMI, pp 35\u201342","DOI":"10.1007\/978-3-642-24319-6_5"},{"key":"2642_CR11","doi-asserted-by":"crossref","unstructured":"Lee LH, Bradburn E, Papageorghiou AT, Noble JA (2020) Calibrated Bayesian neural networks to estimate gestational age and its uncertainty on fetal brain ultrasound images. In: ASMUS, pp 13\u201322","DOI":"10.1007\/978-3-030-60334-2_2"},{"key":"2642_CR12","doi-asserted-by":"crossref","unstructured":"Bano S, Dromey B, Vasconcelos F, Napolitano R, David AL, Peebles DM, Stoyanov D (2021) AutoFB: Automating fetal biometry estimation from standard ultrasound planes. In: MICCAI, pp 228\u2013238","DOI":"10.1007\/978-3-030-87234-2_22"},{"key":"2642_CR13","doi-asserted-by":"crossref","unstructured":"Salomon LJ, Alfirevic Z, Berghella V, Bilardo C, Hernandez-Andrade E, Johnsen S, Kalache K, Leung K-Y, Malinger G, Munoz H, PREFUMO F, TOI A, LEE W (2011) Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan. Ultrasound Obstetr Gynecol 37(1):116\u2013126","DOI":"10.1002\/uog.8831"},{"key":"2642_CR14","doi-asserted-by":"crossref","unstructured":"Jiao J, Cai Y, Alsharid M, Drukker L, Papageorghiou AT, Noble JA (2020) Self-supervised contrastive video-speech representation learning for ultrasound. In: MICCAI, pp 534\u2013543","DOI":"10.1007\/978-3-030-59716-0_51"},{"issue":"6","key":"2642_CR15","doi-asserted-by":"publisher","first-page":"819","DOI":"10.1007\/s00034-009-9130-7","volume":"28","author":"J-C Yoo","year":"2009","unstructured":"Yoo J-C, Han TH (2009) Fast normalized cross-correlation. Circuits Syst Signal Process 28(6):819\u2013843","journal-title":"Circuits Syst Signal Process"},{"issue":"4","key":"2642_CR16","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600\u2013612","journal-title":"IEEE Trans Image Process"},{"key":"2642_CR17","doi-asserted-by":"publisher","unstructured":"Drukker L, Sharma H, Droste R, Alsharid M, Chatelain P, Noble JA, Papageorghiou AT (2021) Transforming obstetric ultrasound into data science using eye tracking, voice recording, transducer motion and ultrasound video. Sci Rep https:\/\/doi.org\/10.1038\/s41598-021-92829-1","DOI":"10.1038\/s41598-021-92829-1"},{"key":"2642_CR18","doi-asserted-by":"crossref","unstructured":"Sarris I, Ioannou C, Chamberlain P, Ohuma E, Roseman F, Hoch L, Altman D, Papageorghiou A, Fetal I (2012) for the\u00a021st Century\u00a0(INTERGROWTH-21st), N.G.C.: Intra-and interobserver variability in fetal ultrasound measurements. Ultrasound Obstetr Gynecol 39(3):266\u2013273","DOI":"10.1002\/uog.10082"},{"issue":"11","key":"2642_CR19","doi-asserted-by":"publisher","first-page":"2204","DOI":"10.1109\/TMI.2017.2712367","volume":"36","author":"CF Baumgartner","year":"2017","unstructured":"Baumgartner CF, Kamnitsas K, Matthew J, Fletcher TP, Smith S, Koch LM, Kainz B, Rueckert D (2017) SonoNET: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans Med Imag 36(11):2204\u20132215","journal-title":"IEEE Trans Med Imag"},{"key":"2642_CR20","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556"},{"key":"2642_CR21","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"2642_CR22","doi-asserted-by":"crossref","unstructured":"Saeed SU, Fu Y, Baum Z, Yang Q, Rusu M, Fan RE, Sonn GA, Barratt DC, Hu Y (2021) Learning image quality assessment by reinforcing task amenable data selection. In: IPMI, pp 755\u2013766","DOI":"10.1007\/978-3-030-78191-0_58"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-022-02642-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-022-02642-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-022-02642-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T03:41:59Z","timestamp":1658547719000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-022-02642-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,12]]},"references-count":22,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2022,8]]}},"alternative-id":["2642"],"URL":"https:\/\/doi.org\/10.1007\/s11548-022-02642-y","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,12]]},"assertion":[{"value":"8 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study is approved by the UK Research Ethics Committee (reference 18\/WS\/0051).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Informed consent was obtained by all operators and women participated in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}