{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T13:03:06Z","timestamp":1785762186092,"version":"3.56.0"},"reference-count":37,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T00:00:00Z","timestamp":1675987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>In Germany, progress assessments in head and neck ultrasonography training have been carried out mainly theoretically and lack standardisation. Thus, quality assurance and comparisons between certified courses from various course providers are difficult. This study aimed to develop and integrate a direct observation of procedural skills (DOPS) in head and neck ultrasound education and explore the perceptions of both participants and examiners. Five DOPS tests oriented towards assessing basic skills were developed for certified head and neck ultrasound courses on national standards. DOPS tests were completed by 76 participants from basic and advanced ultrasound courses (n = 168 documented DOPS tests) and evaluated using a 7-point Likert scale. Ten examiners performed and evaluated the DOPS after detailed training. The variables of \u201cgeneral aspects\u201d (6.0 Scale Points (SP) vs. 5.9 SP; p = 0.71), \u201ctest atmosphere\u201d (6.3 SP vs. 6.4 SP; p = 0.92), and \u201ctest task setting\u201d (6.2 SP vs. 5.9 SP; p = 0.12) were positively evaluated by all participants and examiners. There were no significant differences between a basic and advanced course in relation to the overall results of DOPS tests (p = 0.81). Regardless of the courses, there were significant differences in the total number of points achieved between individual DOPS tests. DOPS tests are accepted by participants and examiners as an assessment tool in head and neck ultrasound education. In view of the trend toward \u201ccompetence-based\u201d teaching, this type of test format should be applied and validated in the future.<\/jats:p>","DOI":"10.3390\/diagnostics13040661","type":"journal-article","created":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T04:48:03Z","timestamp":1676004483000},"page":"661","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Development and Integration of DOPS as Formative Tests in Head and Neck Ultrasound Education: Proof of Concept Study for Exploration of Perceptions"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3759-8176","authenticated-orcid":false,"given":"Johannes Matthias","family":"Weimer","sequence":"first","affiliation":[{"name":"Rudolf Frey Teaching Department, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maximilian","family":"Rink","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Regensburg University Hospital, 93053 Regensburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klaus","family":"Dirks","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Internal Medicine, Rems-Murr-Klinikum, 71364 Winnenden, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carlotta","family":"Ille","sequence":"additional","affiliation":[{"name":"Rudolf Frey Teaching Department, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9648-8834","authenticated-orcid":false,"given":"Alessandro","family":"Bozzato","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, University of Saarland, 66123 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8637-9255","authenticated-orcid":false,"given":"Christoph","family":"Sproll","sequence":"additional","affiliation":[{"name":"Department of Oral and Maxillofacial Surgery, University Hospital D\u00fcsseldorf, Heinrich-Heine-University D\u00fcsseldorf, 40225 D\u00fcsseldorf, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas Michael","family":"Weimer","sequence":"additional","affiliation":[{"name":"Department of Orthopedics, Trauma Surgery, and Spinal Cord Injury, Heidelberg University Hospital, 69120 Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Neubert","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, University of Saarland, 66123 Homburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Holger","family":"Buggenhagen","sequence":"additional","affiliation":[{"name":"Rudolf Frey Teaching Department, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4557-2948","authenticated-orcid":false,"given":"Benjamin Philipp","family":"Ernst","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, University Medical Center Bonn (UKB), 53127 Bonn, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luisa","family":"Symeou","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Regensburg University Hospital, 93053 Regensburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liv Annebritt","family":"Lorenz","sequence":"additional","affiliation":[{"name":"Department of Radiooncology and Radiotherapy, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anke","family":"Hollinderb\u00e4umer","sequence":"additional","affiliation":[{"name":"Rudolf Frey Teaching Department, Mainz University Hospital, 55131 Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4402-0287","authenticated-orcid":false,"given":"Julian","family":"K\u00fcnzel","sequence":"additional","affiliation":[{"name":"Department of Otorhinolaryngology, Regensburg University Hospital, 93053 Regensburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,10]]},"reference":[{"key":"ref_1","unstructured":"Bundes\u00e4rztekammer [Federal Medical Association] (2021). (Muster-)Weiterbildungsordnung 2018 in der Fassung vom 26.06.2021, Bundes\u00e4rztekammer."},{"key":"ref_2","first-page":"225","article-title":"Current state of under- and postgraduate education in abdominal ultrasonography at German university hospitals. A panel study over 20 years","volume":"59","author":"Welle","year":"2021","journal-title":"Z. Gastroenterol."},{"key":"ref_3","first-page":"4","article-title":"Ultraschallausbildung an deutschsprachigen Universit\u00e4ten: Wo stehen wir und wo soll es hingehen?","volume":"35","author":"Recker","year":"2014","journal-title":"Ultraschall Med.\u2014Eur. J. Ultrasound."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1055\/a-1699-7473","article-title":"Clinical practice guidance and education in ultrasound: Evidence and experience are two sides of one coin!","volume":"43","author":"Cantisani","year":"2022","journal-title":"Ultraschall Med."},{"key":"ref_5","unstructured":"(2022, October 10). Sektion-Kopf-Hals D. Kurscurriculum der Sektion Kopf\u2014Hals Degum.de: Deutsche Gesellschaft f\u00fcr Ultraschall in der Medizin. Available online: https:\/\/www.degum.de\/fileadmin\/dokumente\/sektionen\/kopf-hals\/KPF_2015__Kurscurriculum_2015-05-18.pdf."},{"key":"ref_6","unstructured":"Kassen\u00e4rztliche Bundesvereinigung (KBV) (2022, March 21). Vereinbarung von Qualit\u00e4tssicherungsmassnahmen nach \u00a7 135 Abs. 2 SGB V zur Ultraschalldiagnostik (Ultraschall-Vereinbarung) vom 31.10.2008 in der ab dem 01.10.2021 geltenden Fassung. Available online: https:\/\/www.kbv.de\/media\/sp\/Ultraschallvereinbarung.pdf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"K\u00fcnzel, J., Bozzato, V., Ernst, B.P., and Bozzato, A. (2022). Musterbefund zur sonografischen Diagnostik im Kopf-Hals-Bereich (Sektion Kopf-Hals der DEGUM). Ultraschall Med.\u2014Eur. J. Ultrasound.","DOI":"10.1055\/a-1810-7173"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"E29","DOI":"10.1055\/a-1922-6778","article-title":"Head and Neck Ultrasound\u2014EFSUMB Training Recommendations for the Practice of Medical Ultrasound in Europe","volume":"8","author":"Todsen","year":"2022","journal-title":"Ultrasound Int. Open."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1208","DOI":"10.1111\/j.1365-2929.2004.02055.x","article-title":"Merging views on assessment","volume":"38","author":"Schuwirth","year":"2004","journal-title":"Med. Educ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1080\/01421590050110731","article-title":"The role of assessment in expanding professional horizons","volume":"22","year":"2000","journal-title":"Med. Teach."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"S63","DOI":"10.1097\/00001888-199009000-00045","article-title":"The assessment of clinical skills\/competence\/performance","volume":"65","author":"Miller","year":"1990","journal-title":"Acad. Med. J. Assoc. Am. Med. Coll."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1111\/j.1365-2923.1979.tb00918.x","article-title":"Assessment of clinical competence using an objective structured clinical examination (OSCE)","volume":"13","author":"Harden","year":"1979","journal-title":"Med. Educ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"476","DOI":"10.7326\/0003-4819-138-6-200303180-00012","article-title":"The mini-CEX: A method for assessing clinical skills","volume":"138","author":"Norcini","year":"2003","journal-title":"Ann. Intern. Med."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Heinzow, H.S., Friederichs, H., Lenz, P., Schmedt, A., Becker, J.C., Hengst, K., Marschall, B., and Domagk, D. (2013). Teaching ultrasound in a curricular course according to certified EFSUMB standards during undergraduate medical education: A prospective study. BMC Med. Educ., 13.","DOI":"10.1186\/1472-6920-13-84"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"254","DOI":"10.14196\/mjiri.32.45","article-title":"Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence","volume":"32","year":"2018","journal-title":"Med. J. Islam Repub Iran."},{"key":"ref_16","first-page":"1207","article-title":"Procedural skills","volume":"25","author":"Jamil","year":"2018","journal-title":"Prof. Med. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"91","DOI":"10.4103\/jehp.jehp_672_19","article-title":"The effect of direct observation of procedural skills method on learning clinical skills of midwifery students of medical sciences","volume":"9","author":"Mohamadirizi","year":"2020","journal-title":"J. Educ. Health Promot."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"871957","DOI":"10.3389\/fmed.2022.871957","article-title":"Assessment Methods in Medical Ultrasound Education","volume":"9","author":"Recker","year":"2022","journal-title":"Front. Med."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1097\/SLA.0000000000000552","article-title":"Reliable and valid assessment of point-of-care ultrasonography","volume":"261","author":"Todsen","year":"2015","journal-title":"Ann Surg."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1055\/s-0029-1246049","article-title":"Evaluation of an OSCE assessment tool for abdominal ultrasound courses","volume":"32","author":"Hofer","year":"2011","journal-title":"Ultraschall Med."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tolsgaard, M.G., Todsen, T., Sorensen, J.L., Ringsted, C., Lorentzen, T., Ottesen, B., and Tabor, A. (2013). International multispecialty consensus on how to evaluate ultrasound competence: A Delphi consensus survey. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0057687"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1002\/lary.26841","article-title":"Competency-based assessment in surgeon-performed head and neck ultrasonography: A validity study","volume":"128","author":"Todsen","year":"2018","journal-title":"Laryngoscope"},{"key":"ref_23","first-page":"166","article-title":"Using DOPS (directly observed procedural skills) for pre call assessment of ultrasound proficiency of first year radiology residents: Development and initial analysis","volume":"29","author":"Rehman","year":"2019","journal-title":"Pak J. Radiol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"7","DOI":"10.5152\/tao.2018.3065","article-title":"Direct observation of procedural skills in otorhinolaryngology training","volume":"56","author":"Kara","year":"2018","journal-title":"Turk. Arch. Otorhinolaryngol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1007\/s12070-019-01624-y","article-title":"Introduction of directly observed procedural skills (DOPS) as a part of competency-based medical education in otorhinolaryngology","volume":"71","author":"Bansal","year":"2019","journal-title":"Indian J. Otolaryngol. Head Neck Surg."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1111\/coa.12251","article-title":"Does direct observation of procedural skills reflect trainee\u2019s progress in otolaryngology?","volume":"39","author":"Awad","year":"2014","journal-title":"Clin. Otolaryngol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1055\/a-1667-8675","article-title":"Sonographic diagnostics in the head and neck area, part 2\u2014Transcervical sonography","volume":"101","author":"Weimer","year":"2022","journal-title":"Laryngorhinootologie"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Streiner, D.L., Norman, G.R., and Cairney, J. (2015). Health Measurement Scales: A Practical Guide to Their Development and Use, Oxford University Press. [5th ed.].","DOI":"10.1093\/med\/9780199685219.001.0001"},{"key":"ref_29","unstructured":"(2022, March 21). Arbeitskreis \u201cLehrevaluation\u201d im Fach Psychologie (Gl\u00e4\u00dfer, E., Gollwitzer, M., Kranz, D., Meiniger, C., Schlotz, W., Schnell, T. & Vo\u00df, A.) in Zusammenarbeit mit dem Zentrum f\u00fcr Psychologische Diagnostik, Begutachtung und Evaluation (ZDiag). 2002. TRIL. Trierer Inventar zur Lehrevaluation [Verfahrensdokumentation, Fragebogen f\u00fcr je Weibliche und M\u00e4nnliche Dozierende]. In Leibniz-Institut f\u00fcr Psychologie (ZPID) (Hrsg.), Open Test Archive. Trier: ZPID. Available online: https:\/\/www.psycharchives.org\/en\/item\/b6adcb29-8e57-45b0-91cb-f1931c0fb552."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Pierre, R.B., Wierenga, A., Barton, M., Branday, J.M., and Christie, C.D.C. (2004). Student evaluation of an OSCE in paediatrics at the University of the West Indies, Jamaica. BMC Med. Educ., 4.","DOI":"10.1186\/1472-6920-4-22"},{"key":"ref_31","first-page":"237","article-title":"Objective Structured Clinical Examination (OSCE): Eine neue Pr\u00fcfungsform in der notfallmedizinischen Ausbildung","volume":"7","author":"Weisser","year":"2004","journal-title":"Notf Rett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e111074","DOI":"10.5812\/aapm.111074","article-title":"The effect of repeated direct observation of procedural skills (R-DOPS) assessment method on the clinical skills of anesthesiology residents","volume":"11","author":"Dabir","year":"2021","journal-title":"Anesthesiol. Pain Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.ajem.2004.07.025","article-title":"Minimum training for right upper quadrant ultrasonography","volume":"22","author":"Jang","year":"2004","journal-title":"Am. J. Emerg. Med."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s13089-019-0118-7","article-title":"Correlation of OSCE performance and point-of-care ultrasound scan numbers among a cohort of emergency medicine residents","volume":"11","author":"Duanmu","year":"2019","journal-title":"Ultrasound J."},{"key":"ref_35","unstructured":"DEGUM (2022, March 21). Pr\u00fcfung Stufe II An\u00e4sthesiologie Deutsche Gesellschaft f\u00fcr Ultraschall in der Medizin. Available online: https:\/\/www.degum.de\/fachgebiete\/sektionen\/anaesthesiologie\/mehrstufenkonzept-zertifizierung\/pruefung-stufe-ii-anae.html."},{"key":"ref_36","first-page":"Doc41","article-title":"Electronic acquisition of OSCE performance using tablets","volume":"32","author":"Hochlehnert","year":"2015","journal-title":"GMS Z. Med. Ausbild."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/s00103-017-2673-z","article-title":"Digital learning and teaching in medical education: Already there or still at the beginning?","volume":"61","author":"Kuhn","year":"2018","journal-title":"Bundesgesundheitsblatt Gesundh. Gesundh."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/4\/661\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:29:51Z","timestamp":1760120991000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/4\/661"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,10]]},"references-count":37,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["diagnostics13040661"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics13040661","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,10]]}}}