{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T23:10:38Z","timestamp":1781910638955,"version":"3.54.5"},"reference-count":74,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T00:00:00Z","timestamp":1718755200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Vet. Sci."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>This study investigates the log data and response behavior from invigilated in-person electronic timed exams at the University of Veterinary Medicine Hannover, Foundation, Germany. The primary focus is on understanding how various factors influence the time needed per exam item, including item format, item difficulty, item discrimination and character count. The aim was to use these results to derive recommendations for designing timed online distance examinations, an examination format that has become increasingly important in recent years.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Data from 216,625 log entries of five electronic exams, taken by a total of 1,241 veterinary medicine students in 2021 and 2022, were analyzed. Various statistical methods were employed to assess the correlations between the recorded parameters.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The analysis revealed that different item formats require varying amounts of time. For instance, image-based question formats and Kprim necessitated more than 60 s per item, whereas one-best-answer multiple-choice questions (MCQs) and individual Key Feature items were effectively completed in less than 60 s. Furthermore, there was a positive correlation between character count and response time, suggesting that longer items require more time. A negative correlation could be verified for the parameters \u201cdifficulty\u201d and \u201cdiscrimination index\u201d towards response time, indicating that more challenging items and those that are less able to differentiate between high- and low-performing students take longer to answer.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The findings highlight the need for careful consideration of the ratio of item formats when defining time limits for exams. Regarding exam design, the literature mentions that time pressure is a critical factor, since it can negatively impact students\u2019 exam performance and some students, such as those with disabilities, are particularly disadvantaged. Therefore, this study emphasizes finding the right time limits to provide sufficient time for answering questions and reducing time pressure. In the context of unsupervised online exams, the findings of this study support previous recommendations that implementation of a stringent time limit might be a useful strategy to reduce cheating.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fvets.2024.1385681","type":"journal-article","created":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T05:26:13Z","timestamp":1718774773000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Identification of parameters for electronic distance examinations"],"prefix":"10.3389","volume":"11","author":[{"given":"Robin","family":"Richter","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Tipold","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elisabeth","family":"Schaper","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,6,19]]},"reference":[{"key":"ref1","first-page":"1","article-title":"The impact of Covid-19 on higher education around the world","volume":"23","author":"Marinoni","year":"2020","journal-title":"IAU Global Survey Rep"},{"key":"ref2","doi-asserted-by":"crossref","DOI":"10.5771\/9783828876484","volume-title":"Vom Corona-Shutdown Zur Blended University?: Expertinnenbefragung Digitales Sommersemester","author":"Seyfeli","year":"2020"},{"key":"ref3","article-title":"Digitale Lehre f\u00fcr alle: Voraussetzungen, Machbarkeit und Optionen im Human- und Zahnmedizinstudium","author":"Wissing","year":"2020","journal-title":"Medizinischer Fakult\u00e4tentag"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1818","DOI":"10.3390\/ani11061818","article-title":"Veterinary education during Covid-19 and beyond-challenges and mitigating approaches","volume":"11","author":"Routh","year":"2021","journal-title":"Animals"},{"key":"ref5","volume-title":"Digitalisierung der Lehre\u2013 Situationsanalyse und Perspektiven in der Veterin\u00e4rmedizin","author":"Gnewuch","year":"2023"},{"key":"ref6","volume-title":"Untersuchung der Hybridsemester-Lehre w\u00e4hrend der Covid-19 Pandemie an der Stiftung Tier\u00e4rztliche Hochschule Hannover","author":"Naundorf","year":"2023"},{"key":"ref7","author":"Gatti","year":"2020"},{"key":"ref8","author":"Thiel","year":"2021"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1080\/01587919.2021.1911629","article-title":"The right amount of pressure: implementing time pressure in online exams","volume":"42","author":"Stadler","year":"2021","journal-title":"Distance Educ"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"24","DOI":"10.3217\/zfhe-4-03\/02","article-title":"Blended Assessment: M\u00fcndliche Und Elektronische Pr\u00fcfungen Im Klinischen Kontext","volume":"4","author":"Ehlers","year":"2010","journal-title":"Zeitschrift f\u00fcr Hochschulentwicklung"},{"key":"ref11","first-page":"544","volume-title":"Measurement and evaluation in psychology and education","author":"Thorndike","year":"1991"},{"key":"ref12","doi-asserted-by":"crossref","DOI":"10.1024\/85092-000","volume-title":"Pr\u00fcfen Mit Multiple Choice. Kompetent Planen, Entwickeln, Durchf\u00fchren Und Auswerten","author":"Krebs","year":"2019"},{"key":"ref13","volume-title":"Item analysis for criterion-referenced tests","author":"McCowan","year":"1999"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1111\/j.1365-2923.1979.tb01517.x","article-title":"Constructing multiple choice questions of the multiple true\/false type","volume":"13","author":"Harden","year":"1979","journal-title":"Med Educ"},{"key":"ref15","doi-asserted-by":"crossref","first-page":"47","DOI":"10.4324\/9781351064781-4","article-title":"Extended time testing accommodations for students with disabilities: impact on score meaning and construct representation","volume-title":"Integrating timing considerations to improve testing practices","author":"Lovett","year":"2020"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"49","DOI":"10.3109\/01421598609036845","article-title":"Optimising marks obtained in multiple choice question examinations","volume":"8","author":"Mitchell","year":"1986","journal-title":"Med Teach"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1207\/s15324818ame1701_2","article-title":"Effect of extra time on verbal and quantitative Gre scores","volume":"17","author":"Bridgeman","year":"2004","journal-title":"Appl Meas Educ"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1097\/00001888-200610001-00026","article-title":"A multilevel analysis of the relationships between selected examinee characteristics and United States medical licensing examination step 2 clinical knowledge performance: revisiting old findings and asking new questions","volume":"81","author":"Cuddy","year":"2006","journal-title":"Acad Med"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1111\/jedm.12177","article-title":"A comparison of experimental and observational approaches to assessing the effects of time constraints in a medical licensing examination","volume":"55","author":"Harik","year":"2018","journal-title":"J Educ Meas"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1080\/00220973.1995.9943816","article-title":"The effect of time constraints and statistics test anxiety on test performance in a statistics course","volume":"63","author":"Onwuegbuzie","year":"1995","journal-title":"J Exp Educ"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1080\/08923647.2016.1234301","article-title":"Timed online tests: do students perform better with more time?","volume":"30","author":"Portolese","year":"2016","journal-title":"Am J Dist Educ"},{"key":"ref22","first-page":"2","volume-title":"In Paper presented at the Annual Meeting of the National Council on Measurement in Education","author":"Schnipke","year":"1995"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1027\/1015-5759\/a000400","article-title":"Why time constraints increase the gender gap in measured numerical intelligence in academically high achieving samples","volume":"35","author":"Steinmayr","year":"2019","journal-title":"Eur J Psychol Assess"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"267","DOI":"10.3758\/s13423-010-0042-0","article-title":"Time limits and gender differences on paper-and-pencil tests of mental rotation: a meta-analysis","volume":"18","author":"Voyer","year":"2011","journal-title":"Psychon Bull Rev"},{"key":"ref25","first-page":"81","article-title":"Digitale Fernpr\u00fcfungen \/ Online-Pr\u00fcfungen au\u00dferhalb der Hochschule","volume-title":"Digitale Pr\u00fcfungen in der Hochschule. Whitepaper einer Community Working Group aus Deutschland, \u00d6sterreich und der Schweiz","author":"Persike","year":"2021"},{"key":"ref26","volume-title":"Inclusive assessment in higher education: a resource for change","author":"Waterfield","year":"2006"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1111\/emip.12420","article-title":"On the cover: time spent on multiple-choice items","volume":"40","author":"Cui","year":"2021","journal-title":"Educ Meas Issues Pract"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1177\/0098628312456630","article-title":"Time limits on tests: updating the 1-minute rule","volume":"39","author":"Brothen","year":"2012","journal-title":"Teach Psychol"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1020","DOI":"10.1111\/medu.12525","article-title":"Reducing the number of options on multiple-choice questions: response time, psychometrics and standard setting","volume":"48","author":"Schneid","year":"2014","journal-title":"Med Educ"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3946\/kjme.2019.113","article-title":"The relationship between classical item characteristics and item response time on computer-based testing","volume":"31","author":"Chae","year":"2019","journal-title":"Korean J Med Educ"},{"key":"ref31","first-page":"43","article-title":"How to create a good exam","volume-title":"Lessons learned: practical advice for teaching of psychology","author":"Renner","year":"1999"},{"key":"ref32","volume-title":"Teaching tips","author":"McKeachie","year":"2002"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"1006","DOI":"10.1111\/j.1365-2929.2004.01932.x","article-title":"Reliability: on the reproducibility of assessment data","volume":"38","author":"Downing","year":"2004","journal-title":"Med Educ"},{"key":"ref34","first-page":"11","article-title":"Grundlegende quantitative analysen medizinischer pr\u00fcfungen","volume":"23","author":"M\u00f6ltner","year":"2006","journal-title":"GMS Z Med Ausbild"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"53","DOI":"10.5116\/ijme.4dfb.8dfd","article-title":"Making sense of Cronbach\u2019s alpha","volume":"2","author":"Tavakol","year":"2011","journal-title":"Int J Med Educ"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"Doc34","DOI":"10.3205\/zma000926","article-title":"Recommendations of the German Society for Medical Education and the German Association of Medical Faculties regarding university-specific assessments during the study of human, dental and veterinary medicine","volume":"31","author":"J\u00fcnger","year":"2014","journal-title":"GMS Z Med Ausbild"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1152\/advan.00116.2016","article-title":"Best practices in summative assessment","volume":"41","author":"Kibble","year":"2017","journal-title":"Adv Physiol Educ"},{"key":"ref38","first-page":"1","article-title":"Thwarting online exam cheating without proctor supervision","volume":"4","author":"Cluskey","year":"2011","journal-title":"J Acad Bus Ethics"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1037\/h0086821","article-title":"Context effects on examinations: the effects of time, item order and item difficulty","volume":"39","author":"Perlini","year":"1998","journal-title":"Can Psychol"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1024\/1010-0652\/a000229","article-title":"Studentische Bewertung und Pr\u00e4ferenz von Hochschulpr\u00fcfungen mit Aufgaben im offenen und geschlossenen Antwortformat","volume":"32","author":"Lindner","year":"2018","journal-title":"Zeitschrift f\u00fcr P\u00e4dagogische Psychol"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/ets2.12337","article-title":"Methods for measuring speededness: chronology, classification, and ensuing research and development","volume":"2021","author":"Cintron","year":"2021","journal-title":"ETS Res Rep Ser"},{"key":"ref42","first-page":"847","article-title":"Time needed for undergraduate biomechanics exams","volume":"36","author":"Hsieh","year":"2018","journal-title":"ISBS Proc Arch"},{"key":"ref43","year":"2017"},{"key":"ref44","doi-asserted-by":"crossref","first-page":"32","DOI":"10.4324\/9781351064781-3","article-title":"Timing considerations in test development and administration","volume-title":"Integrating timing considerations to improve testing practices","author":"Sireci","year":"2020"},{"key":"ref45","volume-title":"The test analysis manual (ETS SR 74-06)","author":"Swineford","year":"1974"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1111\/bjet.13270","article-title":"Beyond item analysis: connecting student behaviour and performance using E-assessment logs","volume":"54","author":"Lahza","year":"2023","journal-title":"Br J Educ Technol"},{"key":"ref47","first-page":"97","article-title":"Innovative applications of classroom response systems: investigating students\u2019 item response times in relation to final course grade, gender, general point average, and high school act scores","volume":"6","author":"Gonz\u00e1lez-Espada","year":"2007","journal-title":"Electron J Integr Technol Educ"},{"key":"ref48","first-page":"42","volume-title":"Preparation and use of new-type examinations; a manual for teachers","author":"Paterson","year":"1924"},{"key":"ref49","volume-title":"Essentials of psychological testing","author":"Cronbach","year":"1984"},{"key":"ref50","first-page":"13","article-title":"Constructing written test questions for the basic and clinical sciences","author":"Case","year":"2002","journal-title":"Natl Board Exam"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1207\/S15324818AME1503_5","article-title":"A review of multiple-choice item-writing guidelines for classroom assessment","volume":"15","author":"Haladyna","year":"2002","journal-title":"Appl Meas Educ"},{"key":"ref52","doi-asserted-by":"crossref","DOI":"10.4324\/9780203850381","volume-title":"Developing and validating test items","author":"Haladyna","year":"2013"},{"key":"ref53","volume-title":"A taxonomy for learning, teaching, and assessing: a revision of Bloom\u2019s taxonomy of educational objectives","author":"Anderson","year":"2001"},{"key":"ref54","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1097\/ACM.0000000000000977","article-title":"Comparing open-book and closed-book examinations: a systematic review","volume":"91","author":"Durning","year":"2016","journal-title":"Acad Med"},{"key":"ref55","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1177\/009862838401100303","article-title":"Staying with initial answers on objective tests: is it a myth?","volume":"11","author":"Benjamin","year":"1984","journal-title":"Teach Psychol"},{"key":"ref56","doi-asserted-by":"publisher","first-page":"890","DOI":"10.1111\/j.1365-2929.2005.02243.x","article-title":"Answering multiple-choice questions in high-stakes medical examinations","volume":"39","author":"Fischer","year":"2005","journal-title":"Med Educ"},{"key":"ref57","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1037\/0022-3514.88.5.725","article-title":"Counterfactual thinking and the first instinct fallacy","volume":"88","author":"Kruger","year":"2005","journal-title":"J Pers Soc Psychol"},{"key":"ref58","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s11409-015-9140-8","article-title":"The instinct fallacy: the metacognition of answering and revising during college exams","volume":"11","author":"Couchman","year":"2015","journal-title":"Metacogn Learn"},{"key":"ref59","volume-title":"Beantwortungszeiten von Fragen bei einem online durchgef\u00fchrten Progresstest: Abh\u00e4ngigkeit von Schwierigkeit, Studienjahr und Korrektheit der Antwort und die First Instinct Fallacy","author":"M\u00f6ltner","year":"2016"},{"key":"ref60","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1504\/IJTEL.2019.098777","article-title":"Medical student question answering behaviour during high-stakes multiple choice examinations","volume":"11","author":"AlMahmoud","year":"2019","journal-title":"Int J Technol Enhanc Learn"},{"key":"ref61","volume-title":"Untersuchungen zu kompetenzorientierten Pr\u00fcfungen an der Stiftung Tier\u00e4rztliche Hochschule","author":"Ehrich","year":"2019"},{"key":"ref62","first-page":"261","article-title":"Fallbasiertes, elektronisches Lernen und Pr\u00fcfen in der Tiermedizin - Auf der Suche nach einer Alternative zu Multiple-Choice Pr\u00fcfungen","volume":"66","author":"Schaper","year":"2011","journal-title":"Tierarztl Umsch"},{"key":"ref63","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3108\/beej.2004.03000004","article-title":"Evaluation: is an open book examination easier?","volume":"3","author":"Brightwell","year":"2015","journal-title":"Biosci Educ"},{"key":"ref64","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1007\/s10459-006-9038-y","article-title":"Open-book tests to complement assessment-programmes: analysis of open and closed-book tests","volume":"13","author":"Heijne-Penninga","year":"2008","journal-title":"Adv Health Sci Educ Theory Pract"},{"key":"ref65","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1111\/medu.14247","article-title":"High-stakes, remote-access, open-book examinations","volume":"54","author":"Sam","year":"2020","journal-title":"Med Educ"},{"key":"ref66","doi-asserted-by":"publisher","first-page":"387","DOI":"10.2139\/ssrn.4335905","article-title":"Chatgpt goes to law school","volume":"71","author":"Choi","year":"2022","journal-title":"J Legal Educ"},{"key":"ref67","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"ref68","doi-asserted-by":"publisher","first-page":"373","DOI":"10.3238\/arztebl.m2023.0113","article-title":"Chatgpt passes German state examination in medicine with picture questions omitted","volume":"120","author":"Jung","year":"2023","journal-title":"Dtsch Arztebl Int"},{"key":"ref69","volume-title":"Gpt-4 technical report","year":"2023"},{"key":"ref70","author":"Terwiesch","year":"2023"},{"key":"ref71","doi-asserted-by":"publisher","first-page":"2220920","DOI":"10.1080\/10872981.2023.2220920","article-title":"Chatgpt in medical school: how successful is AI in progress testing?","volume":"28","author":"Friederichs","year":"2023","journal-title":"Med Educ"},{"key":"ref72","volume-title":"ChatGPT: the end of online exam integrity?","author":"Susnjak","year":"2022"},{"key":"ref73","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1080\/14703297.2023.2190148","article-title":"Chatting and cheating: ensuring academic integrity in the era of ChatGPT","volume":"61","author":"Cotton","year":"2023","journal-title":"Innov Educ Teach Int"},{"key":"ref74","first-page":"213","article-title":"Artificial intelligence implications for academic cheating: expanding the dimensions of responsible human-AI collaboration with ChatGPT","volume":"34","author":"Oravec","year":"2023","journal-title":"J Interact Learn Res"}],"container-title":["Frontiers in Veterinary Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fvets.2024.1385681\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T05:26:43Z","timestamp":1718774803000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fvets.2024.1385681\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,19]]},"references-count":74,"alternative-id":["10.3389\/fvets.2024.1385681"],"URL":"https:\/\/doi.org\/10.3389\/fvets.2024.1385681","relation":{},"ISSN":["2297-1769"],"issn-type":[{"value":"2297-1769","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,19]]},"article-number":"1385681"}}