{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T14:57:08Z","timestamp":1776092228430,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T00:00:00Z","timestamp":1657152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,7]]},"DOI":"10.1145\/3502718.3524806","type":"proceedings-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T10:28:30Z","timestamp":1657189710000},"page":"470-476","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Impact of Capture The Flag (CTF)-style vs. Traditional Exercises in an Introductory Computer Security Class"],"prefix":"10.1145","author":[{"given":"Stephen V.","family":"Cole","sequence":"first","affiliation":[{"name":"Washington University in St. Louis, St. Louis, MO, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"C.C. Abt. 1987. Serious Games. University Press of America. https:\/\/books. google.com\/books?id=axUs9HA-hF8C  C.C. Abt. 1987. Serious Games. University Press of America. https:\/\/books. google.com\/books?id=axUs9HA-hF8C"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1108\/IJILT-02-2017-0009"},{"key":"e_1_3_2_1_3_1","volume-title":"Edurange: Meeting the pedagogical challenges of student participation in cybertraining environments. In 7th Workshop on Cyber Security Experimentation and Test (CSET'14)","author":"Boesen Stefan","year":"2014","unstructured":"Stefan Boesen , Richard Weiss , James Sullivan , Michael E Locasto , Jens Mache , and Erik Nilsen . 2014 . Edurange: Meeting the pedagogical challenges of student participation in cybertraining environments. In 7th Workshop on Cyber Security Experimentation and Test (CSET'14) . Stefan Boesen, Richard Weiss, James Sullivan, Michael E Locasto, Jens Mache, and Erik Nilsen. 2014. Edurange: Meeting the pedagogical challenges of student participation in cybertraining environments. In 7th Workshop on Cyber Security Experimentation and Test (CSET'14)."},{"key":"e_1_3_2_1_4_1","volume-title":"Analysis and Exercises for Engaging Beginners in Online CTF Competitions for Security Education. In 2017 USENIX Workshop on Advances in Security Education (ASE 17)","author":"Burns Tanner J.","year":"2017","unstructured":"Tanner J. Burns , Samuel C. Rios , Thomas K. Jordan , Qijun Gu , and Trevor Underwood . 2017 . Analysis and Exercises for Engaging Beginners in Online CTF Competitions for Security Education. In 2017 USENIX Workshop on Advances in Security Education (ASE 17) . USENIX Association, Vancouver, BC. https: \/\/www.usenix.org\/conference\/ase17\/workshop-program\/presentation\/burns Tanner J. Burns, Samuel C. Rios, Thomas K. Jordan, Qijun Gu, and Trevor Underwood. 2017. Analysis and Exercises for Engaging Beginners in Online CTF Competitions for Security Education. In 2017 USENIX Workshop on Advances in Security Education (ASE 17). USENIX Association, Vancouver, BC. https: \/\/www.usenix.org\/conference\/ase17\/workshop-program\/presentation\/burns"},{"key":"e_1_3_2_1_5_1","unstructured":"Martin Carlisle Michael Chiaramonte and David Caswell. 2015. Using ctfs for an undergraduate cyber education. In 2015 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 15).  Martin Carlisle Michael Chiaramonte and David Caswell. 2015. Using ctfs for an undergraduate cyber education. In 2015 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 15)."},{"key":"e_1_3_2_1_6_1","unstructured":"Kevin Chung and Julian Cohen. 2014. Learning obstacles in the capture the flag model. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14).  Kevin Chung and Julian Cohen. 2014. Learning obstacles in the capture the flag model. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14)."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/1046878120933312"},{"key":"e_1_3_2_1_8_1","unstructured":"Andy Davis Tim Leek Michael Zhivich Kyle Gwinnup and William Leonard. 2014. The Fun and Future of CTF. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14).  Andy Davis Tim Leek Michael Zhivich Kyle Gwinnup and William Leonard. 2014. The Fun and Future of CTF. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14)."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2181037.2181040"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2011.139"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3430665.3456376"},{"key":"e_1_3_2_1_12_1","unstructured":"Wu-chang Feng and Ashley L. Podhradsky (Eds.). 2018. 2018 USENIXWorkshop on Advances in Security Education (ASE) Baltimore MD USA. USENIX Association. https:\/\/www.usenix.org\/conference\/ase18  Wu-chang Feng and Ashley L. Podhradsky (Eds.). 2018. 2018 USENIXWorkshop on Advances in Security Education (ASE) Baltimore MD USA. USENIX Association. https:\/\/www.usenix.org\/conference\/ase18"},{"key":"e_1_3_2_1_13_1","volume-title":"2016 USENIX Workshop on Advances in Security Education (ASE)","author":"Mark","unstructured":"Mark A. Gondree and Zachary N. J. Peterson (Eds.). 2016 . 2016 USENIX Workshop on Advances in Security Education (ASE) , Austin, TX, USA. USENIX Association. https:\/\/www.usenix.org\/conference\/ase16 Mark A. Gondree and Zachary N. J. Peterson (Eds.). 2016. 2016 USENIX Workshop on Advances in Security Education (ASE), Austin, TX, USA. USENIX Association. https:\/\/www.usenix.org\/conference\/ase16"},{"key":"e_1_3_2_1_14_1","volume-title":"2017 USENIX Workshop on Advances in Security Education (ASE)","author":"Mark","unstructured":"Mark A. Gondree and Ashley L. Podhradsky (Eds.). 2017 . 2017 USENIX Workshop on Advances in Security Education (ASE) , Vancouver, BC, Canada. USENIX Association. https:\/\/www.usenix.org\/conference\/ase17 Mark A. Gondree and Ashley L. Podhradsky (Eds.). 2017. 2017 USENIX Workshop on Advances in Security Education (ASE), Vancouver, BC, Canada. USENIX Association. https:\/\/www.usenix.org\/conference\/ase17"},{"key":"e_1_3_2_1_15_1","volume-title":"KSU Proceedings on Cybersecurity Education, Research, and Practice","author":"Hill Winston Anthony","year":"2020","unstructured":"Winston Anthony Hill Jr , Mesafint Fanuel , Xiaohong Yuan , Jinghua Zhang , and Sajad Sajad . 2020 . A Survey of Serious Games for Cybersecurity Education and Training . KSU Proceedings on Cybersecurity Education, Research, and Practice (2020). Winston Anthony Hill Jr, Mesafint Fanuel, Xiaohong Yuan, Jinghua Zhang, and Sajad Sajad. 2020. A Survey of Serious Games for Cybersecurity Education and Training. KSU Proceedings on Cybersecurity Education, Research, and Practice (2020)."},{"key":"e_1_3_2_1_16_1","unstructured":"Inc. (ISC)2. 2021. (ISC)2 Cybersecurity Workforce Study 2021. https:\/\/www.isc2.org\/\/-\/media\/ISC2\/Research\/2021\/ISC2-Cybersecurity- Workforce-Study-2021.ashx Accessed 2021--12--18.  Inc. (ISC)2. 2021. (ISC)2 Cybersecurity Workforce Study 2021. https:\/\/www.isc2.org\/\/-\/media\/ISC2\/Research\/2021\/ISC2-Cybersecurity- Workforce-Study-2021.ashx Accessed 2021--12--18."},{"key":"e_1_3_2_1_17_1","volume-title":"Motivational design of instruction. Instructional design theories and models: An overview of their current status 1","author":"Keller John M","year":"1983","unstructured":"John M Keller . 1983. Motivational design of instruction. Instructional design theories and models: An overview of their current status 1 , 1983 (1983), 383--434. John M Keller. 1983. Motivational design of instruction. Instructional design theories and models: An overview of their current status 1, 1983 (1983), 383--434."},{"key":"e_1_3_2_1_18_1","volume-title":"Motivational design for learning and performance: The ARCS model approach","author":"Keller John M","unstructured":"John M Keller . 2009. Motivational design for learning and performance: The ARCS model approach . Springer Science & Business Media . John M Keller. 2009. Motivational design for learning and performance: The ARCS model approach. Springer Science & Business Media."},{"key":"e_1_3_2_1_19_1","volume-title":"Cyber Taxi: A Taxonomy of Interactive Cyber Training and Education Systems. In Model-driven Simulation and Training Environments for Cybersecurity","author":"Kn\u00fcpfer Marcus","year":"2020","unstructured":"Marcus Kn\u00fcpfer , Tore Bierwirth , Lars Stiemert , Matthias Schopp , Sebastian Seeber , Daniela P\u00f6hn , and Peter Hillmann . 2020 . Cyber Taxi: A Taxonomy of Interactive Cyber Training and Education Systems. In Model-driven Simulation and Training Environments for Cybersecurity . Springer International Publishing , 3--21. https: \/\/doi.org\/10.1007\/978--3-030--62433-0_1 10.1007\/978--3-030--62433-0_1 Marcus Kn\u00fcpfer, Tore Bierwirth, Lars Stiemert, Matthias Schopp, Sebastian Seeber, Daniela P\u00f6hn, and Peter Hillmann. 2020. Cyber Taxi: A Taxonomy of Interactive Cyber Training and Education Systems. In Model-driven Simulation and Training Environments for Cybersecurity. Springer International Publishing, 3--21. https: \/\/doi.org\/10.1007\/978--3-030--62433-0_1"},{"key":"e_1_3_2_1_20_1","volume-title":"Italian Conference on Cybersecurity (ITASEC'21)","author":"Lagorio Giovanni","year":"2021","unstructured":"Giovanni Lagorio , Marina Ribaudo , and Alessandro Armando . 2021 . Capture the Flag competitions for Higher Education . In Italian Conference on Cybersecurity (ITASEC'21) . CEUR Workshop Proceedings. Giovanni Lagorio, Marina Ribaudo, and Alessandro Armando. 2021. Capture the Flag competitions for Higher Education. In Italian Conference on Cybersecurity (ITASEC'21). CEUR Workshop Proceedings."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3125659.3125686"},{"key":"e_1_3_2_1_22_1","volume-title":"Validation of the Instructional Materials Motivation Survey (IMMS) in a selfdirected instructional setting aimed at working with technology. British journal of educational technology 46, 1","author":"Loorbach Nicole","year":"2015","unstructured":"Nicole Loorbach , Oscar Peters , Joyce Karreman , and Micha\u00ebl Steehouder . 2015. Validation of the Instructional Materials Motivation Survey (IMMS) in a selfdirected instructional setting aimed at working with technology. British journal of educational technology 46, 1 ( 2015 ), 204--218. Nicole Loorbach, Oscar Peters, Joyce Karreman, and Micha\u00ebl Steehouder. 2015. Validation of the Instructional Materials Motivation Survey (IMMS) in a selfdirected instructional setting aimed at working with technology. British journal of educational technology 46, 1 (2015), 204--218."},{"key":"e_1_3_2_1_23_1","unstructured":"Jelena Mirkovic and Peter AH Peterson. 2014. Class capture-the-flag exercises. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14).  Jelena Mirkovic and Peter AH Peterson. 2014. Class capture-the-flag exercises. In 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE 14)."},{"key":"e_1_3_2_1_24_1","volume-title":"Engaging Novices in Cybersecurity Competitions: A Vision and Lessons Learned at ACM Tapia","author":"Mirkovic Jelena","year":"2015","unstructured":"Jelena Mirkovic , Aimee Tabor , Simon Woo , and Portia Pusey . 2015. Engaging Novices in Cybersecurity Competitions: A Vision and Lessons Learned at ACM Tapia 2015 . In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education ( 3GSE 15). Jelena Mirkovic, Aimee Tabor, Simon Woo, and Portia Pusey. 2015. Engaging Novices in Cybersecurity Competitions: A Vision and Lessons Learned at ACM Tapia 2015. In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education (3GSE 15)."},{"key":"e_1_3_2_1_25_1","unstructured":"National Initiative for Cybersecurity Education. 2021. Cybersecurity Workforce Demand. Online poster report.. https:\/\/www.nist.gov\/system\/files\/documents\/ 2021\/12\/03\/NICEFactSheet_WorkforceDemand_Final_20211202.pdf Accessed 2021--12--18.  National Initiative for Cybersecurity Education. 2021. Cybersecurity Workforce Demand. Online poster report.. https:\/\/www.nist.gov\/system\/files\/documents\/ 2021\/12\/03\/NICEFactSheet_WorkforceDemand_Final_20211202.pdf Accessed 2021--12--18."},{"key":"e_1_3_2_1_26_1","unstructured":"Zachary N. J. Peterson (Ed.). 2014. 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE) San Diego CA USA. USENIX Association. https:\/\/www.usenix.org\/conference\/3gse14  Zachary N. J. Peterson (Ed.). 2014. 2014 USENIX Summit on Gaming Games and Gamification in Security Education (3GSE) San Diego CA USA. USENIX Association. https:\/\/www.usenix.org\/conference\/3gse14"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2016.119"},{"key":"e_1_3_2_1_28_1","volume-title":"2016 USENIX Workshop on Advances in Security Education (ASE 16)","author":"Cliffe Schreuders Z","year":"2016","unstructured":"Z Cliffe Schreuders and Emlyn Butterfield . 2016 . Gamification for teaching and learning computer security in higher education . In 2016 USENIX Workshop on Advances in Security Education (ASE 16) . Z Cliffe Schreuders and Emlyn Butterfield. 2016. Gamification for teaching and learning computer security in higher education. In 2016 USENIX Workshop on Advances in Security Education (ASE 16)."},{"key":"e_1_3_2_1_29_1","volume-title":"2017 USENIX Workshop on Advances in Security Education (ASE 17)","author":"Schreuders Z Cliffe","year":"2017","unstructured":"Z Cliffe Schreuders , Thomas Shaw , Mohammad Shan-A- Khuda , Gajendra Ravichandran , Jason Keighley , and Mihai Ordean . 2017 . Security Scenario Generator (SecGen): A Framework for Generating Randomly Vulnerable Rich-scenario VMs for Learning Computer Security and Hosting CTF Events . In 2017 USENIX Workshop on Advances in Security Education (ASE 17) . Z Cliffe Schreuders, Thomas Shaw, Mohammad Shan-A-Khuda, Gajendra Ravichandran, Jason Keighley, and Mihai Ordean. 2017. Security Scenario Generator (SecGen): A Framework for Generating Randomly Vulnerable Rich-scenario VMs for Learning Computer Security and Hosting CTF Events. In 2017 USENIX Workshop on Advances in Security Education (ASE 17)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2591741"},{"key":"e_1_3_2_1_31_1","unstructured":"Hector Suarez Hooper Kincannon and Li Yang. 2017. SSETGami: Secure software education through gamification. (2017).  Hector Suarez Hooper Kincannon and Li Yang. 2017. SSETGami: Secure software education through gamification. (2017)."},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM. https:\/\/doi.org\/10","author":"Vykopal Jan","year":"2020","unstructured":"Jan Vykopal . 2020 . What Are Cybersecurity Education Papers About? . In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM. https:\/\/doi.org\/10 .1145\/3328778.3366816 10.1145\/3328778.3366816 Jan Vykopal. 2020. What Are Cybersecurity Education Papers About?. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. ACM. https:\/\/doi.org\/10.1145\/3328778.3366816"},{"key":"e_1_3_2_1_33_1","volume-title":"CTF: State-of-the-Art and Building the Next Generation. In 2017 USENIX Workshop on Advances in Security Education (ASE 17)","author":"Taylor Clark","year":"2017","unstructured":"Clark Taylor , Pablo Arias , Jim Klopchic , Celeste Matarazzo , and Evi Dube . 2017 . CTF: State-of-the-Art and Building the Next Generation. In 2017 USENIX Workshop on Advances in Security Education (ASE 17) . Clark Taylor, Pablo Arias, Jim Klopchic, Celeste Matarazzo, and Evi Dube. 2017. CTF: State-of-the-Art and Building the Next Generation. In 2017 USENIX Workshop on Advances in Security Education (ASE 17)."},{"key":"e_1_3_2_1_34_1","unstructured":"vulc@n of DDTek. 2021. A History of Capture The Flag at DEFCON. DEFCON website. https:\/\/defcon.org\/html\/links\/dc-ctf-history.html Retrieved 2021--12--18.  vulc@n of DDTek. 2021. A History of Capture The Flag at DEFCON. DEFCON website. https:\/\/defcon.org\/html\/links\/dc-ctf-history.html Retrieved 2021--12--18."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366893"},{"key":"e_1_3_2_1_36_1","volume-title":"2016 USENIX Workshop on Advances in Security Education (ASE 16)","author":"Weiss Richard","year":"2016","unstructured":"Richard Weiss , Frankly Turbak , Jens Mache , Erik Nilsen , and Michael E Locasto . 2016 . Finding the balance between guidance and independence in cybersecurity exercises . In 2016 USENIX Workshop on Advances in Security Education (ASE 16) . Richard Weiss, Frankly Turbak, Jens Mache, Erik Nilsen, and Michael E Locasto. 2016. Finding the balance between guidance and independence in cybersecurity exercises. In 2016 USENIX Workshop on Advances in Security Education (ASE 16)."},{"key":"e_1_3_2_1_37_1","volume-title":"A Study of Cybersecurity Education Using a Present-Test-Practice-Assess Model","author":"Workman Michael D","year":"2021","unstructured":"Michael D Workman , J Anthony Lu\u00e9vanos , and Bin Mai . 2021. A Study of Cybersecurity Education Using a Present-Test-Practice-Assess Model . IEEE Transactions on Education ( 2021 ). Michael D Workman, J Anthony Lu\u00e9vanos, and Bin Mai. 2021. A Study of Cybersecurity Education Using a Present-Test-Practice-Assess Model. IEEE Transactions on Education (2021)."}],"event":{"name":"ITiCSE 2022: Innovation and Technology in Computer Science Education","location":"Dublin Ireland","acronym":"ITiCSE 2022","sponsor":["SIGCSE ACM Special Interest Group on Computer Science Education"]},"container-title":["Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502718.3524806","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3502718.3524806","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:47Z","timestamp":1750183787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502718.3524806"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,7]]},"references-count":37,"alternative-id":["10.1145\/3502718.3524806","10.1145\/3502718"],"URL":"https:\/\/doi.org\/10.1145\/3502718.3524806","relation":{},"subject":[],"published":{"date-parts":[[2022,7,7]]},"assertion":[{"value":"2022-07-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}