{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,26]],"date-time":"2025-11-26T16:38:30Z","timestamp":1764175110377,"version":"3.41.0"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T00:00:00Z","timestamp":1635724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation\u2019s Improving Undergraduate STEM Education","award":["DUE-1225742 and DUE-1525112 (FIU), DUE-1525555 (FAMU), and DUE-1562773 (FGCU)"],"award-info":[{"award-number":["DUE-1225742 and DUE-1525112 (FIU), DUE-1525555 (FAMU), and DUE-1562773 (FGCU)"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>There continues to be an increase in enrollments in various computing programs at academic institutions due to many job opportunities available in the information, communication, and technology sectors. This enrollment surge has presented several challenges in many Computer Science (CS), Information Technology (IT), and Software Engineering (SE) programs at universities and colleges. One such challenge is that many instructors in CS\/IT\/SE programs continue to use learning approaches that are not learner centered and therefore are not adequately preparing students to be proficient in the ever-changing computing industry. To mitigate this challenge, instructors need to use evidence-based pedagogical approaches, e.g., active learning, to improve student learning and engagement in the classroom and equip students with the skills necessary to be lifelong learners.<\/jats:p>\n          <jats:p>\n            This article presents an approach that combines learning and engagement strategies (LESs) in learning environments using different teaching modalities to improve student learning and engagement. We describe how LESs are integrated into face-to-face (F2F) and online class activities. The LESs currently used are\n            <jats:italic>collaborative learning<\/jats:italic>\n            ,\n            <jats:italic>gamification<\/jats:italic>\n            ,\n            <jats:italic>problem-based learning<\/jats:italic>\n            , and\n            <jats:italic>social interaction<\/jats:italic>\n            . We describe an approach used to quantify each LES used during class activities based on a set of characteristics for LESs and the traditional lecture-style pedagogical approaches. To demonstrate the impact of using LESs in F2F class activities, we report on a study conducted over seven semesters in a software testing class at a large urban minority serving institution. The study uses a posttest-only study design, the scores of two midterm exams, and approximate class times dedicated to each LES and traditional lecture style to quantify their usage in a face-to-face software testing class. The study results showed that increasing the time dedicated to collaborative learning, gamification, and social interaction and decreasing the traditional lecture-style approach resulted in a statistically significant improvement in student learning, as reflected in the exam scores.\n          <\/jats:p>","DOI":"10.1145\/3469131","type":"journal-article","created":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T19:12:52Z","timestamp":1635793972000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Combining Learning and Engagement Strategies in a Software Testing Learning Environment"],"prefix":"10.1145","volume":"22","author":[{"given":"Peter J.","family":"Clarke","sequence":"first","affiliation":[{"name":"Florida International University, Miami, FL, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7328-8517","authenticated-orcid":false,"given":"Debra L.","family":"Davis","sequence":"additional","affiliation":[{"name":"Florida International University, Miami, FL, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2736-718X","authenticated-orcid":false,"given":"Ingrid A.","family":"Buckley","sequence":"additional","affiliation":[{"name":"Florida Gulf Coast University, Fort Myers, FL, USA"}]},{"given":"Geoff","family":"Potvin","sequence":"additional","affiliation":[{"name":"Florida International University, Miami, FL, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1427-3144","authenticated-orcid":false,"given":"Mandayam","family":"Thirunarayanan","sequence":"additional","affiliation":[{"name":"Florida International University, Miami, FL, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6680-5878","authenticated-orcid":false,"given":"Edward L.","family":"Jones","sequence":"additional","affiliation":[{"name":"Florida A&amp;M University, Tallahassee, FL, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,11]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Social Learning Theory (2nd ed.)","author":"Bandura Albert","year":"1977","unstructured":"Albert Bandura. 1977. Social Learning Theory (2nd ed.). Prentice-Hall, Englewood Cliffs. NJ."},{"key":"e_1_3_2_3_2","volume-title":"Problem-based Learning: An Approach to Medical Education","volume":"1","author":"Barrows Howard S.","year":"1980","unstructured":"Howard S. Barrows and Robyn M. Tamblyn. 1980. Problem-based Learning: An Approach to Medical Education. Vol. 1. Springer, New York."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324888"},{"key":"e_1_3_2_5_2","unstructured":"Blackboard Inc.2019. Blackboard. Retrieved September 2019 from http:\/\/www.blackboard.com\/."},{"key":"e_1_3_2_6_2","volume-title":"What\u2019s the Use of Lectures? (5th ed.)","author":"Bligh Donald A.","year":"1998","unstructured":"Donald A. Bligh. 1998. What\u2019s the Use of Lectures? (5th ed.). Intellect, Exeter, England."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3230697"},{"key":"e_1_3_2_8_2","first-page":"21","volume-title":"Harv. Educ. Rev.","volume":"32","author":"Bruner Jerome S","year":"1961","unstructured":"Jerome S Bruner. 1961. The act of discovery. Harv. Educ. Rev. 32 (1961), 21\u201332."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18687\/LACCEI2018.1.1.377"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.7821\/naer.2017.7.237"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1207\/s1532690xci0804_2"},{"key":"e_1_3_2_12_2","unstructured":"Raymond Chang-lau and Peter J. Clarke. 2018. Software Engineering and Programming Cyberlearning Environment (SEP-CyLE). Retrieved from https:\/\/stem-cyle.cis.fiu.edu\/instances."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.3102\/0034654318791584"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1002\/pfi.4930450920"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2648787"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.18260\/1-2--32057"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3068324"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEET.2017.1"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.5840\/thinking19888129"},{"key":"e_1_3_2_20_2","unstructured":"Computing Research Association. 2017. Generation CS: Computer Science Undergraduate Enrollments Surge Since 2006. Retrieved August 2020 from https:\/\/cra.org\/data\/Generation-CS\/."},{"key":"e_1_3_2_21_2","volume-title":"Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (4th ed.)","author":"Creswell John W.","year":"2014","unstructured":"John W. Creswell. 2014. Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (4th ed.). Prentice Hall, Upper Saddle River, NJ."},{"issue":"3","key":"e_1_3_2_22_2","first-page":"7","article-title":"Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning.","volume":"25","author":"Davidson Neil","year":"2014","unstructured":"Neil Davidson and Claire Howell Major. 2014. Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning. J. Excell. Coll. Teach. 25, 3&4 (2014), 7\u201355.","journal-title":"J. Excell. Coll. Teach."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-2271-7"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2181037.2181040"},{"key":"e_1_3_2_25_2","volume-title":"Experience and Education","author":"Dewey John","year":"1963","unstructured":"John Dewey. 1963. Experience and Education. Macmillan, New York."},{"issue":"3","key":"e_1_3_2_26_2","first-page":"75","article-title":"Gamification in education: A systematic mapping study.","volume":"18","author":"Dicheva Darina","year":"2015","unstructured":"Darina Dicheva, Christo Dichev, Gennady Agre, and Galia Angelova. 2015. Gamification in education: A systematic mapping study. Educ. Technol. Soc. 18, 3 (2015), 75\u201388.","journal-title":"Educ. Technol. Soc."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2012.12.020"},{"key":"e_1_3_2_28_2","volume-title":"Proceedings of the 126th ASEE Annual Conference & Exposition\u2014Computing and Information Technology Division","author":"Duke Debra M.","year":"2019","unstructured":"Debra M. Duke, Mandayam Thirunarayanan, Abigail Byram, and Peter J. Clarke. 2019. Students\u2019 perceptions of the implementation of a cyberlearning tool. In Proceedings of the 126th ASEE Annual Conference & Exposition\u2014Computing and Information Technology Division. ASEE, Washington, D.C."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1319030111"},{"key":"e_1_3_2_30_2","unstructured":"E. Gamma and K. Beck. 2018. JUnit. Retrieved August 2018 from http:\/\/www.junit.org\/."},{"key":"e_1_3_2_31_2","volume-title":"Proceedings of the ASEE Annual Conference & Exposition","author":"Gehringer Edward F.","year":"2007","unstructured":"Edward F. Gehringer. 2007. Active and collaborative learning strategies for teaching computing. In Proceedings of the ASEE Annual Conference & Exposition. ASEE, Washington DC, 13."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1080\/87567555.1994.9926845"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2014.377"},{"key":"e_1_3_2_34_2","first-page":"1","volume-title":"Proceedings of the IEEE Frontiers in Education Conference (FIE\u201917)","author":"Valle Pedro Henrique Dias","year":"2017","unstructured":"Pedro Henrique Dias Valle, Armando Maciel Toda, Ellen Francine Barbosa, and Jos\u00e9 Carlos Maldonado. 2017. Educational games: A contribution to software testing education. In Proceedings of the IEEE Frontiers in Education Conference (FIE\u201917). IEEE, Los Alamitos, CA, 1\u20138. https:\/\/doi.org\/10.1109\/FIE.2017.8190470"},{"key":"e_1_3_2_35_2","unstructured":"Marc R. Hoffmann Brock Janiczak and Evgeny Mandrikov. 2018. EclEmma. Retrieved August 2018 from http:\/\/www.eclemma.org\/."},{"issue":"4","key":"e_1_3_2_36_2","first-page":"375","article-title":"The impact of social interaction on student learning","volume":"52","author":"Hurst Beth","year":"2013","unstructured":"Beth Hurst, Randall Wallace, and Sarah B. Nixon. 2013. The impact of social interaction on student learning. Read. Horiz. 52, 4 (2013), 375\u2013398.","journal-title":"Read. Horiz."},{"key":"e_1_3_2_37_2","unstructured":"IBM. 2018. Rational Functional Tester. Retrieved August 2018 from https:\/\/www.ibm.com\/us-en\/marketplace\/rational-functional-tester."},{"key":"e_1_3_2_38_2","unstructured":"Instructure. 2019. Canvas. Retrieved September 2019 from https:\/\/www.instructure.com\/canvas."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.18260\/1-2--19772"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-1428-6_1785"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-010-9150-7"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1076\/0899-3408(200008)10:2;1-C;FT109"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0747-5632(02)00057-2"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.17583\/ijep.2015.832"},{"key":"e_1_3_2_45_2","first-page":"1","volume-title":"UDIT (The Norwegian Conference on Didactics in IT Education)","author":"Lauv\u00e5s Per","year":"2018","unstructured":"Per Lauv\u00e5s and Andrea Arcuri. 2018. Recent trends in software testing education: A systematic literature review. In UDIT (The Norwegian Conference on Didactics in IT Education). Bibsys Open Journal Systems, Norway, 1\u201311."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511815355"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/1345375.1345442"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/800088.802839"},{"key":"e_1_3_2_49_2","volume-title":"Foundations of Software Testing (2nd ed.)","author":"Mathur Aditya P.","year":"2014","unstructured":"Aditya P. Mathur. 2014. Foundations of Software Testing (2nd ed.). Pearson Education, India."},{"key":"e_1_3_2_50_2","volume-title":"Evaluation of Evidence-based Practices in Online Learning: A Meta-analysis and Review of Online Learning Studies","author":"Means Barbara","year":"2009","unstructured":"Barbara Means, Yuki Toyama, Robert Murphy, Marianne Bakia, and Karla Jones. 2009. Evaluation of Evidence-based Practices in Online Learning: A Meta-analysis and Review of Online Learning Studies. Technical Report. U.S. Department of Education."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287413"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.18260\/1-2--30084"},{"key":"e_1_3_2_53_2","unstructured":"National Science Board. 2018. Science and Engineering Indicators 2018. Retrieved August 2020 from https:\/\/nsf.gov\/statistics\/2018\/nsb20181\/."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-1428-6_1770"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1037\/11494-000"},{"key":"e_1_3_2_56_2","volume-title":"Freedom to Learn (3rd ed.)","author":"Rogers Carl R.","year":"1994","unstructured":"Carl R. Rogers and H. Jerome Freiberg. 1994. Freedom to Learn (3rd ed.). Merrill, New York."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1006\/ceps.1999.1020"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.7771\/1541-5015.1002"},{"issue":"5","key":"e_1_3_2_59_2","first-page":"31","article-title":"Problem based learning: An instructional model and its constructivist framework","volume":"35","author":"Savery John R","year":"1995","unstructured":"John R Savery and Thomas M Duffy. 1995. Problem based learning: An instructional model and its constructivist framework. Educ. Technol. 35, 5 (1995), 31\u201338.","journal-title":"Educ. Technol."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.1993.tb00296.x"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1201\/b18453-6"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3344429.3372502"},{"key":"e_1_3_2_63_2","volume-title":"Collaborative Learning: A Sourcebook for Higher Education.","author":"Smith Barbara Leigh","year":"1992","unstructured":"Barbara Leigh Smith and Jean T. MacGregor. 1992. What is collaborative learning? In Collaborative Learning: A Sourcebook for Higher Education., A. S. Goodsell, M. R. Maher, and V. Tinto (Eds.). National Center on Postsecondary Teaching, Learning, and Assessment, University Park, PA."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1187\/cbe.13-08-0154"},{"key":"e_1_3_2_65_2","volume-title":"The New Media Consortium","author":"Smith Rachel S.","year":"2004","unstructured":"Rachel S. Smith. 2004. Guidelines for authors of learning objects. The New Media Consortium. http:\/\/archive2.nmc.org\/guidelines\/NMCLOGuidelines.pdf.Retrieved from"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.5555\/2819009.2819060"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.31274\/etd-180810-507"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aap8892"},{"key":"e_1_3_2_69_2","volume-title":"New York, NY","author":"Stephenson C.","year":"2018","unstructured":"C. Stephenson, A. Derbenwick Miller, C. Alvarado, L. Barker, V. Barr, T. Camp, C. Frieze, C. Lewis, E. Cannon Mindell, L. Limbird, D. Richardson, M. Sahami, E. Villa, H. Walker, and S. Zweben. 2018. Retention in Computer Science Undergraduate Programs in the U.S.: Data Challenges and Promising Interventions. ACM, New York, NY."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1734263.1734297"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1111\/jcal.12423"},{"key":"e_1_3_2_72_2","unstructured":"U.S. Bureau of Labor Statistics. 2020. Occupational Outlook Handbook (OOH). Retrieved August 2020 from https:\/\/www.bls.gov\/ooh\/computer-and-information-technology\/home.htm."},{"key":"e_1_3_2_73_2","volume-title":"Thought and Language","author":"Vygotsky Lev S.","year":"2012","unstructured":"Lev S. Vygotsky. 2012. Thought and Language. MIT Press, Cambridge, MA."},{"key":"e_1_3_2_74_2","volume-title":"Essential Readings in Problem-based Learning: Exploring and Extending the Legacy of Howard S. Barrows","author":"Walker Andrew","year":"2015","unstructured":"Andrew Walker, Heather Leary, Cindy Hmelo-Silver, and Peggy A. Ertmer (Eds.). 2015. Essential Readings in Problem-based Learning: Exploring and Extending the Legacy of Howard S. Barrows. Purdue University Press, West Lafayette, IN."},{"key":"e_1_3_2_75_2","volume-title":"Computing Research Association","author":"Zweben Stuart","year":"2020","unstructured":"Stuart Zweben and Betsy Bizot. 2020. 2019 Taulbee Survey. Computing Research Association. Retrieved from https:\/\/cra.org\/wp-content\/uploads\/2020\/05\/2019-Taulbee-Survey.pdf."}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469131","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3469131","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:29Z","timestamp":1750191509000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11]]},"references-count":74,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3469131"],"URL":"https:\/\/doi.org\/10.1145\/3469131","relation":{},"ISSN":["1946-6226","1946-6226"],"issn-type":[{"type":"print","value":"1946-6226"},{"type":"electronic","value":"1946-6226"}],"subject":[],"published":{"date-parts":[[2021,11]]},"assertion":[{"value":"2021-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}