{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T17:25:25Z","timestamp":1785173125794,"version":"3.55.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2020,3,30]],"date-time":"2020-03-30T00:00:00Z","timestamp":1585526400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Ministerio de Econom\u00eda"},{"name":"Industria y Competitividad of Spain","award":["EDU2016-79838-P"],"award-info":[{"award-number":["EDU2016-79838-P"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2020,6,30]]},"abstract":"<jats:p>A capstone project is an extensive learning experience traditionally developed during a student's final academic year. Assessing such a complex assignment involves several challenges and is usually based upon the evaluations of at least two different people: the capstone project advisor, and one or more other assessors. Quantitative studies comparing only different assessors' grades and qualitative studies investigating the origin of possible discrepancies have been conducted. In both cases, contradictory conclusions were reached. The objective of this study is to analyze the factors that are given consideration by assessors of engineering capstone projects and the influence of these factors on the discrepancies between different assessors\u2019 opinions of the same project. This study quantitatively examined 162 computer science engineering capstone projects developed by one student and supervised by one advisor. Each project was assessed by the project advisor and a committee. For each project, the advisor and the committee were asked to complete an additional questionnaire on product characteristics, student competences, and project supervision. Competences demonstrated by the student were found to be the most relevant element when a capstone project was evaluated by the advisor and the committee; product characteristics were found to be second in influence. Furthermore, advisors grant minor significance to the advisor-involvement component. Discrepancies between grades seem to be associated with those aspects to which one assessor has access, while the other does not, such as student skills demonstrated during project development or their performance in the oral defense. Both the advisor's and the committee's perspectives are important in the assessment of this complex task and they complement one another.<\/jats:p>","DOI":"10.1145\/3381836","type":"journal-article","created":{"date-parts":[[2020,3,30]],"date-time":"2020-03-30T18:04:11Z","timestamp":1585591451000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Factors Considered in the Assessment of Computer Science Engineering Capstone Projects and Their Influence on Discrepancies Between Assessors"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2081-7523","authenticated-orcid":false,"given":"C\u00e9sar","family":"Dom\u00ednguez","sequence":"first","affiliation":[{"name":"Universidad de La Rioja, Logro\u00f1o, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arturo","family":"Jaime","sequence":"additional","affiliation":[{"name":"Universidad de La Rioja, Logro\u00f1o, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francisco J.","family":"Garc\u00eda-Izquierdo","sequence":"additional","affiliation":[{"name":"Universidad de La Rioja, Logro\u00f1o, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan J.","family":"Olarte","sequence":"additional","affiliation":[{"name":"Universidad de La Rioja, Logro\u00f1o, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,3,30]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Retrieved","author":"ABET.","year":"2018"},{"key":"e_1_2_1_2_1","volume-title":"Retrieved","author":"CE.","year":"2016"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 2011 Conference on Frontiers of Information Technology (FIT\u201911)","author":"Ahmad E."},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"J. Bettany\u2010Saltikov S. Kilinc and K. Stow. 2009. Bones boys bombs and booze: An exploratory study of the reliability of marking dissertations across disciplines. Assessment 8 Evaluation in Higher Education 34 6 (2009) 621--639.  J. Bettany\u2010Saltikov S. Kilinc and K. Stow. 2009. Bones boys bombs and booze: An exploratory study of the reliability of marking dissertations across disciplines. Assessment 8 Evaluation in Higher Education 34 6 (2009) 621--639.","DOI":"10.1080\/02602930802302196"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.18261\/issn.1893-8981-2018-03-11"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/13.940997"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the IEEE 9th International Conference on Engineering Education (ICEED\u201917)","author":"Chin R. K. Y."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1595453.1595468"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/572139.572179"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-017-9760-7"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.3926\/jotse.427"},{"key":"e_1_2_1_12_1","volume-title":"Research and Development in Higher Education: Connections in Higher Education","volume":"35","author":"Dong C.","year":"2012"},{"key":"e_1_2_1_13_1","volume-title":"Meta-Analysis, and the Interpretation of Research Results Textbook","author":"Ellis P. D."},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education. 168--173","author":"Farrell V."},{"key":"e_1_2_1_15_1","volume-title":"Discovering Statistics Using SPSS","author":"Field A.","edition":"3"},{"key":"e_1_2_1_16_1","first-page":"997","article-title":"A systematic approach to the pedagogic design of final year projects: Learning outcomes, supervision and assessment","volume":"26","author":"Fraile R.","year":"2010","journal-title":"International Journal of Engineering Education"},{"key":"e_1_2_1_17_1","unstructured":"J. F. Hair W. C. Black B. J. Babin and R. E. Anderson. 2009. Multivariate Data Analysis (7th ed). Pearson.  J. F. Hair W. C. Black B. J. Babin and R. E. Anderson. 2009. Multivariate Data Analysis (7th ed). Pearson."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 20th Annual Conference for the Australasian Association for Engineering Education.","author":"Henderson A."},{"key":"e_1_2_1_19_1","first-page":"912","article-title":"Surveying and benchmarking techniques to analyse DNA gel fingerprint images","volume":"17","author":"Heras C.","year":"2016","journal-title":"Briefings in Bioinformatics"},{"key":"e_1_2_1_20_1","first-page":"1","article-title":"Is a rubric worth the time and effort? Conditions for its success","volume":"10","author":"Ito H.","year":"2015","journal-title":"International Journal of Learning, Teaching and Educational Research"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 2008 International Conference on Computer Science and Software Engineering (CSSE\u201908)","author":"Lan Y."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.publhealth.23.100901.140546"},{"key":"e_1_2_1_23_1","first-page":"116","article-title":"Halos and horns in the assessment of undergraduate medical students: A consistency-based approach","volume":"3","author":"MacDougall M.","year":"2008","journal-title":"Journal of Applied Quantitative Methods"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.1999.tb00405.x"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6920-4-28"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0191-8869(00)00201-4"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.11120\/ened.2012.07010040"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TE.2015.2427118"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1108\/09684880710773165"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.2012.tb01124.x"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3309157"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.17763\/haer.74.3.wq16l67103324520"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/03075070500339988"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3229882"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3152098"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.1993.tb00082.x"},{"key":"e_1_2_1_37_1","volume-title":"Statistics in Plain English","author":"Urdan T. C.","edition":"3"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TE.2002.808258"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2168-9830.2003.tb00773.x"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3381836","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3381836","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:33:07Z","timestamp":1750199587000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3381836"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,30]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,6,30]]}},"alternative-id":["10.1145\/3381836"],"URL":"https:\/\/doi.org\/10.1145\/3381836","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,30]]},"assertion":[{"value":"2019-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-03-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}