{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T03:45:51Z","timestamp":1781581551978,"version":"3.54.5"},"reference-count":171,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T00:00:00Z","timestamp":1615248000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2021,7,31]]},"abstract":"<jats:p>Understanding how developers carry out different computer science activities with objective measures can help to improve productivity and guide the use and development of supporting tools in software engineering. In this article, we present two controlled experiments involving 112 students to explore multiple computing activities (code comprehension, code review, and data structure manipulations) using three different objective measures including neuroimaging (functional near-infrared spectroscopy (fNIRS) and functional magnetic resonance imaging (fMRI)) and eye tracking. By examining code review and prose review using fMRI, we find that the neural representations of programming languages vs. natural languages are distinct. We can classify which task a participant is undertaking based solely on brain activity, and those task distinctions are modulated by expertise. We leverage insights from the psychological notion of spatial ability to decode the neural representations of several fundamental data structures and their manipulations using fMRI, fNIRS, and eye tracking. We examine list, array, tree, and mental rotation tasks and find that data structure and spatial operations use the same focal regions of the brain but to different degrees: they are related but distinct neural tasks. We demonstrate best practices and describe the implication and tradeoffs between fMRI, fNIRS, eye tracking, and self-reporting for software engineering research.<\/jats:p>","DOI":"10.1145\/3434643","type":"journal-article","created":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T12:09:29Z","timestamp":1615291769000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Toward an Objective Measure of Developers\u2019 Cognitive Activities"],"prefix":"10.1145","volume":"30","author":[{"given":"Zohreh","family":"Sharafi","sequence":"first","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Huang","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"Leach","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Westley","family":"Weimer","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, Michigan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,3,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Arne Moller, Andreas Roepstorff, and Torben Lund.","author":"Aamand Rasmus","year":"2013","unstructured":"Rasmus Aamand, Thomas Dalsgaard, Yi-Ching Lynn Ho, Arne Moller, Andreas Roepstorff, and Torben Lund. 2013. A NO way to BOLD?: Dietary nitrate alters the hemodynamic response to visual stimulation. NeuroImage 83 (July 2013)."},{"key":"e_1_2_1_2_1","volume-title":"Maletic","author":"Abid Nahla J.","year":"2019","unstructured":"Nahla J. Abid, Bonita Sharif, Natalia Dragan, Hend Alrasheed, and Jonathan I. Maletic. 2019. Developer reading behavior while summarizing java methods: Size and context matters. In Proceedings of the 41st International Conference on Software Engineering. IEEE Press, 384--395."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1002\/brb3.242"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.28121"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/331795.331804"},{"key":"e_1_2_1_6_1","volume-title":"PME International Conference","volume":"11","author":"Aharoni Dan","year":"2000","unstructured":"Dan Aharoni. 2000. What you see is what you get: The influence of visualization on the perception of data structures. In PME International Conference, Vol. 11. 1--8."},{"key":"e_1_2_1_7_1","unstructured":"Marwa Abdul-Monem Al-Shandawely. 2010. Impacts of Data Structures and Algorithms on Multi-Core Efficiency. Skolan f\u00f6r datavetenskap och kommunikation Kungliga Tekniska h\u00f6gskolan Leganes Spain."},{"key":"e_1_2_1_8_1","first-page":"51","article-title":"Effect of instruction on spatial visualization ability in civil engineering students","volume":"3","author":"Alias Maizam","year":"2002","unstructured":"Maizam Alias, Thomas R. Black, and David E. Gray. 2002. Effect of instruction on spatial visualization ability in civil engineering students. International Education Journal 3, 1 (2002), 51--71.","journal-title":"International Education Journal"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8535.2007.00721.x"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1486"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10339-013-0550-8"},{"key":"e_1_2_1_12_1","volume-title":"Bender","author":"Aumann Yonatan","year":"1996","unstructured":"Yonatan Aumann and Michael A. Bender. 1996. Fault tolerant data structures. In Foundations of Computer Science. IEEE, Burlington, VT, 580--589."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1364\/BOE.4.001366"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055094"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2011.09.003"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1053-8119(09)71202-9"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5408\/1089-9995-53.4.402"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.11.033"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(77)80039-4"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 34th International Conference on Software Engineering (ICSE\u201912)","author":"Raymond P.","unstructured":"Raymond P. L. Buse and Thomas Zimmermann. 2012. Information needs for software development analytics. In Proceedings of the 34th International Conference on Software Engineering (ICSE\u201912). IEEE Press, 987--996."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2015.36"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2004.07.013"},{"key":"e_1_2_1_23_1","volume-title":"Retraining Nearly Half its Workforce for Jobs of the Future. Retrieved","author":"Caminiti Susan","year":"2020","unstructured":"Susan Caminiti. 2018. AT&T\u2019\u2019s $1 Billion Gambit: Retraining Nearly Half its Workforce for Jobs of the Future. Retrieved February 2020 from https:\/\/www.cnbc.com\/2018\/03\/13\/atts-1-billion-gambit-retraining-nearly-half-its-workforce.html."},{"key":"e_1_2_1_24_1","volume-title":"The role of the insula in intuitive expert bug detection in computer code: An fMRI study. Brain Imaging and Behavior (May","author":"Castelhano Jo\u00e3o","year":"2018","unstructured":"Jo\u00e3o Castelhano, Isabel C. Duarte, Carlos Ferreira, Jo\u00e3o Duraes, Henrique Madeira, and Miguel Castelo-Branco. 2018. The role of the insula in intuitive expert bug detection in computer code: An fMRI study. Brain Imaging and Behavior (May 2018)."},{"key":"e_1_2_1_25_1","volume-title":"So What\u2019s a Coil? Retrieved","author":"Center for Diagnostic Imaging. 2016.","year":"2020","unstructured":"Center for Diagnostic Imaging. 2016. I\u2019m Getting an MRI, So What\u2019s a Coil? Retrieved February 2020 from https:\/\/www.mycdi.com\/viewpoints\/im_getting_an_mri_so_whats_a_coil_103."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2015.02.037"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1093\/brain\/119.1.89"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1006\/brln.1997.1835"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2829945"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0959-4388(00)00191-4"},{"key":"e_1_2_1_31_1","volume-title":"Amazon to retrain a third of its U.S. workforce. Retrieved","author":"Cutter Chip","year":"2020","unstructured":"Chip Cutter. 2019. Amazon to retrain a third of its U.S. workforce. Retrieved February 2020 from https:\/\/www.wsj.com\/articles\/amazon-to-retrain-a-third-of-its-u-s-workforce-11562841120."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainres.2006.11.030"},{"key":"e_1_2_1_33_1","volume-title":"Cognitive neuroscience meets mathematics education.Educational Research Review 5, 1","author":"Smedt Bert De","year":"2010","unstructured":"Bert De Smedt, Daniel Ansari, Roland H. Grabner, Minna M. Hannula, Michael Schneider, and Lieven Verschaffel. 2010. Cognitive neuroscience meets mathematics education.Educational Research Review 5, 1 (2010), 97--105."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/382204.382524"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMIP.2019.00010"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208589"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2005.04.039"},{"key":"e_1_2_1_38_1","first-page":"387","article-title":"Impact of cognitive imaging and sex differences on the development of laparoscopic suturing skills","volume":"48","author":"Donnon Tyrone","year":"2005","unstructured":"Tyrone Donnon, Jean-Gaston DesC\u00f4teaux, and Claudio Violato. 2005. Impact of cognitive imaging and sex differences on the development of laparoscopic suturing skills. Canadian Journal of Surgery 48, 5 (2005), 387.","journal-title":"Canadian Journal of Surgery"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2003.1207450"},{"key":"e_1_2_1_40_1","volume-title":"Bugs in Writing, a Guide to Debugging Your Prose","author":"Dupr\u00e9 Lyn","unstructured":"Lyn Dupr\u00e9. 1995. Bugs in Writing, a Guide to Debugging Your Prose. ACM Press\/Addison-Wesley Publishing Co.."},{"key":"e_1_2_1_41_1","volume-title":"International Symposium on Software Reliability Engineering. IEEE, 87--92","author":"Duraes J.","unstructured":"J. Duraes, H. Madeira, J. Castelhano, C. Duarte, and M. C. Branco. 2016. WAP: Understanding the brain at software debugging. In International Symposium on Software Reliability Engineering. IEEE, 87--92."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.03.067"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1602413113"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/358589.358596"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1080\/0031383930370305"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1147\/sj.382.0258"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3275426"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196321.3196347"},{"key":"e_1_2_1_50_1","volume-title":"Proceedings of the 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP\u201913)","author":"Faria Nuno","year":"2013","unstructured":"Nuno Faria, Rui Silva, and Joao L. Sobral. 2013. Impact of data structure layout on performance. In Proceedings of the 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP\u201913). IEEE Computer Society, 116--120. DOI:https:\/\/doi.org\/10.1109\/PDP.2013.24"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.24"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","unstructured":"Denae Ford Justin Smith Philip J. Guo and Chris Parnin. 2016. Paradise unplugged: Identifying barriers for female participation on stack overflow. Foundations of Software Engineering. 846--857. DOI:https:\/\/doi.org\/10.1145\/2950290.2950331","DOI":"10.1145\/2950290.2950331"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568266"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2338965.2336775"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1002\/hbm.10091"},{"key":"e_1_2_1_56_1","first-page":"133","article-title":"Overview of functional magnetic resonance imaging","volume":"22","author":"Glover Gary H.","year":"2011","unstructured":"Gary H. Glover. 2011. Overview of functional magnetic resonance imaging. Neurosurgery Clinics 22, 2 (2011), 133--139.","journal-title":"Neurosurgery Clinics"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuropsychologia.2009.10.013"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the 3rd BELIV\u201910 Workshop: BEyond Time and Errors: Novel evaLuation Methods for Information Visualization (BELIV\u201910)","author":"Joseph","unstructured":"Joseph H. Goldberg and Jonathan I. Helfman. 2010. Comparing information graphics: A critical look at eye tracking. In Proceedings of the 3rd BELIV\u201910 Workshop: BEyond Time and Errors: Novel evaLuation Methods for Information Visualization (BELIV\u201910). ACM, New York, NY, 71--78. DOI:https:\/\/doi.org\/10.1145\/2110192.2110203"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-8141(98)00068-7"},{"key":"e_1_2_1_60_1","doi-asserted-by":"crossref","unstructured":"V. Grigoreanu J. Brundage E. Bahna M. Burnett P. ElRif and J. Snover. 2009. Males and females script debugging strategies. End-User Development (2009) 205--224.","DOI":"10.1007\/978-3-642-00427-8_12"},{"key":"e_1_2_1_61_1","volume-title":"Choosing a proxy for academic aptitude. Journal of Economic Education","author":"Grove Wayne A.","year":"2006","unstructured":"Wayne A. Grove, Tim Wasserman, and Andrew Grodner. 2006. Choosing a proxy for academic aptitude. Journal of Economic Education (2006), 131--147."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/279437.279470"},{"key":"e_1_2_1_64_1","volume-title":"Simultaneous fMRI and eye gaze recordings during prolonged natural stimulation-a studyforrest extension. BioRxiv","author":"Hanke Michael","year":"2016","unstructured":"Michael Hanke, Nico Adelh\u00f6fer, Daniel Kottke, Vittorio Iacovella, Ayan Sengupta, Falko R. Kaule, Roland Nigbur, Alexander Q. Waite, Florian J. Baumgartner, and J\u00f6rg Stadler. 2016. Simultaneous fMRI and eye gaze recordings during prolonged natural stimulation-a studyforrest extension. BioRxiv (2016), 046581."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1093\/brain\/123.1.65"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.91.4.684"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1006\/nimg.2001.0940"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.16910\/jemr.6.4.4"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337322"},{"key":"e_1_2_1_70_1","volume-title":"International Symposium on the Foundations of Software Engineering (ESEC\/FSE\u201920)","author":"Huang Yu","year":"2020","unstructured":"Yu Huang, Kevin Leach, Zohreh Sharafi, Nicholas McKay, Tyler Santander, and Westley Weimer. 2020. Investigating gender bias and differences in code review: Using medical imaging and eye-tracking. In International Symposium on the Foundations of Software Engineering (ESEC\/FSE\u201920). ACM\/SIGSOFT."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00053"},{"key":"e_1_2_1_72_1","volume-title":"Automated Defect Prevention: Best Practices in Software Management","author":"Huizinga Dorota","unstructured":"Dorota Huizinga and Adam Kolawa. 2007. Automated Defect Prevention: Best Practices in Software Management (1st ed.). Wiley.","edition":"1"},{"key":"e_1_2_1_73_1","volume-title":"Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics 3 (March","author":"Huppert Theodore J.","year":"2016","unstructured":"Theodore J. Huppert. 2016. Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. Neurophotonics 3 (March 2016), 010401."},{"key":"e_1_2_1_74_1","volume-title":"Artificial Intelligence, Networking and Parallel\/Distributed Computing","author":"Ikutani Yoshiharu","unstructured":"Yoshiharu Ikutani and Hidetake Uwano. 2014. Brain activity measurement during program comprehension with NIRS. Software Engineering, Artificial Intelligence, Networking and Parallel\/Distributed Computing. IEEE, 1--6."},{"key":"e_1_2_1_75_1","first-page":"4","article-title":"Eye tracking in human-computer interaction and usability research: Ready to deliver the promises","volume":"2","author":"Jacob Robert J. K.","year":"2003","unstructured":"Robert J. K. Jacob and Keith S. Karn. 2003. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind 2, 3 (2003), 4.","journal-title":"Mind"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1080\/13803395.2015.1029440"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/1743666.1743718"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.87.4.329"},{"key":"e_1_2_1_79_1","volume-title":"Google Mondrian: Web-based code review and storage. Retrieved","author":"Kennedy Niall","year":"2006","unstructured":"Niall Kennedy. 2006. Google Mondrian: Web-based code review and storage. Retrieved February 2020 from https:\/\/www.niallkennedy.com\/blog\/2006\/11\/google-mondrian.html."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.03.030"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1038\/jcbfm.2012.23"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1348\/000712607X251243"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/163359.163366"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1093\/comjnl\/27.2.97"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380348"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1194901"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2005.01.048"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2011.5"},{"key":"e_1_2_1_89_1","volume-title":"Mining biometric data to predict programmer expertise and task difficulty. Cluster Computing","author":"Lee Seolhwa","year":"2017","unstructured":"Seolhwa Lee, Danial Hooshyar, Hyesung Ji, Kichun Nam, and Heuiseok Lim. 2017. Mining biometric data to predict programmer expertise and task difficulty. Cluster Computing (2017), 1--11."},{"key":"e_1_2_1_90_1","volume-title":"Proceedings of the International Conference on Bioinformatics and Bioengineering. IEEE, 350--355","author":"Lee S.","year":"2016","unstructured":"S. Lee, A. Matteson, D. Hooshyar, S. Kim, J. Jung, G. Nam, and H. Lim. 2016. Comparing programming language comprehension between novice and expert programmers using EEG analysis. In Proceedings of the International Conference on Bioinformatics and Bioengineering. IEEE, 350--355. DOI:https:\/\/doi.org\/10.1109\/BIBE.2016.30"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2010.10.058"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2008.10.065"},{"key":"e_1_2_1_93_1","volume-title":"Petersen","author":"Linn Marcia C.","year":"1985","unstructured":"Marcia C. Linn and Anne C. Petersen. 1985. Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development (1985), 1479--1498."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1162\/jocn.2009.21203"},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neubiorev.2009.07.008"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.67.3.280"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.070039597"},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1361-8415(98)80001-7"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2009.10.072"},{"key":"e_1_2_1_100_1","doi-asserted-by":"crossref","unstructured":"Pawel J. Matusz Suzanne Dikker Alexander G. Huth and Catherine Perrodin. 2019. Are we ready for real-world neuroscience?","DOI":"10.1162\/jocn_e_01276"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1162\/089892902317236821"},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1090-5138(97)00104-9"},{"key":"e_1_2_1_103_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa0905370"},{"key":"e_1_2_1_104_1","volume-title":"Companion Proceedings of the 36th International Conference on Software Engineering (ICSE Companion\u201914)","author":"Nakagawa Takao","unstructured":"Takao Nakagawa, Yasutaka Kamei, Hidetake Uwano, Akito Monden, Kenichi Matsumoto, and Daniel M. German. 2014. Quantifying programmers\u2019 mental workload during program comprehension based on cerebral blood flow measurement: A controlled experiment. In Companion Proceedings of the 36th International Conference on Software Engineering (ICSE Companion\u201914). Association for Computing Machinery, New York, NYA, 448--451. DOI:https:\/\/doi.org\/10.1145\/2591062.2591098"},{"key":"e_1_2_1_105_1","volume-title":"Software reuse survey. Retrieved on","author":"NASA Software Reuse Working Group","year":"2020","unstructured":"NASA Software Reuse Working Group. 2005. Software reuse survey. Retrieved on February 2020 from http:\/\/www.esdswg.com\/softwarereuse\/Resources\/library\/working_group_documents\/survey2005."},{"key":"e_1_2_1_106_1","doi-asserted-by":"publisher","DOI":"10.1145\/3145904"},{"key":"e_1_2_1_107_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.03.045"},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.1196\/annals.1390.022"},{"key":"e_1_2_1_109_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.87.24.9868"},{"key":"e_1_2_1_110_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2005.12.021"},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1992.242525"},{"key":"e_1_2_1_112_1","doi-asserted-by":"publisher","DOI":"10.1145\/2857491.2857544"},{"key":"e_1_2_1_113_1","volume-title":"A cognitive neuroscience perspective on memory for programming tasks","author":"Parnin Chris","year":"2010","unstructured":"Chris Parnin. 2010. A cognitive neuroscience perspective on memory for programming tasks. Programming Interest Group (2010), 27."},{"key":"e_1_2_1_114_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2011.49"},{"key":"e_1_2_1_115_1","doi-asserted-by":"publisher","DOI":"10.1016\/0732-118X(84)90018-7"},{"key":"e_1_2_1_116_1","doi-asserted-by":"publisher","DOI":"10.1145\/3239235.3240495"},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bandc.2007.09.003"},{"key":"e_1_2_1_118_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bandl.2011.05.003"},{"key":"e_1_2_1_119_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1751-228X.2007.00011.x"},{"key":"e_1_2_1_120_1","doi-asserted-by":"publisher","DOI":"10.1177\/014920638601200408"},{"key":"e_1_2_1_121_1","volume-title":"Ball","author":"Poole Alex","year":"2005","unstructured":"Alex Poole and Linden J. Ball. 2005. Eye tracking in human-computer interaction and usability research: Current status and future. In Prospects\u201d, C. Ghaoui (Ed.), Encyclopedia of Human-Computer Interaction. Idea Group, Inc., Pennsylvania. Information Science Reference - Imprint of IGI Publishing, Hershey, PA, 1--5."},{"key":"e_1_2_1_122_1","article-title":"Gaussian processes for machine learning (GPML) toolbox","author":"Rasmussen Carl Edward","year":"2010","unstructured":"Carl Edward Rasmussen and Hannes Nickisch. 2010. Gaussian processes for machine learning (GPML) toolbox. Journal of Machine Learning Research 11 (Dec. 2010), 3011--3015.","journal-title":"Journal of Machine Learning Research 11"},{"key":"e_1_2_1_123_1","doi-asserted-by":"publisher","DOI":"10.5555\/962111.962113"},{"key":"e_1_2_1_124_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.85.3.618"},{"key":"e_1_2_1_125_1","doi-asserted-by":"publisher","DOI":"10.1093\/cercor"},{"key":"e_1_2_1_126_1","first-page":"2833","article-title":"Password security: What users know and what they actually do","volume":"8","author":"Riley Shannon","year":"2006","unstructured":"Shannon Riley. 2006. Password security: What users know and what they actually do. Usability News 8, 1 (2006), 2833--2836.","journal-title":"Usability News"},{"key":"e_1_2_1_127_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018976708691"},{"key":"e_1_2_1_128_1","volume-title":"Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ETRA\u201900)","author":"Dario","unstructured":"Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ETRA\u201900). ACM, New York, NY, 71--78. DOI:https:\/\/doi.org\/10.1145\/355017.355028"},{"key":"e_1_2_1_129_1","volume-title":"Image Processing, and GIS. Addison-Wesley Longman Publishing Co.","author":"Samet Hanan","unstructured":"Hanan Samet. 1990. Applications of Spatial Data Structures: Computer Graphics, Image Processing, and GIS. Addison-Wesley Longman Publishing Co., Inc., Boston, MA."},{"key":"e_1_2_1_130_1","volume-title":"Smola","author":"Scholkopf Bernhard","year":"2001","unstructured":"Bernhard Scholkopf and Alexander J. Smola. 2001. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA."},{"key":"e_1_2_1_131_1","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2412"},{"key":"e_1_2_1_132_1","doi-asserted-by":"publisher","DOI":"10.1109\/PRNI.2013.40"},{"key":"e_1_2_1_133_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0035860"},{"key":"e_1_2_1_134_1","volume-title":"IgNobel Prize in Neuroscience: The Dead Salmon Study. Scientific American Blog Network (Sept","year":"2012","unstructured":"Scicurious. 2012. IgNobel Prize in Neuroscience: The Dead Salmon Study. Scientific American Blog Network (Sept. 2012). Retrieved on February 2020 from https:\/\/blogs.scientificamerican.com\/scicurious-brain\/ignobel-prize-in-neuroscience-the-dead-salmon-study\/."},{"key":"e_1_2_1_135_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2803188"},{"key":"e_1_2_1_136_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2015.53"},{"key":"e_1_2_1_137_1","volume-title":"A practical guide on conducting eye tracking studies in software engineering. Empirical Software Engineering","author":"Sharafi Zohreh","year":"2020","unstructured":"Zohreh Sharafi, Bonita Sharif, Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc, Andrew Begel, Roman Bednarik, and Martha Crosby. 2020. A practical guide on conducting eye tracking studies in software engineering. Empirical Software Engineering (2020), 1--47."},{"key":"e_1_2_1_138_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2015.06.008"},{"key":"e_1_2_1_139_1","volume-title":"Kernel Methods for Pattern Analysis","author":"Shawe-Taylor John","unstructured":"John Shawe-Taylor and Nello Cristianini. 2004. Kernel Methods for Pattern Analysis. Cambridge University Press."},{"key":"e_1_2_1_140_1","volume-title":"Shepard and Jacqueline Metzler","author":"Roger","year":"1971","unstructured":"Roger N. Shepard and Jacqueline Metzler. 1971. Mental rotation of three-dimensional objects. Science 171, 3972 (1971), 701--703."},{"key":"e_1_2_1_141_1","doi-asserted-by":"publisher","DOI":"10.5555\/381473.381612"},{"key":"e_1_2_1_142_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568252"},{"key":"e_1_2_1_143_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106268"},{"key":"e_1_2_1_144_1","doi-asserted-by":"publisher","DOI":"10.1145\/1181775.1181779"},{"key":"e_1_2_1_145_1","volume-title":"fMRI 2.0: Functional magnetic resonance imaging is growing from showy adolescence into a workhorse of brain imaging. Nature 484, 7392","author":"Smith Kerri","year":"2012","unstructured":"Kerri Smith. 2012. fMRI 2.0: Functional magnetic resonance imaging is growing from showy adolescence into a workhorse of brain imaging. Nature 484, 7392 (2012), 24--27."},{"key":"e_1_2_1_146_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0905267106"},{"key":"e_1_2_1_147_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2007.04.003"},{"key":"e_1_2_1_148_1","volume-title":"Software Engineering","author":"Sommerville Ian","unstructured":"Ian Sommerville. 2010. Software Engineering (9th ed.). Vol. 137035152. Pearson.","edition":"9"},{"key":"e_1_2_1_149_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2011.08.065"},{"key":"e_1_2_1_150_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1986.6312929"},{"key":"e_1_2_1_151_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837101.1837106"},{"key":"e_1_2_1_152_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.06.016"},{"key":"e_1_2_1_153_1","volume-title":"Gender differences and bias in open source: Pull request acceptance of women versus men. PeerJ Computer Science 3, e111","author":"Terrell Josh","year":"2017","unstructured":"Josh Terrell, Andrew Kofink, Justin Middleton, Clarissa Rainear, Emerson Murphy-Hill, Chris Parnin, and Jon Stallings. 2017. Gender differences and bias in open source: Pull request acceptance of women versus men. PeerJ Computer Science 3, e111 (2017)."},{"key":"e_1_2_1_154_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2012.6240505"},{"key":"e_1_2_1_155_1","volume-title":"The Official SAT Study Guide (Redesigned SAT)","author":"The College Board. 2016.","unstructured":"The College Board. 2016. The Official SAT Study Guide (Redesigned SAT). College Board."},{"key":"e_1_2_1_156_1","volume-title":"How Three Fundamental Data Structures Impact Storage and Retrieval. Retrieved","author":"Tkachenko Vadim","year":"2020","unstructured":"Vadim Tkachenko. 2016. How Three Fundamental Data Structures Impact Storage and Retrieval. Retrieved February 2020 from https:\/\/dzone.com\/articles\/how-three-fundamental-data-structures-impact-stora."},{"key":"e_1_2_1_157_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00287620"},{"key":"e_1_2_1_158_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2005.01.007"},{"key":"e_1_2_1_159_1","volume-title":"Meet Phabricator, the Witty Code Review Tool Built Inside Facebook. Retrieved","author":"Tsotsis Alexia","year":"2011","unstructured":"Alexia Tsotsis. 2011. Meet Phabricator, the Witty Code Review Tool Built Inside Facebook. Retrieved February 2020 from https:\/\/techcrunch.com\/2011\/08\/07\/oh-what-noble-scribe-hath-penned-these-words\/."},{"key":"e_1_2_1_160_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2012.00331"},{"key":"e_1_2_1_161_1","doi-asserted-by":"publisher","DOI":"10.1006\/nimg.2001.0978"},{"key":"e_1_2_1_162_1","doi-asserted-by":"publisher","DOI":"10.14786\/flr.v6i3.366"},{"key":"e_1_2_1_163_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.euroneuro.2010.03.008"},{"key":"e_1_2_1_164_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0016127"},{"key":"e_1_2_1_165_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2007.13"},{"key":"e_1_2_1_166_1","doi-asserted-by":"publisher","DOI":"10.1145\/356583.356584"},{"key":"e_1_2_1_167_1","volume-title":"Rational Software Developer Conference.","author":"Williamson Leigh","year":"2008","unstructured":"Leigh Williamson. 2008. IBM rational software analyzer: Beyond source code. In Rational Software Developer Conference."},{"key":"e_1_2_1_168_1","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 143--146","author":"Wobbrock Jacob O.","unstructured":"Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 143--146."},{"key":"e_1_2_1_169_1","doi-asserted-by":"publisher","DOI":"10.1080\/09500690210126784"},{"key":"e_1_2_1_170_1","volume-title":"Moving From Static to Dynamic Modeling of Expertise for Question Routing in CQA Sites. Retrieved on","author":"Yeniterzi Reyyan","year":"2020","unstructured":"Reyyan Yeniterzi and Jamie Callan. 2015. Moving From Static to Dynamic Modeling of Expertise for Question Routing in CQA Sites. Retrieved on February 2020 from https:\/\/www.aaai.org\/ocs\/index.php\/ICWSM\/ICWSM15\/paper\/view\/10548."},{"key":"e_1_2_1_171_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025121"},{"key":"e_1_2_1_172_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2500238"},{"key":"e_1_2_1_173_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11768-010-8043-0"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3434643","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3434643","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:58Z","timestamp":1750195918000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3434643"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,9]]},"references-count":171,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,7,31]]}},"alternative-id":["10.1145\/3434643"],"URL":"https:\/\/doi.org\/10.1145\/3434643","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,9]]},"assertion":[{"value":"2020-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}