{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T00:50:34Z","timestamp":1773363034119,"version":"3.50.1"},"reference-count":78,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,3]],"date-time":"2024-02-03T00:00:00Z","timestamp":1706918400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,3]],"date-time":"2024-02-03T00:00:00Z","timestamp":1706918400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Spanish Government","award":["RTI-2018-101963-B-I00"],"award-info":[{"award-number":["RTI-2018-101963-B-I00"]}]},{"DOI":"10.13039\/100012818","name":"Comunidad de Madrid","doi-asserted-by":"publisher","award":["e-Madrid-CM - P2018\/TCS-4307"],"award-info":[{"award-number":["e-Madrid-CM - P2018\/TCS-4307"]}],"id":[{"id":"10.13039\/100012818","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Spanish Government","award":["RTI-2018-101963-B-I00"],"award-info":[{"award-number":["RTI-2018-101963-B-I00"]}]},{"name":"Spanish Government","award":["e-Madrid-CM - P2018\/TCS-4307"],"award-info":[{"award-number":["e-Madrid-CM - P2018\/TCS-4307"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Context<\/jats:title>\n                <jats:p>Current data visualization interfaces predominantly rely on 2-D screens. However, the emergence of virtual reality (VR) devices capable of immersive data visualization has sparked interest in exploring their suitability for visualizing software development data. Despite this, there is a lack of detailed investigation into the effectiveness of VR devices specifically for interacting with software development data visualizations.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Objective<\/jats:title>\n                <jats:p>Our objective is to investigate the following question: \u201cHow do VR devices compare to traditional screens in visualizing data about software development?\u201d Specifically, we aim to assess the accuracy of conclusions derived from exploring visualizations for understanding the software development process, as well as the time required to reach these conclusions.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Method<\/jats:title>\n                <jats:p>In our controlled experiment, we recruited N=32 volunteers with diverse backgrounds. Participants interacted with similar data visualizations in both VR and traditional screen environments. For the traditional screen setup, we utilized a commercially available set of interactive dashboards based on Kibana, commonly used by Bitergia customers for data insights. In the VR environment, we designed a set of visualizations, tailored to provide an equivalent dataset within a virtual room. Participants answered questions related to software evolution processes, specifically code review and issue tracking, in both VR and traditional screen environments, for two projects. We conducted statistical analyses to compare the correctness of their answers and the time taken for each question.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Our findings indicate that the correctness of answers in both environments is comparable. Regarding time spent, we observed similar durations, except for complex questions that required examining multiple interconnected visualizations. In such cases, participants in the VR environment were able to answer questions more quickly.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Based on our results, we conclude that VR immersion can be equally effective as traditional screen setups for understanding software development processes through visualization of relevant metrics in most scenarios. Moreover, VR may offer advantages in comprehending complex tasks that require navigating through multiple interconnected visualizations. However, further experimentation is necessary to validate and reinforce these conclusions.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s10664-023-10435-3","type":"journal-article","created":{"date-parts":[[2024,2,3]],"date-time":"2024-02-03T11:02:08Z","timestamp":1706958128000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Software development metrics: to VR or not to VR"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5454-7808","authenticated-orcid":false,"given":"David","family":"Moreno-Lumbreras","sequence":"first","affiliation":[]},{"given":"Gregorio","family":"Robles","sequence":"additional","affiliation":[]},{"given":"Daniel","family":"Izquierdo-Cort\u00e1zar","sequence":"additional","affiliation":[]},{"given":"Jesus M.","family":"Gonzalez-Barahona","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,2,3]]},"reference":[{"issue":"3\u20134","key":"10435_CR1","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1016\/S1093-3263(99)00029-7","volume":"17","author":"A Anderson","year":"1999","unstructured":"Anderson A, Weng Z (1999) VRDD: applying virtual reality visualization to protein docking and design. J Mol Graph Model 17(3\u20134):180\u2013186","journal-title":"J Mol Graph Model"},{"key":"10435_CR2","doi-asserted-by":"publisher","unstructured":"Antoniol G, Ayari K, Di\u00a0Penta M, Khomh F, Gu\u00e9h\u00e9neuc YG (2008) Is It a Bug or an Enhancement? A Text-Based Approach to Classify Change Requests. In: Proceedings of the 2008 Conference of the Center for Advanced Studies on Collaborative Research: Meeting of Minds. CASCON \u201908. New York, USA: Association for Computing Machinery. Available from: https:\/\/doi.org\/10.1145\/1463788.1463819","DOI":"10.1145\/1463788.1463819"},{"key":"10435_CR3","doi-asserted-by":"crossref","unstructured":"Bacchelli A, Bird C (2013) Expectations, outcomes, and challenges of modern code review. In: 2013 35th ICSE. IEEE pp. 712\u2013721","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"10435_CR4","doi-asserted-by":"crossref","unstructured":"Batch A, Cunningham A, Cordeil M, Elmqvist N, Dwyer T, Thomas BH et al (2019) There is no spoon: evaluating performance, space use, and presence with expert domain users in immersive analytics. IEEE Trans Vis Comput Graph 26(1):536\u2013546","DOI":"10.1109\/TVCG.2019.2934803"},{"key":"10435_CR5","doi-asserted-by":"publisher","unstructured":"Bayyari A, Tudoreanu ME (2006) The Impact of Immersive Virtual Reality Displays on the Understanding of Data Visualization. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology. VRST \u201906. New York, USA: Association for Computing Machinery, pp 368\u2013371. Available from: https:\/\/doi.org\/10.1145\/1180495.1180570","DOI":"10.1145\/1180495.1180570"},{"key":"10435_CR6","doi-asserted-by":"publisher","unstructured":"Bettenburg N, Just S, Schr\u00f6ter A, Weiss C, Premraj R, Zimmermann T (2008) What Makes a Good Bug Report? In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. SIGSOFT \u201908\/FSE-16. New York, USA: Association for Computing Machinery, pp 308\u2013318. Available from: https:\/\/doi.org\/10.1145\/1453101.1453146","DOI":"10.1145\/1453101.1453146"},{"key":"10435_CR7","doi-asserted-by":"crossref","unstructured":"Bissyand\u00e9 TF, Lo D, Jiang L, R\u00e9veill\u00e8re L, Klein J, Traon YL (2013) Got issues? Who cares about it? A large scale investigation of issue trackers from GitHub. In: 2013 IEEE 24th International symposium on software reliability engineering (ISSRE), pp 188\u2013197","DOI":"10.1109\/ISSRE.2013.6698918"},{"key":"10435_CR8","doi-asserted-by":"publisher","unstructured":"Bowman D, McMahan R (2007) Virtual reality: how much immersion is enough? Computer 08(40):36\u201343. https:\/\/doi.org\/10.1109\/MC.2007.257","DOI":"10.1109\/MC.2007.257"},{"issue":"5","key":"10435_CR9","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1145\/229459.229467","volume":"39","author":"S Bryson","year":"1996","unstructured":"Bryson S (1996) Virtual reality in scientific visualization. Communications of the ACM 39(5):62\u201371","journal-title":"Communications of the ACM"},{"key":"10435_CR10","doi-asserted-by":"crossref","unstructured":"Chen JJ, Cai H, Auchus AP, Laidlaw DH (2012) Effects of stereo and screen size on the legibility of three-dimensional streamtube visualization. IEEE Trans Vis Comput Graph 18:2130\u20132139","DOI":"10.1109\/TVCG.2012.216"},{"key":"10435_CR11","unstructured":"Cleveland WS (1994) The Elements of Graphing Data. Hobart Press"},{"key":"10435_CR12","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"J Cohen","year":"1988","unstructured":"Cohen J (1988) Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates"},{"key":"10435_CR13","doi-asserted-by":"crossref","unstructured":"Cruz-Neira C, Sandin DJ, DeFanti TA (1993) Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In: Proceedings of the 20th annual conference on Computer graphics and interactive techniques, pp 135\u2013142","DOI":"10.1145\/166117.166134"},{"key":"10435_CR14","doi-asserted-by":"publisher","unstructured":"Demiralp C, Jackson C, Karelitz D, Zhang S, Laidlaw D (2006) CAVE and fishtank virtual-reality displays: a qualitative and quantitative comparison. IEEE Trans Vis Comput Graph 12:323\u201330. https:\/\/doi.org\/10.1109\/TVCG.2006.42","DOI":"10.1109\/TVCG.2006.42"},{"key":"10435_CR15","unstructured":"Djorgovski S, Hut P, Knop R, Longo G, McMillan S, Vesperini E et\u00a0al (2013) The MICA experiment: astrophysics in virtual worlds. arXiv:1301.6808"},{"key":"10435_CR16","doi-asserted-by":"crossref","unstructured":"Donalek C, Djorgovski SG, Cioc A, Wang A, Zhang J, Lawler E et\u00a0al (2014) Immersive and collaborative data visualization using virtual reality platforms. In: 2014 IEEE International conference on big data (big data), pp 609\u2013614","DOI":"10.1109\/BigData.2014.7004282"},{"key":"10435_CR17","doi-asserted-by":"crossref","unstructured":"Drogemuller A, Cunningham A, Walsh J, Cordeil M, Ross W, Thomas B (2018) Evaluating navigation techniques for 3D graph visualizations in virtual reality. In: 2018 International symposium on big data visual and immersive analytics (BDVA), pp 1\u201310","DOI":"10.1109\/BDVA.2018.8533895"},{"key":"10435_CR18","doi-asserted-by":"crossref","unstructured":"Due\u00f1as S, Cosentino V, Gonzalez-Barahona JM, del Castillo San Felix A, Izquierdo-Cortazar D, Ca\u00f1as-D\u00edaz L et al (2021) GrimoireLab: a toolset for software development analytics. Accepted, publication pending, PeerJ Computer Science","DOI":"10.7717\/peerj-cs.601"},{"key":"10435_CR19","doi-asserted-by":"crossref","unstructured":"Due\u00f1as S, Cosentino V, Robles G, Gonzalez-Barahona JM (2018) Perceval: software project data at your will. In: Proceedings of the 40th international conference on software engineering: companion proceeedings, pp 1\u20134","DOI":"10.1145\/3183440.3183475"},{"key":"10435_CR20","doi-asserted-by":"crossref","unstructured":"Elliott A, Peiris B, Parnin C (2015) Virtual reality in software engineering: affordances, applications, and challenges. In: 2015 IEEE\/ACM 37th IEEE International conference on software engineering vol 2. IEEE, pp 547\u2013550","DOI":"10.1109\/ICSE.2015.191"},{"key":"10435_CR21","unstructured":"Few S (2004) Show me the numbers. Analytics Pres"},{"key":"10435_CR22","volume-title":"Now You See It: Simple Visualization Techniques for Quantitative Analysis","author":"S Few","year":"2009","unstructured":"Few S (2009) Now You See It: Simple Visualization Techniques for Quantitative Analysis, 1st edn. Analytics Press, Oakland, CA, USA","edition":"1"},{"key":"10435_CR23","doi-asserted-by":"publisher","unstructured":"Friendly M (2008) In: A Brief History of Data Visualization. Berlin, Heidelberg: Springer Berlin Heidelberg, pp 15\u201356. Available from: https:\/\/doi.org\/10.1007\/978-3-540-33037-0_2","DOI":"10.1007\/978-3-540-33037-0_2"},{"key":"10435_CR24","unstructured":"Friendly M (2008) Milestones in the history of thematic cartography, statistical graphics, and data visualization"},{"key":"10435_CR25","doi-asserted-by":"crossref","unstructured":"Garc\u00eda-Hern\u00e1ndez RJ, Anthes C, Wiedemann M, Kranzlm\u00fcller D (2016) Perspectives for using virtual reality to extend visual data mining in information visualization. In: 2016 IEEE Aerospace Conference, pp 1\u201311","DOI":"10.1109\/AERO.2016.7500608"},{"issue":"6","key":"10435_CR26","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1145\/1743546.1743567","volume":"53","author":"J Heer","year":"2010","unstructured":"Heer J, Bostock M, Ogievetsky V (2010) A tour through the visualization zoo. Commun ACM 53(6):59\u201367. https:\/\/doi.org\/10.1145\/1743546.1743567","journal-title":"Commun ACM"},{"key":"10435_CR27","doi-asserted-by":"publisher","unstructured":"Hooimeijer P, Weimer W (2007) Modeling Bug Report Quality. In: Proceedings of the Twenty-Second IEEE\/ACM International Conference on Automated Software Engineering. ASE \u201907. New York, USA: Association for Computing Machinery, pp 34\u201343. Available from: https:\/\/doi.org\/10.1145\/1321631.1321639","DOI":"10.1145\/1321631.1321639"},{"key":"10435_CR28","doi-asserted-by":"crossref","unstructured":"Ibrahim Z, Money AG (2019) Computer mediated reality technologies: a conceptual framework and survey of the state of the art in healthcare intervention systems. J Biomed Inform 90:103102","DOI":"10.1016\/j.jbi.2019.103102"},{"key":"10435_CR29","unstructured":"Jackson D, Gilbert J (2023) WebGL 2.0 Specification. Khronos Group Specification"},{"key":"10435_CR30","doi-asserted-by":"crossref","unstructured":"Jacob RJ, Girouard A, Hirshfield LM, Horn MS, Shaer O, Solovey ET et\u00a0al (2008) Reality-based interaction: a framework for post-WIMP interfaces. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp 201\u2013210","DOI":"10.1145\/1357054.1357089"},{"key":"10435_CR31","doi-asserted-by":"crossref","unstructured":"Jedlitschka A, Pfahl D (2005) Reporting guidelines for controlled experiments in software engineering. In: 2005 International symposium on empirical software engineering, 2005, p 10-pp","DOI":"10.1109\/ISESE.2005.1541818"},{"key":"10435_CR32","unstructured":"Jones B, Goregaokar M (2023) WebXR Device API. W3C Working Draft"},{"key":"10435_CR33","doi-asserted-by":"crossref","unstructured":"Kaiser P, Vasak P, Suorineni F, Thibodeau D (2005) New Dimensions in Seismic Data Interpretation with 3-D Virtual Reality Visualisation for Burst-Prone Mines, pp 33\u201345","DOI":"10.36487\/ACG_repo\/574_0.3"},{"key":"10435_CR34","doi-asserted-by":"crossref","unstructured":"Kendall M (1938) A New Measure of Rank Correlation. Biometrika","DOI":"10.2307\/2332226"},{"key":"10435_CR35","doi-asserted-by":"crossref","unstructured":"Kobayashi K, Kamimura M, Yano K, Kato K, Matsuo A (2013) SArF map: visualizing software architecture from feature and layer viewpoints. In: 2013 21st International conference on program comprehension (ICPC), pp 43\u201352","DOI":"10.1109\/ICPC.2013.6613832"},{"key":"10435_CR36","doi-asserted-by":"crossref","unstructured":"Kononenko O, Rose T, Baysal O, Godfrey M, Theisen D, De\u00a0Water B (2018) Studying pull request merges: a case study of shopify\u2019s active merchant. In: Proceedings of the 40th ICSE SEIP, pp 124\u2013133","DOI":"10.1145\/3183519.3183542"},{"key":"10435_CR37","doi-asserted-by":"publisher","unstructured":"Laha B, Bowman D, Socha J (2014) Effects of VR system fidelity on analyzing isosurface visualization of volume datasets. IEEE Trans Vis Comput Graph 20:513\u201322. https:\/\/doi.org\/10.1109\/TVCG.2014.20","DOI":"10.1109\/TVCG.2014.20"},{"issue":"12","key":"10435_CR38","doi-asserted-by":"publisher","first-page":"1373","DOI":"10.1007\/s00371-013-0892-3","volume":"30","author":"S Liu","year":"2014","unstructured":"Liu S, Cui W, Wu Y, Liu M (2014) A survey on information visualization: recent advances and challenges. The Visual Computer 30(12):1373\u20131393","journal-title":"The Visual Computer"},{"key":"10435_CR39","doi-asserted-by":"crossref","unstructured":"Maddila C, Bansal C, Nagappan N (2019) Predicting pull request completion time: a case study on large scale cloud services. In: Proceedings of the 2019 27th ESEC\/FSE, pp 874\u2013882","DOI":"10.1145\/3338906.3340457"},{"key":"10435_CR40","unstructured":"Maletic JI, Leigh J, Marcus A, Dunlap G (2001) Visualizing object-oriented software in virtual reality. In: Proceedings 9th international workshop on program comprehension. IWPC 2001, pp 26\u201335"},{"key":"10435_CR41","doi-asserted-by":"crossref","unstructured":"Merino L, Fuchs J, Blumenschein M, Anslow C, Ghafari M, Nierstrasz O et\u00a0al (2017) On the impact of the medium in the effectiveness of 3D software visualizations. In: 2017 IEEE Working conference on software visualization (VISSOFT), pp 11\u201321","DOI":"10.1109\/VISSOFT.2017.17"},{"key":"10435_CR42","doi-asserted-by":"crossref","unstructured":"Merino L, Ghafari M, Anslow C, Nierstrasz O (2017) CityVR: gameful software visualization. In: 2017 IEEE International conference on software maintenance and evolution (ICSME), pp 633\u2013637","DOI":"10.1109\/ICSME.2017.70"},{"issue":"12","key":"10435_CR43","first-page":"1321","volume":"77","author":"P Milgram","year":"1994","unstructured":"Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77(12):1321\u20131329","journal-title":"IEICE TRANSACTIONS on Information and Systems"},{"key":"10435_CR44","doi-asserted-by":"crossref","unstructured":"Millais P, Jones SL, Kelly R (2018) Exploring data in virtual reality: comparisons with 2d data visualizations. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, pp 1\u20136","DOI":"10.1145\/3170427.3188537"},{"key":"10435_CR45","doi-asserted-by":"publisher","unstructured":"Millais P, Jones SL, Kelly R (2018) Exploring Data in Virtual Reality: Comparisons with 2D Data visualizations. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. CHI EA \u201918. New York, USA: Association for Computing Machinery, pp 1\u20136. Available from: https:\/\/doi.org\/10.1145\/3170427.3188537","DOI":"10.1145\/3170427.3188537"},{"key":"10435_CR46","doi-asserted-by":"crossref","unstructured":"Misiak M, Schreiber A, Fuhrmann A, Zur S, Seider D, Nafeie L (2018) IslandViz: a Tool for Visualizing Modular Software Systems in Virtual Reality. In: 2018 IEEE Working Conference on Software Visualization (VISSOFT), pp 112\u2013116","DOI":"10.1109\/VISSOFT.2018.00020"},{"key":"10435_CR47","doi-asserted-by":"crossref","unstructured":"Moreno-Lumbreras D, Gonzalez-Barahona JM, Villaverde A (2022) BabiaXR: virtual reality software data visualizations for the web. In: 2022 IEEE Conference on virtual reality and 3D user interfaces abstracts and workshops (VRW). IEEE, pp 71\u201374","DOI":"10.1109\/VRW55335.2022.00026"},{"key":"10435_CR48","doi-asserted-by":"publisher","unstructured":"Moreno-Lumbreras D, Minelli R, Villaverde A, Gonzalez-Barahona JM, Lanza M (2021) CodeCity: On-Screen or in virtual reality? In: Working Conference on Software Visualization, VISSOFT 2021, Luxembourg, September 27-28, 2021. IEEE, pp 12\u201322. Available from: https:\/\/doi.org\/10.1109\/VISSOFT52517.2021.00011","DOI":"10.1109\/VISSOFT52517.2021.00011"},{"key":"10435_CR49","unstructured":"Moreno-Lumbreras D, Robles G, Izquierdo-Cortazar D, Gonz\u00e1lez-Barahona JM (2021) To VR or not to VR: Is virtual reality suitable to understand software development metrics? CoRR. arXiv:2109.13768"},{"key":"10435_CR50","doi-asserted-by":"publisher","DOI":"10.1201\/b17511","volume-title":"Visualization analysis and design","author":"T Munzner","year":"2014","unstructured":"Munzner T (2014) Visualization analysis and design. CRC Press"},{"key":"10435_CR51","doi-asserted-by":"crossref","unstructured":"Natephra W, Motamedi A (2019) Live data visualization of IoT sensors using augmented reality (AR) and BIM. In: 36th International symposium on automation and robotics in construction (ISARC 2019)","DOI":"10.22260\/ISARC2019\/0084"},{"key":"10435_CR52","doi-asserted-by":"publisher","unstructured":"Olshannikova E, Ometov A, Koucheryavy Y, Olsson T (2015) Visualizing big data with augmented and virtual reality: challenges and research agenda. Journal of Big Data 2. https:\/\/doi.org\/10.1186\/s40537-015-0031-2","DOI":"10.1186\/s40537-015-0031-2"},{"key":"10435_CR53","unstructured":"Playfair W (1822) A Letter on Our Agricultural Distresses, Their Causes and Remedies: Accompanied with Tables and Copper-plate Charts, Shewing and Comparing the Prices of Wheat, Bread and Labour from 1565 to 1821. 23431. W. Sams"},{"key":"10435_CR54","doi-asserted-by":"publisher","unstructured":"Ragan ED, Kopper R, Schuchardt P, Bowman DA (2013) Studying the effects of stereo, head tracking, and field of regard on a small-scale spatial judgment task. IEEE Trans Vis Comput Graph 19(5):886\u2013896. https:\/\/doi.org\/10.1109\/TVCG.2012.163","DOI":"10.1109\/TVCG.2012.163"},{"key":"10435_CR55","unstructured":"Raja D, Bowman D, Lucas J, North C (2004) Exploring the benefits of immersion in abstract information visualization. In: Proc. Immersive Projection Technology Workshop vol 61, pp 69"},{"key":"10435_CR56","doi-asserted-by":"publisher","unstructured":"Ralph P (2021) ACM SIGSOFT empirical standards released. SIGSOFT Softw Eng Notes 46(1):19. https:\/\/doi.org\/10.1145\/3437479.3437483","DOI":"10.1145\/3437479.3437483"},{"key":"10435_CR57","doi-asserted-by":"publisher","unstructured":"Romano S, Capece N, Erra U, Scanniello G, Lanza M (2019) On the use of virtual reality in software visualization: the case of the city metaphor. Inf Software Technol 114:92\u2013106. https:\/\/doi.org\/10.1016\/j.infsof.2019.06.007","DOI":"10.1016\/j.infsof.2019.06.007"},{"key":"10435_CR58","doi-asserted-by":"crossref","unstructured":"Rosenbaum R, Bottleson J, Liu Z, Hamann B (2011) Involve me and i will understand!\u2013abstract data visualization in immersive environments. In: International symposium on visual computing. Springer, pp 530\u2013540","DOI":"10.1007\/978-3-642-24028-7_49"},{"key":"10435_CR59","doi-asserted-by":"crossref","unstructured":"R\u00fcdel MO, Ganser J, Koschke R (2018) A controlled experiment on spatial orientation in VR-based software cities. In: 2018 IEEE Working conference on software visualization (VISSOFT), pp 21\u201331","DOI":"10.1109\/VISSOFT.2018.00011"},{"key":"10435_CR60","doi-asserted-by":"crossref","unstructured":"Sadowski C, S\u00f6derberg E, Church L, Sipko M, Bacchelli A (2018) Modern code review: a case study at google. In: Proceedings of the 40th international conference on software engineering: software engineering in practice, pp 181\u2013190","DOI":"10.1145\/3183519.3183525"},{"key":"10435_CR61","unstructured":"Saket B, Endert A, \u00c7agatay Demiralp (2017) Data and task based effectiveness of basic visualizations. arXiv:1709.08546"},{"key":"10435_CR62","doi-asserted-by":"publisher","unstructured":"Schuemie M, Straaten P, Krijn M, Mast C (2001) Research on presence in virtual reality: a survey. cyberpsychology and behavior: the impact of the Internet, multimedia and virtual reality on behavior and society 05(4):183\u2013201. https:\/\/doi.org\/10.1089\/109493101300117884","DOI":"10.1089\/109493101300117884"},{"key":"10435_CR63","doi-asserted-by":"crossref","unstructured":"Sillito J, Murphy GC, De\u00a0Volder K (2006) Questions Programmers Ask during Software Evolution Tasks. In: Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering. SIGSOFT \u201906\/FSE-14. New York, NY, USA: Association for Computing Machinery, pp 23\u201334","DOI":"10.1145\/1181775.1181779"},{"key":"10435_CR64","doi-asserted-by":"publisher","first-page":"647997","DOI":"10.3389\/frvir.2021.647997","volume":"2","author":"R Skarbez","year":"2021","unstructured":"Skarbez R, Smith M, Whitton MC (2021) Revisiting milgram and kishino\u2019s reality-virtuality continuum. Frontiers in Virtual Reality 2:647997","journal-title":"Frontiers in Virtual Reality"},{"key":"10435_CR65","doi-asserted-by":"publisher","unstructured":"Ometov A, Olshannikova E, Olsson T, Koucheryavy Y (2015) Visualizing Big Data with augmented and virtual reality: challenges and research agenda. Journal of Big Data 2. https:\/\/doi.org\/10.1186\/s40537-015-0031-2","DOI":"10.1186\/s40537-015-0031-2"},{"key":"10435_CR66","doi-asserted-by":"crossref","unstructured":"Sun C, Lo D, Khoo SC, Jiang J (2011) Towards more accurate retrieval of duplicate bug reports. In: 2011 26th IEEE\/ACM international conference on automated software engineering (ASE 2011), pp 253\u2013262","DOI":"10.1109\/ASE.2011.6100061"},{"key":"10435_CR67","doi-asserted-by":"crossref","unstructured":"Sun C, Lo D, Wang X, Jiang J, Khoo SC (2010) A discriminative model approach for accurate duplicate bug report retrieval. In: 2010 ACM\/IEEE 32nd International conference on software engineering vol 1, pp 45\u201354","DOI":"10.1145\/1806799.1806811"},{"issue":"2","key":"10435_CR68","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1007\/s10664-016-9452-6","volume":"22","author":"P Thongtanunam","year":"2017","unstructured":"Thongtanunam P, McIntosh S, Hassan AE, Iida H (2017) Review participation in modern code review. Empirical Software Engineering 22(2):768\u2013817","journal-title":"Empirical Software Engineering"},{"key":"10435_CR69","doi-asserted-by":"crossref","unstructured":"Tian Y, Sun C, Lo D (2012) Improved duplicate bug report identification. In: 2012 16th European conference on software maintenance and reengineering, pp 385\u2013390","DOI":"10.1109\/CSMR.2012.48"},{"key":"10435_CR70","unstructured":"Tufte ER (2001) The Visual Display of Quantitative Information, 2nd edn. Graphics Press, Cheshire, CT"},{"key":"10435_CR71","doi-asserted-by":"publisher","unstructured":"Vegas S, Apa C, Juristo N (2016) Crossover designs in software engineering experiments: benefits and perils. IEEE Trans Software Eng 42(2):120\u2013135. https:\/\/doi.org\/10.1109\/TSE.2015.2467378","DOI":"10.1109\/TSE.2015.2467378"},{"key":"10435_CR72","doi-asserted-by":"crossref","unstructured":"Wettel R, Lanza M (2007) Visualizing software systems as cities. In: 2007 4th IEEE International workshop on visualizing software for understanding and analysis, pp 92\u201399","DOI":"10.1109\/VISSOF.2007.4290706"},{"key":"10435_CR73","doi-asserted-by":"crossref","unstructured":"Wettel R, Lanza M, Robbes R (2011) Software systems as cities: a controlled experiment. In: 2011 33rd International conference on software engineering (ICSE), pp 551\u2013560","DOI":"10.1145\/1985793.1985868"},{"key":"10435_CR74","doi-asserted-by":"crossref","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A (2012) Experimentation in software engineering. Springer Science & Business Media","DOI":"10.1007\/978-3-642-29044-2"},{"key":"10435_CR75","doi-asserted-by":"crossref","unstructured":"Yano K, Matsuo A (2017) Data access visualization for legacy application maintenance. In: 2017 IEEE 24th International conference on software analysis, evolution and reengineering (SANER), pp 546\u2013550","DOI":"10.1109\/SANER.2017.7884671"},{"key":"10435_CR76","doi-asserted-by":"crossref","unstructured":"Young P, Munro M (1998) Visualising software in virtual reality. In: Proceedings. 6th international workshop on program comprehension. IWPC\u201998 (Cat. No.98TB100242), pp 19\u201326","DOI":"10.1109\/WPC.1998.693276"},{"key":"10435_CR77","doi-asserted-by":"crossref","unstructured":"Yu Y, Wang H, Filkov V, Devanbu P, Vasilescu B (2015) Wait for it: determinants of pull request evaluation latency on github. In: 2015 IEEE\/ACM 12th working conference on mining software repositories. IEEE, pp 367\u2013371","DOI":"10.1109\/MSR.2015.42"},{"key":"10435_CR78","doi-asserted-by":"crossref","unstructured":"Zhang S, Demiralp C, Keefe DF, DaSilva M, Laidlaw DH, Greenberg BD et\u00a0al (2001) An immersive virtual environment for DT-MRI volume visualization applications: a case study. In: Proceedings visualization, 2001. VIS \u201901, pp 437\u2013584","DOI":"10.1109\/VISUAL.2001.964545"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10435-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10435-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10435-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T02:19:30Z","timestamp":1711160370000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10435-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,3]]},"references-count":78,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10435"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10435-3","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,3]]},"assertion":[{"value":"10 December 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 February 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}}],"article-number":"42"}}