{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T13:00:48Z","timestamp":1777899648147,"version":"3.51.4"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Grant No. 62372219"],"award-info":[{"award-number":["Grant No. 62372219"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Research Grants Council of the Hong Kong Special Administrative Region, China","award":["No. CUHK 14206921 of the General Research Fund"],"award-info":[{"award-number":["No. CUHK 14206921 of the General Research Fund"]}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["Project No. 2023A1515011959"],"award-info":[{"award-number":["Project No. 2023A1515011959"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shenzhen International Science and Technology Cooperation Project","award":["No. GJHZ20220913143008015"],"award-info":[{"award-number":["No. GJHZ20220913143008015"]}]},{"name":"Shenzhen-Hong Kong Joint Funding Project","award":["No. SGDX20230116091246007"],"award-info":[{"award-number":["No. SGDX20230116091246007"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>The quality of Virtual Reality (VR) apps is vital, particularly the rendering quality of the VR Graphical User Interface (GUI). Different from traditional two-dimensional (2D) apps, VR apps create a 3D digital scene for users, by rendering two distinct 2D images for the user\u2019s left and right eyes, respectively. Stereoscopic visual inconsistency (denoted as \u201cSVI\u201d) issues, however, undermine the rendering process of the user\u2019s brain, leading to user discomfort and even adverse health effects. Such issues commonly exist in VR apps but remain underexplored. To comprehensively understand the SVI issues, we conduct an empirical analysis on 282 SVI bug reports collected from 15 VR platforms, summarizing 15 types of manifestations of the issues. The empirical analysis reveals that automatically detecting SVI issues is challenging, mainly because: (1) lack of training data; (2) the manifestations of SVI issues are diverse, complicated, and often application-specific; (3) most accessible VR apps are closed-source commercial software, we have no access to code, scene configurations, etc. for issue detection. Our findings imply that the existing pattern-based supervised classification approaches may be inapplicable or ineffective in detecting the SVI issues.<\/jats:p>\n                  <jats:p>\n                    To counter these challenges, we propose an unsupervised black-box testing framework named\n                    <jats:sc>Stereo<\/jats:sc>\n                    ID to identify the stereoscopic visual inconsistencies, based only on the rendered GUI states.\n                    <jats:sc>Stereo<\/jats:sc>\n                    ID generates a synthetic right-eye image based on the actual left-eye image and computes distances between the synthetic right-eye image and the actual right-eye image to detect SVI issues. We propose a depth-aware conditional stereo image translator to power the image generation process, which captures the expected perspective shifts between left-eye and right-eye images. We build a large-scale unlabeled VR stereo screenshot dataset with larger than 171K images from 288 real-world VR apps, which can be utilized to train our depth-aware conditional stereo image translator and evaluate the whole testing framework\n                    <jats:sc>Stereo<\/jats:sc>\n                    ID. After substantial experiments, depth-aware conditional stereo image translator demonstrates superior performance in generating stereo images, outpacing traditional architectures. It achieved the lowest average L1 and L2 losses and the highest SSIM score, signifying its effectiveness in pixel-level accuracy and structural consistency for VR apps.\n                    <jats:sc>Stereo<\/jats:sc>\n                    ID further demonstrates its power for detecting SVI issues in both user reports and wild VR apps. In summary, this novel framework enables effective detection of elusive SVI issues, benefiting the quality of VR apps.\n                  <\/jats:p>","DOI":"10.1145\/3660803","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"2167-2189","source":"Crossref","is-referenced-by-count":10,"title":["Less Cybersickness, Please: Demystifying and Detecting Stereoscopic Visual Inconsistencies in Virtual Reality Apps"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6323-1402","authenticated-orcid":false,"given":"Shuqing","family":"Li","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4774-2434","authenticated-orcid":false,"given":"Cuiyun","family":"Gao","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8262-9608","authenticated-orcid":false,"given":"Jianping","family":"Zhang","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3869-6344","authenticated-orcid":false,"given":"Yujia","family":"Zhang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8147-8126","authenticated-orcid":false,"given":"Yepang","family":"Liu","sequence":"additional","affiliation":[{"name":"Southern University of Science and Technology, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5831-9474","authenticated-orcid":false,"given":"Jiazhen","family":"Gu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1936-5598","authenticated-orcid":false,"given":"Yun","family":"Peng","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3666-5798","authenticated-orcid":false,"given":"Michael R.","family":"Lyu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"2021. FILM XR https:\/\/vrfilmreview.ru\/."},{"key":"e_1_3_1_3_2","unstructured":"2021. VirtualSkill - Virtual Reality Training https:\/\/virtualskill.com\/."},{"key":"e_1_3_1_4_2","unstructured":"2021. XR Games https:\/\/www.xrgames.io\/."},{"key":"e_1_3_1_5_2","unstructured":"2022. Oculus App Lab https:\/\/developer.oculus.com\/blog\/introducing-app-lab-a-new-way-to-distribute-oculus-quest-apps\/."},{"key":"e_1_3_1_6_2","unstructured":"2022. Oculus App Store https:\/\/www.oculus.com\/experiences\/quest\/."},{"key":"e_1_3_1_7_2","unstructured":"2022. SideQuest https:\/\/sidequestvr.com\/."},{"key":"e_1_3_1_8_2","unstructured":"2023. Epic Developer Community Forums https:\/\/forums.unrealengine.com\/."},{"key":"e_1_3_1_9_2","unstructured":"2023. GitHub https:\/\/github.com\/."},{"key":"e_1_3_1_10_2","unstructured":"2023. GitHub Repository of ValveSoftware\/SteamVR-for-Linux https:\/\/github.com\/ValveSoftware\/SteamVR-for-Linux."},{"key":"e_1_3_1_11_2","unstructured":"2023. Introducing SteamVR Home Beta. https:\/\/steamcommunity.com\/games\/250820\/announcements\/detail\/1256913672017157095."},{"key":"e_1_3_1_12_2","unstructured":"2023. Meta Community Forums https:\/\/communityforums.atmeta.com\/."},{"key":"e_1_3_1_13_2","unstructured":"2023. Post-processing and Full-screen Effects https:\/\/docs.unity3d.com\/Manual\/PostProcessingOverview.html."},{"key":"e_1_3_1_14_2","unstructured":"2023. Post-processing Effects https:\/\/docs.unity3d.com\/Packages\/com.unity.render-pipelines.high-definition@12.0\/manual\/post-processing-effect-list.html."},{"key":"e_1_3_1_15_2","unstructured":"2023. Stack Overflow https:\/\/stackoverflow.com\/."},{"key":"e_1_3_1_16_2","unstructured":"2023. Steam Community https:\/\/steamcommunity.com\/."},{"key":"e_1_3_1_17_2","unstructured":"2023. Unity Discussions https:\/\/discussions.unity.com\/."},{"key":"e_1_3_1_18_2","unstructured":"2023. Unity Forum https:\/\/forum.unity.com\/."},{"key":"e_1_3_1_19_2","unstructured":"2023. Unity Forumml: Right Eye Discrepancies https:\/\/forum.unity.com\/threads\/right-eye-discrepancies-oculus-urp-obi-fluid-shader.1047632\/."},{"key":"e_1_3_1_20_2","unstructured":"2023. Unity Issue Tracker https:\/\/issuetracker.unity3d.com\/."},{"key":"e_1_3_1_21_2","unstructured":"2023. Unreal Engine Forums: Eyes Sometimes Show Different LODs in VR https:\/\/forums.unrealengine.com\/t\/eyes-sometimes-show-different-lods-in-vr\/389839."},{"key":"e_1_3_1_22_2","unstructured":"2023. Unreal Engine Forums: Vive Eyes Displacements https:\/\/forums.unrealengine.com\/t\/vive-eyes-displacements\/101890."},{"key":"e_1_3_1_23_2","unstructured":"2023. Unreal Engine Issues and Bug Tracker https:\/\/issues.unrealengine.com\/."},{"key":"e_1_3_1_24_2","unstructured":"2023. Visual Effect Graph https:\/\/docs.unity3d.com\/2023.2\/Documentation\/Manual\/VFXGraph.html."},{"key":"e_1_3_1_25_2","unstructured":"2023. VIVE Forum https:\/\/forum.htc.com\/."},{"key":"e_1_3_1_26_2","unstructured":"2023. VIVEPORT https:\/\/www.viveport.com\/."},{"key":"e_1_3_1_27_2","unstructured":"2023. VR Content on Steam App Store https:\/\/store.steampowered.com\/search\/?vrsupport=401."},{"key":"e_1_3_1_28_2","unstructured":"2023. VR Playtesting Guide https:\/\/developer.oculus.com\/resources\/playtest-guide\/."},{"key":"e_1_3_1_29_2","first-page":"427","volume-title":"In SOUPS","author":"Devon Adams","year":"2018","unstructured":"DevonAdams, AlsenyBah, CatherineBarwulor, NureliMusaby, KadeemPitkin, and ElissaM. Redmiles. 2018. Ethics Emerging: the Story of Privacy and Security Perceptions in Virtual Reality In SOUPS. USENIX Association, 427\u2013442."},{"key":"e_1_3_1_30_2","first-page":"214","article-title":"Wasserstein generative adversarial networks","author":"Martin Arjovsky","year":"2017","unstructured":"MartinArjovsky, SoumithChintala, and L\u00e9onBottou. 2017. Wasserstein generative adversarial networks. In International Conference on Machine Learning. PMLR, 214\u2013223.","journal-title":"In International Conference on Machine Learning"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1778351"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2019.00058"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","unstructured":"KeChen YufeiLi YingfengChen ChangjieFan ZhipengHu and WeiYang. 2021. GLIB: Towards Automated Test Oracle for Graphically-Rich Applications In ESEC\/FSE. ACM 1093\u20131104. https:\/\/doi.org\/10.1145\/3468264.3468586 10.1145\/3468264.3468586","DOI":"10.1145\/3468264.3468586"},{"key":"e_1_3_1_34_2","volume-title":"Qualitative Inquiry and Research Design: Choosing Among Five Approaches","author":"John W Creswell","year":"2016","unstructured":"JohnW Creswell and CherylN Poth. 2016. Qualitative Inquiry and Research Design: Choosing Among Five Approaches Sage Publications."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.3991\/ijet.v14i03.9289"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TG.2021.3057288"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-60703-6_18"},{"key":"e_1_3_1_38_2","article-title":"Improved Training of Wasserstein GANs","volume":"30","author":"Ishaan Gulrajani","year":"2017","unstructured":"IshaanGulrajani, FarukAhmed, MartinArjovsky, VincentDumoulin, and AaronC Courville. 2017. Improved Training of Wasserstein GANs Advances in Neural Information Processing Systems, 30 (2017).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.3390\/E25081212"},{"key":"e_1_3_1_40_2","first-page":"1125","article-title":"Image-to-Image Translation with Conditional Adversarial Networks","author":"Phillip Isola","year":"2017","unstructured":"PhillipIsola, Jun-YanZhu, TinghuiZhou, and AlexeiA. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1125\u20131134.","journal-title":"In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298762"},{"issue":"23","key":"e_1_3_1_42_2","article-title":"Augmented and virtual reality in surgery-the digital surgical environment: applications, limitations and legal pitfalls","volume":"4","author":"Wee sim Khor","year":"2016","unstructured":"Wee simKhor, BenjaminBaker, KavitAmin, AdrianChan, KetanPatel, and JasonWong. 2016. Augmented and virtual reality in surgery-the digital surgical environment: applications, limitations and legal pitfalls Annals of Translational Medicine, 4(23).","journal-title":"Annals of Translational Medicine"},{"key":"e_1_3_1_43_2","article-title":"Adamml: A method for stochastic optimization","author":"Diederik P Kingma","year":"2014","unstructured":"DiederikP Kingma and JimmyBa. 2014. Adamml: A method for stochastic optimization arXiv preprint arXiv:1412.6980.","journal-title":"arXiv preprint arXiv:1412.6980"},{"issue":"1","key":"e_1_3_1_44_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1145\/333329.333344","article-title":"A Discussion of Cybersickness in Virtual Environments","volume":"32","author":"Joseph J. LaViola","year":"2000","unstructured":"JosephJ. LaViola. 2000. A Discussion of Cybersickness in Virtual Environments ACM SIGCHI Bull., 32(1): 47-56.","journal-title":"ACM SIGCHI Bull."},{"key":"e_1_3_1_45_2","volume-title":"In 2017 IEEE International Conference on Image Processing, ICIP 2017, Beijing, China, September 17-20, 2017","author":"Jiyoung Lee","year":"2017","unstructured":"JiyoungLee, HyungjooJung, YoungjungKim, and KwanghoonSohn. 2017. Automatic 2D-to-3D conversion using multi-scale deep neural network In 2017 IEEE International Conference on Image Processing, ICIP 2017, Beijing, China, September 17-20, 2017, IEEE, 730-734."},{"key":"e_1_3_1_46_2","unstructured":"ShuqingLi BinchangLi CuiyunGao and ichaelR. Lyu. 2024. An Interaction Simulation and Automated Testing Framework for Spatial Computing Extended Reality Applications."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2308.06783"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE5003.2020.00025"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.17"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","unstructured":"LiuZhe ChenChunyang WangJunjie HuangYuekai HuJun and WangQing. 2020. Owl Eyes: Spotting UI Display Issues via Visual Understanding In ASE IEEE 398-409. https:\/\/doi.org\/10.1145\/3324884.3416547 10.1145\/3324884.3416547","DOI":"10.1145\/3324884.3416547"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556913"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1162\/pres.1992.1.3.311"},{"key":"e_1_3_1_53_2","article-title":"Conditional generative adversarial nets","author":"Mirza Mehdi","year":"2014","unstructured":"MirzaMehdi and OsinderoSimon. 2014. Conditional generative adversarial nets arXiv preprint arXiv:1411.1784.","journal-title":"arXiv preprint arXiv:1411.1784"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00221-016-4846-7"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0003-6870(02)00020-0"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00052"},{"key":"e_1_3_1_57_2","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"PaszkeAdam, GrossSam, MassaFrancisco, LererAdam, BradburyJames, ChananGregory, KilleenTrevor, LinZeming, GimelsheinNatalia, AntigaLuca, et-al. 2019. Pytorch: An imperative style, high-performance deep learning library Advances in Neural Information Processing Systems,32:","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376847"},{"key":"e_1_3_1_59_2","article-title":"Making Reality Virtual: How VR \u201cTricks\u201d Your Brain","volume":"6","author":"Penn Rebecca A","year":"2018","unstructured":"PennRebecca A and HoutMichae C. 2018. Making Reality Virtual: How VR \u201cTricks\u201d Your Brain Frontiers for Young Minds,6:","journal-title":"Frontiers for Young Minds"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3561160"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3019967"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.65"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598134"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00805"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1002\/STVR.1690"},{"key":"e_1_3_1_67_2","unstructured":"Statista. 2022. Report of Active Virtual Reality Users Worldwide Statista https:\/\/www.statista.com\/statistics\/426469\/active-virtual-reality-users-worldwide\/."},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510454.3516870"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00197"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3064819"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_51"},{"key":"e_1_3_1_72_2","unstructured":"XuBing WangNaiyan ChenTianqi and LiMu. 2015). Empirical evaluation of rectified activations in convolutional network arXiv preprint arXiv:1505.00853."},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380411"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.244"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIPR49039.2020.00069"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660803","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3660803","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T08:04:49Z","timestamp":1770192289000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660803"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":74,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3660803"],"URL":"https:\/\/doi.org\/10.1145\/3660803","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}