{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T14:42:40Z","timestamp":1775745760635,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,3,8]],"date-time":"2023-03-08T00:00:00Z","timestamp":1678233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Funds through the FCT\u2014Foundation for Science and Technology","award":["UIDB\/00127\/2020"],"award-info":[{"award-number":["UIDB\/00127\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>This work presents a novel framework for web-based environment-aware rendering and interaction in augmented reality based on WebXR and three.js. It aims at accelerating the development of device-agnostic Augmented Reality (AR) applications. The solution allows for a realistic rendering of 3D elements, handles geometry occlusion, casts shadows of virtual objects onto real surfaces, and provides physics interaction with real-world objects. Unlike most existing state-of-the-art systems that are built to run on a specific hardware configuration, the proposed solution targets the web environment and is designed to work on a vast range of devices and configurations. Our solution can use monocular camera setups with depth data estimated by deep neural networks or, when available, use higher-quality depth sensors (e.g., LIDAR, structured light) that provide a more accurate perception of the environment. To ensure consistency in the rendering of the virtual scene a physically based rendering pipeline is used, in which physically correct attributes are associated with each 3D object, which, combined with lighting information captured by the device, enables the rendering of AR content matching the environment illumination. All these concepts are integrated and optimized into a pipeline capable of providing a fluid user experience even on middle-range devices. The solution is distributed as an open-source library that can be integrated into existing and new web-based AR projects. The proposed framework was evaluated and compared in terms of performance and visual features with two state-of-the-art alternatives.<\/jats:p>","DOI":"10.3390\/jimaging9030063","type":"journal-article","created":{"date-parts":[[2023,3,8]],"date-time":"2023-03-08T03:29:05Z","timestamp":1678246145000},"page":"63","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Environment-Aware Rendering and Interaction in Web-Based Augmented Reality"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7111-4070","authenticated-orcid":false,"given":"Jos\u00e9","family":"Ferr\u00e3o","sequence":"first","affiliation":[{"name":"Department of Electronics, Telecommunications, and Informatics (DETI), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3754-2749","authenticated-orcid":false,"given":"Paulo","family":"Dias","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications, and Informatics (DETI), University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Intelligent System Associate Laboratory (LASI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2219-2731","authenticated-orcid":false,"given":"Beatriz Sousa","family":"Santos","sequence":"additional","affiliation":[{"name":"Department of Electronics, Telecommunications, and Informatics (DETI), University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Intelligent System Associate Laboratory (LASI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9288-5058","authenticated-orcid":false,"given":"Miguel","family":"Oliveira","sequence":"additional","affiliation":[{"name":"Intelligent System Associate Laboratory (LASI), Institute of Electronics and Informatics Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Department of Mechanical Engineering (DEM), University of Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Broll, W. (2022). Virtual and Augmented Reality (VR\/AR): Foundations and Methods of Extended Realities (XR), Springer.","DOI":"10.1007\/978-3-030-79062-2_8"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.cag.2021.10.009","article-title":"A vision for contextualized evaluation of remote collaboration supported by AR","volume":"102","author":"Marques","year":"2021","journal-title":"Comput. Graph."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Luo, W., Goebel, E., Reipschl\u00e4ger, P., Mats, O.E., and Dachselt, R. (2021, January 4\u20138). Exploring and Slicing Volumetric Medical Data in Augmented Reality Using a Spatially-Aware Mobile Device. Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bari, Italy.","DOI":"10.1109\/ISMAR-Adjunct54149.2021.00076"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1007\/s10055-020-00492-0","article-title":"Virtual, mixed, and augmented reality: A systematic review for immersive systems research","volume":"25","author":"Liberatore","year":"2021","journal-title":"Virtual Real."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1007\/s10055-017-0319-y","article-title":"Real-time adjustment of contrast saliency for improved information visibility in mobile augmented reality","volume":"22","author":"Ahn","year":"2018","journal-title":"Virtual Real."},{"key":"ref_6","unstructured":"Ungureanu, D., Bogo, F., Galliani, S., Sama, P., Duan, X., Meekhof, C., St\u00fchmer, J., Cashman, T.J., Tekin, B., and Sch\u00f6nberger, J.L. (2020). HoloLens 2 Research Mode as a Tool for Computer Vision Research. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kyt\u00f6, M., Ens, B., Piumsomboon, T., Lee, G.A., and Billinghurst, M. (2018, January 21\u201326). Pinpointing: Precise head- and eye-based target selection for augmented reality. Proceedings of the Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173655"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Battisti, C., Messelodi, S., and Poiesi, F. (2018, January 16\u201320). Seamless Bare-Hand Interaction in Mixed Reality. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Munich, Germany.","DOI":"10.1109\/ISMAR-Adjunct.2018.00066"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3306346.3322987","article-title":"Foveated AR: Dynamically-foveated augmented reality display","volume":"38","author":"Kim","year":"2019","journal-title":"ACM Trans. Graph."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, W., Zhu, D., Wang, X., Hu, Y., Qiu, Y., Wang, C., Hu, Y., Kapoor, A., and Scherer, S. (2020, January 23\u201327). TartanAir: A Dataset to Push the Limits of Visual SLAM. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Kyoto, Japan.","DOI":"10.1109\/IROS45743.2020.9341801"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Maclntyre, B., and Smith, T.F. (2018, January 16\u201320). Thoughts on the Future of WebXR and the Immersive Web. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2018, Munich, Germany.","DOI":"10.1109\/ISMAR-Adjunct.2018.00099"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Herskovitz, J., Wu, J., White, S., Pavel, A., Reyes, G., Guo, A., and Bigham, J.P. (2020, January 18\u201322). Making Mobile Augmented Reality Applications Accessible. Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility, Virtual Event.","DOI":"10.1145\/3373625.3417006"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1007\/s10055-020-00476-0","article-title":"Implementation of escape room system based on augmented reality involving deep convolutional neural network","volume":"25","author":"Zeng","year":"2021","journal-title":"Virtual Real."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pereira, N., Rowe, A., Farb, M.W., Liang, I., Lu, E., and Riebling, E. (2021, January 4\u20138). Hybrid Conference Experiences in the ARENA. Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented RealityAdjunct, Bari, Italy.","DOI":"10.1109\/ISMAR-Adjunct54149.2021.00103"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.4236\/ce.2017.89101","article-title":"A Comparative Analysis of Augmented Reality Frameworks Aimed at the Development of Educational Applications","volume":"08","author":"Herpich","year":"2017","journal-title":"Creat. Educ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1111\/1467-8659.1530011","article-title":"Interactive Occlusion and Automatic Object Placement for Augmented Reality","volume":"15","author":"Breen","year":"1996","journal-title":"Comput. Graph. Forum"},{"key":"ref_17","unstructured":"Tr\u00f6ster, S. (2016). Optimization of Augmented Reality Applications considering the Depth Information with Googles Project Tango, Cologne University of Applied Sciences."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Marder-Eppstein, E. (2016, January 24\u201328). Project Tango. Proceedings of the ACM SIGGRAPH 2016 Real-Time Live!, Anaheim, CA, USA.","DOI":"10.1145\/2933540.2933550"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3272127.3275083","article-title":"Fast depth densification for occlusion-aware augmented reality","volume":"37","author":"Holynski","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cao, D. (2020, January 25\u201326). Real-time multi-user spatial collaboration using ARCore. Proceedings of the IEEE\/ACM 7th International Conference on Mobile Software Engineering and Systems, Seoul, Republic of Korea.","DOI":"10.1145\/3387905.3388601"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1007\/978-3-030-19501-4_36","article-title":"Capabilities of ARCore and ARKit Platforms for AR\/VR Applications","volume":"987","author":"Nowacki","year":"2020","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Feigl, T., Porada, A., Steiner, S., L\u00f6ffler, C., Mutschler, C., and Philippsen, M. (2020, January 27\u201329). Localization limitations of ARCore, ARKit, and hololens in dynamic large-scale industry environments. Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2020, Valletta, Malta.","DOI":"10.5220\/0008989903070318"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Du, R., Turner, E., Dzitsiuk, M., Prasso, L., Duarte, I., Dourgarian, J., Afonso, J., Pascoal, J., Gladstone, J., and Cruces, N. (2020, January 20\u201323). DepthLab: Real-time 3D interaction with depth maps for mobile augmented reality. Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, UIST 2020, Minneapolis, MN, USA.","DOI":"10.1145\/3379337.3415881"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, S., Wu, Y., Liu, Y., Wang, D., Wen, M., Tao, Y., Sui, Y., and Liu, Y. (2020, January 12\u201315). An exploratory study of bugs in extended reality applications on the web. Proceedings of the International Symposium on Software Reliability Engineering, ISSRE, Coimbra, Portugal.","DOI":"10.1109\/ISSRE5003.2020.00025"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"255","DOI":"10.5194\/isprs-archives-XLIII-B2-2020-255-2020","article-title":"Marker-Less Mobile Augmented Reality Application for Massive 3D Point Clouds and Semantics","volume":"XLIII-B2-2","author":"Kharroubi","year":"2020","journal-title":"Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_26","first-page":"1","article-title":"Real-time rendering on a power budget","volume":"35","author":"Wang","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_27","unstructured":"Agusanto, K., Li, L., Zhu, C., and Ng, W.S. (2003, January 7\u201310). Photorealistic rendering for augmented reality using environment illumination. Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR 2003, Washington, DC, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Haller, M. (2004, January 16\u201318). Photorealism and Non-Photorealism in Augmented Reality. Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry, VRCAI 04, Singapore.","DOI":"10.1145\/1044588.1044627"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Legendre, C., Ma, W.C., Fyffe, G., Flynn, J., Charbonnel, L., Busch, J., and Debevec, P. (2019, January 17\u201319). Deeplight: Learning illumination for unconstrained mobile mixed reality. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, PR, USA.","DOI":"10.1109\/CVPR.2019.00607"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lee, D., Shim, W., Lee, M., Lee, S., Jung, K.D., and Kwon, S. (2021). Performance evaluation of ground ar anchor with webxr device api. Appl. Sci., 11.","DOI":"10.3390\/app11177877"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Su, M., Guo, R., Wang, H., Wang, S., and Niu, P. (2017, January 18\u201320). View frustum culling algorithm based on optimized scene management structure. Proceedings of the 2017 IEEE International Conference on Information and Automation, ICIA 2017, Macau, China.","DOI":"10.1109\/ICInfA.2017.8079019"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Blinn, J.F. (1977, January 20\u201322). Models of light reflection for computer synthesized pictures. Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1977, San Jose, CA, USA.","DOI":"10.1145\/563858.563893"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MCG.2006.13","article-title":"BRDF-Shop: Creating physically correct bidirectional reflectance distribution functions","volume":"26","author":"Colbert","year":"2006","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1109\/TPAMI.2007.1162","article-title":"A theory of frequency domain invariants: Spherical harmonic identities for BRDF\/lighting transfer and image consistency","volume":"30","author":"Mahajan","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","first-page":"1","article-title":"Efficient rendering of layered materials using an atomic decomposition with statistical operators","volume":"37","author":"Belcour","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, N., and Pang, M.Y. (2009, January 18\u201320). Shadow mapping algorithms: A complete survey. Proceedings of the 1st International Symposium on Computer Network and Multimedia Technology, CNMT 2009, Wuhan, China.","DOI":"10.1109\/CNMT.2009.5374715"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Dou, H., Yan, Y., Kerzner, E., Dai, Z., and Wyman, C. (2014, January 14\u201316). Adaptive depth bias for shadow maps. Proceedings of the Symposium on Interactive 3D Graphics, San Francisco, CA, USA.","DOI":"10.1145\/2556700.2556706"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1145\/37402.37435","article-title":"Rendering Antialiased Shadows with Depth Maps","volume":"21","author":"Reeves","year":"1987","journal-title":"Comput. Graph. (ACM)"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Donnelly, W., and Lauritzen, A. (2006, January 14\u201317). Variance shadow maps. Proceedings of the Symposium on Interactive 3D Graphics, Redwood City, CA, USA.","DOI":"10.1145\/1111411.1111440"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Fernando, R. (August, January 31). Percentage-closer soft shadows. Proceedings of the ACM SIGGRAPH 2005 Sketches on\u2014SIGGRAPH \u201905, Los Angeles, CA, USA.","DOI":"10.1145\/1187112.1187153"},{"key":"ref_41","unstructured":"Lacoursiere, C. (2007). Ghosts and Machines: Regularized Variational Methods for Interactive Simulations of Multibodies with Dry Frictional Contacts. [Ph.D. Thesis, Umea University]."},{"key":"ref_42","unstructured":"Agha-Mohammadi, A.-A., Heiden, E., Hausman, K., and Sukhatme, G.S. (2017, January 24\u201328). Confidence-aware Occupancy Grids. Proceedings of the IROS Workshop on Vision-Based Agile Autonomous Navigation of UAVs, Vancouver, BC, Canada."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1145\/235815.235821","article-title":"The Quickhull Algorithm for Convex Hulls","volume":"22","author":"Barber","year":"1996","journal-title":"ACM Trans. Math. Softw."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2982439","article-title":"GST: GPU-decodable Supercompressed Textures","volume":"35","author":"Krajcevski","year":"2016","journal-title":"ACM Trans. Graph."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/3\/63\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:50:28Z","timestamp":1760122228000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/3\/63"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,8]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["jimaging9030063"],"URL":"https:\/\/doi.org\/10.3390\/jimaging9030063","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,8]]}}}