{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T11:30:25Z","timestamp":1764588625945,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>Dynamic 3D point clouds enable the immersive user experience and thus have become increasingly more popular in volumetric video streaming applications. When being streamed over best-effort networks, point cloud frames may suffer from lost or late packets, leading to non-trivial quality degradation. To solve this problem, we proposed the very first error concealment pipeline framework, which comprises five stages: pre-processing, matching, motion estimation, prediction, and post-processing. Alternative algorithms can be developed for each stage, while algorithms of different stages could be mixed and matched into pipelines for end-to-end performance evaluations. We discussed the design goal and proposed multiple algorithms for each stage. These algorithms were then quantitatively compared using dynamic 3D point cloud sequences with diverse characteristics. Based on the comparison results, we proposed four representative pipelines for: (i) diverse degrees of motion variance, i.e., minor versus significant, and (ii) different application requirements, i.e., high quality versus low overhead. Extensive end-to-end evaluations of our proposed pipelines demonstrated their superior concealed quality over the 3D frame-copy method in both: (i) 3D metrics, by up to 5.32 dB in GPSNR and 1.7 dB in CPSNR,and (ii) 2D metrics, by up to 2.22 dB in PSNR, 0.06 in SSIM, and 11.67 in VMAF. Adding to that, a user study with 15 subjects indicated that our best-performing pipeline achieved 100% preference winning rate over the state-of-the-art learning-based interpolation algorithms while consuming merely up to 8.55% of running time.<\/jats:p>","DOI":"10.1145\/3731561","type":"journal-article","created":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T14:34:09Z","timestamp":1745418849000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Composing Error Concealment Pipelines for Dynamic 3D Point Cloud Streaming"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9817-9931","authenticated-orcid":false,"given":"I-Chun","family":"Huang","sequence":"first","affiliation":[{"name":"National Tsing Hua University, Hsin-Chu, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7893-8512","authenticated-orcid":false,"given":"Yuang","family":"Shi","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-6069-0002","authenticated-orcid":false,"given":"Yuan-Chun","family":"Sun","sequence":"additional","affiliation":[{"name":"National Tsing Hua University, Hsin-Chu, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8994-1736","authenticated-orcid":false,"given":"Wei Tsang","family":"Ooi","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5503-9541","authenticated-orcid":false,"given":"Chun-Ying","family":"Huang","sequence":"additional","affiliation":[{"name":"National Yang Ming Chiao Tung University, Hsin-Chu, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8116-2591","authenticated-orcid":false,"given":"Cheng-Hsin","family":"Hsu","sequence":"additional","affiliation":[{"name":"National Tsing Hua University, Hsin-Chu, Taiwan"}]}],"member":"320","published-online":{"date-parts":[[2025,7]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1732","volume-title":"Proceedings of the IEEE International Conference on Image Processing (ICIP \u201915)","author":"Aaron Anne","year":"2015","unstructured":"Anne Aaron, Zhi Li, Megha Manohara, Joe Yuchieh Lin, Eddy Chi-Hao Wu, and C.-C. Jay Kuo. 2015. Challenges in cloud based ingest and encoding for high quality streaming media. In Proceedings of the IEEE International Conference on Image Processing (ICIP \u201915), 1732\u20131736."},{"key":"e_1_3_2_3_2","first-page":"2574","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP \u201922)","author":"Akhtar Anique","year":"2022","unstructured":"Anique Akhtar, Zhu Li, Geert Van der Auwera, and Jianle Chen. 2022. Dynamic point cloud interpolation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP \u201922), 2574\u20132578."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.153"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20185029"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1109\/TDPVT.2002.1024098","volume-title":"Proceedings of the First International Symposium on 3D Data Processing Visualization and Transmission (3DPVT \u201902)","author":"Davis J.","year":"2002","unstructured":"J. Davis, S. R. Marschner, M. Garr, and M. Levoy. 2002. Filling holes in complex surfaces using volumetric diffusion. In Proceedings of the First International Symposium on 3D Data Processing Visualization and Transmission (3DPVT \u201902), 428\u2013441."},{"key":"e_1_3_2_7_2","unstructured":"Eugene d\u2019Eon Bob Harrison Taos Myers and Philip Chou. 2017. 8i Voxelized Full Bodies\u2013A Voxelized Point Cloud Dataset. Retrieved from http:\/\/plenodb.jpeg.org\/pc\/8ilabs\/"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.3390\/s23125623"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3068606"},{"issue":"1","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"e13:1","DOI":"10.1017\/ATSIP.2020.12","article-title":"An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC)","volume":"9","author":"Graziosi D.","year":"2020","unstructured":"D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and A. Tabatabai. 2020. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing 9, 1 (2020), e13:1\u2013e13:17.","journal-title":"APSIPA Transactions on Signal and Information Processing"},{"key":"e_1_3_2_11_2","volume":"3","author":"Guennebaud Ga\u00ebl","year":"2010","unstructured":"Ga\u00ebl Guennebaud and Beno\u00eet Jacob. 2010. Eigen v3. Retrieved from http:\/\/eigen.tuxfamily.org","journal-title":"Eigen"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3005434"},{"issue":"5","key":"e_1_3_2_13_2","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1016\/j.jksuci.2020.08.011","article-title":"Stereo matching algorithm based on deep learning: A survey","volume":"34","author":"Saad Hamid Mohd","year":"2022","unstructured":"Mohd Saad Hamid, NurulFajar Abd Manap, Rostam Affendi Hamzah, and Ahmad Fauzan Kadmin. 2022. Stereo matching algorithm based on deep learning: A survey. Journal of King Saud University-Computer and Information Sciences 34, 5 (2022), 1663\u20131673.","journal-title":"Journal of King Saud University-Computer and Information Sciences"},{"key":"e_1_3_2_14_2","volume-title":"Proceedings of the ACM Symposium on Solid and Physical Modeling (SPM \u201908)","volume":"10","author":"Hoppe Hugues","year":"2008","unstructured":"Hugues Hoppe. 2008. Poisson surface reconstruction and its applications. In Proceedings of the ACM Symposium on Solid and Physical Modeling (SPM \u201908), 10."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2906554"},{"key":"e_1_3_2_16_2","unstructured":"I-Chun Huang Yuang Shi Yuan-Chun Sun Wei Tsang Ooi Chun-Ying Huang and Cheng-Hsin Hsu. 2023. Composing Error Concealment Pipelines for Dynamic 3D Point Cloud Streaming. Retrieved from https:\/\/github.com\/AIINS-NTHU\/PCEC"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601205"},{"key":"e_1_3_2_18_2","first-page":"3134","volume-title":"Proceedings of the ACM Multimedia (MM \u201922)","author":"Hung Tzu-Kuan","year":"2022","unstructured":"Tzu-Kuan Hung, I-Chun Huang, Samuel Rhys Cox, Wei Tsang Ooi, and Cheng-Hsin Hsu. 2022. Error concealment of dynamic 3D point cloud streaming. In Proceedings of the ACM Multimedia (MM \u201922), 3134\u20133142."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2020.100338"},{"volume-title":"Methodology for the Subjective Assessment of the Quality of Television Pictures ITU-R Recommendation BT.500-13","author":"ITU-R BT.500-13 2012","key":"e_1_3_2_20_2","unstructured":"ITU-R BT.500-13 2012. Methodology for the Subjective Assessment of the Quality of Television Pictures ITU-R Recommendation BT.500-13. Document. International Telecommunication Union."},{"issue":"3","key":"e_1_3_2_21_2","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/MSP.2019.2900721","article-title":"Video-based point-cloud-compression standard in MPEG: From evidence collection to committee draft","volume":"36","author":"Jang E. S.","year":"2019","unstructured":"E. S. Jang, M. Preda, K. Mammou, A. M. Tourapis, J. Kim, D. B. Graziosi, S. Rhyu, and M. Budagavi. 2019. Video-based point-cloud-compression standard in MPEG: From evidence collection to committee draft. IEEE Signal Processing Magazine 36, 3 (2019), 118\u2013123.","journal-title":"IEEE Signal Processing Magazine"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-020-10333-6"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487237"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"40","DOI":"10.5815\/ijem.2016.04.05","article-title":"A survey on stereo matching techniques for 3D vision in image processing","volume":"4","author":"Kumari Deepika","year":"2016","unstructured":"Deepika Kumari and Kamaljit Kaur. 2016. A survey on stereo matching techniques for 3D vision in image processing. International Journal of Engineering and Manufacturing 4 (2016), 40\u201349.","journal-title":"International Journal of Engineering and Manufacturing"},{"key":"e_1_3_2_25_2","first-page":"4413","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919)","author":"Lee Sungho","year":"2019","unstructured":"Sungho Lee, Seoung Wug Oh, DaeYeun Won, and Seon Joo Kim. 2019. Copy-and-paste networks for deep video inpainting. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV \u201919), 4413\u20134421."},{"key":"e_1_3_2_26_2","first-page":"200","volume-title":"Proceedings of the Eurographics\/ACM SIGGRAPH symposium on Geometry processing (SGP \u201903)","author":"Liepa Peter","year":"2003","unstructured":"Peter Liepa. 2003. Filling holes in meshes. In Proceedings of the Eurographics\/ACM SIGGRAPH symposium on Geometry processing (SGP \u201903), 200\u2013205."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2336548"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2015.2408631"},{"journal-title":"Common Test Conditions For V3C And V-PCC","author":"MPEG\/N0038 2020","key":"e_1_3_2_29_2","unstructured":"MPEG\/N0038 2020. Common Test Conditions For V3C And V-PCC. Document. ISO\/IEC JTC1\/SC29\/WG7 MPEG 3D Graphics Coding."},{"journal-title":"V-PCC Test Model v11","author":"MPEG\/N19519 2020","key":"e_1_3_2_30_2","unstructured":"MPEG\/N19519 2020. V-PCC Test Model v11. Document. ISO\/IEC JTC1\/SC29\/WG11 MPEG 3DG."},{"key":"e_1_3_2_31_2","first-page":"23","volume-title":"Proceedings of the Symposium on Geometry Processing (SGP \u201905)","author":"Pauly Mark","year":"2005","unstructured":"Mark Pauly, Niloy Mitra, Joachim Giesen, Markus Gross, and Leonidas Guibas. 2005. Example-based 3D Scan Completion. In Proceedings of the Symposium on Geometry Processing (SGP \u201905), 23\u201332."},{"key":"e_1_3_2_32_2","volume-title":"Proceedings of the International Conference on Robotics and Automation (ICRA \u201911)","author":"Rusu Radu","year":"2011","unstructured":"Radu Rusu and Steve Cousins. 2011. 3D is here: Point cloud library (PCL). In Proceedings of the International Conference on Robotics and Automation (ICRA \u201911)."},{"key":"e_1_3_2_33_2","first-page":"144","volume-title":"Proceedings of the 15th ACM Multimedia Systems Conference (MMsys \u201924)","author":"Shi Yuang","year":"2024","unstructured":"Yuang Shi, Bennett Clement, and Wei Tsang Ooi. 2024. QV4: QoE-based viewpoint-aware V-PCC-encoded volumetric video streaming. In Proceedings of the 15th ACM Multimedia Systems Conference (MMsys \u201924), 144\u2013154."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587819.3590981"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413535"},{"key":"e_1_3_2_36_2","first-page":"376","volume-title":"Proceedings of the ACM Multimedia Systems Conference (MMsys \u201923)","author":"Sun Yuan-Chun","year":"2023","unstructured":"Yuan-Chun Sun, I-Chun Huang, Yuang Shi, Wei Tsang Ooi, Chun-Ying Huang, and Cheng-Hsin Hsu. 2023. A dynamic 3D point cloud dataset for immersive applications. In Proceedings of the ACM Multimedia Systems Conference (MMsys \u201923), 376\u2013383."},{"key":"e_1_3_2_37_2","volume-title":"Proceedings of the International Conference on Image Processing (ICIP \u201903)","author":"Verdera Joan","year":"2003","unstructured":"Joan Verdera, Vicent Caselles, Marcelo Bertalmio, and Guillermo Sapiro. 2003. Inpainting surface holes. In Proceedings of the International Conference on Image Processing (ICIP \u201903)."},{"key":"e_1_3_2_38_2","unstructured":"Verified Market Research. 2023. Immersive Technology Market Size and Forecast. Retrieved from https:\/\/www.verifiedmarketresearch.com\/product\/immersive-technology-market\/"},{"key":"e_1_3_2_39_2","first-page":"90","volume-title":"Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR \u201919)","author":"Viola Irene","year":"2019","unstructured":"Irene Viola, Jelmer Mulder, Francesca De Simone, and Pablo Cesar. 2019. Temporal interpolation of dynamic digital humans using convolutional neural networks. In Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR \u201919), 90\u201397."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2005.12.006"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2021.103225"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_3_2_43_2","first-page":"1939","volume-title":"Proceedings of the IEEE\/CVF Computer Vision and Pattern Recognition Conference (CVPR \u201920)","author":"Wen Xin","year":"2020","unstructured":"Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. 2020. Point cloud completion by skip-attention network with hierarchical folding. In Proceedings of the IEEE\/CVF Computer Vision and Pattern Recognition Conference (CVPR \u201920), 1939\u20131948."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3154927"},{"key":"e_1_3_2_45_2","first-page":"9","volume-title":"Proceedings of the ACM Network and Operating System Support for Digital Audio and Video (NOSSDAV \u201921)","author":"Wu Cheng-Hao","year":"2021","unstructured":"Cheng-Hao Wu, Xiner Li, Rahul Rajesh, Wei Tsang Ooi, and Cheng-Hsin Hsu. 2021. Dynamic 3D point cloud streaming: Distortion and concealment. In Proceedings of the ACM Network and Operating System Support for Digital Audio and Video (NOSSDAV \u201921), 9\u201314."},{"key":"e_1_3_2_46_2","first-page":"365","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV \u201920)","author":"Xie Haozhe","year":"2020","unstructured":"Haozhe Xie, Hongxun Yao, Shangchen Zhou, Jiageng Mao, Shengping Zhang, and Wenxiu Sun. 2020. GRnet: Gridding residual network for dense point cloud completion. In Proceedings of the European Conference on Computer Vision (ECCV \u201920), 365\u2013381."},{"key":"e_1_3_2_47_2","first-page":"3723","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201919)","author":"Xu Rui","year":"2019","unstructured":"Rui Xu, Xiaoxiao Li, Bolei Zhou, and Chen Change Loy. 2019. Deep flow-guided video inpainting. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201919), 3723\u20133732."},{"key":"e_1_3_2_48_2","first-page":"103675:1","article-title":"Voxel-based representation of 3D point clouds: Methods, applications, and its potential use in the construction industry","volume":"126","author":"Xu Yusheng","year":"2021","unstructured":"Yusheng Xu, Xiaohua Tong, and Uwe Stilla. 2021. Voxel-based representation of 3D point clouds: Methods, applications, and its potential use in the construction industry. Automation in Construction 126 (2021), 103675:1\u2013103675:26.","journal-title":"Automation in Construction"},{"key":"e_1_3_2_49_2","first-page":"1979","volume-title":"Proceedings of the IEEE International Conference on Multimedia and Expo (ICME \u201923)","author":"Yang Mengyu","year":"2023","unstructured":"Mengyu Yang, Di Wu, Zelong Wang, Miao Hu, and Yipeng Zhou. 2023. Understanding and improving perceptual quality of volumetric video streaming. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME \u201923), 1979\u20131984."},{"key":"e_1_3_2_50_2","first-page":"5505","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201918)","author":"Yu Jiahui","year":"2018","unstructured":"Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2018. Generative image inpainting with contextual attention. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201918), 5505\u20135514."},{"key":"e_1_3_2_51_2","first-page":"6338","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922)","author":"Zeng Yiming","year":"2022","unstructured":"Yiming Zeng, Yue Qian, Qijian Zhang, Junhui Hou, Yixuan Yuan, and Ying He. 2022. IDEA-Net: Dynamic 3D point cloud interpolation via deep embedding alignment. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201922), 6338\u20136347."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1109\/QoMEX51781.2021.9465456","volume-title":"Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX \u201921)","author":"Zerman Emin","year":"2021","unstructured":"Emin Zerman, Radhika Kulkarni, and Aljosa Smolic. 2021. User behavior analysis of volumetric video in augmented reality. In Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX \u201921), 129\u2013132."},{"key":"e_1_3_2_53_2","unstructured":"Qian-Yi Zhou Jaesik Park and Vladlen Koltun. 2023. Open3D: A Modern Library for 3D Data Processing. Retrieved from http:\/\/www.open3d.org\/"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3731561","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T14:49:23Z","timestamp":1751381363000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731561"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":52,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3731561"],"URL":"https:\/\/doi.org\/10.1145\/3731561","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"assertion":[{"value":"2024-02-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}