{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T05:59:54Z","timestamp":1790315994696,"version":"4.1.0"},"reference-count":55,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:00:00Z","timestamp":1790294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>\n                    Recent 3D Transformers have become a dominant framework for point-cloud segmentation by modeling spatial context in sparse 3D scenes. However, geometry and color alone provide limited high-level semantic cues, especially for cluttered boundary regions, visually similar objects, and long-tail categories. To address this issue, we propose a segmentation framework guided by Contrastive Language\u2013Image Pre-training (CLIP) that enriches sparse 3D tokens with vision\u2013language semantic priors. Specifically, dense CLIP features extracted from multi-view RGB images are projected onto 3D points through visibility-aware alignment and view pooling, and are fused with relative geometric offsets and color cues to form semantically aware sparse voxel tokens. To better exploit the aligned CLIP semantics during local token interactions, we build on contextual relative signal encoding (cRSE) and introduce a decoupled CLIP-induced semantic residual that forms semantic-geometric attention biases for local window attention. We further adapt block-wise online softmax computation to generate and consume these biases on the fly. Experiments on ScanNet, ScanNet200, and S3DIS demonstrate competitive segmentation performance, improved instance-level discrimination, and a 25.7% reduction in peak\n                    <jats:italic>online 3D-stage<\/jats:italic>\n                    training memory compared with the materialized attention implementation when cached CLIP features are used.\n                  <\/jats:p>","DOI":"10.3389\/frai.2026.1945529","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T05:32:13Z","timestamp":1790314333000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Vision\u2013language guided semantic-geometric transformer for memory-efficient 3D scene understanding"],"prefix":"10.3389","volume":"9","author":[{"given":"Licheng","family":"Liu","sequence":"first","affiliation":[{"name":"School of Electronic and Electrical Engineering, Wuhan Textile University","place":["Wuhan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronic and Electrical Engineering, Wuhan Textile University","place":["Wuhan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fuyong","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Xinjiang College of Science and Technology","place":["Korla, Xinjiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"103730","DOI":"10.1016\/j.jag.2024.103730","article-title":"Preformer: a memory-efficient transformer for point cloud semantic segmentation","volume":"128","author":"Akwensi","year":"2024","journal-title":"Int. J. Appl. Earth Observ. Geoinf"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1534","DOI":"10.1109\/CVPR.2016.170","article-title":"\u201c3D semantic parsing of large-scale indoor spaces,\u201d","author":"Armeni","year":"2016","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B3","doi-asserted-by":"publisher","first-page":"4413","DOI":"10.1109\/CVPR.2018.00464","article-title":"\u201cThe lov\u00e1sz-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks,\u201d","author":"Berman","year":"2018","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B4","doi-asserted-by":"publisher","first-page":"6595","DOI":"10.1109\/CVPR52734.2025.00618","article-title":"\u201cFlash3D: super-scaling point transformers through joint hardware-geometry locality,\u201d","author":"Chen","year":"2025","journal-title":"Proceedings of the computer vision and pattern recognition conference"},{"key":"B5","doi-asserted-by":"publisher","first-page":"7020","DOI":"10.1109\/CVPR52729.2023.00678","article-title":"\u201cCLIP2Scene: towards label-efficient 3D scene understanding by CLIP,\u201d","author":"Chen","year":"2023","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B6","doi-asserted-by":"publisher","first-page":"15467","DOI":"10.1109\/ICCV48922.2021.01518","article-title":"\u201cHierarchical aggregation for 3d instance segmentation,\u201d","author":"Chen","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B7","doi-asserted-by":"publisher","first-page":"3075","DOI":"10.1109\/CVPR.2019.00319","article-title":"\u201c4D spatio-temporal convnets: minkowski convolutional neural networks,\u201d","author":"Choy","year":"2019","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B8","doi-asserted-by":"publisher","first-page":"5828","DOI":"10.1109\/CVPR.2017.261","article-title":"\u201cScanNet: richly-annotated 3D reconstructions of indoor scenes,\u201d","author":"Dai","year":"2017","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B9","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1007\/978-3-030-01249-6_28","article-title":"\u201c3DMV: joint 3D-multi-view prediction for 3D semantic scene segmentation,\u201d","author":"Dai","year":"2018","journal-title":"Proceedings of the European conference on computer vision (ECCV)"},{"key":"B10","article-title":"\u201cFlashAttention-2: faster attention with better parallelism and work partitioning,\u201d","volume-title":"International conference on learning representations","author":"Dao","year":"2024"},{"key":"B11","doi-asserted-by":"publisher","first-page":"16344","DOI":"10.52202\/068431-1189","article-title":"FlashAttention: fast and memory-efficient exact attention with IO-awareness","volume":"35","author":"Dao","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1204445","DOI":"10.3389\/fncom.2023.1204445","article-title":"An improved fused feature residual network for 3D point cloud data","volume":"17","author":"Gezawa","year":"2023","journal-title":"Front. Comput. Neurosci"},{"key":"B13","doi-asserted-by":"publisher","first-page":"9224","DOI":"10.1109\/CVPR.2018.00961","article-title":"\u201c3D semantic segmentation with submanifold sparse convolutional networks,\u201d","author":"Graham","year":"2018","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3034","DOI":"10.1007\/978-3-031-26319-4_24","article-title":"\u201cSWPT: spherical window-based point cloud transformer,\u201d","author":"Guo","year":"2022","journal-title":"Proceedings of the Asian conference on computer vision"},{"key":"B15","doi-asserted-by":"publisher","first-page":"104105","DOI":"10.1016\/j.jag.2024.104105","article-title":"Point cloud semantic segmentation with adaptive spatial structure graph transformer","volume":"133","author":"Han","year":"2024","journal-title":"Int. J. Appl. Earth Observ. Geoinf"},{"key":"B16","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1007\/978-3-031-73397-0_5","article-title":"\u201cScatterFormer: efficient voxel transformer with scattered linear attention,\u201d","author":"He","year":"2024","journal-title":"European Conference on computer vision"},{"key":"B17","doi-asserted-by":"publisher","first-page":"14373","DOI":"10.1109\/CVPR46437.2021.01414","article-title":"\u201cBidirectional projection network for cross dimension scene understanding,\u201d","author":"Hu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B18","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2023.XIX.066","article-title":"\u201cConceptFusion: open-set multimodal 3D mapping,\u201d","volume-title":"Robotics: science and systems","author":"Jatavallabhula","year":"2023"},{"key":"B19","doi-asserted-by":"publisher","first-page":"4867","DOI":"10.1109\/CVPR42600.2020.00492","article-title":"\u201cPointgroup: dual-set point grouping for 3D instance segmentation,\u201d","author":"Jiang","year":"2020","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B20","doi-asserted-by":"publisher","first-page":"110712","DOI":"10.1016\/j.patcog.2024.110712","article-title":"Beyond local patches: preserving global-local interactions by enhancing self-attention via 3D point cloud tokenization","volume":"155","author":"Khan","year":"2024","journal-title":"Pattern Recognit"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1439340","DOI":"10.3389\/frai.2024.1439340","article-title":"MLGCN: an ultra-efficient graph convolutional neural model for 3D point cloud analysis","volume":"7","author":"Khodadad","year":"2024","journal-title":"Front. Artif. Intell"},{"key":"B22","doi-asserted-by":"publisher","first-page":"132746","DOI":"10.1016\/j.neucom.2026.132746","article-title":"Dynamic window transformer for three-dimensional indoor scene segmentation","volume":"671","author":"Kim","year":"2026","journal-title":"Neurocomputing"},{"key":"B23","doi-asserted-by":"publisher","first-page":"8500","DOI":"10.1109\/CVPR52688.2022.00831","article-title":"\u201cStratified transformer for 3D point cloud segmentation,\u201d","author":"Lai","year":"2022","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B24","doi-asserted-by":"publisher","first-page":"103813","DOI":"10.1016\/j.jag.2024.103813","article-title":"Hierarchical local global transformer for point clouds analysis","volume":"129","author":"Li","year":"2024","journal-title":"Int. J. Appl. Earth Observ. Geoinf"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1542813","DOI":"10.3389\/fcomp.2025.1542813","article-title":"Rwaformer: a lightweight road lidar point cloud segmentation network based on transformer","volume":"7","author":"Li","year":"2025","journal-title":"Front. Comput. Sci"},{"key":"B26","doi-asserted-by":"publisher","first-page":"32653","DOI":"10.52202\/079017-1026","article-title":"Pointmamba: a simple state space model for point cloud analysis","volume":"37","author":"Liang","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B27","doi-asserted-by":"publisher","first-page":"10012","DOI":"10.1109\/ICCV48922.2021.00986","article-title":"\u201cSwin transformer: hierarchical vision transformer using shifted windows,\u201d","author":"Liu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1200","DOI":"10.1109\/CVPR52729.2023.00122","article-title":"\u201cFlatFormer: flattened window attention for efficient point cloud transformer,\u201d","author":"Liu","year":"2023","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B29","doi-asserted-by":"publisher","first-page":"104572","DOI":"10.1016\/j.jag.2025.104572","article-title":"3DLST: 3D learnable supertoken transformer for lidar point cloud scene segmentation","volume":"140","author":"Lu","year":"2025","journal-title":"Int. J. Appl. Earth Observ. Geoinf"},{"key":"B30","doi-asserted-by":"publisher","first-page":"46","DOI":"10.4236\/jcc.2026.141004","article-title":"Geo-refined point transformer: coordinate-aware excitation and positional upsampling for 3D scene segmentation","volume":"14","author":"Lu","year":"2026","journal-title":"J. Comput. Commun"},{"key":"B31","doi-asserted-by":"publisher","first-page":"16949","DOI":"10.1109\/CVPR52688.2022.01644","article-title":"\u201cFast point transformer,\u201d","author":"Park","year":"2022","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR)"},{"key":"B32","doi-asserted-by":"publisher","first-page":"21305","DOI":"10.1109\/CVPR52733.2024.02013","article-title":"\u201cOA-CNNS: omni-adaptive sparse cnns for 3D semantic segmentation,\u201d","author":"Peng","year":"2024","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B33","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1109\/CVPR52729.2023.00085","article-title":"\u201cOpenScene: 3D scene understanding with open vocabularies,\u201d","author":"Peng","year":"2023","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B34","article-title":"\u201cPointnet++: deep hierarchical feature learning on point sets in a metric space,\u201d","volume-title":"Advances in neural information processing systems, Vol. 30","author":"Qi","year":"2017"},{"key":"B35","doi-asserted-by":"publisher","first-page":"23192","DOI":"10.52202\/068431-1685","article-title":"Pointnext: revisiting pointnet++ with improved training and scaling strategies","volume":"35","author":"Qian","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B36","first-page":"8748","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"International conference on machine learning","author":"Radford","year":"2021"},{"key":"B37","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","article-title":"\u201cU-Net: convolutional networks for biomedical image segmentation,\u201d","author":"Ronneberger","year":"2015","journal-title":"International Conference on Medical image computing and computer-assisted intervention"},{"key":"B38","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/978-3-031-19827-4_8","article-title":"\u201cLanguage-grounded indoor 3D semantic segmentation in the wild,\u201d","author":"Rozenberszki","year":"2022","journal-title":"European conference on computer vision"},{"key":"B39","doi-asserted-by":"publisher","first-page":"8216","DOI":"10.1109\/ICRA48891.2023.10160590","article-title":"\u201cMask3D: mask transformer for 3D semantic instance segmentation,\u201d","author":"Schult","year":"2023","journal-title":"2023 IEEE international conference on robotics and automation (ICRA)"},{"key":"B40","doi-asserted-by":"publisher","first-page":"10943","DOI":"10.1109\/TCSVT.2026.3680794","article-title":"Open-vocabulary SAM3D: towards training-free open-vocabulary 3D scene understanding","volume":"36","author":"Tai","year":"2026","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"B41","doi-asserted-by":"crossref","DOI":"10.52202\/075280-2989","article-title":"\u201cOpenmask3D: open-vocabulary 3D instance segmentation,\u201d","volume-title":"Advances in neural information processing systems, Vol. 36","author":"Takmaz","year":"2023"},{"key":"B42","doi-asserted-by":"publisher","first-page":"6411","DOI":"10.1109\/ICCV.2019.00651","article-title":"\u201cKPConv: flexible and deformable convolution for point clouds,\u201d","author":"Thomas","year":"2019","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3592131","article-title":"OctFormer: octree-based transformers for 3D point clouds","volume":"42","author":"Wang","year":"2023","journal-title":"ACM Trans. Graphics"},{"key":"B44","doi-asserted-by":"publisher","first-page":"14125","DOI":"10.1109\/CVPR52734.2025.01318","article-title":"\u201cMasked point-entity contrast for open-vocabulary 3D scene understanding,\u201d","author":"Wang","year":"2025","journal-title":"2025 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B45","doi-asserted-by":"publisher","first-page":"7037","DOI":"10.1109\/ICCV51701.2025.00661","article-title":"\u201cOpen-vocabulary octree-graph for 3D scene understanding,\u201d","author":"Wang","year":"2025","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B46","doi-asserted-by":"publisher","first-page":"10033","DOI":"10.1109\/ICCV48922.2021.00988","article-title":"\u201cRethinking and improving relative position encoding for vision transformer,\u201d","author":"Wu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B47","doi-asserted-by":"publisher","first-page":"4840","DOI":"10.1109\/CVPR52733.2024.00463","article-title":"\u201cPoint transformer V3: simpler faster stronger,\u201d","author":"Wu","year":"2024","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B48","doi-asserted-by":"publisher","first-page":"33330","DOI":"10.52202\/068431-2415","article-title":"Point transformer V2: grouped vector attention and partition-based pooling","volume":"35","author":"Wu","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B49","doi-asserted-by":"publisher","first-page":"9411","DOI":"10.1109\/ICRA57147.2024.10610193","article-title":"\u201cOpen-fusion: real-time open-vocabulary 3D mapping and queryable scene representation,\u201d","author":"Yamazaki","year":"2024","journal-title":"2024 IEEE international conference on robotics and automation (ICRA)"},{"key":"B50","doi-asserted-by":"publisher","first-page":"465","DOI":"10.26599\/CVM.2025.9450437","article-title":"Swin3D++: effective multi-source pretraining for 3D indoor scene understanding","volume":"11","author":"Yang","year":"","journal-title":"Comput. Vis. Media"},{"key":"B51","doi-asserted-by":"publisher","first-page":"83","DOI":"10.26599\/CVM.2025.9450383","article-title":"Swin3D: a pretrained transformer backbone for 3D indoor scene understanding","volume":"11","author":"Yang","year":"","journal-title":"Comput. Vis. Media"},{"key":"B52","doi-asserted-by":"publisher","first-page":"23078","DOI":"10.18653\/v1\/2025.acl-long.1126","article-title":"\u201cNative sparse attention: hardware-aligned and natively trainable sparse attention,\u201d","author":"Yuan","year":"2025","journal-title":"Proceedings of the 63rd annual meeting of the association for computational linguistics, Vol. 1 (long papers)"},{"key":"B53","doi-asserted-by":"publisher","first-page":"11985","DOI":"10.1002\/int.23073","article-title":"PVT: point-voxel transformer for point cloud learning","volume":"37","author":"Zhang","year":"2022","journal-title":"Int. J. Intell. Syst"},{"key":"B54","doi-asserted-by":"publisher","first-page":"5070","DOI":"10.1145\/3746027.3755735","article-title":"\u201cPGOV3D: open-vocabulary 3D semantic segmentation with partial-to-global curriculum,\u201d","author":"Zhang","year":"2025","journal-title":"Proceedings of the 33rd ACM international conference on multimedia"},{"key":"B55","doi-asserted-by":"publisher","first-page":"16259","DOI":"10.1109\/ICCV48922.2021.01595","article-title":"\u201cPoint transformer,\u201d","author":"Zhao","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1945529\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T05:32:15Z","timestamp":1790314335000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1945529\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,25]]},"references-count":55,"alternative-id":["10.3389\/frai.2026.1945529"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1945529","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,25]]},"article-number":"1945529"}}