{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:19:47Z","timestamp":1760149187236,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T00:00:00Z","timestamp":1689120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Three-dimensional LiDAR systems that capture point cloud data enable the simultaneous acquisition of spatial geometry and multi-wavelength intensity information, thereby paving the way for three-dimensional point cloud recognition and processing. However, due to the irregular distribution, low resolution of point clouds, and limited spatial recognition accuracy in complex environments, inherent errors occur in classifying and segmenting the acquired target information. Conversely, two-dimensional visible light images provide real-color information, enabling the distinction of object contours and fine details, thus yielding clear, high-resolution images when desired. The integration of two-dimensional information with point clouds offers complementary advantages. In this paper, we present the incorporation of two-dimensional information to form a multi-modal representation. From this, we extract local features to establish three-dimensional geometric relationships and two-dimensional color relationships. We introduce a novel network model, termed MInet (Multi-Information net), which effectively captures features relating to both two-dimensional color and three-dimensional pose information. This enhanced network model improves feature saliency, thereby facilitating superior segmentation and recognition tasks. We evaluate our MInet architecture using the ShapeNet and ThreeDMatch datasets for point cloud segmentation, and the Stanford dataset for object recognition. The robust results, coupled with quantitative and qualitative experiments, demonstrate the superior performance of our proposed method in point cloud segmentation and object recognition tasks.<\/jats:p>","DOI":"10.3390\/s23146327","type":"journal-article","created":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T01:58:11Z","timestamp":1689213491000},"page":"6327","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["MInet: A Novel Network Model for Point Cloud Processing by Integrating Multi-Modal Information"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-7847-9375","authenticated-orcid":false,"given":"Yuhao","family":"Wang","sequence":"first","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Yong","family":"Zuo","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Zhihua","family":"Du","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Xiaohan","family":"Song","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Tian","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Xiaobin","family":"Hong","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]},{"given":"Jian","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Post and Telecommunications, Beijing 100876, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"6640","DOI":"10.1109\/TITS.2021.3059674","article-title":"Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions","volume":"23","author":"Liu","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jiang, J., Liu, D., Gu, J., and S\u00fcsstrunk, S. (2013, January 15\u201317). What is the space of spectral sensitivity functions for digital color cameras?. Proceedings of the 2013 IEEE Workshop on Applications of Computer Vision (WACV), Clearwater Beach, FL, USA.","DOI":"10.1109\/WACV.2013.6475015"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Villa, F., Severini, F., Madonini, F., and Zappa, F. (2021). SPADs and SiPMs arrays for long-range high-speed light detection and ranging (LiDAR). Sensors, 21.","DOI":"10.3390\/s21113839"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Cheng, L., Chen, S., Liu, X., Xu, H., Wu, Y., Li, M., and Chen, Y. (2018). Registration of laser scanning point clouds: A review. Sensors, 18.","DOI":"10.3390\/s18051641"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Schumann, O., Hahn, M., Dickmann, J., and W\u00f6hler, C. (2018, January 10\u201313). Semantic segmentation on radar point clouds. Proceedings of the 2018 21st International Conference on Information Fusion (FUSION), Cambridge, UK.","DOI":"10.23919\/ICIF.2018.8455344"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ilci, V., and Toth, C. (2020). High definition 3D map creation using GNSS\/IMU\/LiDAR sensor integration to support autonomous vehicle navigation. Sensors, 20.","DOI":"10.3390\/s20030899"},{"key":"ref_7","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_8","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, L., Huang, Y., Hou, Y., Zhang, S., and Shan, J. (2019, January 16\u201320). Graph attention convolution for point cloud semantic segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01054"},{"key":"ref_10","first-page":"1","article-title":"Dynamic graph cnn for learning on point clouds","volume":"38","author":"Wang","year":"2019","journal-title":"ACM Trans. Graph."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1016\/j.cad.2004.11.005","article-title":"On the normal vector estimation for point cloud data from smooth surfaces","volume":"37","author":"OuYang","year":"2005","journal-title":"Comput.-Aided Des."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1172","DOI":"10.1109\/TIP.2005.851678","article-title":"Two-dimensional transforms for device color correction and calibration","volume":"14","author":"Bala","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","unstructured":"Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., and Su, H. (2015). Shapenet: An information-rich 3d model repository. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zeng, A., Song, S., Nie\u00dfner, M., Fisher, M., Xiao, J., and Funkhouser, T. (2017, January 21\u201326). 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.29"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Luo, Y., Mi, Z., and Tao, W. (2021, January 2\u20139). DeepDT: Learning geometry from Delaunay triangulation for surface reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.","DOI":"10.1609\/aaai.v35i3.16327"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Klokov, R., and Lempitsky, V. (2017, January 22\u201329). Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.99"},{"key":"ref_17","unstructured":"Li, Y., Bu, R., Sun, M., Wu, W., Di, X., and Chen, B. (2018). Pointcnn: Convolution on x-transformed points. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"109791","DOI":"10.1016\/j.scienta.2020.109791","article-title":"A method for organs classification and fruit counting on pomegranate trees based on multi-features fusion and support vector machine by 3D point cloud","volume":"278","author":"Zhang","year":"2021","journal-title":"Sci. Hortic."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1109\/83.623193","article-title":"The farthest point strategy for progressive image sampling","volume":"6","author":"Eldar","year":"1997","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","unstructured":"Bryant, F.B., and Yarnold, P.R. (1995). Principal-Components Analysis and Exploratory and Confirmatory Factor Analysis, American Psychological Association."},{"key":"ref_21","unstructured":"Pang, T., Xu, K., Dong, Y., Du, C., Chen, N., and Zhu, J. (2019). Rethinking softmax cross-entropy loss for adversarial robustness. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Turpin, A., and Scholer, F. (2006, January 6\u201311). User performance versus precision measures for simple search tasks. Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, WA, USA.","DOI":"10.1145\/1148170.1148176"},{"key":"ref_24","unstructured":"Wang, D.Z., and Posner, I. (2015, January 13\u201317). Voting for voting in online point cloud object detection. Proceedings of the Robotics: Science and Systems, Rome, Italy."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"7380","DOI":"10.1109\/TGRS.2020.2982221","article-title":"Compatibility-guided sampling consensus for 3-d point cloud registration","volume":"58","author":"Quan","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1080\/19479832.2015.1051138","article-title":"Exploring GIS knowledge to improve building extraction and change detection from VHR imagery in urban areas","volume":"7","author":"Guo","year":"2016","journal-title":"Int. J. Image Data Fusion"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Te, G., Hu, W., Zheng, A., and Guo, Z. (2018, January 22\u201326). Rgcnn: Regularized graph cnn for point cloud segmentation. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.","DOI":"10.1145\/3240508.3240621"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1007\/s41095-021-0229-5","article-title":"Pct: Point cloud transformer","volume":"7","author":"Guo","year":"2021","journal-title":"Comput. Vis. Media"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Engelmann, F., Kontogianni, T., Hermans, A., and Leibe, B. (2017, January 22\u201329). Exploring spatial context for 3D semantic segmentation of point clouds. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.90"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Jing, W., Zhang, W., Li, L., Di, D., Chen, G., and Wang, J. (2022). AGNet: An attention-based graph network for point cloud classification and segmentation. Remote Sens., 14.","DOI":"10.3390\/rs14041036"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1109\/TRO.2013.2279412","article-title":"3-D mapping with an RGB-D camera","volume":"30","author":"Endres","year":"2013","journal-title":"IEEE Trans. Robot."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6327\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:11:23Z","timestamp":1760127083000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6327"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,12]]},"references-count":31,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["s23146327"],"URL":"https:\/\/doi.org\/10.3390\/s23146327","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,7,12]]}}}