{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T03:18:11Z","timestamp":1773803891802,"version":"3.50.1"},"reference-count":51,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,4,10]],"date-time":"2021-04-10T00:00:00Z","timestamp":1618012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Attributed to the explosive adoption of large-span spatial structures and infrastructures as a critical damage-sensitive element, there is a pressing need to monitor cable vibration frequency to inspect the structural health. Neither existing acceleration sensor-utilized contact methods nor conventional computer vision-based photogrammetry methods have, to date, addressed the defects of lack in cost-effectiveness and compatibility with real-world situations. In this study, a state-of-the-art method based on modified convolutional neural network semantic image segmentation, which is compatible with extensively varying real-world backgrounds, is presented for cable vibration frequency remote and visual monitoring. Modifications of the underlying network framework lie in adopting simpler feature extractors and introducing class weights to loss function by pixel-wise weighting strategies. Nine convolutional neural networks were established and modified. Discrete images with varying real-world backgrounds were captured to train and validate network models. Continuous videos with different cable pixel-to-total pixel (C-T) ratios were captured to test the networks and derive vibration frequencies. Various metrics were leveraged to evaluate the effectiveness of network models. The optimal C-T ratio was also studied to provide guidelines for the parameter setting of monitoring systems in further research and practical application. Training and validation accuracies of nine networks were all reported higher than 90%. A network model with ResNet-50 as feature extractor and uniform prior weighting showed the most superior learning and generalization ability, of which the Precision reached 0.9973, F1 reached 0.9685, and intersection over union (IoU) reached 0.8226 when utilizing images with the optimal C-T ratio of 0.04 as testing set. Contrasted with that sampled by acceleration sensor, the first two order vibration frequencies derived by the most superior network from video with the optimal C-T ratio had merely ignorable absolute percentage errors of 0.41% and 0.36%, substantiating the effectiveness of the proposed method.<\/jats:p>","DOI":"10.3390\/rs13081466","type":"journal-article","created":{"date-parts":[[2021,4,12]],"date-time":"2021-04-12T05:52:00Z","timestamp":1618206720000},"page":"1466","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Semantic Image Segmentation Based Cable Vibration Frequency Visual Monitoring Using Modified Convolutional Neural Network with Pixel-wise Weighting Strategy"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5855-2172","authenticated-orcid":false,"given":"Han","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, China"}]},{"given":"Hong-Cheng","family":"Xu","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, China"}]},{"given":"Shuang-Jian","family":"Jiao","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, China"}]},{"given":"Feng-De","family":"Yin","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, College of Engineering, Ocean University of China, Qingdao 266100, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,10]]},"reference":[{"key":"ref_1","first-page":"34","article-title":"Research progress of cable force test technology and cable flexural rigidity","volume":"22","author":"Guo","year":"2016","journal-title":"Spat. Struct."},{"key":"ref_2","unstructured":"Sumitro, S., Jarosevic, A., and Wang, M.L. (2002, January 13\u201319). Elasto-magnetic sensor utilization on steel cable stress measurement. Proceedings of the 1st Fib Congress, Concrete Structures in the 21th Century, Osaka, Japan."},{"key":"ref_3","unstructured":"Irvine, H.M. (1981). Cable Structures, The MIT Press."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1061\/(ASCE)BE.1943-5592.0000200","article-title":"Practical formula for cable tension estimation by vibration method","volume":"17","author":"Fang","year":"2012","journal-title":"J. Bridge Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.measurement.2016.12.020","article-title":"Cable tension force estimate using novel noncontact vision-based sensor","volume":"99","author":"Feng","year":"2017","journal-title":"Measurement"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.sna.2012.01.007","article-title":"New simple optical sensor: From nanometer resolution to centimeter displacement range","volume":"176","author":"Missoffe","year":"2012","journal-title":"Sens. Actuator A Phys."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1016\/j.sna.2010.08.021","article-title":"Design, analysis and fabrication of a multidimensional acceleration sensor based on fully decoupled compliant parallel mechanism","volume":"163","author":"Gao","year":"2010","journal-title":"Sens. Actuator A Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1016\/j.engstruct.2004.10.013","article-title":"The determination of bridge displacement using measured acceleration","volume":"27","author":"Park","year":"2005","journal-title":"Eng. Struct."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1177\/1475921720935585","article-title":"A review of computer vision-based structural health monitoring at local and global levels","volume":"20","author":"Dong","year":"2021","journal-title":"Struct. Health Monit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s43251-020-00006-7","article-title":"Fracture acoustic emission signals identification of stay cables in bridge engineering application using deep transfer learning and wavelet analysis","volume":"1","author":"Xin","year":"2020","journal-title":"Adv. Bridge Eng."},{"key":"ref_11","unstructured":"Zhang, Z., Yan, J., Li, L., Pan, H., and Dong, C. (September, January 15). Condition assessment of stay cables through enhanced time series classification using a deep learning approach. Proceedings of the 1st International Project Competition for Structural health monitoring (IPC-SHM), Harbin, China."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jeong, S., Kim, H., Lee, J., and Sim, S.-H. (2020). Automated wireless monitoring system for cable tension forces using deep learning. Struct. Health Monit.","DOI":"10.1177\/1475921720935837"},{"key":"ref_13","unstructured":"Chen, Z.C. (2015). Cable Force Identification Based on Non-Contact Photogrammetry System. [Master\u2019s Thesis, Hunan University]. (In Chinese)."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1061\/(ASCE)1084-0702(2008)13:1(34)","article-title":"Nontarget image-based technique for small cable vibration measurement","volume":"13","author":"Ji","year":"2008","journal-title":"J. Bridge Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1260\/1369-4332.15.6.929","article-title":"Modal frequency identification of stay cables with ambient vibration measurements based on nontarget image processing techniques","volume":"15","author":"Chen","year":"2012","journal-title":"Adv. Struct. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.ndteint.2013.05.002","article-title":"Dynamic characteristics of suspension bridge hanger cables using digital image processing","volume":"59","author":"Kim","year":"2013","journal-title":"NDT E Int."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1177\/1475921713500513","article-title":"Vision-based monitoring system for evaluating cable tensile forces on a cable-stayed bridge","volume":"12","author":"Kim","year":"2013","journal-title":"Struct. Health Monit."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, S.-W., Kim, N.-S., Jeon, B.-G., and Park, J.-C. (2014, January 7\u201311). Vision-based monitoring system for estimating cable tensile forces of cable-stayed bridge. Proceedings of the 7th International Conference on Bridge Maintenance, Safety and Management (IABMAS), Shanghai, China.","DOI":"10.1201\/b17063-141"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/stc.2155","article-title":"A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge","volume":"25","author":"Xu","year":"2018","journal-title":"Struct. Control. Health Monit."},{"key":"ref_20","first-page":"1","article-title":"Experimental Verification for Cable Force Estimation Using Handheld Shooting of Smartphones","volume":"2017","author":"Zhao","year":"2017","journal-title":"J. Sens."},{"key":"ref_21","first-page":"21","article-title":"Review of computer vision-based structural displacement monitoring","volume":"32","author":"Ye","year":"2019","journal-title":"China J. Highw. Transp."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/s13349-017-0261-4","article-title":"Review of machine-vision based methodologies for displacement measurement in civil structures","volume":"8","author":"Xu","year":"2018","journal-title":"J. Civ. Struct. Health Monit."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.engstruct.2017.11.018","article-title":"Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection\u2013A review","volume":"156","author":"Feng","year":"2018","journal-title":"Eng. Struct."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.ymssp.2018.11.015","article-title":"Development and field testing of a vision-based displacement system using a low cost wireless action camera","volume":"121","author":"Lydon","year":"2019","journal-title":"Mech. Syst. Signal Proc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/stc.1852","article-title":"Completely contactless structural health monitoring of real-life structures using cameras and computer vision","volume":"24","author":"Khuc","year":"2017","journal-title":"Struct. Control. Health Monit."},{"key":"ref_26","first-page":"10","article-title":"Communication in the presence of noise","volume":"37","author":"Shannon","year":"1949","journal-title":"Proc. I.R.E."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., and Terzopoulos, D. (2020). Image segmentation using deep learning: A survey. arXiv.","DOI":"10.1109\/TPAMI.2021.3059968"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","article-title":"Mask R-CNN","volume":"42","author":"He","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_34","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 8\u201310). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_36","unstructured":"Howard, A.G., Zhu, M.L., Chen, B., Kalenichenko, D., Wang, W.J., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tekin, B., Sinha, S.N., and Fua, P. (2018, January 18\u201323). Real-time seamless single shot 6D object pose prediction. Proceedings of the 31st IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00038"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kehl, W., Manhardt, F., Tombari, F., Ilic, S., and Navab, N. (2017, January 22\u201329). SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again. Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.169"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H.S. (2016, January 8\u201316). Fully-convolutional siamese networks for object tracking. Proceedings of the 14th European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-48881-3_56"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, Q., Zhang, L., Bertinetto, L., Hu, W.M., and Torr, P.H.S. (2019, January 15\u201320). Fast online object tracking and segmentation: A unifying approach. Proceedings of the 32nd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00142"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F.Y., Xing, J.L., and Yan, J.J. (2019, January 15\u201320). SiamRPN++: Evolution of siamese visual tracking with very deep networks. Proceedings of the 32nd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Chen, L.-C., Zhu, Y.K., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_44","unstructured":"Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the 30th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Weiss, K., Khoshgoftaar, T.M., and Wang, D. (2016). A survey of transfer learning. J. Big Data, 3.","DOI":"10.1186\/s40537-016-0043-6"},{"key":"ref_49","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014, January 8\u201313). How transferable are features in deep neural networks?. Proceedings of the 28th Conference on Neural Information Processing Systems (NIPS), Montreal, QC, Canada."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CPVR), Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_51","unstructured":"Shi, W.A. (2016). Research and Development on Cable Force Test System Based on Android Platform. [Master\u2019s Thesis, South China University of Technology]. (In Chinese)."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/8\/1466\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:12:15Z","timestamp":1760364735000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/8\/1466"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,10]]},"references-count":51,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["rs13081466"],"URL":"https:\/\/doi.org\/10.3390\/rs13081466","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,10]]}}}