{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T22:07:48Z","timestamp":1771366068482,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T00:00:00Z","timestamp":1714694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada (NSERC)","doi-asserted-by":"publisher","award":["RGPIN-2022-04160"],"award-info":[{"award-number":["RGPIN-2022-04160"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada (NSERC)","doi-asserted-by":"publisher","award":["222301493"],"award-info":[{"award-number":["222301493"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000146","name":"Alberta Innovates","doi-asserted-by":"publisher","award":["RGPIN-2022-04160"],"award-info":[{"award-number":["RGPIN-2022-04160"]}],"id":[{"id":"10.13039\/501100000146","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000146","name":"Alberta Innovates","doi-asserted-by":"publisher","award":["222301493"],"award-info":[{"award-number":["222301493"]}],"id":[{"id":"10.13039\/501100000146","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Structural engineers are often required to draw two-dimensional engineering sketches for quick structural analysis, either by hand calculation or using analysis software. However, calculation by hand is slow and error-prone, and the manual conversion of a hand-drawn sketch into a virtual model is tedious and time-consuming. This paper presents a complete and autonomous framework for converting a hand-drawn engineering sketch into an analyzed structural model using a camera and computer vision. In this framework, a computer vision object detection stage initially extracts information about the raw features in the image of the beam diagram. Next, a computer vision number-reading model transcribes any handwritten numerals appearing in the image. Then, feature association models are applied to characterize the relationships among the detected features in order to build a comprehensive structural model. Finally, the structural model generated is analyzed using OpenSees. In the system presented, the object detection model achieves a mean average precision of 99.1%, the number-reading model achieves an accuracy of 99.0%, and the models in the feature association stage achieve accuracies ranging from 95.1% to 99.5%. Overall, the tool analyzes 45.0% of images entirely correctly and the remaining 55.0% of images partially correctly. The proposed framework holds promise for other types of structural sketches, such as trusses and frames. Moreover, it can be a valuable tool for structural engineers that is capable of improving the efficiency, safety, and sustainability of future construction projects.<\/jats:p>","DOI":"10.3390\/s24092923","type":"journal-article","created":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T05:52:02Z","timestamp":1714715522000},"page":"2923","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A Computer Vision Framework for Structural Analysis of Hand-Drawn Engineering Sketches"],"prefix":"10.3390","volume":"24","author":[{"given":"Isaac","family":"Joffe","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada"}]},{"given":"Yuchen","family":"Qian","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-3754-5992","authenticated-orcid":false,"given":"Mohammad","family":"Talebi-Kalaleh","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1409-3562","authenticated-orcid":false,"given":"Qipei","family":"Mei","sequence":"additional","affiliation":[{"name":"Department of Civil and Environmental Engineering, University of Alberta, Edmonton, AB T6G 1H9, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,3]]},"reference":[{"key":"ref_1","unstructured":"Hibbeler, R. (2017). Structural Analysis, Pearson. [10th ed.]."},{"key":"ref_2","unstructured":"(2023). SAP2000, Computers and Structures Inc.. Computer Software."},{"key":"ref_3","unstructured":"Rosenblatt, F. (1957). The Perceptron, a Perceiving and Recognizing Automaton Project Para, Cornell Aeronautical Laboratory. Report: Cornell Aeronautical Laboratory."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3212","DOI":"10.1109\/TNNLS.2018.2876865","article-title":"Object Detection with Deep Learning: A Review","volume":"30","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"23729","DOI":"10.1007\/s11042-020-08976-6","article-title":"A review of object detection based on deep learning","volume":"79","author":"Xiao","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"012015","DOI":"10.1088\/1742-6596\/1848\/1\/012015","article-title":"Offline Handwritten Text Recognition Using Deep Learning: A Review","volume":"1848","author":"Wang","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_8","unstructured":"Touretzky, D. (1989, January 27\u201330). Handwritten digit recognition with a back-propagation network. Proceedings of the Advances in Neural Information Processing Systems (NIPS 1989), Denver, CO, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., and Cai, J. (2017). Recent Advances in Convolutional Neural Networks. arXiv.","DOI":"10.1016\/j.patcog.2017.10.013"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6999","DOI":"10.1109\/TNNLS.2021.3084827","article-title":"A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects","volume":"33","author":"Li","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. arXiv.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_12","unstructured":"Jocher, G. (2020). YOLOv5 by Ultralytics, 7.0, Ultralytics. Computer Software."},{"key":"ref_13","unstructured":"Scheidl, H. (2018). Handwritten Text Recognition in Historical Documents. [Diploma\u2019s Thesis, Vienna University of Technology]."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s11831-022-09793-w","article-title":"Artificial Intelligence, Machine Learning, and Deep Learning in Structural Engineering: A Scientometrics Review of Trends and Best Practices","volume":"30","author":"Tapeh","year":"2023","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"815717","DOI":"10.3389\/fbuil.2022.815717","article-title":"Machine Learning in Structural Design: An Opinionated Review","volume":"8","year":"2022","journal-title":"Front. Built Environ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.engstruct.2018.05.084","article-title":"Emerging artificial intelligence methods in structural engineering","volume":"171","author":"Salehi","year":"2018","journal-title":"Eng. Struct."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"104440","DOI":"10.1016\/j.autcon.2022.104440","article-title":"Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications","volume":"141","author":"Baduge","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_18","unstructured":"Bennett, J., Creary, L., Englemore, R., and Melosh, R. (1978). SACON: A Knowledge-Based Consultant for Structural Analysis, Stanford University. Technical Report."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1061\/(ASCE)0887-3801(1987)1:4(270)","article-title":"Expert Systems for Structural Design","volume":"1","author":"Maher","year":"1987","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1111\/j.1467-8667.1990.tb00377.x","article-title":"Neural Networks in Structural Engineering","volume":"5","author":"Vanluchene","year":"1990","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"122972","DOI":"10.1016\/j.eswa.2023.122972","article-title":"Big data-driven TBM tunnel intelligent construction system with automated-compliance-checking (ACC) optimization","volume":"244","author":"Li","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_22","unstructured":"Sulaiman, S., Zulaiha, P., Ali, M.I., Ramli, N.I., Jamaludin, O., and Shu Ing, D. (2023). IOP Conference Series: Earth and Environmental Science, IOP Publishing."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.compstruc.2019.03.004","article-title":"On the application of machine learning techniques to derive seismic fragility curves","volume":"218","author":"Kiani","year":"2019","journal-title":"Comput. Struct."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.36909\/jer.v8i3.8149","article-title":"Estimation of Vibration Frequency of Structural Floors Using Combined Artificial Intelligence and Finite Element Simulation","volume":"8","author":"Farsangi","year":"2020","journal-title":"J. Eng. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.engstruct.2018.01.008","article-title":"Classification of failure mode and prediction of shear strength for reinforced concrete beam-column joints using machine learning techniques","volume":"160","author":"Mangalathu","year":"2018","journal-title":"Eng. Struct."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110331","DOI":"10.1016\/j.engstruct.2020.110331","article-title":"Data-driven machine-learning-based seismic failure mode identification of reinforced concrete shear walls","volume":"208","author":"Mangalathu","year":"2020","journal-title":"Eng. Struct."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1061\/(ASCE)0887-3801(2007)21:3(175)","article-title":"Sketching finite-element models within a unified two-dimensional framework","volume":"21","author":"Hutchinson","year":"2007","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_28","unstructured":"Peschel, J.M., and Hammond, T.A. (2008, January 4\u20136). STRAT: A Sketched-truss Recognition and Analysis Tool. Proceedings of the Distributed Multimedia Systems, Boston, MA, USA."},{"key":"ref_29","unstructured":"Mohammadi, N., Wang, J., Cao, Y., and Setareh, M. (2013). SMATS: Sketch-based Modeling and Analysis of Truss Systems. ARCC Conf. Repos."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"031009","DOI":"10.1115\/1.4034387","article-title":"FEAsy: A Sketch-Based Tool for Finite Element Analysis","volume":"17","author":"Murugappan","year":"2017","journal-title":"J. Comput. Inf. Sci. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Alamgir, R.M., Shuvro, A.A., Al Mushabbir, M., Raiyan, M.A., Rani, N.J., Rahman, M.M., Kabir, M.H., and Ahmed, S. (2022, January 17\u201319). Performance Analysis of YOLO-based Architectures for Vehicle Detection from Traffic Images in Bangladesh. Proceedings of the 2022 25th International Conference on Computer and Information Technology (ICCIT), Cox\u2019s Bazar, Bangladesh.","DOI":"10.1109\/ICCIT57492.2022.10055758"},{"key":"ref_32","unstructured":"Nazir, A., and Wani, M.A. (2023, January 15\u201317). You Only Look Once\u2014Object Detection Models: A Review. Proceedings of the 2023 10th International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1680","DOI":"10.3390\/make5040083","article-title":"A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS","volume":"5","author":"Terven","year":"2023","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"012024","DOI":"10.1088\/1757-899X\/844\/1\/012024","article-title":"A review: Comparison of performance metrics of pretrained models for object detection using the TensorFlow framework","volume":"844","author":"Romero","year":"2020","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"747","DOI":"10.21105\/joss.00747","article-title":"NN-SVG: Publication-Ready Neural Network Architecture Schematics","volume":"4","author":"LeNail","year":"2019","journal-title":"J. Open Source Softw."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]","volume":"29","author":"Deng","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_37","unstructured":"Zhou, L., Zhao, J., Li, J., Yuan, L., and Feng, J. (2018). Object Relation Detection Based on One-shot Learning. arXiv."},{"key":"ref_38","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016). TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1016\/S0895-4356(96)00236-3","article-title":"A simulation study of the number of events per variable in logistic regression analysis","volume":"49","author":"Peduzzi","year":"1996","journal-title":"J. Clin. Epidemiol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1061\/(ASCE)CP.1943-5487.0000002","article-title":"Nonlinear Finite-Element Analysis Software Architecture Using Object Composition","volume":"24","author":"Mckenna","year":"2010","journal-title":"J. Comput. Civ. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/9\/2923\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:39:12Z","timestamp":1760107152000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/9\/2923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,3]]},"references-count":40,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["s24092923"],"URL":"https:\/\/doi.org\/10.3390\/s24092923","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,3]]}}}