{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T16:50:59Z","timestamp":1781110259555,"version":"3.54.1"},"reference-count":34,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,4,27]],"date-time":"2023-04-27T00:00:00Z","timestamp":1682553600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"IITP","award":["2021-0-0208"],"award-info":[{"award-number":["2021-0-0208"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper delves into image detection based on distributed deep-learning techniques for intelligent traffic systems or self-driving cars. The accuracy and precision of neural networks deployed on edge devices (e.g., CCTV (closed-circuit television) for road surveillance) with small datasets may be compromised, leading to the misjudgment of targets. To address this challenge, TensorFlow and PyTorch were used to initialize various distributed model parallel and data parallel techniques. Despite the success of these techniques, communication constraints were observed along with certain speed issues. As a result, a hybrid pipeline was proposed, combining both dataset and model distribution through an all-reduced algorithm and NVlinks to prevent miscommunication among gradients. The proposed approach was tested on both an edge cluster and Google cluster environment, demonstrating superior performance compared to other test settings, with the quality of the bounding box detection system meeting expectations with increased reliability. Performance metrics, including total training time, images\/second, cross-entropy loss, and total loss against the number of the epoch, were evaluated, revealing a robust competition between TensorFlow and PyTorch. The PyTorch environment\u2019s hybrid pipeline outperformed other test settings.<\/jats:p>","DOI":"10.3390\/s23094347","type":"journal-article","created":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T02:02:23Z","timestamp":1682647343000},"page":"4347","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Comparative Study on Distributed Lightweight Deep Learning Models for Road Pothole Detection"],"prefix":"10.3390","volume":"23","author":[{"given":"Hassam","family":"Tahir","sequence":"first","affiliation":[{"name":"Department of Software & Communications Engineering, Hongik University, Sejong 30016, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1288-7521","authenticated-orcid":false,"given":"Eun-Sung","family":"Jung","sequence":"additional","affiliation":[{"name":"Department of Software & Communications Engineering, Hongik University, Sejong 30016, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"6814","DOI":"10.1109\/TVT.2018.2822762","article-title":"Humanlike Driving: Empirical Decision-Making System for Autonomous Vehicles","volume":"67","author":"Li","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Retallack, A.E., and Ostendorf, B. (2019). Current Understanding of the Effects of Congestion on Traffic Accidents. Int. J. Environ. Res. Public Health, 16.","DOI":"10.3390\/ijerph16183400"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"115158","DOI":"10.1016\/j.engstruct.2022.115158","article-title":"Novel visual crack width measurement based on backbone double-scale features for improved detection automation","volume":"274","author":"Tang","year":"2023","journal-title":"Eng. Struct."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1016\/j.istruc.2021.12.055","article-title":"Seismic performance evaluation of recycled aggregate concrete-filled steel tubular columns with field strain detected via a novel mark-free vision method","volume":"37","author":"Tang","year":"2022","journal-title":"Structures"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1109\/MRA.2018.2874301","article-title":"Highly Automated Vehicles and Self-Driving Cars [Industry Tutorial]","volume":"25","author":"Rudas","year":"2018","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MSSC.2017.2745818","article-title":"Embedded Deep Neural Network Processing: Algorithmic and Processor Techniques Bring Deep Learning to IoT and Edge Devices","volume":"9","author":"Verhelst","year":"2017","journal-title":"IEEE-Solid-State Circuits Mag."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ni, Z., Yuksel, A.C., Ni, X., Mandel, M.I., and Xie, L. (2017, January 20\u201323). Confused or Not Confused? Disentangling Brain Activity from EEG Data Using Bidirectional LSTM Recurrent Neural Networks. Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB\u201917), Boston, MA, USA.","DOI":"10.1145\/3107411.3107513"},{"key":"ref_8","unstructured":"Jin, P.H., Yuan, Q., Iandola, F.N., and Keutzer, K. (2016). How to scale distributed deep learning?. arXiv."},{"key":"ref_9","unstructured":"Yuan, Z., Xue, H., Zhang, C., and Liu, Y. (2023). Hulk: Graph Neural Networks for Optimizing Regionally Distributed Computing Systems. arXiv."},{"key":"ref_10","unstructured":"Alimohammadi, M., Markov, I., Frantar, E., and Alistarh, D. (2022). L-GreCo: An Efficient and General Framework for Layerwise-Adaptive Gradient Compression. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1109\/TSP.2022.3160238","article-title":"Compressed gradient tracking for decentralized optimization over general directed networks","volume":"70","author":"Song","year":"2022","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_12","unstructured":"Charles, Z., Bonawitz, K., Chiknavaryan, S., McMahan, B., and Ag\u00fcera y Arcas, B. (2022). Federated select: A primitive for communication-and memory-efficient federated learning. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1109\/TPDS.2019.2929768","article-title":"Data-parallel hashing techniques for GPU architectures","volume":"31","author":"Lessley","year":"2019","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1466","DOI":"10.1109\/TPDS.2023.3247001","article-title":"Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models","volume":"34","author":"Lai","year":"2023","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"19222","DOI":"10.1109\/ACCESS.2018.2820899","article-title":"An adaptive synchronous parallel strategy for distributed machine learning","volume":"6","author":"Zhang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1109\/TC.2020.2994391","article-title":"SAFA: A semi-asynchronous protocol for fast federated learning with low overhead","volume":"70","author":"Wu","year":"2020","journal-title":"IEEE Trans. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Riba, E., Mishkin, D., Ponsa, D., Rublee, E., and Bradski, G. (2020, January 1\u20135). Kornia: An Open Source Differentiable Computer Vision Library for PyTorch. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093363"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, S., Zhao, Y., Varma, R., Salpekar, O., Noordhuis, P., Li, T., Paszke, A., Smith, J., Vaughan, B., and Damania, P. (2020). PyTorch Distributed: Experiences on Accelerating Data Parallel Training. arXiv.","DOI":"10.14778\/3415478.3415530"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7120","DOI":"10.1109\/TCSVT.2022.3169842","article-title":"Attention in Attention: Modeling Context Correlation for Efficient Video Classification","volume":"32","author":"Hao","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_20","unstructured":"Yan, M., Meisburger, N., Medini, T., and Shrivastava, A. (2022). Distributed SLIDE: Enabling Training Large Neural Networks on Low Bandwidth and Simple CPU-Clusters via Model Parallelism and Sparsity. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kumar, B.V., Sivakumar, P., Surendiran, B., and Ding, J. (2023). Smart Computer Vision, Springer. EAI\/Springer Innovations in Communication and Computing (EAISICC).","DOI":"10.1007\/978-3-031-20541-5"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, D., Liu, Z., Gu, X., Wu, W., Chen, Y., and Wang, L. (2022). Automatic Detection of Pothole Distress in Asphalt Pavement Using Improved Convolutional Neural Networks. Remote Sens., 14.","DOI":"10.3390\/rs14163892"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5799","DOI":"10.1109\/TCYB.2021.3060461","article-title":"Rethinking Road Surface 3-D Reconstruction and Pothole Detection: From Perspective Transformation to Disparity Map Segmentation","volume":"52","author":"Fan","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Musa, A., Hassan, M., Hamada, M., Kakudi, H.A., Amin, M.F.I., and Watanobe, Y. (2022, January 20\u201322). A Lightweight CNN-Based Pothole Detection Model for Embedded Systems Using Knowledge Distillation. Proceedings of the 21st International Conference on New Trends in Intelligent Software Methodologies, Tools and Techniques (SoMeT\u201922), Kitakyushu, Japan.","DOI":"10.3233\/FAIA220281"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kahira, A.N., Nguyen, T.T., Gomez, L.B., Takano, R., Badia, R.M., and Wahib, M. (2021, January 21\u201325). An Oracle for Guiding Large-Scale Model\/Hybrid Parallel Training of Convolutional Neural Networks. Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing (HPDC\u201921), Stockholm, Sweden.","DOI":"10.1145\/3431379.3460644"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ye, X., Lai, Z., Li, S., Cai, L., Sun, D., Qiao, L., and Li, D. (2021, January 9\u201312). Hippie: A Data-Paralleled Pipeline Approach to Improve Memory-Efficiency and Scalability for Large DNN Training. Proceedings of the 50th International Conference on Parallel Processing (ICPP 2021), Lemont, IL, USA.","DOI":"10.1145\/3472456.3472497"},{"key":"ref_27","unstructured":"Sergeev, A., and Balso, M.D. (2018). Horovod: Fast and easy distributed deep learning in TensorFlow. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, S., and Hoefler, T. (2022). Near-Optimal Sparse Allreduce for Distributed Deep Learning. arXiv.","DOI":"10.1145\/3503221.3508399"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mengara Mengara, A.G., Park, E., Jang, J., and Yoo, Y. (2022). Attention-Based Distributed Deep Learning Model for Air Quality Forecasting. Sustainability, 14.","DOI":"10.3390\/su14063269"},{"key":"ref_30","unstructured":"Sung, J., and Jung, E. (2021). Factorial Convolution Neural Networks. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e5923","DOI":"10.1002\/cpe.5923","article-title":"Optimizing execution for pipelined-based distributed deep learning in a heterogeneously networked GPU cluster","volume":"32","author":"Zhang","year":"2020","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"ref_32","unstructured":"(2023, March 11). Getting Started with Distributed Data Parallel. Available online: https:\/\/pytorch.org\/tutorials\/intermediate\/ddp_tutorial.html."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/MM.2022.3148670","article-title":"Accelerating Deep Learning Using Interconnect-Aware UCX Communication for MPI Collectives","volume":"42","author":"Sojoodi","year":"2022","journal-title":"IEEE Micro"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Flynn, P., Yi, X., and Yan, Y. (2022, January 2\u20136). Exploring Source-to-Source Compiler Transformation of OpenMP SIMD Constructs for Intel AVX and Arm SVE Vector Architectures. Proceedings of the Thirteenth International Workshop on Programming Models and Applications for Multicores and Manycores (PMAM\u201922), Seoul, Republic of Korea.","DOI":"10.1145\/3528425.3529100"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4347\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:25:03Z","timestamp":1760124303000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4347"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,27]]},"references-count":34,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23094347"],"URL":"https:\/\/doi.org\/10.3390\/s23094347","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,27]]}}}