{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T12:47:01Z","timestamp":1773492421185,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2022,6,3]],"date-time":"2022-06-03T00:00:00Z","timestamp":1654214400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project \u201cINDTECH 4.0\u2014New technologies for smart manufacturing\u201d","award":["POCI-01-0247-FEDER-026653"],"award-info":[{"award-number":["POCI-01-0247-FEDER-026653"]}]},{"name":"European Regional Development Fund (ERDF)","award":["POCI-01-0247-FEDER-026653"],"award-info":[{"award-number":["POCI-01-0247-FEDER-026653"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>The still prevalent use of paper conformity lists in the automotive industry has a serious negative impact on the performance of quality control inspectors. We propose instead a hybrid quality inspection system, where we combine automated detection with human feedback, to increase worker performance by reducing mental and physical fatigue, and the adaptability and responsiveness of the assembly line to change. The system integrates the hierarchical automatic detection of the non-conforming vehicle parts and information visualization on a wearable device to present the results to the factory worker and obtain human confirmation. Besides designing a novel 3D vehicle generator to create a digital representation of the non conformity list and to collect automatically annotated training data, we apply and aggregate in a novel way state-of-the-art domain adaptation and pseudo labeling methods to our real application scenario, in order to bridge the gap between the labeled data generated by the vehicle generator and the real unlabeled data collected on the factory floor. This methodology allows us to obtain, without any manual annotation of the real dataset, an example-based F1 score of 0.565 in an unconstrained scenario and 0.601 in a fixed camera setup (improvements of 11 and 14.6 percentage points, respectively, over a baseline trained with purely simulated data). Feedback obtained from factory workers highlighted the usefulness of the proposed solution, and showed that a truly hybrid assembly line, where machine and human work in symbiosis, increases both efficiency and accuracy in automotive quality control.<\/jats:p>","DOI":"10.3390\/app12115687","type":"journal-article","created":{"date-parts":[[2022,6,3]],"date-time":"2022-06-03T08:01:18Z","timestamp":1654243278000},"page":"5687","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Hybrid Quality Inspection for the Automotive Industry: Replacing the Paper-Based Conformity List through Semi-Supervised Object Detection and Simulated Data"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2302-8597","authenticated-orcid":false,"given":"Isabel","family":"Rio-Torto","sequence":"first","affiliation":[{"name":"Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1668-1029","authenticated-orcid":false,"given":"Ana Teresa","family":"Campani\u00e7o","sequence":"additional","affiliation":[{"name":"School of Science and Technology, University of Tr\u00e1s-os-Montes and Alto Douro, 5000-801 Vila Real, Portugal"}]},{"given":"Pedro","family":"Pinho","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3747-6577","authenticated-orcid":false,"given":"Vitor","family":"Filipe","sequence":"additional","affiliation":[{"name":"School of Science and Technology, University of Tr\u00e1s-os-Montes and Alto Douro, 5000-801 Vila Real, Portugal"},{"name":"INESC TEC\u2014INESC Technology and Science, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4050-7880","authenticated-orcid":false,"given":"Lu\u00eds F.","family":"Teixeira","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto, 4200-465 Porto, Portugal"},{"name":"INESC TEC\u2014INESC Technology and Science, 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.ifacol.2018.08.333","article-title":"Smart Information Visualization for First-Time Quality within the Automobile Production Assembly Line","volume":"51","author":"Gewohn","year":"2018","journal-title":"IFAC-PapersOnLine"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.apergo.2016.08.030","article-title":"Human-centered design (HCD) of a fault-finding application for mobile devices and its impact on the reduction of time in fault diagnosis in the manufacturing industry","volume":"59","author":"Kluge","year":"2017","journal-title":"Appl. Ergon."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pfeiffer, S. (2016). Robots, Industry 4.0 and humans, or why assembly work is more than routine work. Societies, 6.","DOI":"10.3390\/soc6020016"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.promfg.2017.07.195","article-title":"Virtual commissioning of camera-based quality assurance systems for mixed model assembly lines","volume":"11","author":"Piero","year":"2017","journal-title":"Procedia Manuf."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gewohn, M., Beyerer, J., Usl\u00e4nder, T., and Sutschet, G. (2018, January 7\u20139). A quality visualization model for the evaluation and control of quality in vehicle assembly. Proceedings of the 2018 7th International Conference on Industrial Technology and Management (ICITM), Oxford, UK.","DOI":"10.1109\/ICITM.2018.8333910"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s12599-014-0334-4","article-title":"Industry 4.0","volume":"6","author":"Lasi","year":"2014","journal-title":"Bus. Inf. Syst. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Park, T., Efros, A.A., Zhang, R., and Zhu, J.Y. (2020, January 23\u201328). Contrastive Learning for Unpaired Image-to-Image Translation. Proceedings of the European Conference on Computer Vision, Virtual.","DOI":"10.1007\/978-3-030-58545-7_19"},{"key":"ref_9","unstructured":"Liu, Y.C., Ma, C.Y., He, Z., Kuo, C.W., Chen, K., Zhang, P., Wu, B., Kira, Z., and Vajda, P. (2021, January 3\u20137). Unbiased Teacher for Semi-Supervised Object Detection. Proceedings of the International Conference on Learning Representations (ICLR), Virtual."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Rio-Torto, I., Campani\u00e7o, A.T., Pereira, A., Teixeira, L.F., and Filipe, V. (2021, January 23\u201326). Automatic quality inspection in the automotive industry: A hierarchical approach using simulated data. Proceedings of the 2021 IEEE 8th International Conference on Industrial Engineering and Applications (ICIEA), Virtual.","DOI":"10.1109\/ICIEA52957.2021.9436742"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Luckow, A., Cook, M., Ashcraft, N., Weill, E., Djerekarov, E., and Vorster, B. (2016, January 5\u20138). Deep learning in the automotive industry: Applications and tools. Proceedings of the 2016 IEEE International Conference on Big Data (Big Data), Washington, DC, USA.","DOI":"10.1109\/BigData.2016.7841045"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2491","DOI":"10.1007\/s00170-016-9581-5","article-title":"Fault detection and classification in automated assembly machines using machine vision","volume":"90","author":"Chauhan","year":"2017","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_13","first-page":"3106313","article-title":"Research on Both the Classification and Quality Control Methods of the Car Seat Backrest Based on Machine Vision","volume":"2022","author":"Sun","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kuric, I., Klar\u00e1k, J., Bulej, V., S\u00e1ga, M., Kandera, M., Hajdu\u010d\u00edk, A., and Tucki, K. (2022). Approach to Automated Visual Inspection of Objects Based on Artificial Intelligence. Appl. Sci., 12.","DOI":"10.3390\/app12020864"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Pei, Z., and Chen, L. (2018, January 9\u201311). Welding component identification and solder joint inspection of automobile door panel based on machine vision. Proceedings of the 2018 Chinese Control and Decision Conference (CCDC), Shenyang, China.","DOI":"10.1109\/CCDC.2018.8408283"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Chen, R., Huang, B., Liu, C., Yu, J., and Yu, X. (2019). An automatic surface defect inspection system for automobiles using machine vision methods. Sensors, 19.","DOI":"10.3390\/s19030644"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"125905","DOI":"10.1088\/1361-6501\/ab1467","article-title":"A mobile vision inspection system for tiny defect detection on smooth car-body surfaces based on deep ensemble learning","volume":"30","author":"Chang","year":"2019","journal-title":"Meas. Sci. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sun, X., Gu, J., Huang, R., Zou, R., and Giron Palomares, B. (2019). Surface defects recognition of wheel hub based on improved faster R-CNN. Electronics, 8.","DOI":"10.3390\/electronics8050481"},{"key":"ref_19","first-page":"412","article-title":"Applications of augmented reality for inspection and maintenance process in automotive industry","volume":"10","author":"Halim","year":"2018","journal-title":"J. Fundam. Appl. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chouchene, A., Ventura Carvalho, A., Charrua-Santos, F., and Barhoumi, W. (2022). Augmented Reality-Based Framework Supporting Visual Inspection for Automotive Industry. Appl. Syst. Innov., 5.","DOI":"10.3390\/asi5030048"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.eswa.2017.03.060","article-title":"Markerless tracking system for augmented reality in the automotive industry","volume":"82","author":"Lima","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_22","unstructured":"(2021, December 14). Ford. Available online: https:\/\/media.ford.com\/content\/fordmedia\/fna\/us\/en\/news\/2016\/02\/10\/innovative-smartphone-app-saves-ford-factory-workers.html."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Rega, A., Di Marino, C., Pasquariello, A., Vitolo, F., Patalano, S., Zanella, A., and Lanzotti, A. (2021). Collaborative Workplace Design: A Knowledge-Based Approach to Promote Human\u2013Robot Collaboration and Multi-Objective Layout Optimization. Appl. Sci., 11.","DOI":"10.20944\/preprints202111.0241.v1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"9502692","DOI":"10.1155\/2018\/9502692","article-title":"Designing a human machine interface for quality assurance in car manufacturing: An attempt to address the \u201cfunctionality versus user experience contradiction\u201d in professional production environments","volume":"2018","author":"Borisov","year":"2018","journal-title":"Adv. Hum.-Comput. Interact."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Khamaisi, R.K., Prati, E., Peruzzini, M., Raffaeli, R., and Pellicciari, M. (2021). UX in AR-Supported Industrial Human\u2013Robot Collaborative Tasks: A Systematic Review. Appl. Sci., 11.","DOI":"10.3390\/app112110448"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1007\/978-3-030-58653-9_71","article-title":"Engine Labels Detection for Vehicle Quality Verification in the Assembly Line: A Machine Vision Approach","volume":"Volume 695","author":"Coelho","year":"2021","journal-title":"CONTROLO 2020. Lecture Notes in Electrical Engineering"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1007\/978-3-030-58653-9_56","article-title":"Classification of Car Parts Using Deep Neural Network","volume":"Volume 695","author":"Coelho","year":"2021","journal-title":"CONTROLO 2020. Lecture Notes in Electrical Engineering"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"7375","DOI":"10.1080\/00207543.2018.1428774","article-title":"An optimisation support for the design of hybrid production lines including assembly and disassembly tasks","volume":"56","author":"Mete","year":"2018","journal-title":"Int. J. Prod. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Reis, A., Barroso, J., Lopes, J.B., Mikropoulos, T., and Fan, C.W. (2021). Worker Support and Training Tools to Aid in Vehicle Quality Inspection for the Automotive Industry. Technology and Innovation in Learning, Teaching and Education. TECH-EDU 2020, Springer International Publishing.","DOI":"10.1007\/978-3-030-73988-1"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bebis, G., Athitsos, V., Yan, T., Lau, M., Li, F., Shi, C., Yuan, X., Mousas, C., and Bruder, G. (2021). Improving Automatic Quality Inspection in the Automotive Industry by Combining Simulated and Real Data. Advances in Visual Computing. ISVC 2021. Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-030-90439-5"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nikolenko, S.I. (2021). Introduction: The Data Problem. Synthetic Data for Deep Learning, Springer International Publishing.","DOI":"10.1007\/978-3-030-75178-4"},{"key":"ref_32","unstructured":"Oza, P., Sindagi, V.A., VS, V., and Patel, V.M. (2021). Unsupervised domain adaptation of object detectors: A survey. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"123903","DOI":"10.1109\/ACCESS.2019.2938837","article-title":"Cycle-Consistent Domain Adaptive Faster RCNN","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hsu, H.K., Yao, C.H., Tsai, Y.H., Hung, W.C., Tseng, H.Y., Singh, M., and Yang, M.H. (2020, January 1\u20135). Progressive Domain Adaptation for Object Detection. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093358"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"MacKay, C.T., and Moh, T.S. (2021, January 4\u20136). Learning for Free: Object Detectors Trained on Synthetic Data. Proceedings of the 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM), Seoul, Korea.","DOI":"10.1109\/IMCOM51814.2021.9377353"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"RoyChowdhury, A., Chakrabarty, P., Singh, A., Jin, S., Jiang, H., Cao, L., and Learned-Miller, E. (2019, January 16\u201320). Automatic adaptation of object detectors to new domains using self-training. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00087"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Inoue, N., Furuta, R., Yamasaki, T., and Aizawa, K. (2018, January 18\u201322). Cross-domain weakly-supervised object detection through progressive domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00525"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Deng, J., Li, W., Chen, Y., and Duan, L. (2021, January 20\u201325). Unbiased mean teacher for cross-domain object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00408"},{"key":"ref_39","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2021, November 23). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2016). Aggregated Residual Transformations for Deep Neural Networks. arXiv.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_41","unstructured":"Zhao, S., Liu, Z., Lin, J., Zhu, J.Y., and Han, S. (2020, January 6\u201312). Differentiable Augmentation for Data-Efficient GAN Training. Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), Online."},{"key":"ref_42","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_43","unstructured":"(2021, November 15). ZXing. Available online: https:\/\/github.com\/zxing\/zxing."},{"key":"ref_44","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017, January 4\u20139). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA. NIPS\u201917."}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/11\/5687\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:24:12Z","timestamp":1760138652000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/12\/11\/5687"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,3]]},"references-count":44,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["app12115687"],"URL":"https:\/\/doi.org\/10.3390\/app12115687","relation":{},"ISSN":["2076-3417"],"issn-type":[{"value":"2076-3417","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,3]]}}}