{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T20:00:16Z","timestamp":1779912016647,"version":"3.53.1"},"reference-count":61,"publisher":"ASME International","issue":"3","license":[{"start":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T00:00:00Z","timestamp":1669075200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1925084"],"award-info":[{"award-number":["1925084"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Deep learning-based image segmentation methods have showcased tremendous potential in defect detection applications for several manufacturing processes. Currently, majority of deep learning research for defect detection focuses on manufacturing processes where the defects have well-defined features and there is tremendous amount of image data available to learn such a data-dense model. This makes deep learning unsuitable for defect detection in high-mix low volume manufacturing applications where data are scarce and the features of defects are not well defined due to the nature of the process. Recently, there has been an increased impetus towards automation of high-performance manufacturing processes such as composite prepreg layup. Composite prepreg layup is high-mix low volume in nature and involves manipulation of a sheet-like material. In this work, we propose a deep learning framework to detect wrinkle-like defects during the composite prepreg layup process. Our work focuses on three main technological contributions: (1) generation of physics aware photo-realistic synthetic images with the combination of a thin-shell finite element-based sheet simulation and advanced graphics techniques for texture generation, (2) an open-source annotated dataset of 10,000 synthetic images and 1000 real process images of carbon fiber sheets with wrinkle-like defects, and (3) an efficient two-stage methodology for training the deep learning network on this hybrid dataset. Our method can achieve a mean average precision (mAP) of 0.98 on actual production data for detecting defects.<\/jats:p>","DOI":"10.1115\/1.4056295","type":"journal-article","created":{"date-parts":[[2022,11,22]],"date-time":"2022-11-22T03:10:16Z","timestamp":1669086616000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":13,"title":["Physics Informed Synthetic Image Generation for Deep Learning-Based Detection of Wrinkles and Folds"],"prefix":"10.1115","volume":"23","author":[{"given":"Omey M.","family":"Manyar","sequence":"first","affiliation":[{"name":"Center for Advanced Manufacturing, University of Southern California , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junyan","family":"Cheng","sequence":"additional","affiliation":[{"name":"Center for Advanced Manufacturing, University of Southern California , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Reuben","family":"Levine","sequence":"additional","affiliation":[{"name":"Center for Advanced Manufacturing, University of Southern California , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vihan","family":"Krishnan","sequence":"additional","affiliation":[{"name":"Center for Advanced Manufacturing, University of Southern California , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jernej","family":"Barbi\u010d","sequence":"additional","affiliation":[{"name":"University of Southern California Department of Computer Science, , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Satyandra K.","family":"Gupta","sequence":"additional","affiliation":[{"name":"Center for Advanced Manufacturing, University of Southern California , Los Angeles, CA 90007"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2022,12,9]]},"reference":[{"key":"2023120911400753700_CIT0001","first-page":"769","article-title":"Real-Time Rail Head Surface Defect Detection: A Geometrical Approach","author":"Jie","year":"2009"},{"key":"2023120911400753700_CIT0002","doi-asserted-by":"crossref","DOI":"10.1109\/ICIG.2009.144","article-title":"Automated Pattern Recognition and Defect Inspection System","author":"Cui","year":"2009"},{"key":"2023120911400753700_CIT0003","article-title":"Surface Defect Detection With Histogram-Based Texture Features","author":"Iivarinen","year":"2000"},{"issue":"5","key":"2023120911400753700_CIT0004","doi-asserted-by":"publisher","first-page":"5930","DOI":"10.1016\/j.eswa.2010.11.030","article-title":"A Vision Inspection System for the Surface Defects of Strongly Reflected Metal Based on Multi-Class SVM","volume":"38","author":"Xue-wu","year":"2011","journal-title":"Exp. Syst. Appl."},{"key":"2023120911400753700_CIT0005","first-page":"1","article-title":"Steel Defect Classification With Max-Pooling Convolutional Neural Networks","author":"Masci","year":"2012"},{"key":"2023120911400753700_CIT0006","first-page":"1725","article-title":"Detection of Line Defects in Steel Billets Using Undecimated Wavelet Transform","author":"Yun","year":"2008"},{"key":"2023120911400753700_CIT0007","first-page":"1","article-title":"Fabric Defect Detection Algorithm Using Morphological Processing and DCT","author":"Aziz","year":"2013"},{"key":"2023120911400753700_CIT0008","first-page":"1","article-title":"Terahertz Imaging Method for Composite Insulator Defects Based on Edge Detection Algorithm","volume":"70","author":"Mei","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"issue":"4","key":"2023120911400753700_CIT0009","doi-asserted-by":"publisher","first-page":"040801","DOI":"10.1115\/1.4049535","article-title":"Image-Based Surface Defect Detection Using Deep Learning: A Review","volume":"21","author":"Bhatt","year":"2021","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"4","key":"2023120911400753700_CIT0010","doi-asserted-by":"publisher","first-page":"1493","DOI":"10.1109\/TIM.2019.2915404","article-title":"An End-to-End Steel Surface Defect Detection Approach Via Fusing Multiple Hierarchical Features","volume":"69","author":"He","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"2023120911400753700_CIT0011","first-page":"1","article-title":"Defectnet: Toward Fast and Effective Defect Detection","volume":"70","author":"Li","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"2023120911400753700_CIT0012","doi-asserted-by":"publisher","first-page":"42285","DOI":"10.1109\/ACCESS.2020.2977821","article-title":"A Weakly Supervised Surface Defect Detection Based on Convolutional Neural Network","volume":"8","author":"Xu","year":"2020","journal-title":"IEEE Access"},{"issue":"7","key":"2023120911400753700_CIT0013","doi-asserted-by":"publisher","first-page":"1557","DOI":"10.1016\/j.jmatprotec.2012.03.002","article-title":"Automatic Defect Identification Using Thermal Image Analysis for Online Weld Quality Monitoring","volume":"212","author":"Sreedhar","year":"2012","journal-title":"J. Mater. Process. Technol."},{"key":"2023120911400753700_CIT0014","article-title":"A Synthetic Image Assisted Deep Learning Framework for Detecting Defects During Composite Prepreg Layup","author":"Manyar","year":"2022"},{"key":"2023120911400753700_CIT0015","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.00839","article-title":"Bachgan: High-Resolution Image Synthesis From Salient Object Layout","author":"Li","year":"2020"},{"key":"2023120911400753700_CIT0016","first-page":"969","article-title":"Training Deep Networks With Synthetic Data: Bridging the Reality Gap by Domain Randomization","author":"Tremblay","year":"2018"},{"key":"2023120911400753700_CIT0017","first-page":"14986","article-title":"Anycost Gans for Interactive Image Synthesis and Editing","author":"Lin","year":"2021"},{"key":"2023120911400753700_CIT0018","doi-asserted-by":"crossref","DOI":"10.1117\/12.2533485","article-title":"Physically Based Synthetic Image Generation for Machine Learning: A Review of Pertinent Literature","author":"Schraml","year":"2019"},{"issue":"15","key":"2023120911400753700_CIT0019","doi-asserted-by":"publisher","first-page":"2733","DOI":"10.3390\/math10152733","article-title":"Survey on Synthetic Datageneration, Evaluation Methods and Gans","volume":"10","author":"Figueira","year":"2022","journal-title":"Mathematics"},{"key":"2023120911400753700_CIT0020","doi-asserted-by":"publisher","first-page":"102020","DOI":"10.1016\/j.rcim.2020.102020","article-title":"Automated Planning for Robotic Layup of Composite Prepreg","volume":"67","author":"Malhan","year":"2021","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"2023120911400753700_CIT0021","first-page":"2980","article-title":"Mask R-CNN","author":"He","year":"2017"},{"key":"2023120911400753700_CIT0022","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1016\/j.ultras.2017.11.015","article-title":"Mid-Infrared Pulsed Laser Ultrasonic Testing for Carbon Fiber Reinforced Plastics","volume":"84","author":"Kusano","year":"2018","journal-title":"Ultrasonics"},{"key":"2023120911400753700_CIT0023","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/LMAG.2016.2614248","article-title":"Magnetic Particle Detection System Using Fluxgate Gradiometer on a Permalloy Shielding Disk","volume":"7","author":"Elrefai","year":"2016","journal-title":"IEEE Mag. Lett."},{"issue":"4","key":"2023120911400753700_CIT0024","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1109\/TIM.2018.2792848","article-title":"Fast Eddy Current Testing Defect Classification Using Lissajous Figures","volume":"67","author":"D\u2019Angelo","year":"2018","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"2023120911400753700_CIT0025","doi-asserted-by":"crossref","DOI":"10.1109\/COASE.2018.8560586","article-title":"Hybrid Cells for Multi-Layer Prepreg Composite Sheet Layup","author":"Malhan","year":"2018"},{"key":"2023120911400753700_CIT0026","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA.2019.8794353","article-title":"Identifying Feasible Workpiece Placement With Respect to Redundant Manipulator for Complex Manufacturing Tasks","author":"Malhan","year":"2019"},{"key":"2023120911400753700_CIT0027","doi-asserted-by":"crossref","DOI":"10.1115\/MSEC2019-3003","article-title":"Determining Feasible Robot Placements in Robotic Cells for Composite Prepreg Sheet Layup","author":"Malhan","year":"2019"},{"key":"2023120911400753700_CIT0028","doi-asserted-by":"crossref","DOI":"10.1115\/MSEC2021-63900","article-title":"A Digital Twin for Automated Layup of Prepreg Composite Sheets","author":"Chen","year":"2021"},{"key":"2023120911400753700_CIT0029","first-page":"930","article-title":"A Simulation-Based Grasp Planner for Enabling Robotic Grasping During Composite Sheet Layup","author":"Manyar","year":"2021"},{"issue":"1","key":"2023120911400753700_CIT0030","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1515\/secm-2021-0001","article-title":"Real Time Defect Detection During Composite Layup Via Tactile Shape Sensing","volume":"28","author":"Elkington","year":"2021","journal-title":"Sci. Eng. Comput. Mater."},{"issue":"1","key":"2023120911400753700_CIT0031","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1109\/TIE.1930.896476","article-title":"Computer-Vision-Based Fabric Defect Detection: A Survey","volume":"55","author":"Kumar","year":"2008","journal-title":"IEEE Trans. Ind. Electron."},{"key":"2023120911400753700_CIT0032","doi-asserted-by":"publisher","first-page":"10922","DOI":"10.1109\/TIE.2019.2962437","article-title":"An Efficient Convolutional Neural Network Model Based on Object-Level Attention Mechanism for Casting Defect Detection on Radiography Images","volume":"67","author":"Hu","year":"2019","journal-title":"IEEE Trans. Ind. Electron."},{"issue":"9","key":"2023120911400753700_CIT0033","doi-asserted-by":"publisher","first-page":"3465","DOI":"10.1007\/s00170-017-0882-0","article-title":"A Fast and Robust Convolutional Neural Network-Based Defect Detection Model in Product Quality Control","volume":"94","author":"Wang","year":"2018","journal-title":"Int. J. Adv. Manuf. Technol."},{"issue":"3","key":"2023120911400753700_CIT0034","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1007\/s10845-019-01476-x","article-title":"Segmentation-Based Deep-Learning Approach for Surface-Defect Detection","volume":"31","author":"Tabernik","year":"2020","journal-title":"J. Intell. Manuf."},{"issue":"3","key":"2023120911400753700_CIT0035","doi-asserted-by":"publisher","first-page":"388","DOI":"10.3390\/met11030388","article-title":"Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks","volume":"11","author":"Wang","year":"2021","journal-title":"Metals"},{"key":"2023120911400753700_CIT0036","article-title":"Same Same But Different: Semi-Supervised Defect Detection With Normalizing Flows","author":"Rudolph","year":"2020"},{"issue":"24","key":"2023120911400753700_CIT0037","doi-asserted-by":"publisher","first-page":"5755","DOI":"10.3390\/ma13245755","article-title":"Using Deep Learning to Detect Defects in Manufacturing: A Comprehensive Survey and Current Challenges","volume":"13","author":"Yang","year":"2020","journal-title":"Materials"},{"issue":"7","key":"2023120911400753700_CIT0038","doi-asserted-by":"publisher","first-page":"1364","DOI":"10.3390\/app9071364","article-title":"Sdd-cnn: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection","volume":"9","author":"Xu","year":"2019","journal-title":"Appl. Sci."},{"key":"2023120911400753700_CIT0039","first-page":"473","volume-title":"A Surface Defect Detection Method Based on Positive Samples","author":"Zhao","year":"2018"},{"key":"2023120911400753700_CIT0040","doi-asserted-by":"publisher","first-page":"101105","DOI":"10.1016\/j.aei.2020.101105","article-title":"Anomaly Detection of Defects on Concrete Structures With the Convolutional Autoencoder","volume":"45","author":"Chow","year":"2020","journal-title":"Adv. Eng. Inform."},{"issue":"2","key":"2023120911400753700_CIT0041","doi-asserted-by":"publisher","first-page":"1256","DOI":"10.1109\/TASE.2016.2520955","article-title":"Deformable Patterned Fabric Defect Detection With Fisher Criterion-Based Deep Learning","volume":"14","author":"Li","year":"2017","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"issue":"6","key":"2023120911400753700_CIT0042","doi-asserted-by":"publisher","first-page":"1767","DOI":"10.1007\/s10845-021-01738-7","article-title":"Synthetic Image Data Augmentation for Fibre Layup Inspection Processes: Techniques to Enhance the Data Set","volume":"32","author":"Meister","year":"2021","journal-title":"J. Intell. Manuf."},{"issue":"1","key":"2023120911400753700_CIT0043","first-page":"38","article-title":"Vega: Nonlinear FEM Deformable Object Simulator","volume":"32","author":"Sin","year":"2013","journal-title":"Comput. Graph."},{"key":"2023120911400753700_CIT0044","volume-title":"Blender\u2014A 3D Modelling and Rendering Package","author":"Community","year":"2018"},{"key":"2023120911400753700_CIT0045","article-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","author":"Simonyan","year":"2015"},{"key":"2023120911400753700_CIT0046","first-page":"1","article-title":"Going Deeper With Convolutions","author":"Szegedy","year":"2015"},{"key":"2023120911400753700_CIT0047","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.243","article-title":"Densely Connected Convolutional Networks","author":"Huang","year":"2017"},{"key":"2023120911400753700_CIT0048","first-page":"936","article-title":"Feature Pyramid Networks for Object Detection","author":"Lin","year":"2017"},{"key":"2023120911400753700_CIT0049","first-page":"740","volume-title":"Microsoft Coco: Common Objects in Context","author":"Lin","year":"2014"},{"key":"2023120911400753700_CIT0050","article-title":"MMDetection: Open MMLab Detection Toolbox and Benchmark","author":"Chen","year":"2019"},{"key":"2023120911400753700_CIT0051","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1007\/978-3-030-43089-4_44","volume-title":"Algorithmic Foundations of Robotics XII","author":"Tzeng","year":"2020"},{"key":"2023120911400753700_CIT0052","doi-asserted-by":"crossref","DOI":"10.1109\/IROS40897.2019.8967622","article-title":"Learning to Augment Synthetic Images for Sim2Real Policy Transfer","author":"Pashevich","year":"2019"},{"key":"2023120911400753700_CIT0053","first-page":"3634","article-title":"Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation","author":"Sajjan","year":"2020"},{"issue":"2","key":"2023120911400753700_CIT0054","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (VOC) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"2023120911400753700_CIT0055","first-page":"8024","article-title":"Pytorch: An Imperative Style, High-Performance Deep Learning Library","author":"Paszke","year":"2019"},{"key":"2023120911400753700_CIT0056","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW56347.2022.00309","article-title":"ResNeSt: Split-Attention Networks","author":"Zhang","year":"2022"},{"issue":"5","key":"2023120911400753700_CIT0057","doi-asserted-by":"publisher","first-page":"1483","DOI":"10.1109\/TPAMI.2019.2956516","article-title":"Cascade R-CNN: High Quality Object Detection and Instance Segmentation","volume":"43","author":"Cai","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2023120911400753700_CIT0058","article-title":"Ensemble Deep Learning: A Review","author":"Ganaie","year":"2021"},{"issue":"7","key":"2023120911400753700_CIT0059","doi-asserted-by":"publisher","first-page":"2449","DOI":"10.1109\/TPAMI.2020.2969348","article-title":"Physics-Based Generative Adversarial Models for Image Restoration and Beyond","volume":"43","author":"Pan","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2023120911400753700_CIT0060","first-page":"11130","article-title":"Three Ways to Improve Semantic Segmentation With Self-Supervised Depth Estimation","author":"Hoyer","year":"2021"},{"key":"2023120911400753700_CIT0061","first-page":"12715","article-title":"Semi-Supervised Semantic Image Segmentation With Self-Correcting Networks","author":"Ibrahim","year":"2020"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/23\/3\/030903\/6959176\/jcise_23_3_030903.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/23\/3\/030903\/6959176\/jcise_23_3_030903.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,9]],"date-time":"2023-12-09T11:41:26Z","timestamp":1702122086000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/23\/3\/030903\/1150957\/Physics-Informed-Synthetic-Image-Generation-for"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,9]]},"references-count":61,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,6,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4056295","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,9]]},"article-number":"030903"}}