{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T14:24:15Z","timestamp":1769610255315,"version":"3.49.0"},"reference-count":31,"publisher":"ASME International","issue":"5","license":[{"start":{"date-parts":[[2020,5,26]],"date-time":"2020-05-26T00:00:00Z","timestamp":1590451200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Current manual practices of replacing bolts on structures are time-consuming and costly, especially because of numerous bolts. Thus, an automated method that can visually detect and localize bolt positions would be highly beneficial. We demonstrate the use of deep neural networks using domain randomization for detecting and localizing bolts on a workpiece. In contrast to previous approaches that require training on real images, the use of domain randomization enables all training in simulation. The key idea is to create a wide variety of computer-generated synthetic images by varying the texture, color, camera position and orientation, distractor objects, and noise, and train the neural network on these images such that the neural network is robust to scene variability and hence provides accurate results when deployed on real images. Using domain randomization, we train two neural networks, a faster regional convolutional neural network for detecting the bolt and placing a bounding box, and a regression convolutional neural network for estimating the x- and y-position of the bolts relative to the coordinates fixed to the workpiece. Our results indicate that in the best case, we can detect bolts with 85% accuracy and can predict 75% of bolts within 1.27 cm accuracy. The novelty of this work is in using domain randomization to detect and localize: (1) multiples of a single object and (2) small-sized objects (0.6 cm \u00d7 2.5 cm).<\/jats:p>","DOI":"10.1115\/1.4047074","type":"journal-article","created":{"date-parts":[[2020,5,2]],"date-time":"2020-05-02T06:14:03Z","timestamp":1588400043000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":3,"title":["Domain Randomization Using Deep Neural Networks for Estimating Positions of Bolts"],"prefix":"10.1115","volume":"20","author":[{"given":"Ezra","family":"Ameperosa","sequence":"first","affiliation":[{"name":"Air Force Research Laboratory, Materials and Manufacturing, Directorate (AFRL\/RX), Dayton, OH 45433"}]},{"given":"Pranav A.","family":"Bhounsule","sequence":"additional","affiliation":[{"name":"Department of Mechanical and Industrial Engineering, University of Illinois at Chicago, 842 W. Taylor St., Chicago, IL 60607"}]}],"member":"33","published-online":{"date-parts":[[2020,5,26]]},"reference":[{"issue":"4","key":"2021022704041366400_CIT0001","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1177\/0954409715588119","article-title":"Online Inspection System for the Automatic Detection of Bolt Defects on a Freight Train","volume":"230","author":"Li","year":"2016","journal-title":"Proc. Inst. Mech. Eng., Part F: J. Rail Rapid Transit"},{"key":"2021022704041366400_CIT0002","doi-asserted-by":"crossref","DOI":"10.1117\/12.847709","article-title":"Active Loose Bolt Detection in a Complex Satellite Structure","author":"Reynolds","year":"2010"},{"key":"2021022704041366400_CIT0003","first-page":"135","article-title":"Evaluating the Benefits of Augmented Reality for Task Localization in Maintenance of an Armored Personnel Carrier Turret","author":"Henderson","year":"2009"},{"key":"2021022704041366400_CIT0004","doi-asserted-by":"crossref","DOI":"10.22260\/ISARC2011\/0128","article-title":"Cognition System of Bolt Hole Using Template Matching","author":"Baek","year":"2011"},{"key":"2021022704041366400_CIT0005","first-page":"50","article-title":"Vision Based Pose Estimation of Multiple Peg-in-Hole for Robotic Assembly","author":"Nagarajan","year":"2016"},{"key":"2021022704041366400_CIT0006","first-page":"4821","article-title":"Vision-Based Estimation of Bolt-Hole Location Using Circular Hough Transform","author":"Choe","year":"2009"},{"key":"2021022704041366400_CIT0007","first-page":"855","article-title":"Ikeabot: An Autonomous Multi-Robot Coordinated Furniture Assembly System","author":"Knepper","year":"2013"},{"key":"2021022704041366400_CIT0008","first-page":"69","article-title":"Automatic Assembly Planning with Fasteners","author":"Miller","year":"1989"},{"key":"2021022704041366400_CIT0009","first-page":"2155","article-title":"Fast 3d Recognition and Pose Using the Viewpoint Feature Histogram","author":"Rusu","year":"2010"},{"issue":"2","key":"2021022704041366400_CIT0010","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MMUL.2012.24","article-title":"Microsoft Kinect Sensor and Its Effect","volume":"19","author":"Zhang","year":"2012","journal-title":"IEEE Multimedia"},{"key":"2021022704041366400_CIT0011","first-page":"2050","article-title":"Efficient Multi-View Object Recognition and Full Pose Estimation","author":"Collet","year":"2010"},{"key":"2021022704041366400_CIT0012","first-page":"999","article-title":"Domain Adaptation for Object Recognition: An Unsupervised Approach","author":"Gopalan","year":"2011"},{"issue":"5","key":"2021022704041366400_CIT0013","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1109\/JSTARS.2012.2202881","article-title":"Svm-Based Boosting of Active Learning Strategies for Efficient Domain Adaptation","volume":"5","author":"Matasci","year":"2012","journal-title":"IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens."},{"key":"2021022704041366400_CIT0014","first-page":"23","article-title":"Domain Randomization for Transferring Deep Neural Networks From Simulation to the Real World","author":"Tobin","year":"2017"},{"key":"2021022704041366400_CIT0015","first-page":"4243","article-title":"Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping","author":"Bousmalis","year":"2018"},{"key":"2021022704041366400_CIT0016","first-page":"969","article-title":"Training Deep Networks With Synthetic Data: Bridging the Reality Gap by Domain Randomization","author":"Tremblay","year":"2018"},{"key":"2021022704041366400_CIT0017","article-title":"Domain Randomization and Generative Models for Robotic Grasping","author":"Tobin","year":"2017"},{"key":"2021022704041366400_CIT0018","article-title":"Applying Domain Randomization to Synthetic Data for Object Category Detection","author":"Borrego","year":"2018"},{"key":"2021022704041366400_CIT0019","first-page":"1","article-title":"Sim-to-Real Transfer of Robotic Control With Dynamics Randomization","author":"Peng","year":"2018"},{"key":"2021022704041366400_CIT0020","article-title":"&gt;Sim-to-Real: Learning Agile Locomotion for Quadruped Robots","author":"Tan","year":"2018"},{"key":"2021022704041366400_CIT0021","first-page":"5307","article-title":"Ensemble-CIO: Full-Body Dynamic Motion Planning That Transfers to Physical Humanoids","author":"Mordatch","year":"2015"},{"issue":"5","key":"2021022704041366400_CIT0022","doi-asserted-by":"crossref","first-page":"4392","DOI":"10.1109\/TIE.2017.2764844","article-title":"NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Na\u00efve Bayes Data Fusion","volume":"65","author":"Chen","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"issue":"6","key":"2021022704041366400_CIT0023","doi-asserted-by":"crossref","first-page":"2525","DOI":"10.1007\/s10845-018-1415-x","article-title":"Automated Defect Inspection of Led Chip Using Deep Convolutional Neural Network","volume":"30","author":"Lin","year":"2019","journal-title":"J. Intell. Manuf."},{"issue":"11","key":"2021022704041366400_CIT0024","doi-asserted-by":"crossref","first-page":"7067","DOI":"10.1109\/TIE.2016.2582729","article-title":"Real-Time Motor Fault Detection by 1-d Convolutional Neural Networks","volume":"63","author":"Ince","year":"2016","journal-title":"IEEE Trans. Ind. Electron."},{"issue":"7","key":"2021022704041366400_CIT0025","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1177\/004051759606600710","article-title":"Automatic Inspection of Fabric Defects Using an Artificial Neural Network Technique","volume":"66","author":"Tsai","year":"1996","journal-title":"Text. Res. J."},{"issue":"2","key":"2021022704041366400_CIT0026","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1109\/6104.846932","article-title":"Solder Joints Inspection Using a Neural Network and Fuzzy Rule-Based Classification Method","volume":"23","author":"Ko","year":"2000","journal-title":"IEEE Trans. Electron. Packaging Manuf."},{"issue":"9","key":"2021022704041366400_CIT0027","doi-asserted-by":"crossref","first-page":"11837","DOI":"10.1016\/j.eswa.2011.03.072","article-title":"A Genetic Fuzzy Radial Basis Function Neural Network for Structural Health Monitoring of Composite Laminated Beams","volume":"38","author":"Zheng","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"2021022704041366400_CIT0028","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2019-97393","article-title":"Domain Randomization for Detection and Position Estimation of Multiples of a Single Object With Applications to Localizing Bolts on Structures","author":"Ameperosa","year":"2019"},{"key":"2021022704041366400_CIT0029","article-title":"Airbus a380-842 Horizontal Stabiliser Structure\u2014Bushing Migrated","author":"Australian Government","year":"2018"},{"key":"2021022704041366400_CIT0030","article-title":"Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks","author":"Ren","year":"2015"},{"issue":"3","key":"2021022704041366400_CIT0031","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4047074\/6647499\/jcise_20_5_051006.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4047074\/6647499\/jcise_20_5_051006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,27]],"date-time":"2021-02-27T04:04:27Z","timestamp":1614398667000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4047074\/1083434\/Domain-Randomization-Using-Deep-Neural-Networks"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,26]]},"references-count":31,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,10,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4047074","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,26]]},"article-number":"051006"}}