{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T12:53:02Z","timestamp":1753879982268,"version":"3.41.2"},"reference-count":60,"publisher":"ASME International","issue":"2","license":[{"start":{"date-parts":[[2020,10,13]],"date-time":"2020-10-13T00:00:00Z","timestamp":1602547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The use of flexible and autonomous robotic systems is a possible solution for automation in dynamic and unstructured industrial environments. Pick and place robotic applications are becoming common for the automation of manipulation tasks in an industrial context. This context requires the robot to be aware of its surroundings throughout the whole manipulation task, even after accomplishing the gripping action. This work introduces the deep post gripping perception framework, which includes post gripping perception abilities realized with the help of deep learning techniques, mainly unsupervised learning methods. These abilities help robots to execute a stable and precise placing of the gripped items while respecting the process quality requirements. The framework development is described based on the results of a literature review on post gripping perception functions and frameworks. This results in a modular design using three building components to realize planning, monitoring and verifying modules. Experimental evaluation of the framework shows its advantages in terms of process quality and stability in pick and place applications.<\/jats:p>","DOI":"10.1115\/1.4048204","type":"journal-article","created":{"date-parts":[[2020,8,31]],"date-time":"2020-08-31T16:53:43Z","timestamp":1598892823000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":4,"title":["Design of a Deep Post Gripping Perception Framework for Industrial Robots"],"prefix":"10.1115","volume":"21","author":[{"given":"Firas","family":"Zoghlami","sequence":"first","affiliation":[{"name":"Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich, Germany"}]},{"given":"Philip","family":"Kurrek","sequence":"additional","affiliation":[{"name":"Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich, Germany"}]},{"given":"Mark","family":"Jocas","sequence":"additional","affiliation":[{"name":"Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich, Germany"}]},{"given":"Giovanni","family":"Masala","sequence":"additional","affiliation":[{"name":"Computing and Mathematics, Manchester Metropolitan University, Manchester, United Kingdom"}]},{"given":"Vahid","family":"Salehi","sequence":"additional","affiliation":[{"name":"Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich, Germany"}]}],"member":"33","published-online":{"date-parts":[[2020,10,13]]},"reference":[{"article-title":"Robots and the Workplace of the Future, Technical Report, International Federation of Robotics Frankfurt","year":"2018","author":"IFR","key":"2020101307451101700_CIT0001"},{"key":"2020101307451101700_CIT0002","unstructured":"Lammer, U. A. , 2017, \u201cFunktionsvereinigung in der Lagertechnik,\u201d PhD thesis, Technical University of Munich, Munich, Germany."},{"key":"2020101307451101700_CIT0003","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-54935-3","volume-title":"Die Digitale Transformation Der Automobilindustrie: Treiber - Roadmap - Praxis","author":"Winkelhake","year":"2017"},{"article-title":"Using Point Cloud Technology for Process Simulation in the Context of Digital Factory Based on a Systems Engineering Integrated Approach","year":"2017","author":"Salehi","key":"2020101307451101700_CIT0004"},{"issue":"1","key":"2020101307451101700_CIT0005","doi-asserted-by":"crossref","first-page":"147","DOI":"10.14733\/cadaps.2020.147-166","article-title":"Development of An Agile Concept for MBSE for Future Digital Products Through the Entire Life Cycle Management Called Munich Agile MBSE Concept (MAGIC)","volume":"17","author":"Salehi","year":"2019","journal-title":"Comput.-Aided Design Appl."},{"issue":"1","key":"2020101307451101700_CIT0006","first-page":"3701","article-title":"Munich Agile MBSE Concept (MAGIC)","volume":"1","author":"Salehi","year":"2019","journal-title":"Proc. Design Soc.: Int. Conf. Eng. Design"},{"issue":"6","key":"2020101307451101700_CIT0007","doi-asserted-by":"crossref","first-page":"061009","DOI":"10.1115\/1.4046992","article-title":"Q-model: An Artificial Intelligence Based Methodology for the Development of Autonomous Robots","volume":"20","author":"Kurrek","year":"2020","journal-title":"ASME J. Comput. Inform. Sci. Eng."},{"key":"2020101307451101700_CIT0008","doi-asserted-by":"crossref","DOI":"10.1109\/ICPS48405.2020.9274698","article-title":"Reinforcement Learning Lifecycle for the Design of Advanced Robotic Systems","author":"Kurrek","year":"2020"},{"article-title":"Artificial Intelligence and Machine Learning. KEGA research project, Virtual Program Modules of AI Systems","year":"2005","author":"Saloky","key":"2020101307451101700_CIT0009"},{"key":"2020101307451101700_CIT0010","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell","year":"2009","edition":"3rd ed"},{"issue":"1","key":"2020101307451101700_CIT0011","doi-asserted-by":"crossref","first-page":"011004","DOI":"10.1115\/1.4037435","article-title":"Application of Feature-Learning Methods Toward Product Usage Context Identification and Comfort Prediction","volume":"18","author":"Ghosh","year":"2017","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2020101307451101700_CIT0012","first-page":"1","article-title":"A Survey on Cloud Computing","author":"Shaw","year":"2014"},{"key":"2020101307451101700_CIT0013","unstructured":"Steder, B. ,2013, \u201cFeature-Based 3D Perception for Mobile Robots,\u201d PhD thesis, Uni Freiburg, Freiburg, Germany."},{"issue":"1","key":"2020101307451101700_CIT0014","first-page":"3761","article-title":"Usage Identification of Anomaly Detection in An Industrial Context","volume":"1","author":"Zoghlami","year":"2019","journal-title":"Proc. Design Soc.: Int. Conf. Eng. Design"},{"issue":"1","key":"2020101307451101700_CIT0015","first-page":"3561","article-title":"AI Motion Control a Generic Approach to Develop Control Policies for Robotic Manipulation Tasks","volume":"1","author":"Kurrek","year":"2019","journal-title":"Proc. Design Soc.: Int. Conf. Eng. Design"},{"issue":"1","key":"2020101307451101700_CIT0016","first-page":"2041","article-title":"AI-Based Learning Approach with Consideration of Safety Criteria on Example of a Depalletization Robot","volume":"1","author":"Jocas","year":"2019","journal-title":"Proc. Design Soc.: Int. Conf. Eng. Design"},{"key":"2020101307451101700_CIT0017","first-page":"721","article-title":"Dorapicker: An Autonomous Picking System for General Objects","author":"Zhang","year":"2016"},{"key":"2020101307451101700_CIT0018","doi-asserted-by":"crossref","DOI":"10.1145\/3351917.3351982","article-title":"Enabling Robot Selective Trained Deep Neural Networks for Object Detection Through Intelligent Infrastructure","author":"Poss","year":"2019"},{"key":"2020101307451101700_CIT0019","first-page":"146","article-title":"Perception Based Intelligent Materialhandling in Industrial Logistics Environments","author":"Poss","year":"2019"},{"key":"2020101307451101700_CIT0020","first-page":"717","article-title":"Robust Framework for Intelligent Gripping Point Detection","author":"Poss","year":"2019"},{"key":"2020101307451101700_CIT0021","first-page":"152","article-title":"Perceiving Clutter and Surfaces for Object Placement in Indoor Environments","author":"Schuster","year":"2010"},{"issue":"9","key":"2020101307451101700_CIT0022","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1177\/0278364912438781","article-title":"Learning to Place New Objects in a Scene","volume":"31","author":"Jiang","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"2020101307451101700_CIT0023","first-page":"1414","article-title":"A Fast Pick-and-place Prototype Robot: Design and Control","author":"Teodorescu","year":"2016"},{"key":"2020101307451101700_CIT0024","first-page":"1","article-title":"On-line Learning and Planning in a Pick-and-place Task Demonstrated Through Body Manipulation","author":"Rengerv\u00e9","year":"2011"},{"issue":"1","key":"2020101307451101700_CIT0025","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TASE.2016.2600527","article-title":"Analysis and Observations From the First Amazon Picking Challenge","volume":"15","author":"Correll","year":"2016","journal-title":"IEEE Transactions on Automation Science and Engineering"},{"issue":"12","key":"2020101307451101700_CIT0026","first-page":"242","article-title":"Vision Guided Pick and Place Robotic Arm System Based on SIFT","volume":"4","author":"Patil","year":"2013","journal-title":"Int. J. Sci. Eng. Res."},{"key":"2020101307451101700_CIT0027","first-page":"4627","article-title":"Push Planning for Object Placement on Cluttered Table Surfaces","author":"Cosgun","year":"2011"},{"key":"2020101307451101700_CIT0028","first-page":"299","article-title":"Control of Untethered Soft Grippers for Pick-and-place Tasks","author":"Ongaro","year":"2016"},{"key":"2020101307451101700_CIT0029","first-page":"520","article-title":"Overview of an Ontology-Based Approach for Kit Building Applications","author":"Kootbally","year":"2017"},{"article-title":"What Went Wrong?: Identification of Everyday Object Manipulation Anomalies","year":"2020","author":"Altan","key":"2020101307451101700_CIT0030"},{"volume-title":"Research Methodology : Methods & Techniques","year":"2004","author":"Kothari","key":"2020101307451101700_CIT0031"},{"key":"2020101307451101700_CIT0032","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-84882-587-1","volume-title":"DRM, A Design Research Methodology","author":"Blessing","year":"2009"},{"article-title":"Eleven Lessons: Managing Design in Eleven Global Brands. A Study of the Design Process","year":"2007","author":"Council","key":"2020101307451101700_CIT0033"},{"key":"2020101307451101700_CIT0034","first-page":"102","article-title":"Manipulation in Human Environments","author":"Edsinger","year":"2006"},{"key":"2020101307451101700_CIT0035","first-page":"980","article-title":"Peg-and-Hole Task by Robot With Force Sensor: Simulation and Experiment","author":"Yamashita","year":"1991"},{"key":"2020101307451101700_CIT0036","first-page":"1933","article-title":"Classification Learning Method for PCB Insertion Holes Based on Shape Context","author":"Bo","year":"2017"},{"key":"2020101307451101700_CIT0037","first-page":"1735","article-title":"Robotic Assembly System Guided by Multiple Vision and Laser Sensors for Large Scale Components","author":"Wang","year":"2015"},{"key":"2020101307451101700_CIT0038","first-page":"1351","article-title":"A Practical Simulation Method for Pick-and-place with Vacuum Gripper","author":"Karako","year":"2017"},{"issue":"2","key":"2020101307451101700_CIT0039","doi-asserted-by":"crossref","first-page":"021010","DOI":"10.1115\/1.4001093","article-title":"Toward the Design of a Decoupled, Two-Dimensional, Vision-Based \u03bcN Force Sensor","volume":"2","author":"Cappelleri","year":"2010","journal-title":"ASME J. Mech. Rob."},{"key":"2020101307451101700_CIT0040","first-page":"310","article-title":"Robotic Bolt Insertion and Tightening Based on In-Hand Object Localization and Force Sensing","author":"Nozu","year":"2018"},{"key":"2020101307451101700_CIT0041","first-page":"21118","article-title":"Shaft Insertion for Moving Object by Using Robot Manipulator With One Dimensional PSDs Sensor","author":"Hayakawa","year":"2004"},{"issue":"2","key":"2020101307451101700_CIT0042","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1109\/TASE.2018.2847222","article-title":"Integration of Robotic Vision and Tactile Sensing for Wire-Terminal Insertion Tasks","volume":"16","author":"De Gregorio","year":"2019","journal-title":"IEEE Trans. Auto. Sci. Eng."},{"key":"2020101307451101700_CIT0043","first-page":"4821","article-title":"Visual Task Outcome Verification Using Deep Learning","author":"Erkent","year":"2017"},{"article-title":"Faster R-CNN With Region Proposal Refinement","year":"2017","author":"Yuan","key":"2020101307451101700_CIT0044"},{"key":"2020101307451101700_CIT0045","first-page":"1097","volume-title":"Imagenet Classification With Deep Convolutional Neural Networks","author":"Krizhevsky","year":"2012"},{"issue":"2","key":"2020101307451101700_CIT0046","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1109\/LRA.2016.2640364","article-title":"Robotic Pick-and-Place of Multiple Embryos for Vitrification","volume":"2","author":"Zhang","year":"2017","journal-title":"IEEE Rob. Auto. Lett."},{"key":"2020101307451101700_CIT0047","first-page":"93","article-title":"Automated Robotic Microassembly of Flexible Optical Components","author":"Komati","year":"2016"},{"key":"2020101307451101700_CIT0048","first-page":"980","article-title":"Object Placement Planner for Robotic Pick and Place Tasks","author":"Harada","year":"2012"},{"key":"2020101307451101700_CIT0049","first-page":"4504","article-title":"Unsupervised Learning of Multi-hypothesized Pick-and-Place Task Templates Via Crowdsourcing","author":"Toris","year":"2015"},{"issue":"3","key":"2020101307451101700_CIT0050","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-Vector Networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"2020101307451101700_CIT0051","unstructured":"Baumgartl, J. , 2016, \u201cSchnelle, konservative Greif- und Ablageplanung von unbekannten Objekten\u201d. PhD thesis, Universit \u0308ut Bayreuth, Bayreuth, Germany."},{"key":"2020101307451101700_CIT0052","first-page":"201","volume-title":"An Ontology Based Approach to Action Verification for Agile Manufacturing","author":"Balakirsky","year":"2014"},{"key":"2020101307451101700_CIT0053","first-page":"241","volume-title":"A Simulated Sensor-Based Approach for Kit Building Applications","author":"Kootbally","year":"2014"},{"issue":"8","key":"2020101307451101700_CIT0054","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"issue":"3","key":"2020101307451101700_CIT0055","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly Detection: A Survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surv."},{"issue":"1","key":"2020101307451101700_CIT0056","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.sigpro.2013.12.026","article-title":"A Review of Novelty Detection","volume":"99","author":"Pimentel","year":"2014","journal-title":"Signal Process."},{"article-title":"Yolov3: An Incremental Improvement","year":"2018","author":"Redmon","key":"2020101307451101700_CIT0057"},{"key":"2020101307451101700_CIT0058","first-page":"582","volume-title":"Support Vector Method for Novelty Detection","author":"Sch\u00f6lkopf","year":"2000"},{"key":"2020101307451101700_CIT0059","doi-asserted-by":"crossref","DOI":"10.1109\/ICPS48405.2020.9274705","article-title":"Unsupervised Pose Anomaly Detection for Dynamic Robotic Environments","author":"Zoghlami","year":"2020"},{"key":"2020101307451101700_CIT0060","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-59050-9_12","article-title":"Unsupervised Anomaly Detection With Generative Adversarial Networks to Guide Marker Discovery","author":"Schlegl","year":"2017"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4048204\/6577124\/jcise_21_2_021003.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4048204\/6577124\/jcise_21_2_021003.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T19:08:27Z","timestamp":1668193707000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4048204\/1086511\/Design-of-a-Deep-Post-Gripping-Perception"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,13]]},"references-count":60,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4048204","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"type":"print","value":"1530-9827"},{"type":"electronic","value":"1944-7078"}],"subject":[],"published":{"date-parts":[[2020,10,13]]},"article-number":"021003"}}