{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T20:19:06Z","timestamp":1777407546430,"version":"3.51.4"},"reference-count":70,"publisher":"ASME International","issue":"6","license":[{"start":{"date-parts":[[2020,6,9]],"date-time":"2020-06-09T00:00:00Z","timestamp":1591660800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.asme.org\/publications-submissions\/publishing-information\/legal-policies"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The increasing individualization of products reinforces the importance of decoupled factories in production processes. Artificial intelligence (AI) is a recognized technology for problem solving and accelerates automation by enabling systems to act independently. In the field of robotics, there are new deep learning approaches which make robotic control systems human independent. This work provides a literature overview of the current state of development methodologies, showing that there are only limited methods available for the development of artificial intelligent robots. We present a novel development methodology based on artificial intelligence, particularly deep reinforcement learning. The so-called Q-model can enable robots to learn specific tasks independently. In summary, we show how an AI-based methodology assists the development of autonomous robots along the product lifecycle.<\/jats:p>","DOI":"10.1115\/1.4046992","type":"journal-article","created":{"date-parts":[[2020,4,21]],"date-time":"2020-04-21T15:48:37Z","timestamp":1587484117000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":11,"title":["Q-Model: An Artificial Intelligence Based Methodology for the Development of Autonomous Robots"],"prefix":"10.1115","volume":"20","author":[{"given":"Philip","family":"Kurrek","sequence":"first","affiliation":[{"name":"Department of Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich 80335, Germany"}]},{"given":"Firas","family":"Zoghlami","sequence":"additional","affiliation":[{"name":"Department of Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich 80335, Germany"}]},{"given":"Mark","family":"Jocas","sequence":"additional","affiliation":[{"name":"Department of Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich 80335, Germany"}]},{"given":"Martin","family":"Stoelen","sequence":"additional","affiliation":[{"name":"School of Computing, Electronics, and Mathematics, University of Plymouth, Plymouth PL4 8AA, UK"}]},{"given":"Vahid","family":"Salehi","sequence":"additional","affiliation":[{"name":"Department of Applied Sciences and Mechatronics, Munich University of Applied Sciences, Munich 80335, Germany"}]}],"member":"33","published-online":{"date-parts":[[2020,6,9]]},"reference":[{"key":"2020071611035425100_CIT0001","first-page":"24","article-title":"Structured Megatrends Research As Foundation for Future-Oriented Research-Planning and R&D Roadmapping in Robotics","author":"Boesl","year":"2017"},{"key":"2020071611035425100_CIT0002","article-title":"Industrie 4.0 in Produktion, Automatisierung und logistik. Anwendung \u00b7 Technologien \u00b7 migration","author":"Bauernhansl","year":"2014"},{"key":"2020071611035425100_CIT0003","article-title":"Artificial Intelligence and Life in 2030","author":"Stone","year":"2016"},{"key":"2020071611035425100_CIT0004","unstructured":"Van der Meer, J., Hassan, S., Lofink, O., Jones, R., and Hagedorn, R., 2018, \u201cAi and Robotics Automation in Consumer-Driven Supply Chains\u201d. The Consumer Goods Forum \u2013 AI-Robotics-Report, https:\/\/www.theconsumergoodsforum.com\/wp-content\/uploads\/2018\/04\/201805-CGF-AI-Robotics-Report-with-PA-Consulting.pdf, Accessed June 28, 2019."},{"issue":"75","key":"2020071611035425100_CIT0005","first-page":"1","article-title":"A Review of Future and Ethical Perspectives of Robotics and AI","volume":"4","author":"Torresen","year":"2018","journal-title":"Front. Robotics AI"},{"issue":"1","key":"2020071611035425100_CIT0006","first-page":"1","article-title":"Artificial Intelligence, Robotics and \u2018Autonomous\u2019 Systems","volume":"30","author":"Beer","year":"2018","journal-title":"Eur. Group Ethics Sci. New Technol."},{"key":"2020071611035425100_CIT0007","doi-asserted-by":"crossref","unstructured":"Perez, J. A., Deligianni, F., Rav\u00ec, D., and Yang, G.,2018, Artificial Intelligence and Robotics, UK Artificial Intelligence and Robotics Network, Report No. 1, London.","DOI":"10.31256\/WP2017.1"},{"issue":"2","key":"2020071611035425100_CIT0008","doi-asserted-by":"crossref","first-page":"74","DOI":"10.5898\/JHRI.3.2.Beer","article-title":"Toward a Framework for Levels of Robot Autonomy in Human-Robot Interaction","volume":"3","author":"Beer","year":"2014","journal-title":"J. Hum.-Robot Interact."},{"key":"2020071611035425100_CIT0009","article-title":"Development of a Non-Contextual Model for Determining the Autonomy Level of Intelligent Unmanned Systems","author":"Durst","year":"2013"},{"issue":"1","key":"2020071611035425100_CIT0010","doi-asserted-by":"crossref","first-page":"4","DOI":"10.3390\/robotics8010004","article-title":"Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges","volume":"8","author":"Bhagat","year":"2019","journal-title":"Robotics"},{"key":"2020071611035425100_CIT0011","first-page":"1","article-title":"End-to-end Training of Deep Visuomotor Policies","volume":"17","author":"Levine","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"2020071611035425100_CIT0012","doi-asserted-by":"crossref","DOI":"10.1017\/dsi.2019.363","article-title":"AI Motion Control \u2014 A Generic Approach to Develop Control Policies for Robotic Manipulation Tasks","author":"Kurrek","year":"2019"},{"key":"2020071611035425100_CIT0013","author":"Blessing","year":"2009"},{"key":"2020071611035425100_CIT0014","first-page":"7","volume-title":"Systems Engineering Guidebook for Intelligent Transportation Systems","author":"Woolley","year":"2009","edition":"3"},{"key":"2020071611035425100_CIT0015","article-title":"Vdi 2206: Entwicklungsmethodik fuer mechatronische systeme","author":"VDI2206","year":"2004"},{"key":"2020071611035425100_CIT0016","article-title":"Methods and Tools for Rapid Application Development","author":"Kralev","year":"2017"},{"key":"2020071611035425100_CIT0017","first-page":"3","volume-title":"SEI series in Software Engineering","author":"Bass","year":"2015"},{"key":"2020071611035425100_CIT0018","unstructured":"Saravia, S., Slen, E., and Pendse, G., 2018, \u201cArtificial Intelligence and Robotics: Industry Report and Investment Case\u201d. NASDAQ Global Information Services, March. https:\/\/indexes.nasdaqomx.com\/docs\/NQROBO%20Research.pdf"},{"key":"2020071611035425100_CIT0019","author":"Coombs","year":"2017"},{"key":"2020071611035425100_CIT0020","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2012.6385560","article-title":"Improving Robot Manipulation Through Fingertip Perception","author":"Maldonado","year":"2012"},{"issue":"4\u20135","key":"2020071611035425100_CIT0021","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1177\/0278364914549607","article-title":"Deep Learning for Detecting Robotic Grasps","volume":"34","author":"Lenz","year":"2015","journal-title":"Int. J. Robotics Res."},{"key":"2020071611035425100_CIT0022","doi-asserted-by":"crossref","DOI":"10.1017\/dsi.2019.383","article-title":"Usage Identification of Anomaly Detection in An Industrial Context","author":"Zoghlami","year":"2019"},{"issue":"4","key":"2020071611035425100_CIT0023","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s10339-011-0404-1","article-title":"Model Learning for Robot Control: a Survey","volume":"12","author":"Nguyen-Tuong","year":"2011","journal-title":"Cogn. Process."},{"key":"2020071611035425100_CIT0024","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA.2014.6906922","article-title":"Combined Task and Motion Planning Through An Extensible Planner-Independent Interface Layer","author":"Srivastava","year":"2014"},{"issue":"3","key":"2020071611035425100_CIT0025","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1147\/rd.33.0210","article-title":"Some Studies in Machine Learning Using the Game of Checkers","volume":"3","author":"Samuel","year":"1959","journal-title":"IBM. J. Res. Dev."},{"key":"2020071611035425100_CIT0026","article-title":"Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations","author":"Laskey","year":"2016"},{"key":"2020071611035425100_CIT0027","article-title":"Multi-Modal Scene Understanding for Robotic Grasping","author":"Bohg","year":"2011"},{"key":"2020071611035425100_CIT0028","doi-asserted-by":"crossref","DOI":"10.1109\/HUMANOIDS.2015.7363434","article-title":"A Visuo-Tactile Control Framework for Manipulation and Exploration of Unknown Objects","author":"Li","year":"2015"},{"key":"2020071611035425100_CIT0029","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.robot.2016.08.022","article-title":"Visual Motor Integration of Robot\u2019s Drawing Behavior Using Recurrent Neural Network","volume":"86","author":"Sasaki","year":"2016","journal-title":"Rob. Auton. Syst."},{"key":"2020071611035425100_CIT0030","doi-asserted-by":"crossref","DOI":"10.1109\/HUMANOIDS.2015.7363593","article-title":"Multi-Model Approach Based on 3D Functional Features for Tool Affordance Learning in Robotics","author":"Mar","year":"2015"},{"key":"2020071611035425100_CIT0031","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8206046","article-title":"Deep Predictive Policy Training Using Reinforcement Learning","author":"Ghadirzadeh","year":"2017"},{"issue":"1","key":"2020071611035425100_CIT0032","first-page":"237","article-title":"Reinforcement Learning: A Survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Int. Res."},{"key":"2020071611035425100_CIT0033","volume-title":"Introduction to Reinforcement Learning","author":"Sutton","year":"1998","edition":"1st ed"},{"key":"2020071611035425100_CIT0034","article-title":"Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping","author":"Bousmalis","year":"2017"},{"key":"2020071611035425100_CIT0035","article-title":"Sim-to-Real Robot Learning from Pixels with Progressive Nets","author":"Rusu","year":"2016"},{"key":"2020071611035425100_CIT0036","doi-asserted-by":"crossref","DOI":"10.1017\/dsi.2019.210","article-title":"AI-Based Learning Approach With Consideration of Safety Criteria on Example of a Depalletization Robot","author":"Jocas","year":"2019"},{"key":"2020071611035425100_CIT0037","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2018-86013","article-title":"Design of Transfer Reinforcement Learning Under Low Task Similarity","author":"Liu","year":"2018"},{"key":"2020071611035425100_CIT0038","unstructured":"Sommerville, I.\n          , 2019, \u201cArtificial Intelligence and Systems Engineering.\u201d https:\/\/pdfs.semanticscholar.org\/63a6\/8192ed57c629192d4e7058cf7fa0b08ef89c.pdf, Accessed June 28, 2019."},{"issue":"4","key":"2020071611035425100_CIT0039","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1109\/MRA.2015.2460911","article-title":"Distributed and Adaptive Shared Control Systems: Methodology for the Replication of Experiments","volume":"22","author":"Stoelen","year":"2015","journal-title":"IEEE Rob. Autom. Mag."},{"issue":"2","key":"2020071611035425100_CIT0040","doi-asserted-by":"crossref","first-page":"122","DOI":"10.5772\/55055","article-title":"A Service-Oriented Framework for the Development of Home Robots","volume":"10","author":"Yang","year":"2013","journal-title":"Int. J. Adv. Robotic Systems"},{"key":"2020071611035425100_CIT0041","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2010.5650146","article-title":"Cram-A Cognitive Robot Abstract Machine for Everyday Manipulation in Human Environments","author":"Beetz","year":"2010"},{"key":"2020071611035425100_CIT0042","doi-asserted-by":"crossref","DOI":"10.1109\/SMC.2015.173","article-title":"A Behavior Generation Framework for Robots to Learn From Demonstrations","author":"Tan","year":"2015"},{"key":"2020071611035425100_CIT0043","article-title":"The Teaching-Box: A Universal Robot Learning Framework","author":"Ertel","year":"2009"},{"key":"2020071611035425100_CIT0044","article-title":"Open-Ease\u2014A Knowledge Processing Service for Robots and Robotics\/ai Researchers","author":"Beetz","year":"2015"},{"key":"2020071611035425100_CIT0045","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA.2015.7140065","article-title":"A Framework for End-User Instruction of a Robot Assistant for Manufacturing","author":"Guerin","year":"2015"},{"key":"2020071611035425100_CIT0046","doi-asserted-by":"crossref","DOI":"10.1145\/544741.544798","article-title":"A Hierarchical Architecture for Behavior-Based Robots","author":"Nicolescu","year":"2002"},{"key":"2020071611035425100_CIT0047","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.robot.2017.01.002","article-title":"Tool-Body Assimilation Model Considering Grasping Motion Through Deep Learning","volume":"91","author":"Takahashi","year":"2017","journal-title":"Rob. Auton. Syst."},{"issue":"1","key":"2020071611035425100_CIT0048","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1109\/LRA.2016.2535907","article-title":"A Hierarchical Human-Robot Interaction-Planning Framework for Task Allocation in Collaborative Industrial Assembly Processes","volume":"2","author":"Johannsmeier","year":"2017","journal-title":"IEEE Robotics Automation Lett."},{"key":"2020071611035425100_CIT0049","doi-asserted-by":"crossref","DOI":"10.1109\/ROBIO.2016.7866589","article-title":"An Integrated Probabilistic Framework for Robot Perception, Learning and Memory","author":"Martinez-Hernandez","year":"2016"},{"key":"2020071611035425100_CIT0050","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8206193","article-title":"On-line Simultaneous Learning and Recognition of Everyday Activities From Virtual Reality Performances","author":"Bates","year":"2017"},{"issue":"4","key":"2020071611035425100_CIT0051","doi-asserted-by":"crossref","first-page":"1732","DOI":"10.1109\/TII.2017.2788079","article-title":"Supervisory Control-based Navigation Architecture: A New Framework for Autonomous Robots in Industry 4.0 Environments","volume":"14","author":"Gonzalez","year":"2018","journal-title":"IEEE Trans. Ind. Informatics"},{"key":"2020071611035425100_CIT0052","doi-asserted-by":"crossref","DOI":"10.1109\/SYSOSE.2017.7994967","article-title":"A Framework for An Adaptive Human-Robot Collaboration Approach Through Perception-Based Real-Time Adjustments of Robot Behavior in Industry","author":"Kumar","year":"2017"},{"issue":"1","key":"2020071611035425100_CIT0053","first-page":"11","article-title":"Design of Complex Engineered Systems Using Multi-Agent Coordination","volume":"18","author":"Soria Zurita","year":"2017","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"issue":"1","key":"2020071611035425100_CIT0054","first-page":"11","article-title":"An Industry Case Study: Investigating Early Design Decision Making in Virtual Reality","volume":"17","author":"Berg","year":"2016","journal-title":"ASME J. Comput. Inf. Sci. Eng."},{"key":"2020071611035425100_CIT0055","doi-asserted-by":"crossref","DOI":"10.1017\/dsi.2019.377","article-title":"Munich Agile MBSE Concept (MAGIC)","author":"Salehi","year":"2019"},{"issue":"1","key":"2020071611035425100_CIT0056","doi-asserted-by":"crossref","first-page":"147","DOI":"10.14733\/cadaps.2020.147-166","article-title":"Development of An Agile Concept for Mbse for Future Digital Products Through the Entire Life Cycle Management Called Munich Agle Mbse Concept (magic)","volume":"17","author":"Salehi","year":"2019","journal-title":"Computer-Aided Design Appl."},{"issue":"2017","key":"2020071611035425100_CIT0057","first-page":"2","article-title":"Model-Based Development and Artificial Intelligence","volume":"6","author":"Adachi","year":"2018","journal-title":"Keihin Technical Review"},{"key":"2020071611035425100_CIT0058","article-title":"Rapid Control Prototyping komplexer und flexibler robotersteuerungen auf basis des sbc-ansatzes","author":"Maletzki","year":"2013"},{"issue":"1","key":"2020071611035425100_CIT0059","first-page":"2309","article-title":"Use of Artificial Intelligence in Software Development Life Cycle: A State of the Art Review","volume":"3","author":"Sorte","year":"2015","journal-title":"Int. J. Technol. Manage."},{"issue":"8","key":"2020071611035425100_CIT0060","first-page":"398","article-title":"Use of Artificial Intelligence in Software Development Life Cycle: Requirements and Its Model","volume":"5","author":"Kumari","year":"2018","journal-title":"Int. Res. J. Eng. Technol. (IRJET)"},{"key":"2020071611035425100_CIT0061","doi-asserted-by":"crossref","DOI":"10.1109\/ICDE.2017.192","article-title":"Modelhub: Deep Learning Lifecycle Management","author":"Miao","year":"2017"},{"key":"2020071611035425100_CIT0062","doi-asserted-by":"crossref","DOI":"10.1109\/ICDE.2017.112","article-title":"Towards Unified Data and Lifecycle Management for Deep Learning","author":"Miao","year":"2017"},{"key":"2020071611035425100_CIT0063","doi-asserted-by":"crossref","DOI":"10.11128\/sne.25.tn.10293","article-title":"Matlab\/Simulink based Rapid Control Prototyping for Multivendor Robot Applications","author":"Deatcu","year":"2015"},{"key":"2020071611035425100_CIT0064","article-title":"Learning Deep Structured Models","author":"Chen","year":"2014"},{"issue":"1","key":"2020071611035425100_CIT0065","doi-asserted-by":"crossref","first-page":"011004","DOI":"10.1115\/1.4038684","article-title":"Grasping Force Optimization Approaches for Anthropomorphic Hands","volume":"10","author":"Cloutier","year":"2018","journal-title":"ASME J. Mech. Rob."},{"key":"2020071611035425100_CIT0066","author":"Juliani","year":"2018"},{"key":"2020071611035425100_CIT0067","article-title":"ROS: An Open-Source Robot Operating System","author":"Quigley","year":"2009"},{"key":"2020071611035425100_CIT0068","article-title":"Openai Baselines","author":"Dhariwal","year":"2017"},{"issue":"2017","key":"2020071611035425100_CIT0069","first-page":"1","article-title":"Proximal Policy Optimization Algorithms","volume":"abs\/1707.06347","author":"Schulman","year":"2017","journal-title":"ArXiv"},{"key":"2020071611035425100_CIT0070","article-title":"Sim-to-Real Transfer of Robotic Control With Dynamics Randomization","author":"Peng","year":"2017"}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4046992\/6541900\/jcise_20_6_061006.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4046992\/6541900\/jcise_20_6_061006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,7,16]],"date-time":"2020-07-16T17:35:27Z","timestamp":1594920927000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/doi\/10.1115\/1.4046992\/1082857\/QModel-An-Artificial-Intelligence-Based"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,9]]},"references-count":70,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,12,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4046992","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,9]]},"article-number":"061006"}}