{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T15:13:12Z","timestamp":1784905992315,"version":"3.55.0"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,13]],"date-time":"2021-03-13T00:00:00Z","timestamp":1615593600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Economic Affairs of the state Baden-W\u00fcrttemberg","award":["036-170017"],"award-info":[{"award-number":["036-170017"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Manual inspection of workpieces in highly flexible production facilities with small lot sizes is costly and less reliable compared to automated inspection systems. Reinforcement Learning (RL) offers promising, intelligent solutions for robotic inspection and manufacturing tasks. This paper presents an RL-based approach to determine a high-quality set of sensor view poses for arbitrary workpieces based on their 3D computer-aided design (CAD). The framework extends available open-source libraries and provides an interface to the Robot Operating System (ROS) for deploying any supported robot and sensor. The integration into commonly used OpenAI Gym and Baselines leads to an expandable and comparable benchmark for RL algorithms. We give a comprehensive overview of related work in the field of view planning and RL. A comparison of different RL algorithms provides a proof of concept for the framework\u2019s functionality in experimental scenarios. The obtained results exhibit a coverage ratio of up to 0.8 illustrating its potential impact and expandability. The project will be made publicly available along with this article.<\/jats:p>","DOI":"10.3390\/s21062030","type":"journal-article","created":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T23:52:06Z","timestamp":1615765926000},"page":"2030","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["A Reinforcement Learning Approach to View Planning for Automated Inspection Tasks"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9398-8788","authenticated-orcid":false,"given":"Christian","family":"Landgraf","sequence":"first","affiliation":[{"name":"Fraunhofer Institute for Manufacturing, Engineering and Automation IPA, Nobelstra\u00dfe 12, 70569 Stuttgart, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1413-8321","authenticated-orcid":false,"given":"Bernd","family":"Meese","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Manufacturing, Engineering and Automation IPA, Nobelstra\u00dfe 12, 70569 Stuttgart, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Pabst","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Manufacturing, Engineering and Automation IPA, Nobelstra\u00dfe 12, 70569 Stuttgart, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8963-7627","authenticated-orcid":false,"given":"Georg","family":"Martius","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Intelligent Systems, Max-Planck-Ring 4, 72076 T\u00fcbingen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8250-2092","authenticated-orcid":false,"given":"Marco F.","family":"Huber","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Manufacturing, Engineering and Automation IPA, Nobelstra\u00dfe 12, 70569 Stuttgart, Germany"},{"name":"Institute of Industrial Manufacturing and Management IFF, University of Stuttgart, Allmandring 35, 70569 Stuttgart, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Siciliano, B., and Khatib, O. (2016). Industrial Robotics. Springer Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-319-32552-1"},{"key":"ref_2","unstructured":"(2021, March 12). International Federation of Robotics. Available online: https:\/\/ifr.org\/free-downloads."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1007\/s00138-007-0110-2","article-title":"Model-based view planning","volume":"20","author":"Scott","year":"2009","journal-title":"Mach. Vis. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Engin, S., Mitchell, E., Lee, D., Isler, V., and Lee, D.D. (August, January 31). Higher Order Function Networks for View Planning and Multi-View Reconstruction. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197435"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1145\/641865.641868","article-title":"View planning for automated three-dimensional object reconstruction and inspection","volume":"35","author":"Scott","year":"2003","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1343","DOI":"10.1177\/0278364911410755","article-title":"Active vision in robotic systems: A survey of recent developments","volume":"30","author":"Chen","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1145\/285055.285059","article-title":"A threshold of ln n for approximating set cover","volume":"45","author":"Feige","year":"1998","journal-title":"J. ACM"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1006\/cviu.1995.1007","article-title":"Planning for Complete Sensor Coverage in Inspection","volume":"61","author":"Tarbox","year":"1995","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Martin, R., Rojas, I., Franke, K., and Hedengren, J. (2016). Evolutionary View Planning for Optimized UAV Terrain Modeling in a Simulated Environment. Remote Sens., 8.","DOI":"10.3390\/rs8010026"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1007\/978-3-319-29363-9_19","article-title":"Planning Complex Inspection Tasks Using Redundant Roadmaps","volume":"Volume 100","author":"Christensen","year":"2017","journal-title":"Robotics Research"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kaba, M.D., Uzunbas, M.G., and Lim, S.N. (2017, January 21\u201326). A Reinforcement Learning Approach to the View Planning Problem. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.541"},{"key":"ref_12","unstructured":"Sutton, R.S., and Barto, A. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.]. Adaptive Computation and Machine Learning."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"54854","DOI":"10.1109\/ACCESS.2018.2872693","article-title":"A Computational Framework for Automatic Online Path Generation of Robotic Inspection Tasks via Coverage Planning and Reinforcement Learning","volume":"6","author":"Jing","year":"2018","journal-title":"IEEE Access"},{"key":"ref_14","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double Q-Learning. Proceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI 2016, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_16","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized Experience Replay. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sutton, R.S. (1992). Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Reinforcement Learning, Springer.","DOI":"10.1007\/978-1-4615-3618-5"},{"key":"ref_18","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous methods for deep reinforcement learning. Proceedings of the 33rd International Conference on Machine Learning, ICML 2016, New York, NY, USA."},{"key":"ref_19","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lucchi, M., Zindler, F., M\u00fchlbacher-Karrer, S., and Pichler, H. (2020). robo-gym\u2014An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robots. arXiv.","DOI":"10.1109\/IROS45743.2020.9340956"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1109\/MRA.2012.2205651","article-title":"The Open Motion Planning Library","volume":"19","author":"Sucan","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Koch, S., Matveev, A., Jiang, Z., Williams, F., Artemov, A., Burnaev, E., Alexa, M., Zorin, D., and Panozzo, D. (2019, January 15\u201319). ABC: A Big CAD Model Dataset For Geometric Deep Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00983"},{"key":"ref_23","unstructured":"Zamora, I., Lopez, N.G., Vilches, V.M., and Cordero, A.H. (2017). Extending the OpenAI Gym for robotics: A toolkit for reinforcement learning using ROS and Gazebo. arXiv."},{"key":"ref_24","unstructured":"Koenig, N., and Howard, A. (, January September). Design and use paradigms for gazebo, an open-source multi-robot simulator. Proceedings of the 2004 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"456","DOI":"10.21105\/joss.00456","article-title":"ros_control: A generic and simple control framework for ROS","volume":"2","author":"Chitta","year":"2017","journal-title":"J. Open Source Softw."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., and Cousins, S. (2011, January 9\u201313). 3D is here: Point Cloud Library (PCL). Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980567"},{"key":"ref_27","unstructured":"Zhou, Q.Y., Park, J., and Koltun, V. (2018). Open3D: A Modern Library for 3D Data Processing. arXiv."},{"key":"ref_28","unstructured":"Coleman, D., Sucan, I., Chitta, S., and Correll, N. (2014). Reducing the Barrier to Entry of Complex Robotic Software: A MoveIt! Case Study. arXiv."},{"key":"ref_29","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv."},{"key":"ref_30","unstructured":"Raffin, A., Hill, A., Ernestus, M., Gleave, A., Kanervisto, A., and Dormann, N. (2021, March 12). Stable Baselines3. GitHub. Available online: https:\/\/github.com\/DLR-RM\/stable-baselines3."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wong, C., Mineo, C., Yang, E., Yan, X.T., and Gu, D. (2020). A novel clustering-based algorithm for solving spatially-constrained robotic task sequencing problems. IEEE\/ASME Trans. Mechatron., 1.","DOI":"10.1109\/TMECH.2020.3037158"},{"key":"ref_33","unstructured":"Gumhold, S., Wang, X., and Macleod, R. (2001, January 7\u201310). Feature Extraction from Point Clouds. Proceedings of the 10th International Meshing Roundtable, Newport Beach, CA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., and Bennamoun, M. (2020). Deep Learning for 3D Point Clouds: A Survey. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2020.3005434"},{"key":"ref_35","unstructured":"Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. (2016). Adversarial Autoencoders. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"102921","DOI":"10.1016\/j.cviu.2020.102921","article-title":"Adversarial autoencoders for compact representations of 3D point clouds","volume":"193","author":"Zamorski","year":"2020","journal-title":"Comput. Vis. Image Underst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2030\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:35:05Z","timestamp":1760160905000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2030"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,13]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062030"],"URL":"https:\/\/doi.org\/10.3390\/s21062030","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,13]]}}}