{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T00:34:40Z","timestamp":1772152480597,"version":"3.50.1"},"reference-count":18,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,4,5]],"date-time":"2022-04-05T00:00:00Z","timestamp":1649116800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["2020.06923.BD"],"award-info":[{"award-number":["2020.06923.BD"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004895","name":"European Social Fund","doi-asserted-by":"publisher","award":["NORTE-06-3559-FSE-000116"],"award-info":[{"award-number":["NORTE-06-3559-FSE-000116"]}],"id":[{"id":"10.13039\/501100004895","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Bin picking is a challenging problem that involves using a robotic manipulator to remove, one-by-one, a set of objects randomly stacked in a container. In order to provide ground truth data for evaluating heuristic or machine learning perception systems, this paper proposes using simulation to create bin picking environments in which a procedural generation method builds entangled tubes that can have curvatures throughout their length. The output of the simulation is an annotated point cloud, generated by a virtual 3D depth camera, in which the tubes are assigned with unique colors. A general metric based on micro-recall is proposed to compare the accuracy of point cloud annotations with the ground truth. The synthetic data is representative of a high quality 3D scanner, given that the performance of a tube modeling system when given 640 simulated point clouds was similar to the results achieved with real sensor data. Therefore, simulation is a promising technique for the automated evaluation of solutions for bin picking tasks.<\/jats:p>","DOI":"10.3390\/robotics11020046","type":"journal-article","created":{"date-parts":[[2022,4,5]],"date-time":"2022-04-05T06:01:34Z","timestamp":1649138494000},"page":"46","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Using Simulation to Evaluate a Tube Perception Algorithm for Bin Picking"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9081-9268","authenticated-orcid":false,"given":"Gon\u00e7alo","family":"Le\u00e3o","sequence":"first","affiliation":[{"name":"Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal"},{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8453-4031","authenticated-orcid":false,"given":"Carlos M.","family":"Costa","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal"},{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0317-4714","authenticated-orcid":false,"given":"Armando","family":"Sousa","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal"},{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4709-1718","authenticated-orcid":false,"given":"Lu\u00eds Paulo","family":"Reis","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal"},{"name":"Artificial Intelligence and Computer Science Laboratory (LIACC), University of Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3044-6938","authenticated-orcid":false,"given":"Germano","family":"Veiga","sequence":"additional","affiliation":[{"name":"Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal"},{"name":"Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Le\u00e3o, G., Costa, C.M., Sousa, A., and Veiga, G. (2019, January 20\u201322). Perception of Entangled Tubes for Automated Bin Picking. Proceedings of the Iberian Robotics Conference (ROBOT), Porto, Portugal.","DOI":"10.1007\/978-3-030-35990-4_50"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yue, X., Wu, B., Seshia, S.A., Keutzer, K., and Sangiovanni-Vincentelli, A.L. (2018, January 11\u201314). A LiDAR Point Cloud Generator: From a Virtual World to Autonomous Driving. Proceedings of the International Conference on Multimedia Retrieval (ICMR), Yokohama, Japan.","DOI":"10.1145\/3206025.3206080"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A.M. (2016, January 27\u201330). The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.352"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Niemeyer, M., Mescheder, L., Oechsle, M., and Geiger, A. (2020, January 16\u201318). Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00356"},{"key":"ref_5","unstructured":"Yang, G., Huang, X., Hao, Z., Liu, M.Y., Belongie, S., and Hariharan, B. (November, January 27). Pointflow: 3D point cloud generation with continuous normalizing flows. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Richter, S.R., Vineet, V., Roth, S., and Koltun, V. (2016, January 11\u201314). Playing for Data: Ground Truth from Computer Games. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_7"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2671","DOI":"10.1109\/TIM.2019.2906416","article-title":"Automatic Generation of Synthetic LiDAR Point Clouds for 3-D Data Analysis","volume":"68","author":"Wang","year":"2019","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Schyja, A., and Kuhlenk\u00f6tter, B. (2015, January 17\u201319). Realistic simulation of Industrial Bin-Picking Systems. Proceedings of the International Conference on Automation, Robotics and Applications (ICARA), Queenstown, New Zealand.","DOI":"10.1109\/ICARA.2015.7081137"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kleeberger, K., Landgraf, C., and Huber, M.F. (2019, January 3\u20138). Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8967594"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Matsumura, R., Harada, K., Domae, Y., and Wan, W. (2018, January 11\u201315). Learning based industrial bin-picking trained with approximate physics simulator. Proceedings of the International Conference on Intelligent Autonomous Systems (IAS), Baden-Baden, Germany.","DOI":"10.1007\/978-3-030-01370-7_61"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"102032","DOI":"10.1016\/j.rcim.2020.102032","article-title":"Reconfigurable Grasp Planning Pipeline with Grasp Synthesis and Selection Applied to Picking Operations in Aerospace Factories","volume":"67","author":"Costa","year":"2021","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1109\/MRA.2004.1371616","article-title":"Graspit: A versatile simulator for robotic grasping","volume":"11","author":"Miller","year":"2004","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Le\u00e3o, G., Costa, C.M., Sousa, A., and Veiga, G. (2020). Detecting and Solving Tube Entanglement in Bin Picking Operations. Appl. Sci., 10.","DOI":"10.3390\/app10072264"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1108\/IR-06-2016-0166","article-title":"Benchmarking motion planning algorithms for bin-picking applications","volume":"44","author":"Iversen","year":"2017","journal-title":"Ind. Robot."},{"key":"ref_15","unstructured":"Koenig, N., and Howard, A. (October, January 28). Design and use paradigms for gazebo, an open-source multi-robot simulator. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., and Cousins, S. (2011, January 9\u201313). 3D is here: Point Cloud Library (PCL). Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980567"},{"key":"ref_17","unstructured":"Ramshaw, L., and Tarjan, R.E. (2012). On Minimum-Cost Assignments in Unbalanced Bipartite Graphs, HP Laboratories. Technical Report."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inf. Process. Manag."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/11\/2\/46\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:49:51Z","timestamp":1760136591000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/11\/2\/46"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,5]]},"references-count":18,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["robotics11020046"],"URL":"https:\/\/doi.org\/10.3390\/robotics11020046","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,5]]}}}