{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T04:36:03Z","timestamp":1776918963896,"version":"3.51.2"},"reference-count":28,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,10,22]],"date-time":"2019-10-22T00:00:00Z","timestamp":1571702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Robot table tennis systems require a vision system that can track the ball position with low latency and high sampling rate. Altering the ball to simplify the tracking using, for instance, infrared coating changes the physics of the ball trajectory. As a result, table tennis systems use custom tracking systems to track the ball based on heuristic algorithms respecting the real-time constrains applied to RGB images captured with a set of cameras. However, these heuristic algorithms often report erroneous ball positions, and the table tennis policies typically need to incorporate additional heuristics to detect and possibly correct outliers. In this paper, we propose a vision system for object detection and tracking that focuses on reliability while providing real-time performance. Our assumption is that by using multiple cameras, we can find and discard the errors obtained in the object detection phase by checking for consistency with the positions reported by other cameras. We provide an open source implementation of the proposed tracking system to simplify future research in robot table tennis or related tracking applications with strong real-time requirements. We evaluate the proposed system thoroughly in simulation and in the real system, outperforming previous work. Furthermore, we show that the accuracy and robustness of the proposed system increases as more cameras are added. Finally, we evaluate the table tennis playing performance of an existing method in the real robot using the proposed vision system. We measure a slight increase in performance compared to a previous vision system even after removing all the heuristics previously present to filter out erroneous ball observations.<\/jats:p>","DOI":"10.3390\/robotics8040090","type":"journal-article","created":{"date-parts":[[2019,10,23]],"date-time":"2019-10-23T11:46:59Z","timestamp":1571831219000},"page":"90","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Reliable Real-Time Ball Tracking for Robot Table Tennis"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6521-5203","authenticated-orcid":false,"given":"Sebastian","family":"Gomez-Gonzalez","sequence":"first","affiliation":[{"name":"Max Planck for Intelligent Systems, Max Planck Ring 4, 72072 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yassine","family":"Nemmour","sequence":"additional","affiliation":[{"name":"Max Planck for Intelligent Systems, Max Planck Ring 4, 72072 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bernhard","family":"Sch\u00f6lkopf","sequence":"additional","affiliation":[{"name":"Max Planck for Intelligent Systems, Max Planck Ring 4, 72072 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5266-8091","authenticated-orcid":false,"given":"Jan","family":"Peters","sequence":"additional","affiliation":[{"name":"Max Planck for Intelligent Systems, Max Planck Ring 4, 72072 T\u00fcbingen, Germany"},{"name":"Technische Universit\u00e4t Darmstadt, Computer Science Department; Hochschulstr. 10, D-64289 Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0004-3702(01)00129-1","article-title":"Deep blue","volume":"134","author":"Campbell","year":"2002","journal-title":"Artif. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., and Osawa, E. (1997, January 5\u20138). Robocup: The robot world cup initiative. Proceedings of the first International Conference on Autonomous Agents (AGENTS \u201997), Marina del Rey, CA, USA.","DOI":"10.1145\/267658.267738"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1177\/1059712311419378","article-title":"A biomimetic approach to robot table tennis","volume":"19","author":"Kober","year":"2011","journal-title":"Adapt. Behav."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gomez-Gonzalez, S., Neumann, G., Sch\u00f6lkopf, B., and Peters, J. (2016, January 15\u201317). Using probabilistic movement primitives for striking movements. Proceedings of the International Conference of Humanoid Robots, Cancun, Mexico.","DOI":"10.1109\/HUMANOIDS.2016.7803322"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1177\/0278364912472380","article-title":"Learning to select and generalize striking movements in robot table tennis","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_7","unstructured":"Huang, Y., Sch\u00f6lkopf, B., and Peters, J. (October, January 28). Learning optimal striking points for a ping-pong playing robot. Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/s11554-010-0168-3","article-title":"Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components","volume":"7","author":"Lampert","year":"2012","journal-title":"J. Real-Time Image Process."},{"key":"ref_9","unstructured":"Gomez-Gonzalez, S., Neumann, G., Sch\u00f6lkopf, B., and Peters, J. (2018). Adaptation and Robust Learning of Probabilistic Movement Primitives. arXiv."},{"key":"ref_10","unstructured":"(2019, October 20). Open Source Implementation of the Ball Tracking System. Available online: https:\/\/gitlab.tuebingen.mpg.de\/sgomez\/ball_tracking."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Seo, Y., Choi, S., Kim, H., and Hong, K.S. (1997). Where are the ball and players? Soccer game analysis with color-based tracking and image mosaick. International Conference on Image Analysis and Processing, Springer.","DOI":"10.1007\/3-540-63508-4_123"},{"key":"ref_12","unstructured":"Tong, X.F., Lu, H.Q., and Liu, Q.S. (2004, January 26). An effective and fast soccer ball detection and tracking method. Proceedings of the International Conference on Pattern Recognition, Cambridge, UK."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/j.jvcir.2008.11.008","article-title":"Physics-based ball tracking and 3D trajectory reconstruction with applications to shooting location estimation in basketball video","volume":"20","author":"Chen","year":"2009","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_14","unstructured":"Pingali, G., Opalach, A., and Jean, Y. (2000, January 3\u20137). Ball tracking and virtual replays for innovative tennis broadcasts. Proceedings of the International Conference on Pattern Recognition, Barcelona, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, H., Li, Z., Wang, B., Zhou, Y., and Zhang, Q. (2013, January 12\u201314). Table tennis robot with stereo vision and humanoid manipulator II: Visual measurement of motion-blurred ball. Proceedings of the International Conference on Robotics and Biomimetics, Shenzhen, China.","DOI":"10.1109\/ROBIO.2013.6739835"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, H., Wu, H., Lou, L., K\u00fchnlenz, K., and Ravn, O. (2012, January 5\u20137). Ping-pong robotics with high-speed vision system. Proceedings of the International Conference on Control Automation Robotics & Vision, Guangzhou, China.","DOI":"10.1109\/ICARCV.2012.6485142"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"35","DOI":"10.5772\/60406","article-title":"A robust vision module for humanoid robotic ping-pong game","volume":"12","author":"Chen","year":"2015","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., and Guadarrama, S. (2017, January 21\u201326). Speed\/accuracy trade-offs for modern convolutional object detectors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.351"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single shot multibox detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_20","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pohlen, T., Hermans, A., Mathias, M., and Leibe, B. (2017). Full-resolution residual networks for semantic segmentation in street scenes. arXiv.","DOI":"10.1109\/CVPR.2017.353"},{"key":"ref_22","unstructured":"Heyden, A., and Pollefeys, M. (2005). Multiple view geometry. Emerging topics in computer vision, Prentice Hall PTR."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1023\/B:AIRE.0000045502.10941.a9","article-title":"A survey of outlier detection methodologies","volume":"22","author":"Hodge","year":"2004","journal-title":"Artif. Intell. Rev."},{"key":"ref_24","unstructured":"Chum, O. (2005). Two-View Geometry Estimation by Random Sample and Consensus. [Ph.D. Thesis, Czech Technical University]."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tordoff, B., and Murray, D.W. (2002). Guided sampling and consensus for motion estimation. European Conference on Computer Vision, Springer.","DOI":"10.1007\/3-540-47969-4_6"},{"key":"ref_27","unstructured":"Nasuto, D., and Craddock, J.B.R. (, January September). Napsac: High noise, high dimensional robust estimation-it\u2019s in the bag. Proceedings of the British Machine Vision Conference, University of Cardiff."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2022","DOI":"10.1109\/TPAMI.2012.257","article-title":"USAC: A universal framework for random sample consensus","volume":"35","author":"Raguram","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/8\/4\/90\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:28:29Z","timestamp":1760189309000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/8\/4\/90"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,22]]},"references-count":28,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["robotics8040090"],"URL":"https:\/\/doi.org\/10.3390\/robotics8040090","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,22]]}}}