{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T05:08:09Z","timestamp":1770527289096,"version":"3.49.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2009,11]]},"DOI":"10.1007\/s10514-009-9141-z","type":"journal-article","created":{"date-parts":[[2009,8,28]],"date-time":"2009-08-28T09:22:26Z","timestamp":1251451346000},"page":"449-464","source":"Crossref","is-referenced-by-count":46,"title":["Evaluating maps produced by urban search and rescue robots: lessons learned from RoboCup"],"prefix":"10.1007","volume":"27","author":[{"given":"Benjamin","family":"Balaguer","sequence":"first","affiliation":[]},{"given":"Stephen","family":"Balakirsky","sequence":"additional","affiliation":[]},{"given":"Stefano","family":"Carpin","sequence":"additional","affiliation":[]},{"given":"Arnoud","family":"Visser","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2009,8,28]]},"reference":[{"issue":"1\/2","key":"9141_CR1","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1002\/rob.20180","volume":"24","author":"S. M. Abdallah","year":"2007","unstructured":"Abdallah, S. M., Smar, D. C., & Zelek, J. S. (2007). A benchmark for outdoor vision SLAM systems. Journal of Field Robotics, 24(1\/2), 145\u2013165.","journal-title":"Journal of Field Robotics"},{"key":"9141_CR2","doi-asserted-by":"crossref","unstructured":"Amigoni, F. (2008). Experimental evaluation of some exploration strategies for mobile robots. In IEEE international conference on robotics and automation (pp. 2818\u20132823).","DOI":"10.1109\/ROBOT.2008.4543637"},{"key":"9141_CR3","doi-asserted-by":"crossref","unstructured":"Amigoni, F., & Gallo, A. (2005). A multi-objective exploration strategy for mobile robots. In IEEE international conference on robotics and automation (pp. 3850\u20133855).","DOI":"10.1109\/ROBOT.2005.1570708"},{"key":"9141_CR4","unstructured":"Balaguer, B., & Carpin, S. (2008a). Uc mercenary team description paper: Robocup 2008 virtual robot rescue simulation league. In Team description paper for RoboCup 2008."},{"key":"9141_CR5","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1007\/978-3-540-89076-8_23","volume-title":"Proceedings of the first international conference on simulation, modeling and programming for autonomous robots","author":"B. Balaguer","year":"2008","unstructured":"Balaguer, B., Carpin, S. (2008b). Where Am I? A Simulated GPS Sensor for Outdoor Robotic Applications. In S. Carpin et al. (Eds.), Proceedings of the first international conference on simulation, modeling and programming for autonomous robots (pp.\u00a0222\u2013233). Berlin: Springer."},{"key":"9141_CR6","unstructured":"Balaguer, B., Carpin, S., & Balakirsky, S. (2007). Towards quantitative comparisons of robot algorithms: Experiences with slam in simulation and real world systems. In Workshop on performance evaluation and benchmarking for intelligent robots and systems at IEEE\/RSJ IROS."},{"issue":"11\u201312","key":"9141_CR7","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1002\/rob.20212","volume":"24","author":"S. Balakirsky","year":"2007","unstructured":"Balakirsky, S., Carpin, S., Kleiner, A., Lewis, M., Visser, A., Wang,\u00a0J., & Ziparo, V. A. (2007). Towards heterogeneous robot teams for disaster mitigation: Results and performance metrics from robocup rescue. Journal of Field Robotics, 24(11\u201312), 943\u2013967.","journal-title":"Journal of Field Robotics"},{"key":"9141_CR8","doi-asserted-by":"crossref","unstructured":"Baltes, J. (2000). A benchmark suite for mobile robots. In Proceedings of the IEEE\/RSJ conference on intelligent robots and systems (pp.\u00a01101\u20131106).","DOI":"10.1109\/IROS.2000.893166"},{"key":"9141_CR9","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/S0020-7373(81)80001-6","volume":"15","author":"R. J. Baron","year":"1981","unstructured":"Baron, R. J. (1981). Mechanisms of human facial recognition. International Journal of Man Machine Studies, 15, 137\u2013178.","journal-title":"International Journal of Man Machine Studies"},{"key":"9141_CR10","unstructured":"Basilico, N., & Amigoni, F. (2008). On evaluating performance of exploration strategies for autonomous mobile robots. In Proceedings of the performance evaluation and benchmarking for intelligent robots and systems workshop at IEEE\/RSJ international conference on intelligent robots and systems."},{"key":"9141_CR11","unstructured":"Birk, A. (1996). Learning geometric concepts with an evolutionary algorithm. In The fifth annual conference on evolutionary programming."},{"key":"9141_CR12","series-title":"Lecture notes in artificial intelligence (LNAI)","volume-title":"RoboCup 2008: Robot WorldCup XII","author":"A. Birk","year":"2009","unstructured":"Birk, A., Poppinga, J., Stoyanov, T., & Nevatia, Y. (2009). Planetary exploration in USARsim: a case study including real world data from Mars. In L. Iocchi, H. Matsubara, A. Weitzenfeld, & C. Zhou (Eds.), Lecture notes in artificial intelligence (LNAI). RoboCup 2008: Robot WorldCup XII. Berlin: Springer."},{"key":"9141_CR13","unstructured":"Calisi, D., Iocchi, L., & Nardi, D. (2008). A unified benchmark framework for autonomous mobile robots and vehicles motion algorithms (movema benchmarks). In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008)."},{"key":"9141_CR14","doi-asserted-by":"crossref","unstructured":"Carlson, J., Murphy, R., Christopher, S., & Casper, J. (2005). Conflict metric as a measure of sensing quality. In IEEE international conference on robotics and automation.","DOI":"10.1109\/ROBOT.2005.1570412"},{"key":"9141_CR15","unstructured":"Carpin, S., Stoyanov, T., Nevatia, Y., Lewis, M., & Wang, J. (2006a). Quantitative assessments of usarsim accuracy. In Proceedings of the performance metrics for intelligent systems workshop."},{"key":"9141_CR16","series-title":"LNCS","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/11780519_27","volume-title":"Robocup 2005: Robot Soccer World Cup IX","author":"S. Carpin","year":"2006","unstructured":"Carpin, S., Wang, J., Lewis, M., Birk, A., & Jacoff, A. (2006b). High fidelity tools for rescue robotics: results and perspectives. In LNCS. Robocup 2005: Robot Soccer World Cup IX (pp. 301\u2013311). Berlin: Springer."},{"key":"9141_CR17","series-title":"LNCS","first-page":"1","volume-title":"Robocup 2006: Robot Soccer World Cup X","author":"S. Carpin","year":"2007","unstructured":"Carpin, S., Lewis, M., Wang, J., Balakirsky, S., & Scrapper, C. (2007a). Bridging the gap between simulation and reality in urban search and rescue. In LNCS : Vol. 4434. Robocup 2006: Robot Soccer World Cup X (pp. 1\u201312). Berlin: Springer."},{"key":"9141_CR18","doi-asserted-by":"crossref","unstructured":"Carpin, S., Lewis, M., Wang, J., Balakirsky, S., & Scrapper, C. (2007b). USARSim: a robot simulator for research and education. In Proceedings of the IEEE international conference on robotics and automation (pp. 1400\u20131405).","DOI":"10.1109\/ROBOT.2007.363180"},{"key":"9141_CR19","unstructured":"Collins, J. J., & O\u2019Sullivan, S. (2004). Developing a benchmarking framework for map building paradigms. In Ninth international symposium on artificial life and robots."},{"key":"9141_CR20","first-page":"235","volume-title":"Proceedings of AI 2006: advances in artificial intelligence","author":"T. Collins","year":"2005","unstructured":"Collins, T., Collins, J. J., Mansfield, M., & O\u2019Sullivan, S. (2005). Evaluating techniques for resolving redundant information and specularity in occupancy grids. In S. Zhang, & R. Jarvis (Eds.), Proceedings of AI 2006: advances in artificial intelligence (pp. 235\u2013244). Berlin: Springer."},{"key":"9141_CR21","doi-asserted-by":"crossref","unstructured":"Collins, T., Collins, J. J., & Ryan, C. (2007). Occupancy grid mapping: and empirical evaluation. In Proceedings of the Mediterranean conference on control and automation (pp. 1\u20136).","DOI":"10.1109\/MED.2007.4433772"},{"key":"9141_CR22","unstructured":"Egerton, S. J., & Callaghan, V. (2004). A benchmark for measuring mobile robot environment modelling performacne. In Proceedings of the IEEE conference on robotics, automation and mechatronics (pp. 407\u2013412)."},{"key":"9141_CR23","unstructured":"Fontana, G., Matteucci, M., & Sorrenti, D. G. (2008). The RAWSEED proposal for representation-independent benchmarking of SLAM. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008)."},{"key":"9141_CR24","unstructured":"Frontoni, E., Ascani, A., Mancini, A., & Zingaretti, P. (2008). Performance metric for vision based robot localization. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008)."},{"key":"9141_CR25","unstructured":"The GeoTIFF website (2009). http:\/\/trac.osgeo.org\/geotiff ."},{"issue":"1","key":"9141_CR26","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/TRO.2006.889486","volume":"23","author":"G. Grisetti","year":"2007","unstructured":"Grisetti, G., Stachniss, C., & Burgard, W. (2007). Improved techniques for grid mapping iwht Rao-Blackwellized particle filters. IEEE Transactions on Robotics, 23(1), 36\u201346.","journal-title":"IEEE Transactions on Robotics"},{"key":"9141_CR27","series-title":"Series studies in fuzziness and soft computing","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-35488-8","volume-title":"Feature extraction, foundations and applications","author":"I. Guyon","year":"2006","unstructured":"Guyon, I., Gunn, S., Nikravesh, M., & Zadeh, L. (2006). Feature extraction, foundations and applications. Series studies in fuzziness and soft computing. Berlin: Springer."},{"key":"9141_CR28","doi-asserted-by":"crossref","unstructured":"Jacoff, A., Messina, E., Weiss, B., Tadokoro, S., & Nakagawa, Y. (2003). Test arenas and performance metrics for urban search and rescue robots. In International conference on intelligent robots and systems (pp. 3396\u20133403).","DOI":"10.1109\/IROS.2003.1249681"},{"key":"9141_CR29","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1207\/s15516709cog0202_3","volume":"2","author":"B. Kuipers","year":"1978","unstructured":"Kuipers, B. (1978). Modeling spatial knowledge. Cognitive Science, 2, 129\u2013153.","journal-title":"Cognitive Science"},{"key":"9141_CR30","unstructured":"Martin, M. C., & Moravec, H. P. (1996). Robot evidence grids. Technical Report CMU-RI-TR-96-06, Robotics Institute\u2013Carnegie Mellon University, March 1996."},{"key":"9141_CR31","unstructured":"Michel, O., Bourquin, Y., & Baillie, J. C. (2008). Robotstadium: Online humanoid robot soccer simulation competition. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008)."},{"key":"9141_CR32","unstructured":"MOAST project (2009). http:\/\/sourceforge.net\/projects\/moast ."},{"issue":"3","key":"9141_CR33","first-page":"61","volume":"9","author":"H. P. Moravec","year":"1988","unstructured":"Moravec, H. P. (1988). Sensor fusion in certainty grids for mobile robots. AI Magazine, 9(3), 61\u201374.","journal-title":"AI Magazine"},{"key":"9141_CR34","unstructured":"OpenSlam (2009). http:\/\/www.openslam.org ."},{"key":"9141_CR35","unstructured":"O\u2019Sullivan, S. (2003). An empirical evaluation of map building methodologies in mobile robotics using the feature prediction sonar noise filter and metric grip map benchmarking suite. Master\u2019s thesis, University of Limerick."},{"key":"9141_CR36","unstructured":"O\u2019Sullivan, S., Collins, J. J., Mansfield, M., Haskett, D., & Eaton, M. (2004a). Linear feature prediction for confidence estimation of sonar readings in map building. In Ninth international symposium on artificial life and robots."},{"key":"9141_CR37","unstructured":"O\u2019Sullivan, S., Collins, J. J., Mansfield, M., Haskett, D., & Eaton, M. (2004b). A quantitative evaluation of sonar models and mathematical update methods for map building with mobile robots. In Ninth international symposium on artificial life and robots."},{"key":"9141_CR38","unstructured":"Radish\u2014the robotics data set repository (2009). http:\/\/radish.sourceforge.net ."},{"key":"9141_CR39","unstructured":"Sato, K., Yotsukura, S., & Takahashi, T. (2008). To a rescue simulation applicable to anywhere. Team Description Hinomiyagura, July 2008. Proceedings CD."},{"key":"9141_CR40","unstructured":"Schmits, T., & Visser, A. (2008). An omnidirectional camera simulation for the USARSim World. In Proceedings of the 12th RoboCup international symposium, July 2008. Proceedings CD. To be published in the Lecture Notes on Artificial Intelligence series."},{"key":"9141_CR41","doi-asserted-by":"crossref","unstructured":"Sim, R., Dudek, G., & Roy, N. (2004). Online control policy optimization for minimizing map uncertainty during exploration. In IEEE international conference on robotics and automation.","DOI":"10.1109\/ROBOT.2004.1308078"},{"key":"9141_CR42","unstructured":"Stachniss, C., & Burgard, W. (2003). Exploring unknown environments with mobile robots using coverage maps. In 18th joint conference on artificial intelligence."},{"key":"9141_CR43","unstructured":"Varsadan, I., Birk, A., Pingsthorn, M., Schwertfeger, S., & Pathak, K. (2008). The Jacobs map analysis toolkit. In Workshop on experimental methodology and benchmarking in robotics research (RSS 2008)."}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-009-9141-z.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,4,7]],"date-time":"2019-04-07T12:08:58Z","timestamp":1554638938000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10514-009-9141-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,8,28]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2009,11]]}},"alternative-id":["9141"],"URL":"https:\/\/doi.org\/10.1007\/s10514-009-9141-z","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"value":"0929-5593","type":"print"},{"value":"1573-7527","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,8,28]]}}}