{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T00:18:26Z","timestamp":1783988306710,"version":"3.55.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1-3","license":[{"start":{"date-parts":[[2017,6,9]],"date-time":"2017-06-09T00:00:00Z","timestamp":1496966400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2017,12]]},"DOI":"10.1007\/s11263-017-1020-z","type":"journal-article","created":{"date-parts":[[2017,6,9]],"date-time":"2017-06-09T10:54:13Z","timestamp":1497005653000},"page":"95-109","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":50,"title":["How Good Is My Test Data? Introducing Safety Analysis for Computer Vision"],"prefix":"10.1007","volume":"125","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8097-1226","authenticated-orcid":false,"given":"Oliver","family":"Zendel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Murschitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Humenberger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolfgang","family":"Herzner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2017,6,9]]},"reference":[{"key":"1020_CR1","volume-title":"Integration of visual modules: An extension of the Marr paradigm","author":"JY Aloimonos","year":"1989","unstructured":"Aloimonos, J. Y., & Shulman, D. (1989). Integration of visual modules: An extension of the Marr paradigm. Boston: Academic Press Professional Inc."},{"key":"1020_CR2","doi-asserted-by":"crossref","unstructured":"Bleyer, M., Rhemann, C., & Rother, C. (2011). Patchmatch stereo\u2013stereo matching with slanted support windows. In British machine vision conference.","DOI":"10.5244\/C.25.14"},{"key":"1020_CR3","volume-title":"Empirical evaluation techniques in computer vision","author":"K Bowyer","year":"1998","unstructured":"Bowyer, K., & Phillips, P. J. (1998). Empirical evaluation techniques in computer vision. Los Alamitos, CA: IEEE Computer Society Press."},{"issue":"1","key":"1020_CR4","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1006\/cviu.2001.0931","volume":"84","author":"K Bowyer","year":"2001","unstructured":"Bowyer, K., Kranenburg, C., & Dougherty, S. (2001). Edge detector evaluation using empirical ROC curves. Computer Vision and Image Understanding, 84(1), 77\u2013103.","journal-title":"Computer Vision and Image Understanding"},{"key":"1020_CR5","unstructured":"Center for Chemical Process Safety. (1992). Guidelines for hazard evaluation procedures, with worked examples (2nd ed.). Hoboken: Wiley."},{"key":"1020_CR6","unstructured":"Department of Defense. (1949). Procedures for Performing a Failure Mode, Effects and Criticality Analysis, MIL-STD-1629A."},{"key":"1020_CR7","unstructured":"Department of Defense. (1991). Reliability Prediction of Electronic Equipment: MIL-HDBK-217F."},{"key":"1020_CR8","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1\u201330.","journal-title":"Journal of Machine Learning Research"},{"key":"1020_CR9","unstructured":"Fenelon, P., & Hebbron, B. (1994). Applying HAZOP to software engineering models. In Risk management and critical protective systems: proceedings of SARSS (pp. 11\u2013116)."},{"issue":"6","key":"1020_CR10","doi-asserted-by":"crossref","first-page":"381395","DOI":"10.1145\/358669.358692","volume":"24","author":"MA Fischler","year":"1981","unstructured":"Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381395.","journal-title":"Communications of the ACM"},{"key":"1020_CR11","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2012.6248074"},{"issue":"10","key":"1020_CR12","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1109\/TSE.2003.1237174","volume":"29","author":"K Goseva-Popstojanova","year":"2003","unstructured":"Goseva-Popstojanova, K., Hassan, A., Guedem, A., Abdelmoez, W., Nassar, D. E. M., Ammar, H., et al. (2003). Architectural-level risk analysis using UML. IEEE Transactions on Software Engineering, 29(10), 946\u2013960.","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"6","key":"1020_CR13","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.1214\/aoms\/1177693054","volume":"42","author":"FR Hampel","year":"1971","unstructured":"Hampel, F. R. (1971). A general qualitative definition of robustness. The Annals of Mathematical Statistics, 42(6), 1887\u20131896.","journal-title":"The Annals of Mathematical Statistics"},{"issue":"2","key":"1020_CR14","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1109\/TPAMI.2007.1166","volume":"30","author":"H Hirschm\u00fcller","year":"2008","unstructured":"Hirschm\u00fcller, H. (2008). Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), 328\u2013341.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1020_CR15","doi-asserted-by":"crossref","unstructured":"Honauer, K., Maier-Hein, L., & Kondermann, D. (2015). The HCI stereo metrics: Geometry-aware performance analysis of stereo algorithms. In The IEEE international conference on computer vision (ICCV).","DOI":"10.1109\/ICCV.2015.245"},{"issue":"1","key":"1020_CR16","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1214\/aoms\/1177703732","volume":"35","author":"P Huber","year":"1964","unstructured":"Huber, P. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73\u2013101.","journal-title":"The Annals of Mathematical Statistics"},{"key":"1020_CR17","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1016\/j.cviu.2010.03.012","volume":"114","author":"M Humenberger","year":"2010","unstructured":"Humenberger, M., Zinner, C., Weber, M., Kubinger, W., & Vincze, M. (2010). A fast stereo matching algorithm suitable for embedded real-time systems. Computer Vision and Image Understanding, 114, 1180\u20131202.","journal-title":"Computer Vision and Image Understanding"},{"key":"1020_CR18","unstructured":"International Electrotechnical Commission. (2010). Functional safety of electrical\/electronic\/programmable electronic safety-related systems\u2014part 4: Definitions and abbreviations: IEC 61508-4."},{"key":"1020_CR19","doi-asserted-by":"crossref","unstructured":"Kadiofsky, T., Weichselbaum, J., & Zinner, C. (2012). Off-road terrain mapping based on dense hierarchical real-time stereo vision. In Advances in visual computing. Lecture Notes in Computer Science (Vol. 7431, pp. 404\u2013415). Berlin: Springer.","DOI":"10.1007\/978-3-642-33179-4_39"},{"key":"1020_CR20","doi-asserted-by":"crossref","unstructured":"Kajiya, J. T. (1986). The rendering equation. In SIGGRAPH conference proceedings (Vol. 20, No. 4, pp. 143\u2013150).","DOI":"10.1145\/15922.15902"},{"key":"1020_CR21","unstructured":"Kletz, T. A. (1983). HAZOP and HAZAN notes on the identification and assessment of hazards. The Institution of Chemical Engineers."},{"key":"1020_CR22","doi-asserted-by":"crossref","unstructured":"Kondermann, D. (2013). Ground truth design principles: An overview. In Proceedings of the international workshop on video and image ground truth in computer vision applications, VIGTA \u201913 (pp. 5:1\u20135:4). ACM, New York, NY, USA","DOI":"10.1145\/2501105.2501114"},{"key":"1020_CR23","doi-asserted-by":"crossref","unstructured":"Kondermann, D., Nair, R., Meister, S., Mischler, W., G\u00fcssefeld, B., Honauer, K., et al. (2015). Stereo ground truth with error bars. In Asian conference on computer vision.","DOI":"10.1007\/978-3-319-16814-2_39"},{"key":"1020_CR24","doi-asserted-by":"crossref","unstructured":"Konolige, K. (1998). Small vision systems: Hardware and implementation. In Robotics research. Berlin: Springer.","DOI":"10.1007\/978-1-4471-1580-9_19"},{"key":"1020_CR25","unstructured":"Laprie, J. (1992). Dependability: Basic concepts and terminology. In Dependable computing and fault-tolerant systems (Vol.\u00a05). Berlin: Springer."},{"key":"1020_CR26","volume-title":"Vision: A computational investigation into the human representation and processing of visual information","author":"D Marr","year":"1982","unstructured":"Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. San Francisco: W. H. Freeman."},{"key":"1020_CR27","unstructured":"Matthias, B., Oberer-Treitz, S., Staab, H., Schuller, E., & Peldschus, S. (2010). Injury risk quantification for industrial robots in collaborative operation with humans. In Proceedings of the of 41st international symposium on robotics and 6th German conference on robotics."},{"key":"1020_CR28","doi-asserted-by":"crossref","unstructured":"Mei, X., Sun, X., Dong, W., Wang, H., & Zhang, X. (2013). Segment-tree based cost aggregation for stereo matching. In Computer vision and pattern recognition (pp. 313\u2013320).","DOI":"10.1109\/CVPR.2013.47"},{"key":"1020_CR29","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TSMCB.2003.811118","volume":"34","author":"J Min","year":"2004","unstructured":"Min, J., Powell, M., & Bowyer, K. W. (2004). Automated performance evaluation of range image segmentation algorithms. IEEE Transactions on Systems Man and Cybernetics Part B Cybernetics, 34, 263\u2013271.","journal-title":"IEEE Transactions on Systems Man and Cybernetics Part B Cybernetics"},{"issue":"2","key":"1020_CR30","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1109\/T-AIEE.1928.5055024","volume":"47","author":"H Nyquist","year":"1928","unstructured":"Nyquist, H. (1928). Certain topics in telegraph transmission theory. Transactions of the American Institute of Electrical Engineers, 47(2), 617\u2013644.","journal-title":"Transactions of the American Institute of Electrical Engineers"},{"key":"1020_CR31","doi-asserted-by":"crossref","first-page":"e27","DOI":"10.1371\/journal.pcbi.0040027","volume":"4","author":"N Pinto","year":"2008","unstructured":"Pinto, N., Cox, D. D., & DiCarlo, J. J. (2008). Why is real-world visual object recognition hard? PLOS Computational Biology, 4, e27.","journal-title":"PLOS Computational Biology"},{"key":"1020_CR32","doi-asserted-by":"crossref","unstructured":"Ponce, J., Berg, T. L., Everingham, M., Forsyth, D. A., Hebert, M., Lazebnik, et\u00a0al. (2006). Dataset issues in object recognition. In Toward category-level object recognition (pp. 29\u201348). Springer.","DOI":"10.1007\/11957959_2"},{"key":"1020_CR33","doi-asserted-by":"crossref","unstructured":"Rhemann, C., Hosni, A., Bleyer, M., Rother, C., & Gelautz, M. (2011). Fast cost-volume filtering for visual correspondence and beyond. In Computer vision and pattern recognition (pp. 3017\u20133024).","DOI":"10.1109\/CVPR.2011.5995372"},{"key":"1020_CR34","doi-asserted-by":"crossref","unstructured":"Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7\u201342.","DOI":"10.1023\/A:1014573219977"},{"key":"1020_CR35","doi-asserted-by":"crossref","unstructured":"Scharstein, D., & Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2003.1211354"},{"key":"1020_CR36","doi-asserted-by":"crossref","unstructured":"Scharstein, D., Hirschm\u00fcller, H., Kitajima, Y., Krathwohl, G., Nesic, N., Wang, X., & Westling, P. (2014). High-resolution stereo datasets with subpixel-accurate ground truth. In Pattern recognition (pp. 31\u201342). Springer.","DOI":"10.1007\/978-3-319-11752-2_3"},{"key":"1020_CR37","doi-asserted-by":"crossref","unstructured":"Schlick, R., Herzner, W., & J\u00f6bstl, E. (2011). Fault-based generation of test cases from UML-models approach and some experiences. In Computer safety, reliability, and security. Lecture Notes in Computer Science (Vol. 6894, pp. 270\u2013283). Berlin: Springer.","DOI":"10.1007\/978-3-642-24270-0_20"},{"issue":"1","key":"1020_CR38","first-page":"10","volume":"37","author":"CE Shannon","year":"1949","unstructured":"Shannon, C. E. (1949). Communication in the presence of noise. Proceedings of the Institute of Radio Engineers, 37(1), 10\u201321.","journal-title":"Proceedings of the Institute of Radio Engineers"},{"key":"1020_CR39","volume-title":"The mathematical theory of communication","author":"CE Shannon","year":"1949","unstructured":"Shannon, C. E., & Weaver, W. (1949). The mathematical theory of communication. Champaign: University of Illinois Press."},{"key":"1020_CR40","doi-asserted-by":"crossref","unstructured":"Strecha, C., von Hansen, W., Van\u00a0Gool, L., Fua, P., & Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In Computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2008.4587706"},{"key":"1020_CR41","unstructured":"Takanen, A., DeMott, J., & Miller, C. (2008). Fuzzing for software security testing and quality assurance. Artech House on Demand."},{"issue":"3","key":"1020_CR42","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1016\/j.cviu.2007.04.006","volume":"109","author":"N Thacker","year":"2008","unstructured":"Thacker, N., Clark, A., Barron, J., Ross Beveridge, J., Courtney, P., Crum, W., et al. (2008). Performance characterization in computer vision: A guide to best practices. Computer Vision and Image Understanding, 109(3), 305\u2013334.","journal-title":"Computer Vision and Image Understanding"},{"key":"1020_CR43","doi-asserted-by":"crossref","unstructured":"Torralba, A., & Efros, A. A. (2011). Unbiased look at dataset bias. In Computer vision and pattern recognition (pp. 1521\u20131528).","DOI":"10.1109\/CVPR.2011.5995347"},{"key":"1020_CR44","unstructured":"Vesely, W. E., Goldberg, F. F., Roberts, N. H., & Haasl, D. F. (1981). Fault tree handbook. In Systems and reliability research. Office of Nuclear Regulatory Research: NRC."},{"key":"1020_CR45","unstructured":"Von Bertalanffy, L. (1968). General systems theory. New York 41973, 40."},{"key":"1020_CR46","unstructured":"Van\u00a0der Spek, R., & Spijkervet, A. (1997). Knowledge management: Dealing intelligently with knowledge. In Knowledge management and its integrative elements (pp. 31\u201360)."},{"issue":"6","key":"1020_CR47","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","volume":"1","author":"F Wilcoxon","year":"1945","unstructured":"Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80\u201383.","journal-title":"Biometrics Bulletin"},{"key":"1020_CR48","doi-asserted-by":"crossref","unstructured":"Zhang, K., Fang, Y., Min, D., Sun, L., Yang, S., Yan, S., & Tian, Q. (2014). Cross-scale cost aggregation for stereo matching. In Computer vision and pattern recognition.","DOI":"10.1109\/CVPR.2014.206"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11263-017-1020-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-017-1020-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-017-1020-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2017,10,29]],"date-time":"2017-10-29T00:34:42Z","timestamp":1509237282000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11263-017-1020-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,9]]},"references-count":48,"journal-issue":{"issue":"1-3","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["1020"],"URL":"https:\/\/doi.org\/10.1007\/s11263-017-1020-z","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,9]]}}}