{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T00:55:17Z","timestamp":1767142517489,"version":"build-2238731810"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["308036"],"award-info":[{"award-number":["308036"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2020,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    We study the problem of automatically detecting if a given multi-class classifier operates\n                    <jats:italic>outside of its specifications (out-of-specs)<\/jats:italic>\n                    , i.e. on input data from a different distribution than what it was trained for. This is an important problem to solve on the road towards creating reliable computer vision systems for real-world applications, because the quality of a classifier\u2019s predictions cannot be guaranteed if it operates out-of-specs. Previously proposed methods for out-of-specs detection make decisions on the level of single inputs. This, however, is insufficient to achieve low false positive rate and high false negative rates at the same time. In this work, we describe a new procedure named KS(conf), based on statistical reasoning. Its main component is a classical Kolmogorov\u2013Smirnov test that is applied to the set of predicted confidence values for batches of samples. Working with batches instead of single samples allows increasing the true positive rate without negatively affecting the false positive rate, thereby overcoming a crucial limitation of single sample tests. We show by extensive experiments using a variety of convolutional network architectures and datasets that KS(conf) reliably detects out-of-specs situations even under conditions where other tests fail. It furthermore has a number of properties that make it an excellent candidate for practical deployment: it is easy to implement, adds almost no overhead to the system, works with any classifier that outputs confidence scores, and requires no a priori knowledge about how the data distribution could change.\n                  <\/jats:p>","DOI":"10.1007\/s11263-019-01232-x","type":"journal-article","created":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T06:19:32Z","timestamp":1570688372000},"page":"970-995","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["KS(conf): A Light-Weight Test if a Multiclass Classifier Operates Outside of Its Specifications"],"prefix":"10.1007","volume":"128","author":[{"given":"R\u00e9my","family":"Sun","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8622-7887","authenticated-orcid":false,"given":"Christoph H.","family":"Lampert","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"issue":"7","key":"1232_CR1","doi-asserted-by":"publisher","first-page":"1425","DOI":"10.1109\/TPAMI.2015.2487986","volume":"38","author":"Z Akata","year":"2016","unstructured":"Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2016). Label-embedding for image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 38(7), 1425\u20131438.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)"},{"key":"1232_CR2","unstructured":"Bansal, A., Farhadi, A., & Parikh, D. (2014). Towards transparent systems: Semantic characterization of failure modes. In European conference on computer vision (ECCV). New York: Springer."},{"key":"1232_CR3","volume-title":"Detection of abrupt changes: Theory and application","author":"M Basseville","year":"1993","unstructured":"Basseville, M., Nikiforov, I. V., et al. (1993). Detection of abrupt changes: Theory and application (Vol. 104). Englewood Cliffs, NJ: Prentice Hall."},{"key":"1232_CR4","doi-asserted-by":"crossref","unstructured":"Bendale, A. & Boult, T. (2016). Towards open set deep networks. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2016.173"},{"key":"1232_CR5","doi-asserted-by":"crossref","unstructured":"Bendale, A., & Boult, T. (2015). Towards open world recognition. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2015.7298799"},{"issue":"1\u20132","key":"1232_CR6","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/s10994-009-5152-4","volume":"79","author":"S Ben-David","year":"2010","unstructured":"Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010). A theory of learning from different domains. Machine Learning, 79(1\u20132), 151\u2013175.","journal-title":"Machine Learning"},{"key":"1232_CR7","doi-asserted-by":"crossref","unstructured":"Blitzer, J., McDonald, R., & Pereira, F. (2006). Domain adaptation with structural correspondence learning. In Empirical methods in natural language processing.","DOI":"10.3115\/1610075.1610094"},{"key":"1232_CR8","doi-asserted-by":"publisher","first-page":"15:1","DOI":"10.1145\/1541880.1541882","volume":"41","author":"V Chandola","year":"2009","unstructured":"Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41, 15:1\u201315:58.","journal-title":"ACM Computing Surveys (CSUR)"},{"issue":"4","key":"1232_CR9","doi-asserted-by":"publisher","first-page":"268","DOI":"10.2345\/0899-8205-46.4.268","volume":"46","author":"M Cvach","year":"2012","unstructured":"Cvach, M. (2012). Monitor alarm fatigue: An integrative review. Biomedical Instrumentation & Technology, 46(4), 268\u2013277.","journal-title":"Biomedical Instrumentation & Technology"},{"key":"1232_CR10","doi-asserted-by":"crossref","unstructured":"Daftry, S., Zeng, S., Bagnell, J. A., & Hebert, M. (2016). Introspective perception: Learning to predict failures in vision systems. In International conference on intelligent robots (IROS).","DOI":"10.1109\/IROS.2016.7759279"},{"key":"1232_CR11","unstructured":"DeVries, T., & Taylor, G. W. (2018). Learning confidence for out-of-distribution detection in neural networks. \narXiv:1802.04865"},{"key":"1232_CR12","unstructured":"dos Reis, D. M., Flach, P., Matwin, S., & Batista, G. (2016). Fast unsupervised online drift detection using incremental Kolmogorov\u2013Smirnov test. In International conference on knowledge discovery and data mining (KDD)."},{"key":"1232_CR13","unstructured":"Dunning, T., & Ertl, O. (2014). Computing extremely accurate quantiles using t-digests. github.com"},{"issue":"4","key":"1232_CR14","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1016\/0003-6870(94)90001-9","volume":"25","author":"J Edworthy","year":"1994","unstructured":"Edworthy, J. (1994). The design and implementation of non-verbal auditory warnings. Applied Ergonomics, 25(4), 202\u2013210.","journal-title":"Applied Ergonomics"},{"issue":"4","key":"1232_CR15","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1145\/2523813","volume":"46","author":"JA Gama","year":"2014","unstructured":"Gama, J. A., \u017dliobait\u0117, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys (CSUR), 46(4), 44.","journal-title":"ACM Computing Surveys (CSUR)"},{"key":"1232_CR16","unstructured":"Ganin, Y., & Lempitsky, V. (2015). Unsupervised domain adaptation by backpropagation. In International conference on machine learing (ICML)."},{"key":"1232_CR17","unstructured":"Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In International conference on machine learing (ICML)."},{"key":"1232_CR18","unstructured":"H\u00e4kkinen, M. (2010). Why alarms fail: A cognitive explanatory model. Ph.D. thesis, University of Jyv\u00e4skyl\u00e4."},{"key":"1232_CR19","unstructured":"Harel, M., Mannor, S., El-Yaniv, R., & Crammer, K. (2014). Concept drift detection through resampling. In International conference on machine learing (ICML)."},{"key":"1232_CR20","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-015-3994-4","volume-title":"Identification of outliers. Monographs on applied probability and statistics","author":"D Hawkins","year":"1980","unstructured":"Hawkins, D. (1980). Identification of outliers. Monographs on applied probability and statistics. London: Chapman and Hall."},{"key":"1232_CR21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2016.90"},{"key":"1232_CR22","unstructured":"Hendrycks, D., & Gimpel, K. (2017). A baseline for detecting misclassifier and out-of-distribution examples in neural networks. In International conference on learning representations (ICLR)."},{"issue":"2","key":"1232_CR23","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1023\/B:AIRE.0000045502.10941.a9","volume":"22","author":"V Hodge","year":"2004","unstructured":"Hodge, V., & Austin, J. (2004). A survey of outlier detection methodologies. Artificial Intelligence Review, 22(2), 85\u2013126.","journal-title":"Artificial Intelligence Review"},{"key":"1232_CR24","unstructured":"Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. \narXiv:1704.04861"},{"key":"1232_CR25","unstructured":"Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and $$<$$0.5 MB model size. \narXiv:1602.07360"},{"key":"1232_CR26","doi-asserted-by":"crossref","unstructured":"Jain, L. P., Scheirer, W. J., & Boult, T. E. (2014). Multi-class open set recognition using probability of inclusion. In European conference on computer vision (ECCV).","DOI":"10.1007\/978-3-319-10578-9_26"},{"issue":"1","key":"1232_CR27","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1109\/TCOM.1967.1089532","volume":"15","author":"T Kailath","year":"1967","unstructured":"Kailath, T. (1967). The divergence and bhattacharyya distance measures in signal selection. IEEE Transactions on Communication Technology (T-COM), 15(1), 52\u201360.","journal-title":"IEEE Transactions on Communication Technology (T-COM)"},{"key":"1232_CR28","doi-asserted-by":"crossref","unstructured":"Khosla, A., Zhou, T., Malisiewicz, T., Efros, A., & Torralba, A. (2012). Undoing the damage of dataset bias. In European conference on computer vision (ECCV).","DOI":"10.1007\/978-3-642-33718-5_12"},{"key":"1232_CR29","doi-asserted-by":"crossref","unstructured":"Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., & Grabska-Barwinska, A., et\u00a0al. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences (PNAS).","DOI":"10.1073\/pnas.1611835114"},{"key":"1232_CR30","unstructured":"Knorr, E.M., & Ng, R.T. (1997). A unified notion of outliers: Properties and computation. In International conference on knowledge discovery and data mining (KDD), vol.\u00a097."},{"key":"1232_CR31","unstructured":"Konstantinov, N., & Lampert, C. H. (2019). Robust learning from untrusted sources. In International conference on machine learing (ICML)."},{"issue":"1","key":"1232_CR32","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1109\/TNNLS.2013.2248094","volume":"25","author":"LI Kuncheva","year":"2014","unstructured":"Kuncheva, L. I., & Faithfull, W. J. (2014). PCA feature extraction for change detection in multidimensional unlabeled data. IEEE Transactions on Neural Networks (T-NN), 25(1), 69\u201380.","journal-title":"IEEE Transactions on Neural Networks (T-NN)"},{"key":"1232_CR33","doi-asserted-by":"crossref","unstructured":"Kuzborskij, I., Orabona, F., & Caputo, B. (2013). From $$n$$ to $$n+1$$: Multiclass transfer incremental learning. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2013.431"},{"key":"1232_CR34","unstructured":"Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. In Conference on neural information processing systems (NIPS)."},{"issue":"3","key":"1232_CR35","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1109\/TPAMI.2013.140","volume":"36","author":"CH Lampert","year":"2014","unstructured":"Lampert, C. H., Nickisch, H., & Harmeling, S. (2014). Attribute-based classification for zero-shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(3), 453\u2013465.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1232_CR36","unstructured":"Lee, K., Lee, H., Lee, K., & Shin, J. (2018). Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International conference on learning representations (ICLR)."},{"key":"1232_CR37","unstructured":"Li, Y., & Gal, Y. (2017). Dropout inference in Bayesian neural networks with alpha-divergences. In International conference on machine learing (ICML)."},{"key":"1232_CR38","unstructured":"Liang, S., Li, Y., & Srikant, R. (2018). Enhancing the reliability of out-of-distribution image detection in neural networks. In International conference on learning representations (ICLR)."},{"key":"1232_CR39","unstructured":"Long, M., Cao, Y., Wang, J., & Jordan, M.I. (2015). Learning transferable features with deep adaptation networks. In International conference on machine learing (ICML)."},{"key":"1232_CR40","unstructured":"Louizos, C., & Welling, M. (2017). Multiplicative normalizing flows for variational bayesian neural networks. In International conference on machine learning (ICML)."},{"issue":"18","key":"1232_CR41","first-page":"1","volume":"8","author":"G Marsaglia","year":"2003","unstructured":"Marsaglia, G., Tsang, W. W., & Wang, J. (2003). Evaluating Kolmogorov\u2019s distribution. Journal of Statistical Software, Articles, 8(18), 1\u20134.","journal-title":"Journal of Statistical Software, Articles"},{"issue":"253","key":"1232_CR42","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1080\/01621459.1951.10500769","volume":"46","author":"FJ Massey Jr","year":"1951","unstructured":"Massey, F. J, Jr. (1951). The Kolmogorov\u2013Smirnov test for goodness of fit. Journal of the American Statistical Association, 46(253), 68\u201378.","journal-title":"Journal of the American Statistical Association"},{"key":"1232_CR43","unstructured":"Palatucci, M., Pomerleau, D., Hinton, G. E., & Mitchell, T. M. (2009). Zero-shot learning with semantic output codes. In Conference on neural information processing systems (NIPS)."},{"issue":"10","key":"1232_CR44","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","volume":"22","author":"SJ Pan","year":"2009","unstructured":"Pan, S. J., & Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345\u20131359.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"3","key":"1232_CR45","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1109\/MSP.2014.2347059","volume":"32","author":"VM Patel","year":"2015","unstructured":"Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015). Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Magazine, 32(3), 53\u201369.","journal-title":"IEEE Signal Processing Magazine"},{"key":"1232_CR46","doi-asserted-by":"crossref","unstructured":"Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., & Sorkine-Hornung, A. (2016). A benchmark dataset and evaluation methodology for video object segmentation. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2016.85"},{"key":"1232_CR47","unstructured":"Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in large margin classifiers. Cambridge: Cambridge University Press."},{"issue":"1","key":"1232_CR48","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1214\/aos\/1176346587","volume":"13","author":"M Pollak","year":"1985","unstructured":"Pollak, M., et al. (1985). Optimal detection of a change in distribution. The Annals of Statistics, 13(1), 206\u2013227.","journal-title":"The Annals of Statistics"},{"key":"1232_CR49","unstructured":"Rebuffi, S. A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2016). iCaRL: Incremental classifier and representation learning. In Conference on computer vision and pattern recognition (CVPR)."},{"key":"1232_CR50","doi-asserted-by":"crossref","unstructured":"Royer, A., & Lampert, C. H. (2015). Classifier adaptation at prediction time. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2015.7298746"},{"issue":"3","key":"1232_CR51","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision (IJCV), 115(3), 211\u2013252.","journal-title":"International Journal of Computer Vision (IJCV)"},{"key":"1232_CR52","doi-asserted-by":"crossref","unstructured":"Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010). Adapting visual category models to new domains. In European conference on computer vision (ECCV).","DOI":"10.1007\/978-3-642-15561-1_16"},{"issue":"8","key":"1232_CR53","doi-asserted-by":"publisher","first-page":"1689","DOI":"10.1109\/TPAMI.2011.54","volume":"33","author":"WJ Scheirer","year":"2011","unstructured":"Scheirer, W. J., Rocha, A., Micheals, R. J., & Boult, T. E. (2011). Meta-recognition: The theory and practice of recognition score analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 33(8), 1689\u20131695.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)"},{"issue":"1","key":"1232_CR54","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s10844-015-0358-3","volume":"46","author":"TS Sethi","year":"2016","unstructured":"Sethi, T. S., Kantardzic, M., & Hu, H. (2016). A grid density based framework for classifying streaming data in the presence of concept drift. Journal of Intelligent Information Systems, 46(1), 179\u2013211.","journal-title":"Journal of Intelligent Information Systems"},{"key":"1232_CR55","unstructured":"Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. \narXiv:1409.1556"},{"key":"1232_CR56","unstructured":"Sun, R., & Lampert, C.H. (2018). KS(conf): A light-weight test if a ConvNet operates outside of its specifications. In German conference on pattern recognition (GCPR)."},{"key":"1232_CR57","doi-asserted-by":"publisher","unstructured":"Tange, O. (2018). GNU Parallel 2018. \nhttps:\/\/doi.org\/10.5281\/zenodo.1146014\n\n.","DOI":"10.5281\/zenodo.1146014"},{"key":"1232_CR58","unstructured":"Tax, D. M. J. (2001). One-class classification: Concept learning in the absence of counter-examples. Ph.D. thesis, Technische Universiteit Delft."},{"key":"1232_CR59","unstructured":"Tax, D.M., & Duin, R.P. (1998). Outlier detection using classifier instability. In Joint IAPR international workshops on statistical techniques in pattern recognition (SPR) and structural and syntactic pattern recognition (SSPR). New York: Springer."},{"key":"1232_CR60","unstructured":"Tommasi, T., Quadrianto, N., Caputo, B., & Lampert, C. H. (2012). Beyond dataset bias: Multi-task unaligned shared knowledge transfer. In Asian conference on computer vision (ACCV)."},{"issue":"3","key":"1232_CR61","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1109\/TCOM.1972.1091157","volume":"20","author":"GT Toussaint","year":"1972","unstructured":"Toussaint, G. T. (1972). Comments on the divergence and Bhattacharyya distance measures in signal selection. IEEE Transactions on Communication Technology (T-COM), 20(3), 485\u2013485.","journal-title":"IEEE Transactions on Communication Technology (T-COM)"},{"key":"1232_CR62","doi-asserted-by":"crossref","unstructured":"Wang, H., & Abraham, Z. (2015). Concept drift detection for streaming data. In International joint conference on neural networks (IJCNN).","DOI":"10.1109\/IJCNN.2015.7280398"},{"key":"1232_CR63","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1016\/j.neucom.2018.05.083","volume":"312","author":"M Wang","year":"2018","unstructured":"Wang, M., & Deng, W. (2018). Deep visual domain adaptation: A survey. Neurocomputing, 312, 135\u2013153.","journal-title":"Neurocomputing"},{"key":"1232_CR64","unstructured":"Xian, Y., Lampert, C.H., Schiele, B., & Akata, Z. (2018). Zero-shot learning: A comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI)."},{"key":"1232_CR65","doi-asserted-by":"crossref","unstructured":"Zhang, P., Wang, J., Farhadi, A., Hebert, M., & Parikh, D. (2014). Predicting failures of vision systems. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2014.456"},{"key":"1232_CR66","doi-asserted-by":"crossref","unstructured":"Zliobaite, I. (2010). Change with delayed labeling: When is it detectable? In International conference on data mining (ICDM) workshops.","DOI":"10.1109\/ICDMW.2010.49"},{"key":"1232_CR67","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., & Le, Q.V. (2018). Learning transferable architectures for scalable image recognition. In Conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2018.00907"}],"updated-by":[{"DOI":"10.1007\/s11263-019-01262-5","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2019,11,9]],"date-time":"2019-11-09T00:00:00Z","timestamp":1573257600000}}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-019-01232-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11263-019-01232-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-019-01232-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T19:28:55Z","timestamp":1602185335000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11263-019-01232-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":67,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,4]]}},"alternative-id":["1232"],"URL":"https:\/\/doi.org\/10.1007\/s11263-019-01232-x","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"1 March 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 September 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 October 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 November 2019","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The original version of this article contained a mistake in the denominator of equation (1).","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}}]}}