{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T22:42:14Z","timestamp":1761864134118,"version":"build-2065373602"},"reference-count":82,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,23]],"date-time":"2020-06-23T00:00:00Z","timestamp":1592870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"PSC\/CUNY","award":["TRAD-A"],"award-info":[{"award-number":["TRAD-A"]}]},{"name":"Hugo Carlssons stiftelse f\u00f6r vetenskaplig forskning","award":["1"],"award-info":[{"award-number":["1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Predictive models are used in many different fields of science and engineering and are always prone to make faulty predictions. These faulty predictions can be more or less malignant depending on the model application. We describe fibers of failure (FiFa), a method to classify failure modes of predictive processes. Our method uses Mapper, an algorithm from topological data analysis (TDA), to build a graphical model of input data stratified by prediction errors. We demonstrate two ways to use the failure mode groupings: either to produce a correction layer that adjusts predictions by similarity to the failure modes; or to inspect members of the failure modes to illustrate and investigate what characterizes each failure mode. We demonstrate FiFa on two scenarios: a convolutional neural network (CNN) predicting MNIST images with added noise, and an artificial neural network (ANN) predicting the electrical energy consumption of an electric arc furnace (EAF). The correction layer on the CNN model improved its prediction accuracy significantly while the inspection of failure modes for the EAF model provided guiding insights into the domain-specific reasons behind several high-error regions.<\/jats:p>","DOI":"10.3390\/a13060150","type":"journal-article","created":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T07:14:19Z","timestamp":1592982859000},"page":"150","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Fibers of Failure: Classifying Errors in Predictive Processes"],"prefix":"10.3390","volume":"13","author":[{"given":"Leo S.","family":"Carlsson","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology, Brinellv\u00e4gen 8, 114 28 Stockholm, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6322-7542","authenticated-orcid":false,"given":"Mikael","family":"Vejdemo-Johansson","sequence":"additional","affiliation":[{"name":"Department of Mathematics, CUNY College of Staten Island, 2800 Victory Blvd, Staten Island, NY 10314, USA"},{"name":"Computer Science, CUNY Graduate Center, 365 5th Ave, New York, NY 10016, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gunnar","family":"Carlsson","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Stanford University, 450 Serra Mall, Stanford, CA 94305, USA"},{"name":"Unbox AI, Stanford, CA 94305, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9775-0382","authenticated-orcid":false,"given":"P\u00e4r G.","family":"J\u00f6nsson","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Brinellv\u00e4gen 8, 114 28 Stockholm, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Box, G.E. (1979). Robustness in the strategy of scientific model building. Robustness in Statistics, Elsevier.","DOI":"10.1016\/B978-0-12-438150-6.50018-2"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"7265","DOI":"10.1073\/pnas.1102826108","article-title":"Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival","volume":"108","author":"Nicolau","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"311ra174","DOI":"10.1126\/scitranslmed.aaa9364","article-title":"Identification of type 2 diabetes subgroups through topological analysis of patient similarity","volume":"7","author":"Li","year":"2015","journal-title":"Sci. Transl. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.jaci.2015.11.020","article-title":"Multidimensional endotyping in patients with severe asthma reveals inflammatory heterogeneity in matrix metalloproteinases and chitinase 3\u2013like protein 1","volume":"138","author":"Hinks","year":"2016","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Schneider, D.S., Torres, B.Y., Oliveira, J.H.M., Tate, A.T., Rath, P., and Cumnock, K. (2016). Tracking resilience to infections by mapping disease space. PLoS Biol., 14.","DOI":"10.1371\/journal.pbio.1002436"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4904","DOI":"10.1002\/hbm.22521","article-title":"Topological methods reveal high and low functioning neuro-phenotypes within fragile X syndrome","volume":"35","author":"Romano","year":"2014","journal-title":"Hum. Brain Mapp."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.coisb.2016.12.012","article-title":"The shape of biomedical data","volume":"1","author":"Carlsson","year":"2017","journal-title":"Curr. Opin. Syst. Biol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.coisb.2016.12.007","article-title":"Topological methods for genomics: Present and future direction","volume":"1","year":"2017","journal-title":"Curr. Opin. Syst. Biol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1091","DOI":"10.1016\/j.scitotenv.2017.02.095","article-title":"Topological data analysis (TDA) applied to reveal pedogenetic principles of European topsoil system","volume":"586","author":"Savir","year":"2017","journal-title":"Sci. Total Environ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9676","DOI":"10.1021\/ja8032857","article-title":"Structural Insight into RNA Hairpin Folding Intermediates","volume":"130","author":"Bowman","year":"2008","journal-title":"JACS Commun."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.aca.2017.11.029","article-title":"Exploring hyperspectral imaging data sets with topological data analysis","volume":"1000","author":"Duponchel","year":"2018","journal-title":"Anal. Chim. Acta"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"a1","DOI":"10.1255\/jsi.2018.a1","article-title":"When remote sensing meets topological data analysis","volume":"7","author":"Duponchel","year":"2018","journal-title":"J. Spectr. Imaging"},{"key":"ref_13","first-page":"1","article-title":"Quantifying similarity of pore-geometry in nanoporous materials","volume":"8","author":"Lee","year":"2017","journal-title":"Nat. Commun."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lum, P.Y., Singh, G., Lehman, A., Ishkanov, T., Vejdemo-Johansson, M., Alagappan, M., Carlsson, J., and Carlsson, G. (2013). Extracting insights from the shape of complex data using topology. Sci. Rep., 3.","DOI":"10.1038\/srep01236"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Br\u00fcel Gabrielsson, R., and Carlsson, G. (2019, January 16\u201319). Exposition and interpretation of the topology of neural networks. Proceedings of the 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), Boca Raton, FL, USA.","DOI":"10.1109\/ICMLA.2019.00180"},{"key":"ref_16","unstructured":"Saul, N., and Arendt, D.L. (2020, May 25). Machine Learning Explanations with Topological Data Analysis. Available online: https:\/\/sauln.github.io\/blog\/tda_explanations\/."},{"key":"ref_17","unstructured":"Carri\u00e8re, M., and Michel, B. (2019). Approximation of Reeb spaces with Mappers and Applications to Stochastic Filters. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Song, S., and Cheung, N.M. (2017). On Classification of Distorted Images with Deep Convolutional Neural Networks. arXiv.","DOI":"10.1109\/ICASSP.2017.7952349"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Dodge, S., and Karam, L. (2016, January 6\u20138). Understanding How Image Quality Affects Deep Neural Networks. Proceedings of the 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal.","DOI":"10.1109\/QoMEX.2016.7498955"},{"key":"ref_20","unstructured":"Cisse, M., Adi, Y., Neverova, N., and Keshet, J. (2017, January 4\u20139). Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples. Proceedings of the Advances in Neural Information Processing Systems 30, Long Beach, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2805","DOI":"10.1109\/TNNLS.2018.2886017","article-title":"Adversarial Examples: Attacks and Defenses for Deep Learning","volume":"30","author":"Yuan","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Moosavi-Dezfooli, S.M., Fawzi, A., and Frossard, P. (2015). DeepFool: A simple and accurate method to fool deep neural networks. arXiv.","DOI":"10.1109\/CVPR.2016.282"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.cose.2017.11.007","article-title":"Automated poisoning attacks and defenses in malware detection systems: An adversarial machine learning approach","volume":"73","author":"Chen","year":"2018","journal-title":"Comput. Secur."},{"key":"ref_24","unstructured":"Wilson, A.G., Kim, B., and Herlands, W. (2016). Interpretable Machine Learning for Complex Systems, NIPS 2016 Workshop. arXiv."},{"key":"ref_25","unstructured":"Tosi, A., Vellido, A., and Alvarez, M. (2020, May 25). Transparent and Interpretable Machine Learning in Safety Critical Environments. Available online: https:\/\/sites.google.com\/view\/timl-nips2017."},{"key":"ref_26","unstructured":"Wilson, A.G., Yosinski, J., Simard, P., Caruana, R., and Herlands, W. (2017). Interpretable ML Symposium. arXiv."},{"key":"ref_27","unstructured":"Varshney, K., Weller, A., Kim, B., and Malioutov, D. (2017). Human Interpretability in Machine Learning, ICML 2017 Workshop. arXiv."},{"key":"ref_28","unstructured":"Gunning, D. (2020, May 25). Explainable Artificial Intelligence (XAI). DARPA Broad Agency Announcement DARPA-BAA-16-53. Available online: https:\/\/www.aaai.org\/ojs\/index.php\/aimagazine\/article\/view\/2850."},{"key":"ref_29","unstructured":"Hara, S., and Maehara, T. (2016). Finding Alternate Features in Lasso. arXiv."},{"key":"ref_30","unstructured":"Wisdom, S., Powers, T., Pitton, J., and Atlas, L. (2016). Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery. arXiv."},{"key":"ref_31","unstructured":"Hayete, B., Valko, M., Greenfield, A., and Yan, R. (2016). MDL-motivated compression of GLM ensembles increases interpretability and retains predictive power. arXiv."},{"key":"ref_32","unstructured":"Tansey, W., Thomason, J., and Scott, J.G. (2017). Interpretable Low-Dimensional Regression via Data-Adaptive Smoothing. arXiv."},{"key":"ref_33","unstructured":"Smilkov, D., Thorat, N., Nicholson, C., Reif, E., Vi\u00e9gas, F.B., and Wattenberg, M. (2016). Embedding Projector: Interactive Visualization and Interpretation of Embeddings. arXiv."},{"key":"ref_34","unstructured":"Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., and Batra, D. (2016). Grad-CAM: Why did you say that?. arXiv."},{"key":"ref_35","unstructured":"Thiagarajan, J.J., Kailkhura, B., Sattigeri, P., and Ramamurthy, K.N. (2016). TreeView: Peeking into Deep Neural Networks Via Feature-Space Partitioning. arXiv."},{"key":"ref_36","unstructured":"Gallego-Ortiz, C., and Martel, A.L. (2016). Interpreting extracted rules from ensemble of trees: Application to computer-aided diagnosis of breast MRI. arXiv."},{"key":"ref_37","unstructured":"Krause, J., Perer, A., and Bertini, E. (2016). Using Visual Analytics to Interpret Predictive Machine Learning Models. arXiv."},{"key":"ref_38","unstructured":"Zrihem, N.B., Zahavy, T., and Mannor, S. (2016). Visualizing Dynamics: From t-SNE to SEMI-MDPs. arXiv."},{"key":"ref_39","unstructured":"Handler, A., Blodgett, S.L., and O\u2019Connor, B. (2016). Visualizing textual models with in-text and word-as-pixel highlighting. arXiv."},{"key":"ref_40","unstructured":"Krakovna, V., and Doshi-Velez, F. (2016). Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models. arXiv."},{"key":"ref_41","unstructured":"Reing, K., Kale, D.C., Steeg, G.V., and Galstyan, A. (2016). Toward Interpretable Topic Discovery via Anchored Correlation Explanation. arXiv."},{"key":"ref_42","unstructured":"Samek, W., Montavon, G., Binder, A., Lapuschkin, S., and M\u00fcller, K.R. (2016). Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation. arXiv."},{"key":"ref_43","unstructured":"Hechtlinger, Y. (2016). Interpretation of Prediction Models Using the Input Gradient. arXiv."},{"key":"ref_44","unstructured":"Lundberg, S., and Lee, S.I. (2016). An unexpected unity among methods for interpreting model predictions. arXiv."},{"key":"ref_45","unstructured":"Vidovic, M.M.C., G\u00f6rnitz, N., M\u00fcller, K.R., and Kloft, M. (2016). Feature Importance Measure for Non-linear Learning Algorithms. arXiv."},{"key":"ref_46","unstructured":"Whitmore, L.S., George, A., and Hudson, C.M. (2016). Mapping chemical performance on molecular structures using locally interpretable explanations. arXiv."},{"key":"ref_47","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016). Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance. arXiv."},{"key":"ref_48","unstructured":"Singh, S., Ribeiro, M.T., and Guestrin, C. (2016). Programs as Black-Box Explanations. arXiv."},{"key":"ref_49","unstructured":"Phillips, R.L., Chang, K.H., and Friedler, S.A. (2017). Interpretable Active Learning. arXiv."},{"key":"ref_50","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016). Model-Agnostic Interpretability of Machine Learning. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016). \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. arXiv.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_52","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30, Curran Associates, Inc."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Carlsson, L.S., Samuelsson, P.B., and J\u00f6nsson, P.G. (2020, May 25). Interpretable Machine Learning\u2014Tools to Interpret the Predictions of a Machine Learning Model Predicting the Electrical Energy Consumption of an Electric Arc Furnace. Steel Research International. 2000053, Available online: http:\/\/xxx.lanl.gov\/abs\/https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/srin.202000053.","DOI":"10.1002\/srin.202000053"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.aca.2015.12.037","article-title":"Topological data analysis: A promosing big data exploration tool in biology, analytical chemistry and physical chemistry","volume":"910","author":"Offroy","year":"2016","journal-title":"Anal. Chim. Acta"},{"key":"ref_55","unstructured":"Singh, G., M\u00e9moli, F., and Carlsson, G. (2020, May 25). Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition. Available online: https:\/\/research.math.osu.edu\/tgda\/mapperPBG.pdf."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1090\/S0273-0979-09-01249-X","article-title":"Topology and data","volume":"46","author":"Carlsson","year":"2009","journal-title":"Am. Math. Soc."},{"key":"ref_57","unstructured":"M\u00fcllner, D., and Babu, A. (2018, September 10). Python Mapper: An Open-Source Toolchain for Data Exploration, Analysis and Visualization. Available online: http:\/\/danifold.net\/Mapper."},{"key":"ref_58","unstructured":"Saul, N., and van Veen, H.J. (2017). MLWave\/Kepler-Mapper: 186f (Version 1.0.1), Zenodo."},{"key":"ref_59","unstructured":"Pearson, P., Muellner, D., and Singh, G. (2015). TDAMapper: Analyze High-Dimensional Data Using Discrete Morse Theory, CRAN."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"362","DOI":"10.2307\/2528096","article-title":"A method for cluster analysis","volume":"21","author":"Edwards","year":"1965","journal-title":"Biometrics"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1002\/widm.53","article-title":"Algorithms for hierarchical clustering: An overview","volume":"2","author":"Murtagh","year":"2012","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"P10008","DOI":"10.1088\/1742-5468\/2008\/10\/P10008","article-title":"Fast unfolding of communities in large networks","volume":"2008","author":"Blondel","year":"2008","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_63","unstructured":"Sexton, H., and Kloke, J. (2018). Systems and Methods for Capture of Relationships Within Information. (10,042,959), U.S. Patent."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s001800000040","article-title":"Type S error rates for classical and Bayesian single and multiple comparison procedures","volume":"15","author":"Gelman","year":"2000","journal-title":"Comput. Stat."},{"key":"ref_65","unstructured":"Dodge, Y. (2009). The Concise Encyclopedia of Statistics, Springer."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Carlsson, L.S., Samuelsson, P.B., and J\u00f6nsson, P.G. (2020). Using Statistical Modeling to Predict the Electrical Energy Consumption of an Electric Arc Furnace Producing Stainless Steel. Metals, 10.","DOI":"10.3390\/met10010036"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Pratt, J., and Gibbons, J. (1981). Concepts of Nonparametric Theory, Springer.","DOI":"10.1007\/978-1-4612-5931-2"},{"key":"ref_68","unstructured":"LeCun, Y., Cortes, C., and Burges, C. (1999). The MNIST Dataset of Handwritten Digits(Images), NYU."},{"key":"ref_69","unstructured":"Mu, N., and Gilmer, J. (2019). MNIST-C: A robustness benchmark for computer vision. arXiv."},{"key":"ref_70","unstructured":"World Steel Association (2020, April 29). Steel Statistical Yearbook 2018. Available online: https:\/\/www.worldsteel.org\/steel-by-topic\/statistics\/steel-statistical-yearbook.html."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"6146","DOI":"10.1016\/j.energy.2011.07.050","article-title":"Influence of Direct Reduced Iron on the Energy Balance of the Electric Arc Furnace in Steel Industry","volume":"36","author":"Kirschen","year":"2011","journal-title":"Energy"},{"key":"ref_72","unstructured":"Sandberg, E. (2005). Energy and Scrap Optimisation of Electric Arc Furnaces by Statistical Analysis of Process Data. [Ph.D. Thesis, Lule\u00e5 University of Technology]."},{"key":"ref_73","unstructured":"Pfeifer, H., and Kirschen, M. (2002, January 26\u201329). Thermodynamic analysis of EAF electrical energy demand. Proceedings of the European Electric Steelmaking Conference, Venice, Italy."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1002\/srin.201300228","article-title":"Electric Arc Furnace Off-Gas Heat Recovery and Experience with a Testing Plant","volume":"85","author":"Steinparzer","year":"2014","journal-title":"Steel Res. Int."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"1800009","DOI":"10.1002\/srin.201800009","article-title":"Modeling, Simulation, and Validation with Measurements of a Heat Recovery Hot Gas Cooling Line for Electric Arc Furnaces","volume":"89","author":"Keplinger","year":"2018","journal-title":"Steel Res. Int."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Carlsson, L.S., Samuelsson, P.B., and J\u00f6nsson, P.G. (2019). Predicting the Electrical Energy Consumption of Electric Arc Furnaces Using Statistical Modeling. Metals, 9.","DOI":"10.3390\/met9090959"},{"key":"ref_77","unstructured":"Zeiler, M.D. (2012). ADADELTA: An Adaptive Learning Rate Method. arXiv."},{"key":"ref_78","unstructured":"Vejdemo-Johansson, M., Carlsson, G., and Carlsson, L. (2018). Supplementary Material for Fibres of Failure, Figshare."},{"key":"ref_79","unstructured":"Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv."},{"key":"ref_80","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_81","unstructured":"(2020, May 25). Anaconda Distribution for Python. Available online: https:\/\/www.anaconda.com\/products\/individual."},{"key":"ref_82","unstructured":"(2018, September 10). Ayasdi Python SDK Documentation Suite. Available online: https:\/\/platform.ayasdi.com\/sdkdocs\/."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/6\/150\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:41:54Z","timestamp":1760175714000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/13\/6\/150"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,23]]},"references-count":82,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["a13060150"],"URL":"https:\/\/doi.org\/10.3390\/a13060150","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2020,6,23]]}}}