{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T18:43:23Z","timestamp":1777401803756,"version":"3.51.4"},"reference-count":34,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,11,30]],"date-time":"2024-11-30T00:00:00Z","timestamp":1732924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>In this tutorial, we present a compact and holistic discussion of Deep Learning with a focus on Convolutional Neural Networks (CNNs) and supervised regression. While there are numerous books and articles on the individual topics we cover, comprehensive and detailed tutorials that address deep learning from a foundational yet rigorous and accessible perspective are rare. Most resources on CNNs are either too advanced, focusing on cutting-edge architectures, or too narrow, addressing only specific applications like image classification. This tutorial not only summarizes the most relevant concepts but also provides an in-depth exploration of each, offering a complete yet agile set of ideas. Moreover, we highlight the powerful synergy between learning theory, statistics, and machine learning, which together underpin the deep learning and CNN frameworks. We aim for this tutorial to serve as an optimal resource for students, professors, and anyone interested in understanding the foundations of deep learning.<\/jats:p>","DOI":"10.3390\/make6040132","type":"journal-article","created":{"date-parts":[[2024,12,5]],"date-time":"2024-12-05T09:30:18Z","timestamp":1733391018000},"page":"2753-2782","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep Learning with Convolutional Neural Networks: A Compact Holistic Tutorial with Focus on Supervised Regression"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1002-3815","authenticated-orcid":false,"given":"Yansel","family":"Gonzalez Tejeda","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence and Human Interfaces, Paris Lodron University of Salzburg, 5020 Salzburg, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2428-0962","authenticated-orcid":false,"given":"Helmut A.","family":"Mayer","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence and Human Interfaces, Paris Lodron University of Salzburg, 5020 Salzburg, Austria"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,30]]},"reference":[{"key":"ref_1","unstructured":"Rusell, S.J., and Norvig, P. (2010). Artificial Intelligence. A Modern Approach, Pearson Education, Inc."},{"key":"ref_2","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Shalev-Shwartz, S., and Ben-David, S. (2014). Understanding Machine Learning, Cambridge University Press.","DOI":"10.1017\/CBO9781107298019"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer. [2nd ed.].","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"McElreath, R. (2020). Statistical Rethinking: A Bayesian Course with Examples in R and Stan, Chapman and Hall\/CRC.","DOI":"10.1201\/9780429029608"},{"key":"ref_6","unstructured":"Moody, J., Hanson, S., and Lippmann, R. (1991, January 2\u20135). A Simple Weight Decay Can Improve Generalization. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF02478259","article-title":"A logical calculus of the ideas immanent in nervous activity","volume":"5","author":"McCulloch","year":"1943","journal-title":"Bull. Math. Biophys."},{"key":"ref_8","unstructured":"Hebb, D.O. (1949). The Organization of Behavior: A Neuropsychological Theory, Psychology Press."},{"key":"ref_9","unstructured":"Rosenblatt, F. (1957). The Perceptron\u2014A Perceiving and Recognizing Automaton, Cornell Aeronautical Laboratory. Technical Report 85-460-1."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. (October, January 29). What is the best multi-stage architecture for object recognition?. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.","DOI":"10.1109\/ICCV.2009.5459469"},{"key":"ref_11","unstructured":"Minsky, M., and Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry, MIT Press."},{"key":"ref_12","first-page":"6232","article-title":"The Expressive Power of Neural Networks: A View from the Width","volume":"30","author":"Lu","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/BF02551274","article-title":"Approximation by superpositions of a sigmoidal function","volume":"2","author":"Cybenko","year":"1989","journal-title":"Math. Control. Signals Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/S0021-9045(03)00078-9","article-title":"Approximation by neural networks with a bounded number of nodes at each level","volume":"122","author":"Gripenberg","year":"2003","journal-title":"J. Approx. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/S0925-2312(98)00111-8","article-title":"Lower bounds for approximation by MLP neural networks","volume":"25","author":"Maiorov","year":"1999","journal-title":"Neurocomputing"},{"key":"ref_16","unstructured":"Balcan, M.F., and Weinberger, K.Q. (2016, January 20\u201322). Train faster, generalize better: Stability of stochastic gradient descent. Proceedings of the the 33rd International Conference on Machine Learning, New York, NY, USA. Proceedings of Machine Learning Research."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A Stochastic Approximation Method","volume":"22","author":"Robbins","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Newton, D., Yousefian, F., and Pasupathy, R. (2018). Stochastic Gradient Descent: Recent Trends. Recent Advances in Optimization and Modeling of Contemporary Problems, INFORMS.","DOI":"10.1287\/educ.2018.0191"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e6","DOI":"10.23915\/distill.00006","article-title":"Why Momentum Really Works","volume":"2","author":"Goh","year":"2017","journal-title":"Distill"},{"key":"ref_20","unstructured":"Bishop, C.M. (2007). Pattern Recognition and Machine Learning (Information Science and Statistics), Springer. [1st ed.]."},{"key":"ref_21","unstructured":"Nielsen, M.A. (2015). Neural Networks and Deep Learning, Springer."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_23","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"Volume 9","author":"Teh","year":"2010","journal-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics"},{"key":"ref_24","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Automated Machine Learning\u2014Methods, Systems, Challenges, Springer.","DOI":"10.1007\/978-3-030-05318-5"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/BF00344251","article-title":"Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position","volume":"36","author":"Fukushima","year":"1980","journal-title":"Biol. Cybern."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez Tejeda, Y., and Mayer, H.A. (2021, January 5\u20137). A Neural Anthropometer Learning from Body Dimensions Computed on Human 3D Meshes. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Virtual.","DOI":"10.1109\/SSCI50451.2021.9660069"},{"key":"ref_28","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_29","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Aggarwal, C.C. (2018). Neural Networks and Deep Learning. A Text book., Springer International Publishing.","DOI":"10.1007\/978-3-319-94463-0"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bishop, C. (1995). Neural Networks for Pattern Recognition, Oxford University Press.","DOI":"10.1093\/oso\/9780198538493.001.0001"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"LeCun, Y.A., Bottou, L., Orr, G.B., and M\u00fcller, K.R. (2012). Efficient BackProp. Neural Networks: Tricks of the Trade, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-642-35289-8_3"},{"key":"ref_33","unstructured":"Bach, F., and Blei, D. (2015, January 7\u20139). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning, Lille, France. Proceedings of Machine Learning Research."},{"key":"ref_34","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018, January 3\u20138). How Does Batch Normalization Help Optimization?. Proceedings of the Advances in Neural Information Processing Systems, Montr\u00e9al, QC, Canada."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/4\/132\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:43:27Z","timestamp":1760114607000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/6\/4\/132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,30]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["make6040132"],"URL":"https:\/\/doi.org\/10.3390\/make6040132","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,30]]}}}