{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T14:55:20Z","timestamp":1773413720945,"version":"3.50.1"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"23","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006360","name":"Bundesministerium f\u00fcr Wirtschaft und Energie","doi-asserted-by":"publisher","award":["ZF4483101ED7"],"award-info":[{"award-number":["ZF4483101ED7"]}],"id":[{"id":"10.13039\/501100006360","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008977","name":"Universit\u00e4t Ulm","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100008977","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Density estimation plays a crucial role in many data analysis tasks, as it infers a continuous probability density function (PDF) from discrete samples. Thus, it is used in tasks as diverse as analyzing population data, spatial locations in 2D sensor readings, or reconstructing scenes from 3D scans. In this paper, we introduce a learned, data-driven deep density estimation (DDE) to infer PDFs in an accurate and efficient manner, while being independent of domain dimensionality or sample size. Furthermore, we do not require access to the original PDF during estimation, neither in parametric form, nor as priors, or in the form of many samples. This is enabled by training an unstructured convolutional neural network on an infinite stream of synthetic PDFs, as unbound amounts of synthetic training data generalize better across a deck of natural PDFs than any natural finite training data will do. Thus, we hope that our publicly available DDE method will be beneficial in many areas of data analysis, where continuous models are to be estimated from discrete observations.<\/jats:p>","DOI":"10.1007\/s00521-021-06281-3","type":"journal-article","created":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T11:14:27Z","timestamp":1626866067000},"page":"16773-16807","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Data-driven deep density estimation"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0654-0433","authenticated-orcid":false,"given":"Patrik","family":"Puchert","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pedro","family":"Hermosilla","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tobias","family":"Ritschel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Timo","family":"Ropinski","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,7,21]]},"reference":[{"key":"6281_CR1","unstructured":"Attias H (2000) A variational Bayesian framework for graphical models. In: In advances in neural information processing systems, vol 12. MIT Press, pp 209\u2013215"},{"key":"6281_CR2","doi-asserted-by":"publisher","unstructured":"Baird L, Smalenberger D, Ingkiriwang S (2005) One-step neural network inversion with PDF learning and emulation. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol\u00a02. IEEE, Montreal, Quebec, Canada, pp 966\u2013971. https:\/\/doi.org\/10.1109\/IJCNN.2005.1555983","DOI":"10.1109\/IJCNN.2005.1555983"},{"key":"6281_CR3","doi-asserted-by":"publisher","first-page":"102053","DOI":"10.1016\/j.aquaeng.2020.102053","volume":"89","author":"A Banan","year":"2020","unstructured":"Banan A, Nasiri A, Taheri-Garavand A (2020) Deep learning-based appearance features extraction for automated carp species identification. Aquacult Eng 89:102053","journal-title":"Aquacult Eng"},{"issue":"1","key":"6281_CR4","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1214\/06-BA104","volume":"1","author":"DM Blei","year":"2006","unstructured":"Blei DM, Jordan MI et al (2006) Variational inference for dirichlet process mixtures. Bayesian Anal 1(1):121\u2013143","journal-title":"Bayesian Anal"},{"issue":"2","key":"6281_CR5","doi-asserted-by":"publisher","first-page":"353","DOI":"10.2307\/2336252","volume":"71","author":"AW Bowman","year":"1984","unstructured":"Bowman AW (1984) An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71(2):353\u2013360. https:\/\/doi.org\/10.2307\/2336252","journal-title":"Biometrika"},{"key":"6281_CR6","doi-asserted-by":"publisher","unstructured":"Caleb-Solly P, Gupta P, McClatchey R (2020) Tracking changes in user activity from unlabelled smart home sensor data using unsupervised learning methods. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-020-04737-6","DOI":"10.1007\/s00521-020-04737-6"},{"issue":"2","key":"6281_CR7","doi-asserted-by":"publisher","first-page":"807","DOI":"10.5705\/ss.2011.036a","volume":"21","author":"JE Chac\u00f3n","year":"2011","unstructured":"Chac\u00f3n JE, Duong T, Wand MP (2011) Asymptotics for general multivariate kernel density derivative estimators. Stat Sin 21(2):807. https:\/\/doi.org\/10.5705\/ss.2011.036a","journal-title":"Stat Sin"},{"issue":"1","key":"6281_CR8","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1109\/TIT.1968.1054098","volume":"14","author":"T Cover","year":"1968","unstructured":"Cover T (1968) Estimation by the nearest neighbor rule. IEEE Trans Inf Theory 14(1):50\u201355","journal-title":"IEEE Trans Inf Theory"},{"issue":"1","key":"6281_CR9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","volume":"39","author":"AP Dempster","year":"1977","unstructured":"Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Roy Stat Soc Ser B (Methodol) 39(1):1\u201322","journal-title":"J Roy Stat Soc Ser B (Methodol)"},{"key":"6281_CR10","volume-title":"Pattern classification and scene analysis","author":"RO Duda","year":"1973","unstructured":"Duda RO, Hart PE et al (1973) Pattern classification and scene analysis, vol 3. Wiley, New York"},{"issue":"3","key":"6281_CR11","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1111\/j.1467-9469.2005.00445.x","volume":"32","author":"T Duong","year":"2005","unstructured":"Duong T, Hazelton ML (2005) Cross-validation bandwidth matrices for multivariate kernel density estimation. Scand J Stat 32(3):485\u2013506. https:\/\/doi.org\/10.1111\/j.1467-9469.2005.00445.x","journal-title":"Scand J Stat"},{"key":"6281_CR12","doi-asserted-by":"publisher","first-page":"25111","DOI":"10.1109\/ACCESS.2020.2970836","volume":"8","author":"Y Fan","year":"2020","unstructured":"Fan Y, Xu K, Wu H, Zheng Y, Tao B (2020) Spatiotemporal modeling for nonlinear distributed thermal processes based on kl decomposition, MLP and LSTM network. IEEE Access 8:25111\u201325121","journal-title":"IEEE Access"},{"issue":"5","key":"6281_CR13","doi-asserted-by":"publisher","first-page":"e0196937","DOI":"10.1371\/journal.pone.0196937","volume":"13","author":"J Farmer","year":"2018","unstructured":"Farmer J, Jacobs D (2018) High throughput nonparametric probability density estimation. PLoS ONE 13(5):e0196937","journal-title":"PLoS ONE"},{"key":"6281_CR14","unstructured":"Germain M, Gregor K, Murray I, Larochelle H (2015) Made: masked autoencoder for distribution estimation. In: International conference on machine learning, pp 881\u2013889"},{"key":"6281_CR15","doi-asserted-by":"crossref","unstructured":"Ghosh S, Burnham KP, Laubscher NF, Dallal GE, Wilkinson L, Morrison DF, Loyer MW, Eisenberg B, Kullback S, Jolliffe IT, Simonoff JS (1987) Letters to the editor. Am Stat 41(4):338\u2013341","DOI":"10.1080\/00031305.1987.10475510"},{"key":"6281_CR16","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"issue":"4","key":"6281_CR17","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s10182-013-0216-y","volume":"97","author":"NB Heidenreich","year":"2013","unstructured":"Heidenreich NB, Schindler A, Sperlich S (2013) Bandwidth selection for kernel density estimation: a review of fully automatic selectors. AStA Adv Stat Anal 97(4):403\u2013433. https:\/\/doi.org\/10.1007\/s10182-013-0216-y","journal-title":"AStA Adv Stat Anal"},{"key":"6281_CR18","unstructured":"Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. PMLR, Lille, France, Proceedings of machine learning research, vol\u00a037, pp 448\u2013456. http:\/\/proceedings.mlr.press\/v37\/ioffe15.html"},{"key":"6281_CR19","doi-asserted-by":"publisher","unstructured":"Jarrett K, Kavukcuoglu K, Ranzato MA, LeCun Y (2009) What is the best multi-stage architecture for object recognition? In: 2009 IEEE 12th international conference on computer vision. IEEE, Kyoto, pp 2146\u20132153. https:\/\/doi.org\/10.1109\/ICCV.2009.5459469","DOI":"10.1109\/ICCV.2009.5459469"},{"issue":"4","key":"6281_CR20","doi-asserted-by":"publisher","first-page":"1919","DOI":"10.1214\/aos\/1176348378","volume":"19","author":"MC Jones","year":"1991","unstructured":"Jones MC, Marron JS, Park BU (1991) A simple root \\$n\\$ bandwidth selector. Ann Stat 19(4):1919\u20131932. https:\/\/doi.org\/10.1214\/aos\/1176348378","journal-title":"Ann Stat"},{"key":"6281_CR21","unstructured":"Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. CoRR abs\/1412.6980"},{"issue":"1","key":"6281_CR22","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1214\/aoms\/1177729694","volume":"22","author":"S Kullback","year":"1951","unstructured":"Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79\u201386","journal-title":"Ann Math Stat"},{"issue":"2","key":"6281_CR23","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/S0010-4655(00)00235-6","volume":"135","author":"A Likas","year":"2001","unstructured":"Likas A (2001) Probability density estimation using artificial neural networks. Comput Phys Commun 135(2):167\u2013175. https:\/\/doi.org\/10.1016\/S0010-4655(00)00235-6","journal-title":"Comput Phys Commun"},{"key":"6281_CR24","unstructured":"Magdon-Ismail M, Atiya AF (1998) Neural networks for density estimation. In: NIPS, p\u00a07"},{"issue":"3","key":"6281_CR25","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1109\/72.286931","volume":"5","author":"DS Modha","year":"1994","unstructured":"Modha DS, Fainman Y (1994) A learning law for density estimation. IEEE Trans Neural Netw 5(3):519\u2013523. https:\/\/doi.org\/10.1109\/72.286931","journal-title":"IEEE Trans Neural Netw"},{"key":"6281_CR26","unstructured":"Papamakarios G, Pavlakou T, Murray I (2017) Masked autoregressive flow for density estimation. In: Advances in neural information processing systems, pp 2338\u20132347"},{"issue":"3","key":"6281_CR27","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1214\/aoms\/1177704472","volume":"33","author":"E Parzen","year":"1962","unstructured":"Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065\u20131076. https:\/\/doi.org\/10.1214\/aoms\/1177704472","journal-title":"Ann Math Stat"},{"key":"6281_CR28","doi-asserted-by":"crossref","unstructured":"Rhodes AD, Quinn MH, Mitchell M (2017) Fast on-line kernel density estimation for active object localization. In: 2017 international joint conference on neural networks (IJCNN), pp 454\u2013462","DOI":"10.1109\/IJCNN.2017.7965889"},{"issue":"3","key":"6281_CR29","doi-asserted-by":"publisher","first-page":"832","DOI":"10.1214\/aoms\/1177728190","volume":"27","author":"M Rosenblatt","year":"1956","unstructured":"Rosenblatt M (1956) Remarks on some nonparametric estimates of a density function. Ann Math Stat 27(3):832\u2013837. https:\/\/doi.org\/10.1214\/aoms\/1177728190","journal-title":"Ann Math Stat"},{"issue":"2","key":"6281_CR30","first-page":"65","volume":"9","author":"M Rudemo","year":"1982","unstructured":"Rudemo M (1982) Empirical choice of histograms and kernel density estimators. Scand J Stat 9(2):65\u201378","journal-title":"Scand J Stat"},{"issue":"3","key":"6281_CR31","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211\u2013252. https:\/\/doi.org\/10.1007\/s11263-015-0816-y","journal-title":"Int J Comput Vis (IJCV)"},{"issue":"3","key":"6281_CR32","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1093\/biomet\/66.3.605","volume":"66","author":"DW Scott","year":"1979","unstructured":"Scott DW (1979) On optimal and data-based histograms. Biometrika 66(3):605\u2013610","journal-title":"Biometrika"},{"key":"6281_CR33","doi-asserted-by":"publisher","first-page":"164650","DOI":"10.1109\/ACCESS.2019.2951750","volume":"7","author":"S Shamshirband","year":"2019","unstructured":"Shamshirband S, Rabczuk T, Chau KW (2019) A survey of deep learning techniques: application in wind and solar energy resources. IEEE Access 7:164650\u2013164666","journal-title":"IEEE Access"},{"key":"6281_CR34","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3324-9","volume-title":"Density estimation for statistics and data analysis","author":"BW Silverman","year":"1986","unstructured":"Silverman BW (1986) Density estimation for statistics and data analysis. Chapman & Hall, London"},{"issue":"2","key":"6281_CR35","first-page":"97","volume":"9","author":"MP Wand","year":"1994","unstructured":"Wand MP, Jones C (1994) Multivariate plug-in bandwidth selection. Comput Stat 9(2):97\u2013116","journal-title":"Comput Stat"},{"issue":"3","key":"6281_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1117\/1.JMI.5.3.036501","volume":"5","author":"K Yan","year":"2018","unstructured":"Yan K, Wang X, Lu L, Summers RM (2018) DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J Med Imaging 5(3):1\u201311. https:\/\/doi.org\/10.1117\/1.JMI.5.3.036501","journal-title":"J Med Imaging"},{"issue":"2","key":"6281_CR37","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1016\/S0167-9473(97)00032-7","volume":"26","author":"J \u0106wik","year":"1997","unstructured":"\u0106wik J, Koronacki J (1997) A combined adaptive-mixtures\/plug-in estimator of multivariate probability densities. Comput Stat Data Anal 26(2):199\u2013218. https:\/\/doi.org\/10.1016\/S0167-9473(97)00032-7","journal-title":"Comput Stat Data Anal"},{"key":"6281_CR38","doi-asserted-by":"publisher","unstructured":"\u0141ukasik S (2007) Parallel computing of kernel density estimates with MPI. In: Shi Y, van Albada GD, Dongarra J, Sloot PMA (eds) Computational science\u2014ICCS 2007. ICCS 2007. Lecture notes in computer science, vol 4489. Springer, Berlin. https:\/\/doi.org\/10.1007\/978-3-540-72588-6_120","DOI":"10.1007\/978-3-540-72588-6_120"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-021-06281-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-021-06281-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-021-06281-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T18:05:58Z","timestamp":1725473158000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-021-06281-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":38,"journal-issue":{"issue":"23","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["6281"],"URL":"https:\/\/doi.org\/10.1007\/s00521-021-06281-3","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,21]]},"assertion":[{"value":"4 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 July 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The code for the proposed DDE method, including density estimation, model training and synthetic data generation along with the trained models is available onas well as in the python package deep_density_estimation.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}