{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T17:42:10Z","timestamp":1774374130070,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,8,20]],"date-time":"2022-08-20T00:00:00Z","timestamp":1660953600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,8,20]],"date-time":"2022-08-20T00:00:00Z","timestamp":1660953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010661","name":"EC | Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["826421"],"award-info":[{"award-number":["826421"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010661","name":"EC | Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["826421"],"award-info":[{"award-number":["826421"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["442326535"],"award-info":[{"award-number":["442326535"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["442326535"],"award-info":[{"award-number":["442326535"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Individual organizations, such as hospitals, pharmaceutical companies, and health insurance providers, are currently limited in their ability to collect data that are fully representative of a disease population. This can, in turn, negatively impact the generalization ability of statistical models and scientific insights. However, sharing data across different organizations is highly restricted by legal regulations. While federated data access concepts exist, they are technically and organizationally difficult to realize. An alternative approach would be to exchange synthetic patient data instead. In this work, we introduce the Multimodal Neural Ordinary Differential Equations (MultiNODEs), a hybrid, multimodal AI approach, which allows for generating highly realistic synthetic patient trajectories on a continuous time scale, hence enabling smooth interpolation and extrapolation of clinical studies. Our proposed method can integrate both static and longitudinal data, and implicitly handles missing values. We demonstrate the capabilities of MultiNODEs by applying them to real patient-level data from two independent clinical studies and simulated epidemiological data of an infectious disease.<\/jats:p>","DOI":"10.1038\/s41746-022-00666-x","type":"journal-article","created":{"date-parts":[[2022,8,19]],"date-time":"2022-08-19T23:24:48Z","timestamp":1660951488000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Generation of realistic synthetic data using Multimodal Neural Ordinary Differential Equations"],"prefix":"10.1038","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7170-0915","authenticated-orcid":false,"given":"Philipp","family":"Wendland","sequence":"first","affiliation":[]},{"given":"Colin","family":"Birkenbihl","sequence":"additional","affiliation":[]},{"given":"Marc","family":"Gomez-Freixa","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0862-3806","authenticated-orcid":false,"given":"Meemansa","family":"Sood","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3977-9982","authenticated-orcid":false,"given":"Maik","family":"Kschischo","sequence":"additional","affiliation":[]},{"given":"Holger","family":"Fr\u00f6hlich","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,8,20]]},"reference":[{"key":"666_CR1","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-018-1122-7","volume":"16","author":"H Fr\u00f6hlich","year":"2018","unstructured":"Fr\u00f6hlich, H. et al. From hype to reality: data science enabling personalized medicine. BMC Med. 16, 150 (2018).","journal-title":"BMC Med."},{"key":"666_CR2","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1002\/alz.12387","volume":"18","author":"C Birkenbihl","year":"2022","unstructured":"Birkenbihl, C., Salimi, Y. & Fr\u00f6hlich, H. Japanese Alzheimer's Disease Neuroimaging Initiative; Alzheimer's Disease Neuroimaging Initiative Unraveling the heterogeneity in Alzheimer\u2019s disease progression across multiple cohorts and the implications for data\u2010driven disease modeling. Alzheimers Dement. 18, 251\u2013261 (2022).","journal-title":"Alzheimers Dement."},{"key":"666_CR3","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1007\/s13167-020-00216-z","volume":"11","author":"C Birkenbihl","year":"2020","unstructured":"Birkenbihl, C. et al. Differences in cohort study data affect external validation of artificial intelligence models for predictive diagnostics of dementia \u2013 lessons for translation into clinical practice. EPMA J. 11, 367\u2013376 (2020).","journal-title":"EPMA J."},{"key":"666_CR4","doi-asserted-by":"publisher","first-page":"16","DOI":"10.3389\/fdata.2020.00016","volume":"3","author":"L Gootjes-Dreesbach","year":"2020","unstructured":"Gootjes-Dreesbach, L., Sood, M., Sahay, A., Hofmann-Apitius, M. & Fr\u00f6hlich, H. Variational Autoencoder Modular Bayesian Networks for simulation of heterogeneous clinical study data. Front. Big Data 3, 16 (2020).","journal-title":"Front. Big Data"},{"key":"666_CR5","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-67398-4","volume":"10","author":"M Sood","year":"2020","unstructured":"Sood, M. et al. Realistic simulation of virtual multi-scale, multi-modal patient trajectories using Bayesian networks and sparse auto-encoders. Sci. Rep. 10, 10971 (2020).","journal-title":"Sci. Rep."},{"key":"666_CR6","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1038\/s41551-021-00751-8","volume":"5","author":"RJ Chen","year":"2021","unstructured":"Chen, R. J., Lu, M. Y., Chen, T. Y., Williamson, D. F. K. & Mahmood, F. Synthetic data in machine learning for medicine and healthcare. Nat. Biomed. Eng. 5, 493\u2013497 (2021).","journal-title":"Nat. Biomed. Eng."},{"key":"666_CR7","doi-asserted-by":"publisher","first-page":"457","DOI":"10.2147\/CLEP.S242097","volume":"12","author":"K Thorlund","year":"2020","unstructured":"Thorlund, K., Dron, L., Park, J. J. & Mills, E. J. Synthetic and external controls in clinical trials \u2013 a primer for researchers. Clin. Epidemiol. 12, 457\u2013467 (2020).","journal-title":"Clin. Epidemiol."},{"key":"666_CR8","doi-asserted-by":"publisher","first-page":"3565","DOI":"10.1002\/mp.13617","volume":"46","author":"Y Lei","year":"2019","unstructured":"Lei, Y. et al. MRI\u2010only based synthetic CT generation using dense cycle consistent generative adversarial networks. Med. Phys. 46, 3565\u20133581 (2019).","journal-title":"Med. Phys."},{"key":"666_CR9","doi-asserted-by":"publisher","first-page":"1310","DOI":"10.1109\/TMI.2017.2785879","volume":"37","author":"G Yang","year":"2018","unstructured":"Yang, G. et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imaging 37, 1310\u20131321 (2018).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"666_CR10","doi-asserted-by":"publisher","unstructured":"Lin, Z., Jain, A., Wang, C., Fanti, G. & Sekar, V. Using GANs for sharing networked time series data: challenges, initial promise, and open questions. in Proceedings of the ACM Internet Measurement Conference 464\u2013483 (ACM, 2020). https:\/\/doi.org\/10.1145\/3419394.3423643.","DOI":"10.1145\/3419394.3423643"},{"key":"666_CR11","first-page":"563","volume":"25","author":"H Bae","year":"2020","unstructured":"Bae, H., Jung, D., Choi, H.-S. & Yoon, S. AnomiGAN: Generative Adversarial Networks for anonymizing private medical data. Pac. Symp. Biocomput. Pac. Symp. Biocomput. 25, 563\u2013574 (2020).","journal-title":"Pac. Symp. Biocomput. Pac. Symp. Biocomput."},{"key":"666_CR12","unstructured":"Jordon, J. & Yoon, J. PATE-GAN: generating synthetic data with differential privacy guarantees. in International Conference on Learning Representations 21 (2019)."},{"key":"666_CR13","doi-asserted-by":"publisher","first-page":"e005122","DOI":"10.1161\/CIRCOUTCOMES.118.005122","volume":"12","author":"BK Beaulieu-Jones","year":"2019","unstructured":"Beaulieu-Jones, B. K. et al. Privacy-preserving generative deep neural networks support clinical data sharing. Circ. Cardiovasc. Qual. Outcomes 12, e005122 (2019).","journal-title":"Circ. Cardiovasc. Qual. Outcomes"},{"key":"666_CR14","unstructured":"Chen, R. T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. Neural ordinary differential equations. in Advances in Neural Information Processing Systems (eds Bengio, S. et al.) vol. 31 (Curran Associates, Inc., 2018)."},{"key":"666_CR15","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1016\/j.pneurobio.2011.09.005","volume":"95","author":"K Marek","year":"2011","unstructured":"Marek, K. et al. The Parkinson Progression Marker Initiative (PPMI). Prog. Neurobiol. 95, 629\u2013635 (2011).","journal-title":"Prog. Neurobiol."},{"key":"666_CR16","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1097\/WAD.0000000000000279","volume":"32","author":"L Besser","year":"2018","unstructured":"Besser, L. et al. Version 3 of the National Alzheimer\u2019s Coordinating Center\u2019s Uniform Data Set. Alzheimer Dis. Assoc. Disord. 32, 351\u2013358 (2018).","journal-title":"Alzheimer Dis. Assoc. Disord."},{"key":"666_CR17","doi-asserted-by":"crossref","unstructured":"Nazabal, A., Olmos, P. M., Ghahramani, Z. & Valera, I. Handling incomplete heterogeneous data using VAEs. Preprint at ArXiv180703653 Cs Stat (2020).","DOI":"10.1016\/j.patcog.2020.107501"},{"key":"666_CR18","doi-asserted-by":"publisher","DOI":"10.1093\/gigascience\/giz134","volume":"8","author":"J de Jong","year":"2019","unstructured":"de Jong, J. et al. Deep learning for clustering of multivariate clinical patient trajectories with missing values. GigaScience 8, giz134 (2019).","journal-title":"GigaScience"},{"key":"666_CR19","unstructured":"Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at http:\/\/arxiv.org\/abs\/1312.6114 (2014)."},{"key":"666_CR20","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-020-00977-1","volume":"20","author":"A Goncalves","year":"2020","unstructured":"Goncalves, A. et al. Generation and evaluation of synthetic patient data. BMC Med. Res. Methodol. 20, 108 (2020).","journal-title":"BMC Med. Res. Methodol."},{"key":"666_CR21","doi-asserted-by":"publisher","first-page":"1071","DOI":"10.14778\/3231751.3231757","volume":"11","author":"N Park","year":"2018","unstructured":"Park, N. et al. Data synthesis based on generative adversarial networks. Proc. VLDB Endow. 11, 1071\u20131083 (2018).","journal-title":"Proc. VLDB Endow."},{"key":"666_CR22","doi-asserted-by":"publisher","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770\u2013778 (IEEE, 2016). https:\/\/doi.org\/10.1109\/CVPR.2016.90.","DOI":"10.1109\/CVPR.2016.90"},{"key":"666_CR23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput 9, 1735\u20131780 (1997).","journal-title":"Neural Comput"},{"key":"666_CR24","unstructured":"Dupont, E., Doucet, A. & Teh, Y. W. Augmented neural ODEs. in Advances in Neural Information Processing Systems (eds Wallach, H. et al.) vol. 32 (Curran Associates, Inc., 2019)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00666-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00666-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00666-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T03:16:27Z","timestamp":1669346187000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00666-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,20]]},"references-count":24,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["666"],"URL":"https:\/\/doi.org\/10.1038\/s41746-022-00666-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.09.26.21263968","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,20]]},"assertion":[{"value":"3 November 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"122"}}