{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T08:52:16Z","timestamp":1791535936410,"version":"4.3.4"},"reference-count":43,"publisher":"Wiley","issue":"23-24","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["499552394"],"award-info":[{"award-number":["499552394"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Statistics in Medicine"],"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Modeling of longitudinal cohort data typically involves complex temporal dependencies between multiple variables. There, the transformer architecture, which has been highly successful in language and vision applications, allows us to account for the fact that the most recently observed time points in an individual's history may not always be the most important for the immediate future. This is achieved by assigning attention weights to observations of an individual based on a transformation of their values. One reason why these ideas have not yet been fully leveraged for longitudinal cohort data is that typically, large datasets are required. Therefore, we present a simplified transformer architecture that retains the core attention mechanism while reducing the number of parameters to be estimated, to be more suitable for small datasets with few time points. Guided by a statistical perspective on transformers, we use an autoregressive model as a starting point and incorporate attention as a kernel\u2010based operation with temporal decay, where aggregation of multiple transformer heads, that is, different candidate weighting schemes, is expressed as accumulating evidence on different types of underlying characteristics of individuals. This also enables a permutation\u2010based statistical testing procedure for identifying contextual patterns. In simulation studies, including a controlled simulation on real clinical predictor data, the approach is shown to recover contextual dependencies even with a small number of individuals and time points. In an application to data from a resilience study, we identify temporal patterns in the dynamics of stress and mental health. This indicates that properly adapted transformers can not only achieve competitive predictive performance, but also uncover complex context dependencies in small data settings.<\/jats:p>","DOI":"10.1002\/sim.70744","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T23:52:18Z","timestamp":1790207538000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Statistical Perspective on Transformers for Small Longitudinal Cohort Data"],"prefix":"10.1002","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0518-8597","authenticated-orcid":false,"given":"Kiana","family":"Farhadyar","sequence":"first","affiliation":[{"name":"Institute of Medical Biometry and Statistics (IMBI), University of Freiburg  Freiburg Germany"},{"name":"Freiburg Center for Data Analysis, Modeling and AI University of Freiburg  Freiburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maren","family":"Hackenberg","sequence":"additional","affiliation":[{"name":"Institute of Medical Biometry and Statistics (IMBI), University of Freiburg  Freiburg Germany"},{"name":"Freiburg Center for Data Analysis, Modeling and AI University of Freiburg  Freiburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kira","family":"Ahrens","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Psychosomatic Medicine and Psychotherapy University Hospital Frankfurt  Frankfurt Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charlotte","family":"Schenk","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Psychosomatic Medicine and Psychotherapy University Hospital Frankfurt  Frankfurt Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bianca","family":"Kollmann","sequence":"additional","affiliation":[{"name":"Department of Neuropsychology and Psychological Resilience Research Central Institute of Mental Health (ZI)  Mannheim Germany"},{"name":"Leibniz Institute for Resilience Research (LIR)  Mainz Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oliver","family":"T\u00fcscher","sequence":"additional","affiliation":[{"name":"Leibniz Institute for Resilience Research (LIR)  Mainz Germany"},{"name":"Department of Psychiatry and Psychotherapy Johannes Gutenberg University Medical Center Mainz  Mainz Germany"},{"name":"Department of Psychiatry, Psychotherapy and Psychosomatic Medicine University Medicine Halle, Martin\u2010Luther University Halle\u2010Wittenberg  Halle Germany"},{"name":"German Center for Mental Health (DZPG), Partner Site Halle\u2010Jena\u2010Magdeburg  Halle (Saale) Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klaus","family":"Lieb","sequence":"additional","affiliation":[{"name":"Leibniz Institute for Resilience Research (LIR)  Mainz Germany"},{"name":"Department of Psychiatry and Psychotherapy Johannes Gutenberg University Medical Center Mainz  Mainz Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael M.","family":"Plichta","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Psychosomatic Medicine and Psychotherapy University Hospital Frankfurt  Frankfurt Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Reif","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Psychosomatic Medicine and Psychotherapy University Hospital Frankfurt  Frankfurt Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raffael","family":"Kalisch","sequence":"additional","affiliation":[{"name":"Leibniz Institute for Resilience Research (LIR)  Mainz Germany"},{"name":"Neuroimaging Center (NIC), Focus Program Translational Neuroscience (FTN) Johannes Gutenberg University Medical Center  Mainz Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Wolkewitz","sequence":"additional","affiliation":[{"name":"Institute of Medical Biometry and Statistics (IMBI), University of Freiburg  Freiburg Germany"},{"name":"Freiburg Center for Data Analysis, Modeling and AI University of Freiburg  Freiburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Moritz","family":"Hess","sequence":"additional","affiliation":[{"name":"Institute of Medical Biometry and Statistics (IMBI), University of Freiburg  Freiburg Germany"},{"name":"Freiburg Center for Data Analysis, Modeling and AI University of Freiburg  Freiburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harald","family":"Binder","sequence":"additional","affiliation":[{"name":"Institute of Medical Biometry and Statistics (IMBI), University of Freiburg  Freiburg Germany"},{"name":"Freiburg Center for Data Analysis, Modeling and AI University of Freiburg  Freiburg Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"e_1_2_11_2_1","first-page":"6000","volume-title":"Advances in Neural Information Processing Systems, NIPS'17","author":"Vaswani A.","year":"2017"},{"key":"e_1_2_11_3_1","unstructured":"A.RadfordandK.Narasimhan \u201cImproving Language Understanding by Generative Pre\u2010Training \u201d 2018."},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2023\/759"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467\u2010023\u201043715\u2010z"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010020\u201062922\u2010y"},{"key":"e_1_2_11_7_1","unstructured":"A.Hassani S.Walton N.Shah A.Abuduweili J.Li andH.Shi \u201cEscaping the Big Data Paradigm With Compact Transformers \u201d 2022arXiv:2104.05704 [cs]."},{"key":"e_1_2_11_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2022.10.001"},{"key":"e_1_2_11_9_1","unstructured":"A.Zeng M.Chen L.Zhang andQ.Xu \u201cAre Transformers Effective for Time Series Forecasting? \u201d 2022arXiv:2205.13504 [cs]."},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-27752-1"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/07350015.1986.10509491"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/0033553053327452"},{"issue":"2","key":"e_1_2_11_13_1","first-page":"411","article-title":"Threshold Autoregressive Modelling in Continuous Time","volume":"1","author":"Tong H.","year":"1991","journal-title":"Statistica Sinica"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-51684-9_2"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.2307\/2669861"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.2307\/2669476"},{"key":"e_1_2_11_17_1","unstructured":"Z. C.Lipton J.Berkowitz andC.Elkan \u201cA Critical Review of Recurrent Neural Networks for Sequence Learning \u201d 2015arXiv:1506.00019 [cs]."},{"key":"e_1_2_11_18_1","doi-asserted-by":"crossref","unstructured":"G.Lai W. C.Chang Y.Yang andH.Liu \u201cModeling Long\u2010 and Short\u2010Term Temporal Patterns With Deep Neural Networks \u201d 2018arXiv:1703.07015 [cs].","DOI":"10.1145\/3209978.3210006"},{"key":"e_1_2_11_19_1","first-page":"40848","volume-title":"Proceedings of the 42nd International Conference on Machine Learning, ICML'25","author":"Lu J.","year":"2025"},{"key":"e_1_2_11_20_1","first-page":"937","volume-title":"Proceedings of the 28th International Conference on Artificial Intelligence and Statistics","author":"Cui Y.","year":"2025"},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.17"},{"key":"e_1_2_11_22_1","unstructured":"S.JainandB. C.Wallace \u201cAttention is Not Explanation \u201d 2019arXiv:1902.10186 [cs]."},{"issue":"170","key":"e_1_2_11_23_1","first-page":"1","article-title":"Scalable and Efficient Hypothesis Testing With Random Forests","volume":"23","author":"Coleman T.","year":"2022","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_11_24_1","unstructured":"D.Miwa T.Shiraishi V. N. L.Duy T.Katsuoka andI.Takeuchi \u201cStatistical Test for Anomaly Detections by Variational Auto\u2010Encoders \u201d 2024arXiv:2402.03724 [stat]."},{"key":"e_1_2_11_25_1","first-page":"45079","volume-title":"Proceedings of the 41st International Conference on Machine Learning, ICML'24","author":"Shiraishi T.","year":"2025"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-7908-2604-3_16"},{"key":"e_1_2_11_27_1","unstructured":"M.Hackenberg M.Grodd C.Kreutz et al. \u201cUsing Differentiable Programming for Flexible Statistical Modeling \u201d 2020ADS Bibcode: 2020arXiv201205722H."},{"key":"e_1_2_11_28_1","first-page":"15487","volume-title":"Advances in Neural Information Processing Systems, NIPS'20","author":"Barcel\u00f3 P.","year":"2020"},{"key":"e_1_2_11_29_1","unstructured":"P.Shaw J.Uszkoreit andA.Vaswani \u201cSelf\u2010Attention With Relative Position Representations \u201d 2018arXiv:1803.02155."},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/met0000211"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1037\/pag0000667"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2022.104880"},{"key":"e_1_2_11_33_1","first-page":"4768","volume-title":"Advances in Neural Information Processing Systems NIPS'17","author":"Lundberg S. M.","year":"2017"},{"key":"e_1_2_11_34_1","first-page":"3319","volume-title":"Proceedings of the 34th International Conference on Machine Learning, ICML'17","author":"Sundararajan M.","year":"2017"},{"key":"e_1_2_11_35_1","first-page":"2907","volume-title":"Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics","author":"Janzing D.","year":"2020"},{"key":"e_1_2_11_36_1","volume-title":"International Conference on Learning Representations","author":"Liu Y.","year":"2024"},{"key":"e_1_2_11_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.127063"},{"key":"e_1_2_11_38_1","doi-asserted-by":"publisher","DOI":"10.1002\/hep.1840200120"},{"key":"e_1_2_11_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00406\u2010020\u201001159\u20102"},{"key":"e_1_2_11_40_1","doi-asserted-by":"publisher","DOI":"10.2196\/14566"},{"key":"e_1_2_11_41_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0601674103"},{"key":"e_1_2_11_42_1","doi-asserted-by":"publisher","DOI":"10.1017\/s0033291796004242"},{"key":"e_1_2_11_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jenvp.2016.03.003"},{"key":"e_1_2_11_44_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.02165"}],"container-title":["Statistics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.70744","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/sim.70744","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/sim.70744","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T08:32:36Z","timestamp":1791534756000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/sim.70744"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":43,"journal-issue":{"issue":"23-24","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/sim.70744"],"URL":"https:\/\/doi.org\/10.1002\/sim.70744","archive":["Portico"],"relation":{},"ISSN":["0277-6715","1097-0258"],"issn-type":[{"value":"0277-6715","type":"print"},{"value":"1097-0258","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]},"assertion":[{"value":"2026-02-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70744"}}