{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T17:50:28Z","timestamp":1774029028616,"version":"3.50.1"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T00:00:00Z","timestamp":1728864000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T00:00:00Z","timestamp":1728864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Hochschule f\u00fcr Angewandte Wissenschaften Hamburg (HAW Hamburg)"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Datenbank Spektrum"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Incomplete time series present a\u00a0significant challenge for downstream analysis. In the field of time series, Large Language Models are already being used for prediction, classification, and, in rare cases, imputation. This study thoroughly examines the imputation of time series using Large Language Models. Within a\u00a0defined experimental setup, current state-of-the-art time series imputation methods are compared with the performance of Large Language Models. Parameter-efficient fine-tuning methods are applied to adapt the Large Language Models to the imputation task. The results indicate that the models are suitable for time series imputation. The performance of these models depends on the number of parameters and the type of pre-training. Small specialized models, such as BERT, compete with models like Llama2 and outperform them on selected datasets. Furthermore, it becomes clear that the attention and feedforward network components of Large Language Models are particularly well-suited for adaptation to imputation, and parameter-efficient methods are also performance-enhancing.<\/jats:p>","DOI":"10.1007\/s13222-024-00488-z","type":"journal-article","created":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T17:02:47Z","timestamp":1728925367000},"page":"197-207","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Imputation Strategies in Time Series Based on Language Models"],"prefix":"10.1007","volume":"24","author":[{"given":"Michel","family":"Jacobsen","sequence":"first","affiliation":[]},{"given":"Marina","family":"Tropmann-Frick","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,10,14]]},"reference":[{"issue":"1","key":"488_CR1","doi-asserted-by":"publisher","first-page":"767","DOI":"10.32604\/cmc.2022.019369","volume":"70","author":"H Ahn","year":"2022","unstructured":"Ahn\u00a0H, Sun\u00a0K, Kim\u00a0PK (2022) Comparison of missing data imputation methods in time series forecasting. Comput Mater Continua 70(1):767\u2013779. https:\/\/doi.org\/10.32604\/cmc.2022.019369","journal-title":"Comput Mater Continua"},{"key":"488_CR2","volume-title":"The twelfth international conference on learning representations","author":"D Cao","year":"2024","unstructured":"Cao\u00a0D, Jia\u00a0F, Arik\u00a0SO et\u00a0al (2024) Tempo: prompt-based generative pre-trained transformer for time series forecasting. In: The twelfth international conference on learning representations"},{"key":"488_CR3","volume-title":"Llm4ts: aligning pre-trained llms as data-efficient time-series forecasters","author":"C Chang","year":"2023","unstructured":"Chang\u00a0C, Peng\u00a0WC, Chen\u00a0TF (2023) Llm4ts: aligning pre-trained llms as data-efficient time-series forecasters"},{"key":"488_CR4","volume-title":"Qlora: efficient finetuning of quantized llms","author":"T Dettmers","year":"2023","unstructured":"Dettmers\u00a0T, Pagnoni\u00a0A, Holtzman\u00a0A et\u00a0al (2023) Qlora: efficient finetuning of quantized llms"},{"key":"488_CR5","first-page":"4171","volume-title":"North American chapter of the association for computational linguistics","author":"J Devlin","year":"2019","unstructured":"Devlin\u00a0J, Chang\u00a0MW, Lee\u00a0K et\u00a0al (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: North American chapter of the association for computational linguistics. Association for Computational Linguistics, pp\u00a04171\u20134186"},{"key":"488_CR6","doi-asserted-by":"publisher","first-page":"119619","DOI":"10.1016\/j.eswa.2023.119619","volume":"219","author":"W Du","year":"2023","unstructured":"Du\u00a0W, C\u00f4t\u00e9\u00a0D, Liu\u00a0Y (2023) SAITS: Self-attention-based imputation for time series. Expert Syst Appl 219:119619. https:\/\/doi.org\/10.1016\/j.eswa.2023.119619","journal-title":"Expert Syst Appl"},{"key":"488_CR7","volume-title":"Thirty-seventh conference on neural information processing systems","author":"N Gruver","year":"2023","unstructured":"Gruver\u00a0N, Finzi\u00a0M, Qiu\u00a0S et\u00a0al (2023) Large language models are zero-shot time series forecasters. In: Thirty-seventh conference on neural information processing systems"},{"key":"488_CR8","volume-title":"International conference on learning representations","author":"EJ Hu","year":"2022","unstructured":"Hu\u00a0EJ, Shen\u00a0Y, Wallis\u00a0P et\u00a0al (2022) Lora: Low-rank adaptation of large language models. In: International conference on learning representations"},{"key":"488_CR9","doi-asserted-by":"publisher","first-page":"5254","DOI":"10.18653\/v1\/2023.emnlp-main.319","volume-title":"Proceedings of the 2023 conference on empirical methods in natural language processing","author":"Z Hu","year":"2023","unstructured":"Hu\u00a0Z, Wang\u00a0L, Lan\u00a0Y et\u00a0al (2023) Llm-adapters: an adapter family for parameter-efficient fine-tuning of large language models. In: Proceedings of the 2023 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp\u00a05254\u20135276 https:\/\/doi.org\/10.18653\/v1\/2023.emnlp-main.319"},{"key":"488_CR10","unstructured":"Javaheripi M, Bubeck S (2023) Phi-2: The surprising power of small language models. https:\/\/www.microsoft.com\/en-us\/research\/blog\/phi-2-the-surprising-power-of-small-language-models\/. Accessed 19 Feb 2024"},{"key":"488_CR11","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2307.03759","volume-title":"A\u00a0survey on graph neural networks for time series: forecasting, classification, imputation, and anomaly detection","author":"M Jin","year":"2023","unstructured":"Jin\u00a0M, Koh\u00a0HY, Wen\u00a0Q et\u00a0al (2023) A\u00a0survey on graph neural networks for time series: forecasting, classification, imputation, and anomaly detection https:\/\/doi.org\/10.48550\/ARXIV.2307.03759"},{"key":"488_CR12","doi-asserted-by":"publisher","first-page":"104440","DOI":"10.1016\/j.jbi.2023.104440","volume":"144","author":"M Kazijevs","year":"2023","unstructured":"Kazijevs\u00a0M, Samad\u00a0MD (2023) Deep imputation of missing values in time series health data: a\u00a0review with benchmarking. J\u00a0Biomed Inform 144:104440. https:\/\/doi.org\/10.1016\/j.jbi.2023.104440","journal-title":"J Biomed Inform"},{"key":"488_CR13","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","volume-title":"Proceedings of the 58th annual meeting of the association for computational linguistics","author":"M Lewis","year":"2020","unstructured":"Lewis\u00a0M, Liu\u00a0Y, Goyal\u00a0N et\u00a0al (2020) Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th annual meeting of the association for computational linguistics. Association for Computational Linguistics, pp\u00a07871\u20137880"},{"key":"488_CR14","unstructured":"Liu H, Tam D, Mohammed M et\u00a0al (2022) Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Adv Neural Inf Process Syst: 1950\u20131965"},{"key":"488_CR15","doi-asserted-by":"publisher","first-page":"63279","DOI":"10.1109\/access.2018.2877269","volume":"6","author":"MS Osman","year":"2018","unstructured":"Osman\u00a0MS, Abu-Mahfouz\u00a0AM, Page\u00a0PR (2018) A\u00a0survey on data imputation techniques: water distribution system as a\u00a0use case. IEEE Access 6:63279\u201363291. https:\/\/doi.org\/10.1109\/access.2018.2877269","journal-title":"IEEE Access"},{"key":"488_CR16","volume-title":"ICLR 2023 workshop on mathematical and empirical understanding of foundation models","author":"G Pu","year":"2023","unstructured":"Pu\u00a0G, Jain\u00a0A, Yin\u00a0J et\u00a0al (2023) Empirical analysis of the strengths and weaknesses of peft techniques for llms. In: ICLR 2023 workshop on mathematical and empirical understanding of foundation models"},{"key":"488_CR17","volume-title":"Language models are unsupervised multitask learners","author":"A Radford","year":"2019","unstructured":"Radford\u00a0A, Wu\u00a0J, Child\u00a0R et\u00a0al (2019) Language models are unsupervised multitask learners"},{"key":"488_CR18","unstructured":"Raffel C, Shazeer N, Roberts A et\u00a0al (2020) Exploring the limits of transfer learning with a\u00a0unified text-to-text transformer. J\u00a0Mach Learn Res 21(1)"},{"key":"488_CR19","volume-title":"Llama 2: open foundation and fine-tuned chat models","author":"H Touvron","year":"2023","unstructured":"Touvron\u00a0H, Martin\u00a0L, Stone\u00a0K et\u00a0al (2023) Llama 2: open foundation and fine-tuned chat models"},{"key":"488_CR20","first-page":"6000","volume-title":"Proceedings of the 31st international conference on neural information processing systems, NIPS\u201917","author":"A Vaswani","year":"2017","unstructured":"Vaswani\u00a0A, Shazeer\u00a0N, Parmar\u00a0N et\u00a0al (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, NIPS\u201917, vol 30. Curran Associates Inc, pp\u00a06000\u20136010"},{"key":"488_CR21","volume-title":"International conference on learning representations","author":"H Wu","year":"2023","unstructured":"Wu\u00a0H, Hu\u00a0T, Liu\u00a0Y et\u00a0al (2023) Timesnet: temporal 2d-variation modeling for general time series analysis. In: International conference on learning representations"},{"key":"488_CR22","volume-title":"The eleventh international conference on learning representations","author":"Q Zhang","year":"2023","unstructured":"Zhang\u00a0Q, Chen\u00a0M, Bukharin\u00a0A et\u00a0al (2023) Adaptive budget allocation for parameter-efficient fine-tuning. In: The eleventh international conference on learning representations"},{"key":"488_CR23","volume-title":"The twelfth international conference on learning representations. arxiv","author":"B Zhao","year":"2023","unstructured":"Zhao\u00a0B, Tu\u00a0H, Wei\u00a0C et\u00a0al (2023) Tuning layernorm in attention: towards efficient multi-modal llm finetuning. In: The twelfth international conference on learning representations. arxiv"},{"key":"488_CR24","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2303.18223","volume-title":"A\u00a0survey of large language models","author":"WX Zhao","year":"2023","unstructured":"Zhao\u00a0WX, Zhou\u00a0KLJ (2023) A\u00a0survey of large language models https:\/\/doi.org\/10.48550\/ARXIV.2303.18223"},{"key":"488_CR25","doi-asserted-by":"publisher","first-page":"11106","DOI":"10.48550\/ARXIV.2012.07436","volume-title":"The thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, virtual conference","author":"H Zhou","year":"2021","unstructured":"Zhou\u00a0H, Zhang\u00a0S, Peng\u00a0J et\u00a0al (2021) Informer: beyond efficient transformer for long sequence time-series forecasting. In: The thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, virtual conference, vol 35. AAAI Press, pp\u00a011106\u201311115 https:\/\/doi.org\/10.48550\/ARXIV.2012.07436"},{"key":"488_CR26","first-page":"43322","volume-title":"Advances in neural information processing systems","author":"T Zhou","year":"2023","unstructured":"Zhou\u00a0T, Niu\u00a0P, Wang\u00a0X et\u00a0al (2023) One fits all: power general time series analysis by pretrained lm. In: Advances in neural information processing systems, vol 36. Curran Associates, Inc, pp\u00a043322\u201343355"}],"container-title":["Datenbank-Spektrum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13222-024-00488-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13222-024-00488-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13222-024-00488-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T16:43:42Z","timestamp":1732034622000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13222-024-00488-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,14]]},"references-count":26,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["488"],"URL":"https:\/\/doi.org\/10.1007\/s13222-024-00488-z","relation":{},"ISSN":["1618-2162","1610-1995"],"issn-type":[{"value":"1618-2162","type":"print"},{"value":"1610-1995","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,14]]},"assertion":[{"value":"30 May 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 October 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}