{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T15:51:28Z","timestamp":1768405888872,"version":"3.49.0"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643685489","type":"electronic"}],"license":[{"start":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T00:00:00Z","timestamp":1729036800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,10,16]]},"abstract":"<jats:p>Transformer with self-attention was initially crafted to model language sequences, where discrete tokens (i.e., words) showcase high semantic density. However, when applied to time series token inputs (i.e., datapoints) with weak-density semantics and temporal redundancy, it faces challenges as these time-domain tokens impede its ability to capture the intricate latent properties of time series dynamics. While time-frequency transformation presents a viable solution by bringing forth a new space with heightened expressive power, existing approaches fall short of fully exploiting its potential. In response to these limitations, we propose a general-purpose transformer-based model, named Scattering Transformer, for multivariate time series forecasting and self-supervised representation learning. It is based on two innovative components: i) scattering self-attention mechanism incorporating wavelet key\/value and standard query to unify the learning of cross-domain relationships between the time and wavelet domains; and ii) stochastic scaling positional encoding scheme that relies solely on order information, emulating longer sequence positions to generalize up to ultra-long horizon case. Extensive experiments on eight real-world benchmarks show the potential of our Scattering Transformer as a robust and versatile solution, showcasing its quadruple efficacy of non-stationary forecasting, ultra-long horizons forecasting, representation learning, and reduction in time and space complexity.<\/jats:p>","DOI":"10.3233\/faia240645","type":"book-chapter","created":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T12:59:53Z","timestamp":1729169993000},"source":"Crossref","is-referenced-by-count":1,"title":["Breaking the Weak Semantics Bottleneck of Transformers in Time Series Forecasting"],"prefix":"10.3233","author":[{"given":"Ziang","family":"Yang","sequence":"first","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China"},{"name":"School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Biyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuehai","family":"Tang","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruixuan","family":"Li","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Songlin","family":"Hu","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China"},{"name":"School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2024"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA240645","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T12:59:53Z","timestamp":1729169993000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA240645"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,16]]},"ISBN":["9781643685489"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia240645","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,16]]}}}