{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T17:05:35Z","timestamp":1780765535400,"version":"3.54.1"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    Linearization of attention using various kernel approximation and kernel learning techniques has shown promise. Past methods used a subset of combinations of component functions and weight matrices within the random feature paradigm. We identify the need for a systematic comparison of different combinations of weight matrices and component functions for attention learning in Transformer. Hence, we introduce\n                    <jats:italic toggle=\"yes\">Spectraformer<\/jats:italic>\n                    , a unified framework for approximating and learning the kernel function in the attention mechanism of the Transformer. Our empirical results demonstrate, for the first time, that a random feature-based approach can achieve performance comparable to top-performing sparse and low-rank methods on the challenging Long-Range Arena benchmark. Thus, we establish a new state-of-the-art for random feature-based efficient Transformers. The framework also produces many variants that offer different advantages in accuracy, training time, and memory consumption. Our code is available at:\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/cruiseresearchgroup\/spectraformer\">https:\/\/github.com\/cruiseresearchgroup\/spectraformer<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3768161","type":"journal-article","created":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T18:43:57Z","timestamp":1758221037000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Spectraformer: A Unified Random Feature Framework for Transformer"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-4445-1534","authenticated-orcid":false,"given":"Duke","family":"Nguyen","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, University of New South Wales, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2345-0683","authenticated-orcid":false,"given":"Du","family":"Yin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of New South Wales, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2200-9703","authenticated-orcid":false,"given":"Aditya","family":"Joshi","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of New South Wales, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1237-1664","authenticated-orcid":false,"given":"Flora","family":"Salim","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of New South Wales, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,3,20]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.19"},{"issue":"120","key":"e_1_3_2_3_2","first-page":"1","article-title":"Quasi-Monte Carlo feature maps for shift-invariant kernels","volume":"17","author":"Avron Haim","year":"2016","unstructured":"Haim Avron, Vikas Sindhwani, Jiyan Yang, and Michael W. Mahoney. 2016. Quasi-Monte Carlo feature maps for shift-invariant kernels. Journal of Machine Learning Research 17, 120 (2016), 1\u201338. Retrieved from http:\/\/jmlr.org\/papers\/v17\/14-538.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_4_2","unstructured":"Iz Beltagy Matthew E. Peters and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:200405150. Retrieved from https:\/\/arxiv.org\/abs\/2004.05150"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.38"},{"key":"e_1_3_2_6_2","volume-title":"International Conference on Learning Representations","author":"Bullins Brian","year":"2018","unstructured":"Brian Bullins, Cyril Zhang, and Yi Zhang. 2018. Not-so-random features. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=Hk8XMWgRb"},{"key":"e_1_3_2_7_2","volume-title":"Advances in Neural Information Processing Systems","author":"Chen Yifan","year":"2021","unstructured":"Yifan Chen, Qi Zeng, Heng Ji, and Yun Yang. 2021. Skyformer: Remodel Self-Attention with Gaussian kernel and Nystr\u00f6m method. In Advances in Neural Information Processing Systems. A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (Eds.). Retrieved from https:\/\/openreview.net\/forum?id=pZCYG7gjkKz"},{"key":"e_1_3_2_8_2","first-page":"2502","volume-title":"33rd International Conference on Machine Learning","author":"Choromanski Krzysztof","year":"2016","unstructured":"Krzysztof Choromanski and Vikas Sindhwani. 2016. Recycling randomness with structure for sublinear time kernel expansions. In 33rd International Conference on Machine Learning. Maria Florina Balcan and Kilian Q. Weinberger (Eds.), Proceedings of Machine Learning Research, Vol. 48, PMLR, New York, New York, 2502\u20132510. Retrieved from https:\/\/proceedings.mlr.press\/v48\/choromanski16.html"},{"key":"e_1_3_2_9_2","volume-title":"International Conference on Learning Representations","author":"Marcin Choromanski Krzysztof","year":"2021","unstructured":"Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. 2021. Rethinking attention with performers. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=Ua6zuk0WRH"},{"key":"e_1_3_2_10_2","volume-title":"International Conference on Learning Representations","author":"Marcin Choromanski Krzysztof","year":"2022","unstructured":"Krzysztof Marcin Choromanski, Han Lin, Haoxian Chen, Arijit Sehanobish, Yuanzhe Ma, Deepali Jain, Jake Varley, Andy Zeng, Michael S. Ryoo, Valerii Likhosherstov, et al. 2022. Hybrid random features. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=EMigfE6ZeS"},{"key":"e_1_3_2_11_2","volume-title":"Advances in Neural Information Processing Systems","author":"Choromanski Krzysztof M.","year":"2017","unstructured":"Krzysztof M. Choromanski, Mark Rowland, and Adrian Weller. 2017. The unreasonable effectiveness of structured random orthogonal embeddings. In Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/bf8229696f7a3bb4700cfddef19fa23f-Paper.pdf"},{"key":"e_1_3_2_12_2","article-title":"Learning the transformer kernel","author":"Pal Chowdhury Sankalan","year":"2022","unstructured":"Sankalan Pal Chowdhury, Adamos Solomou, Kumar AvinavaDubey, and Mrinmaya Sachan. 2022. Learning the transformer kernel. Transactions on Machine Learning Research (2022). Retrieved from https:\/\/openreview.net\/forum?id=tLIBAEYjcv","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_13_2","volume-title":"Advances in Neural Information Processing Systems","author":"Dao Tri","year":"2017","unstructured":"Tri Dao, Christopher M. De Sa, and Christopher R\u00e9. 2017. Gaussian quadrature for kernel features. In Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/62f91ce9b820a491ee78c108636db089-Paper.pdf"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/WINCOM59760.2023.10322921"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832581.2832735"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3019893"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1133"},{"key":"e_1_3_2_18_2","first-page":"8330","volume-title":"39th International Conference on Machine Learning","author":"Han Insu","year":"2022","unstructured":"Insu Han, Amir Zandieh, and Haim Avron. 2022. Random Gegenbauer features for scalable kernel methods. In 39th International Conference on Machine Learning. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (Eds.), Proceedings of Machine Learning Research, Vol. 162, PMLR, 8330\u20138358. Retrieved from https:\/\/proceedings.mlr.press\/v162\/han22g.html"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3152247"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2025.3528347"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-024-06626-8"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1214\/009053607000000677"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1031"},{"key":"e_1_3_2_24_2","first-page":"566","volume-title":"31st International Conference on Machine Learning","author":"Hsieh Cho-Jui","year":"2014","unstructured":"Cho-Jui Hsieh, Si Si, and Inderjit Dhillon. 2014. A divide-and-conquer solver for kernel support vector machines. In 31st International Conference on Machine Learning. Eric P. Xing and Tony Jebara (Eds.), Proceedings of Machine Learning Research, Vol. 32, PMLR, Bejing, China, 566\u2013574. Retrieved from https:\/\/proceedings.mlr.press\/v32\/hsieha14.html"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00361"},{"key":"e_1_3_2_26_2","first-page":"5156","volume-title":"37th International Conference on Machine LearningProceedings of Machine Learning Research","author":"Katharopoulos Angelos","year":"2020","unstructured":"Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and Fran\u00e7ois Fleuret. 2020. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention. In 37th International Conference on Machine Learning. Hal Daum\u00e9 Aarti Singh (Eds.), Proceedings of Machine Learning Research, Vol. 119, PMLR, 5156\u20135165. Retrieved from https:\/\/proceedings.mlr.press\/v119\/katharopoulos20a.html"},{"key":"e_1_3_2_27_2","unstructured":"Junkyung Kim Drew Linsley Kalpit Thakkar and Thomas Serre. 2020. Disentangling neural mechanisms for perceptual grouping. In International Conference on Learning Representations."},{"key":"e_1_3_2_28_2","unstructured":"Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv:1312.6114. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:216078090"},{"key":"e_1_3_2_29_2","volume-title":"International Conference on Learning Representations","author":"Kitaev Nikita","year":"2020","unstructured":"Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The efficient transformer. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=rkgNKkHtvB"},{"key":"e_1_3_2_30_2","unstructured":"Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report Computer Science Department University of Toronto Toronto. Retrieved from https:\/\/archive.ics.uci.edu\/dataset\/691\/cifar+10"},{"key":"e_1_3_2_31_2","first-page":"244","volume-title":"30th International Conference on Machine Learning","author":"Le Quoc","year":"2013","unstructured":"Quoc Le, Tamas Sarlos, and Alexander Smola. 2013. Fastfood - Computing Hilbert space expansions in loglinear time. In 30th International Conference on Machine Learning. Sanjoy Dasgupta and David McAllester (Eds.), Proceedings of Machine Learning Research, Vol. 28, PMLR, Atlanta, Georgia, 244\u2013252. Retrieved from https:\/\/proceedings.mlr.press\/v28\/le13.html"},{"key":"e_1_3_2_32_2","first-page":"34559","volume-title":"Advances in Neural Information Processing Systems","author":"Likhosherstov Valerii","year":"2022","unstructured":"Valerii Likhosherstov, Krzysztof M. Choromanski, Kumar Avinava Dubey, Frederick Liu, Tamas Sarlos, and Adrian Weller. 2022. Chefs\u2019 random tables: Non-trigonometric random features. In Advances in Neural Information Processing Systems. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 34559\u201334573. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/file\/df2d62b96a4003203450cf89cd338bb7-Paper-Conference.pdf"},{"key":"e_1_3_2_33_2","volume-title":"37th Conference on Neural Information Processing Systems","author":"Likhosherstov Valerii","year":"2023","unstructured":"Valerii Likhosherstov, Krzysztof Marcin Choromanski, Kumar Avinava Dubey, Frederick Liu, Tamas Sarlos, and Adrian Weller. 2023. Dense-exponential random features: Sharp positive estimators of the Gaussian kernel. In 37th Conference on Neural Information Processing Systems. Retrieved from https:\/\/openreview.net\/forum?id=S0xrBMFihS"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2022.10.001"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.32470\/CCN.2018.1116-0"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5446"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3097011"},{"key":"e_1_3_2_38_2","first-page":"2256","volume-title":"34th International Conference on Machine Learning","author":"Lyu Yueming","year":"2017","unstructured":"Yueming Lyu. 2017. Spherical structured feature maps for kernel approximation. In 34th International Conference on Machine Learning. Doina Precup and Yee Whye Teh (Eds.), Proceedings of Machine Learning Research, Vol. 70, PMLR, 2256\u20132264. Retrieved from https:\/\/proceedings.mlr.press\/v70\/lyu17a.html"},{"key":"e_1_3_2_39_2","first-page":"6269","volume-title":"Advances in Neural Information Processing Systems","author":"Lyu Yueming","year":"2020","unstructured":"Yueming Lyu, Yuan Yuan, and Ivor Tsang. 2020. Subgroup-based rank-1 lattice Quasi-Monte Carlo. In Advances in Neural Information Processing Systems. H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 6269\u20136280. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/456048afb7253926e1fbb7486e699180-Paper.pdf"},{"key":"e_1_3_2_40_2","volume-title":"Advances in Neural Information Processing Systems","author":"Munkhoeva Marina","year":"2018","unstructured":"Marina Munkhoeva, Yermek Kapushev, Evgeny Burnaev, and Ivan Oseledets. 2018. Quadrature-based features for kernel approximation. In Advances in Neural Information Processing Systems. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2018\/file\/6e923226e43cd6fac7cfe1e13ad000ac-Paper.pdf"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-4013"},{"key":"e_1_3_2_42_2","first-page":"29319","volume-title":"Advances in Neural Information Processing Systems","author":"Nguyen Tan","year":"2022","unstructured":"Tan Nguyen, Minh Pham, Tam Nguyen, Khai Nguyen, Stanley Osher, and Nhat Ho. 2022. FourierFormer: Transformer meets generalized fourier integral theorem. In Advances in Neural Information Processing Systems. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 29319\u201329335. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/file\/bc968adbdff4a2551649d464b83f264a-Paper-Conference.pdf"},{"key":"e_1_3_2_43_2","first-page":"1078","volume-title":"19th International Conference on Artificial Intelligence and Statistics","author":"Oliva Junier B.","year":"2016","unstructured":"Junier B. Oliva, Avinava Dubey, Andrew G. Wilson, Barnabas Poczos, Jeff Schneider, and Eric P. Xing. 2016. Bayesian nonparametric kernel-learning. In 19th International Conference on Artificial Intelligence and Statistics. Arthur Gretton and Christian C. Robert (Eds.), Proceedings of Machine Learning Research, Vol. 51, PMLR, Cadiz, Spain, 1078\u20131086. Retrieved from https:\/\/proceedings.mlr.press\/v51\/oliva16.html"},{"key":"e_1_3_2_44_2","unstructured":"Zhen Qin Weixuan Sun Hui Deng Dongxu Li Yunshen Wei Baohong Lv Junjie Yan Lingpeng Kong and Yiran Zhong. 2022. cosFormer: Rethinking softmax in attention. arXiv:2202.08791. Retrieved from https:\/\/arxiv.org\/abs\/2202.08791"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-012-9211-2"},{"key":"e_1_3_2_46_2","volume-title":"Advances in Neural Information Processing Systems","author":"Rahimi Ali","year":"2007","unstructured":"Ali Rahimi and Benjamin Recht. 2007. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems. J. Platt, D. Koller, Y. Singer, and S. Roweis (Eds.), Vol. 20. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2007\/file\/013a006f03dbc5392effeb8f18fda755-Paper.pdf"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/3618408.3619608"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00353"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118165621.ch1"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/4175.003.0005"},{"key":"e_1_3_2_51_2","first-page":"2520","volume-title":"31st AAAI Conference on Artificial Intelligence (AAAI \u201917)","author":"Shen Weiwei","year":"2017","unstructured":"Weiwei Shen, Zhihui Yang, and Jun Wang. 2017. Random features for shift-invariant kernels with moment matching. In 31st AAAI Conference on Artificial Intelligence (AAAI \u201917). AAAI Press, 2520\u20132526."},{"key":"e_1_3_2_52_2","first-page":"911","volume-title":"17th International Conference on Machine Learning (ICML \u201900)","author":"Smola Alex J.","year":"2000","unstructured":"Alex J. Smola and Bernhard Sch\u00f6kopf. 2000. Sparse greedy matrix approximation for machine learning. In 17th International Conference on Machine Learning (ICML \u201900). Morgan Kaufmann Publishers Inc., San Francisco, CA, 911\u2013918."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17168"},{"key":"e_1_3_2_54_2","volume-title":"International Conference on Learning Representations","author":"Tay Yi","year":"2021","unstructured":"Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. 2021. Long range arena: A benchmark for efficient transformers. In International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=qVyeW-grC2k"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3530811"},{"key":"e_1_3_2_56_2","volume-title":"Advances in Neural Information Processing Systems","author":"Thomas Anna","year":"2018","unstructured":"Anna Thomas, Albert Gu, Tri Dao, Atri Rudra, and Christopher R\u00e9. 2018. Learning compressed transforms with low displacement rank. In Advances in Neural Information Processing Systems. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2018\/file\/8e621619d71d0ae5ef4e631ad586334f-Paper.pdf"},{"key":"e_1_3_2_57_2","first-page":"1427","volume-title":"22nd International Conference on Artificial Intelligence and Statistics","author":"Tompkins Anthony","year":"2019","unstructured":"Anthony Tompkins, Ransalu Senanayake, Philippe Morere, and Fabio Ramos. 2019. Black box quantiles for kernel learning. In 22nd International Conference on Artificial Intelligence and Statistics. Kamalika Chaudhuri and Masashi Sugiyama (Eds.), Proceedings of Machine Learning Research, Vol. 89, PMLR, 1427\u20131437. Retrieved from https:\/\/proceedings.mlr.press\/v89\/tompkins19a.html"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1137\/23M1620478"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1443"},{"key":"e_1_3_2_60_2","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141. Ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"e_1_3_2_61_2","unstructured":"Sinong Wang Belinda Z. Li Madian Khabsa Han Fang and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv:2006.04768. Retrieved from https:\/\/arxiv.org\/abs\/2006.04768"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-012-9369-4"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00813"},{"key":"e_1_3_2_64_2","unstructured":"Laura Weidinger John Mellor Maribeth Rauh Conor Griffin Jonathan Uesato Po-Sen Huang Myra Cheng Mia Glaese Borja Balle Atoosa Kasirzadeh et al. 2021. Ethical and social risks of harm from Language Models. arXiv:2112.04359. Retrieved from https:\/\/arxiv.org\/abs\/2112.04359"},{"key":"e_1_3_2_65_2","volume-title":"Advances in Neural Information Processing Systems","author":"Williams Christopher","year":"2000","unstructured":"Christopher Williams and Matthias Seeger. 2000. Using the Nystr\u00f6m method to speed up kernel machines. In Advances in Neural Information Processing Systems. T. Leen, T. Dietterich, and V. Tresp (Eds.), Vol. 13. MIT Press. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2000\/file\/19de10adbaa1b2ee13f77f679fa1483a-Paper.pdf"},{"key":"e_1_3_2_66_2","first-page":"1067","volume-title":"30th International Conference on Machine Learning","author":"Wilson Andrew","year":"2013","unstructured":"Andrew Wilson and Ryan Adams. 2013. Gaussian process kernels for pattern discovery and extrapolation. In 30th International Conference on Machine Learning. Sanjoy Dasgupta and David McAllester (Eds.), Proceedings of Machine Learning Research, Vol. 28, PMLR, Atlanta, Georgia, 1067\u20131075. Retrieved from https:\/\/proceedings.mlr.press\/v28\/wilson13.html"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9053878"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i16.17664"},{"key":"e_1_3_2_69_2","first-page":"1098","volume-title":"18th International Conference on Artificial Intelligence and Statistics","author":"Yang Zichao","year":"2015","unstructured":"Zichao Yang, Andrew Wilson, Alex Smola, and Le Song. 2015. A la Carte \u2013 Learning fast kernels. In 18th International Conference on Artificial Intelligence and Statistics. Guy Lebanon and S. V. N. Vishwanathan (Eds.), Proceedings of Machine Learning Research, Vol. 38, PMLR, San Diego, California, 1098\u20131106. Retrieved from https:\/\/proceedings.mlr.press\/v38\/yang15b.html"},{"key":"e_1_3_2_70_2","first-page":"2348","volume-title":"26th International Conference on Artificial Intelligence and Statistics","author":"Yao Junwen","year":"2023","unstructured":"Junwen Yao, N. Benjamin Erichson, and Miles E. Lopes. 2023. Error estimation for random Fourier features. In 26th International Conference on Artificial Intelligence and Statistics. Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent (Eds.), Proceedings of Machine Learning Research, Vol. 206, PMLR, 2348\u20132364. Retrieved from https:\/\/proceedings.mlr.press\/v206\/yao23a.html"},{"key":"e_1_3_2_71_2","unstructured":"Felix X. Yu Sanjiv Kumar Henry Rowley and Shih-Fu Chang. 2015. Compact nonlinear maps and circulant extensions. arXiv:1503.03893. Retrieved from https:\/\/arxiv.org\/abs\/1503.03893"},{"key":"e_1_3_2_72_2","volume-title":"Advances in Neural Information Processing Systems","author":"Yu Felix Xinnan X.","year":"2016","unstructured":"Felix Xinnan X. Yu, Ananda Theertha Suresh, Krzysztof M. Choromanski, Daniel N. Holtmann-Rice, and Sanjiv Kumar. 2016. Orthogonal random features. In Advances in Neural Information Processing Systems. D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2016\/file\/53adaf494dc89ef7196d73636eb2451b-Paper.pdf"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-024-02055-1"},{"key":"e_1_3_2_74_2","first-page":"427","volume-title":"Computer Vision (ECCV \u201924)","author":"Yuan Kaishen","year":"2025","unstructured":"Kaishen Yuan, Zitong Yu, Xin Liu, Weicheng Xie, Huanjing Yue, and Jingyu Yang. 2025. AUFormer: Vision transformers are parameter-efficient facial action unit detectors. In Computer Vision (ECCV \u201924). Ale\u0161 Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and G\u00fcl Varol (Eds.). Springer Nature Switzerland, Cham, 427\u2013445."},{"key":"e_1_3_2_75_2","first-page":"17283","volume-title":"Advances in Neural Information Processing Systems","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020. Big Bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems. H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 17283\u201317297. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/c8512d142a2d849725f31a9a7a361ab9-Paper.pdf"},{"key":"e_1_3_2_76_2","first-page":"592","volume-title":"26th Annual Conference on Learning Theory","author":"Zhang Yuchen","year":"2013","unstructured":"Yuchen Zhang, John Duchi, and Martin Wainwright. 2013. Divide and conquer kernel ridge regression. In 26th Annual Conference on Learning Theory. Shai Shalev-Shwartz and Ingo Steinwart (Eds.), Proceedings of Machine Learning Research, Vol. 30, PMLR, Princeton, NJ, 592\u2013617. Retrieved from https:\/\/proceedings.mlr.press\/v30\/Zhang13.html"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i12.17325"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3768161","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T16:27:56Z","timestamp":1774024076000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3768161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,20]]},"references-count":76,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3768161"],"URL":"https:\/\/doi.org\/10.1145\/3768161","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,20]]},"assertion":[{"value":"2025-02-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}