{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T19:00:15Z","timestamp":1772132415258,"version":"3.50.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T00:00:00Z","timestamp":1772064000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T00:00:00Z","timestamp":1772064000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100011033","name":"Agencia Estatal de Investigaci\u00f3n","doi-asserted-by":"publisher","award":["PID2023-146569NB-C2"],"award-info":[{"award-number":["PID2023-146569NB-C2"]}],"id":[{"id":"10.13039\/501100011033","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004233","name":"Universitat Polit\u00e8cnica de Val\u00e8ncia","doi-asserted-by":"publisher","award":["PAID_06_24"],"award-info":[{"award-number":["PAID_06_24"]}],"id":[{"id":"10.13039\/501100004233","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003359","name":"Generalitat Valenciana","doi-asserted-by":"crossref","award":["CIPROM\/2022\/20"],"award-info":[{"award-number":["CIPROM\/2022\/20"]}],"id":[{"id":"10.13039\/501100003359","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004233","name":"Universitat Polit\u00e8cnica de Val\u00e8ncia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004233","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Supercomput"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Transformer-based models such as BERT and GPT2 have become the foundation of many modern applications, yet their execution requires substantial computational and memory resources. To address these challenges, recent advances in compiler technology and hardware accelerators have introduced new opportunities for performance portability. In this work, we evaluate JAX and TVM as high-level frameworks that combine a NumPy-like programming model with Just-In-Time (JIT) or Ahead-of-Time (AOT) code optimization and compilation, enabling efficient execution across CPUs or GPUs, and in the case of JAX, on TPUs as well. We present systematic implementations of the core Transformer encoder and decoder blocks in JAX and TVM and compare their automatically optimized code against NumPy and CuPy baselines. Our experimental study covers heterogeneous hardware platforms (AMD CPU, NVIDIA GPUs, and Google TPUs) and multiple arithmetic precisions (FP32, BF16, INT8, and INT32). Results show that JAX and TVM deliver significant performance improvements over standard libraries, while reducing the programming effort required to adapt to different hardware. These findings demonstrate the potential of JIT- and AOT-oriented frameworks to serve as a portable and efficient solution for deploying Transformer workloads in diverse computing environments.<\/jats:p>","DOI":"10.1007\/s11227-026-08327-6","type":"journal-article","created":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T18:11:50Z","timestamp":1772129510000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhancing transformer performance and portability through auto-tuning frameworks"],"prefix":"10.1007","volume":"82","author":[{"given":"Patricia","family":"Siwinska","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Lei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adri\u00e1n","family":"Castell\u00f3","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pedro","family":"Alonso-Jord\u00e1","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Enrique S.","family":"Quintana-Ort\u00ed","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,2,26]]},"reference":[{"issue":"9","key":"8327_CR1","doi-asserted-by":"publisher","first-page":"948","DOI":"10.1109\/TC.1972.5009071","volume":"C\u201321","author":"MJ Flynn","year":"1972","unstructured":"Flynn MJ (1972) Some computer organizations and their effectiveness. IEEE Trans Comput C\u201321(9):948\u2013960","journal-title":"IEEE Trans Comput"},{"issue":"13","key":"8327_CR2","doi-asserted-by":"publisher","first-page":"25 676","DOI":"10.1109\/JIOT.2025.3560382","volume":"12","author":"H Mart\u00ednez","year":"2025","unstructured":"Mart\u00ednez H, Catal\u00e1n S, Castell\u00f3 A, Mestre JI, Quintana-Ort\u00ed ES (2025) Latency-critical quantized inference with transformer decoders on arm and risc-v cpus. IEEE Internet Things J 12(13):25 676-25 690","journal-title":"IEEE Internet Things J"},{"issue":"6","key":"8327_CR3","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1177\/10943420251355115","volume":"39","author":"H Mart\u00ednez","year":"2025","unstructured":"Mart\u00ednez H, Catal\u00e1n S, Castell\u00f3 A, Quintana-Ort\u00ed ES (2025) Characterization of quantized inference with transformer encoders on low power CPUs. Int J High Perform Comput Appl 39(6):803\u2013821. https:\/\/doi.org\/10.1177\/10943420251355115","journal-title":"Int J High Perform Comput Appl"},{"issue":"4es","key":"8327_CR4","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1145\/242224.242477","volume":"28","author":"P Hudak","year":"1996","unstructured":"Hudak P (1996) Building domain-specific embedded languages. ACM Comput Surv 28(4es):196. https:\/\/doi.org\/10.1145\/242224.242477","journal-title":"ACM Comput Surv"},{"key":"8327_CR5","unstructured":"NVIDIA corporation, CUDA C programming guide, 2008. Retrieved from https:\/\/docs.nvidia.com\/cuda\/cuda-c-programming-guide\/"},{"key":"8327_CR6","unstructured":"\"Project home page for OpenCL: the open standard for parallel programming of heterogeneous systems. project home page,\" http:\/\/www.khronos.org\/opencl\/"},{"key":"8327_CR7","unstructured":"Chen T, Moreau T, Jiang Z, Shen H, Yan EQ, Wang L, Hu Y, Ceze L, Guestrin C, Krishnamurthy A (2018) \"TVM: end-to-end optimization stack for deep learning,\" CoRR, vol. abs\/1802.04799. [Online]. Available: http:\/\/arxiv.org\/abs\/1802.04799"},{"key":"8327_CR8","unstructured":"Bradbury J, Frostig R, Hawkins P, Johnson MJ, Leary C, Maclaurin D, Necula G, Paszke A, VanderPlas J, Wanderman-Milne S, Zhang Q (2018) JAX: composable transformations of Python+NumPy programs. [Online]. Available: http:\/\/github.com\/jax-ml\/jax"},{"issue":"2","key":"8327_CR9","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1145\/857076.857077","volume":"35","author":"J Aycock","year":"2003","unstructured":"Aycock J (2003) A brief history of just-in-time. ACM Comput Surv 35(2):97\u2013113. https:\/\/doi.org\/10.1145\/857076.857077","journal-title":"ACM Comput Surv"},{"key":"8327_CR10","unstructured":"Li M, Liu Y, Liu X, Sun Q, You X, Yang H, Luan Z, Qian D (2020) The deep learning compiler: A comprehensive survey. CoRR, vol. abs\/2002.03794. [Online]. Available: https:\/\/arxiv.org\/abs\/2002.03794"},{"key":"8327_CR11","doi-asserted-by":"publisher","unstructured":"Ragan-Kelley J, Barnes C, Adams A, Paris S, Durand F, Amarasinghe S (2013) Halide: aof the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, ser. PLDI \u201913. New York, NY, USA: Association for Computing Machinery, p. 519\u2013530. [Online]. Available: https:\/\/doi.org\/10.1145\/2491956.2462176","DOI":"10.1145\/2491956.2462176"},{"key":"8327_CR12","doi-asserted-by":"crossref","unstructured":"Ikarashi Y, Bernstein GL, Reinking A, Genc H, Ragan-Kelley J (2022) Exocompilation for productive programming of hardware accelerators. In: Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 703\u2013718","DOI":"10.1145\/3519939.3523446"},{"key":"8327_CR13","unstructured":"Moreau T, Chen T, Jiang Z, Ceze L, Guestrin C, Krishnamurthy A (2018) VTA: an open hardware-software stack for deep learning. CoRR, vol. abs\/1807.04188. [Online]. Available: http:\/\/arxiv.org\/abs\/1807.04188"},{"key":"8327_CR14","unstructured":"Research G (2020) Xla: optimizing compiler for machine learning. Accessed 16 June 2025. [Online]. Available: https:\/\/www.tensorflow.org\/xla"},{"key":"8327_CR15","unstructured":"Lattner C, Pienaar JA, Amini M, Bondhugula U, Riddle R, Cohen A, Shpeisman T, Davis A, Vasilache N, Zinenko O (2020) MLIR: a compiler infrastructure for the end of Moore\u2019s law. CoRR, vol. abs\/2002.11054. [Online]. Available: https:\/\/arxiv.org\/abs\/2002.11054"},{"key":"8327_CR16","doi-asserted-by":"publisher","unstructured":"Kang D, Kim E, Bae I, Egger B, Ha S (2018) C-good: C-code generation framework for optimized on-device deep learning. In: Proceedings of the International Conference on Computer-Aided Design, ser. ICCAD \u201918. New York, NY, USA: Association for Computing Machinery. [Online]. Available: https:\/\/doi.org\/10.1145\/3240765.3240786","DOI":"10.1145\/3240765.3240786"},{"key":"8327_CR17","unstructured":"Chen T, Moreau T, Jiang Z, Zheng L, Yan E, Shen H, Cowan M, Wang L, Hu Y, Ceze L, Guestrin C, Krishnamurthy A (2018) TVM: an automated End-to-End optimizing compiler for deep learning. In: 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). Carlsbad, CA: USENIX Association, pp. 578\u2013594. [Online]. Available: https:\/\/www.usenix.org\/conference\/osdi18\/presentation\/chen"},{"key":"8327_CR18","unstructured":"Zheng L, Jia C, Sun M, Wu Z, Yu CH, Haj-Ali A, Wang Y, Yang J, Zhuo D, Sen K, et al (2020) Ansor: generating $$\\{$$High-Performance$$\\}$$ tensor programs for deep learning. In: 14th USENIX Symposium on Operating Systems Design aAnd Implementation (OSDI 20), pp. 863\u2013879"},{"key":"8327_CR19","unstructured":"Chen T, Zheng L, Yan EQ, Jiang Z, Moreau T, Ceze L, Guestrin C, Krishnamurthy A (2018) Learning to optimize tensor programs. CoRR, vol. abs\/1805.08166. [Online]. Available: http:\/\/arxiv.org\/abs\/1805.08166"},{"key":"8327_CR20","unstructured":"Frostig R, Johnson M, Leary C (2018) Compiling machine learning programs via high-level tracing. [Online]. Available: https:\/\/mlsys.org\/Conferences\/doc\/2018\/146.pdf"},{"key":"8327_CR21","unstructured":"Heek J, Levskaya A, Oliver A, Ritter M, Rondepierre B, Steiner A, van Zee M (2020) Flax: a neural network library and ecosystem for JAX, 2020. URL http:\/\/github.com\/google\/flax, vol. 1"},{"key":"8327_CR22","unstructured":"Fedus W, Zoph B, ShazeerN (2022) Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. [Online]. Available: https:\/\/arxiv.org\/abs\/2101.03961"},{"issue":"1","key":"8327_CR23","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1007\/s11227-024-06720-7","volume":"81","author":"A Castell\u00f3","year":"2025","unstructured":"Castell\u00f3 A, Mart\u00ednez H, Catal\u00e1n S, Igual FD, Quintana-Ort\u00ed ES (2025) Experience-guided, mixed-precision matrix multiplication with apache TVM for ARM processors. J Supercomput 81(1):214","journal-title":"J Supercomput"},{"key":"8327_CR24","doi-asserted-by":"crossref","unstructured":"Castell\u00f3 A, Bellavita J, Dinh G, Ikarashi Y, Mart\u00ednez H (2024) Tackling the matrix multiplication micro-kernel generation with exo. In: IEEE\/ACM International Symposium on Code Generation and Optimization (CGO) 2024:182\u2013193","DOI":"10.1109\/CGO57630.2024.10444883"},{"issue":"1","key":"8327_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3638532","volume":"50","author":"G Alaejos","year":"2024","unstructured":"Alaejos G, Castell\u00f3 A, Alonso-Jord\u00e1 P, Igual FD, Mart\u00ednez H, Quintana-Ort\u00ed ES (2024) Algorithm 1039: automatic generators for a family of matrix multiplication routines with apache TVM. ACM Trans Math Softw 50(1):1\u201334. https:\/\/doi.org\/10.1145\/3638532","journal-title":"ACM Trans Math Softw"},{"key":"8327_CR26","unstructured":"Micikevicius P, Narang S, Alben J, Diamos G, Elsen E, Garcia D, Ginsburg B, Houston M, Kuchaiev O, Venkatesh G, Wu H (2018) Mixed precision training. [Online]. Available: https:\/\/arxiv.org\/abs\/1710.03740"},{"key":"8327_CR27","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2023) Attention is all you need. [Online]. Available: https:\/\/arxiv.org\/abs\/1706.03762"}],"container-title":["The Journal of Supercomputing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-026-08327-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11227-026-08327-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11227-026-08327-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T18:11:55Z","timestamp":1772129515000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11227-026-08327-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,26]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["8327"],"URL":"https:\/\/doi.org\/10.1007\/s11227-026-08327-6","relation":{},"ISSN":["1573-0484"],"issn-type":[{"value":"1573-0484","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,26]]},"assertion":[{"value":"13 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 February 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"206"}}