{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T00:10:03Z","timestamp":1755907803449,"version":"3.44.0"},"publisher-location":"New York, NY, USA","reference-count":14,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,12,27]],"date-time":"2023-12-27T00:00:00Z","timestamp":1703635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,12,27]]},"DOI":"10.1145\/3639479.3639514","type":"proceedings-article","created":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T07:55:51Z","timestamp":1709106951000},"page":"166-170","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["GPT Rotational Position Embedding for Length Extrapolation"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8662-6468","authenticated-orcid":false,"given":"Zhijie","family":"Qu","sequence":"first","affiliation":[{"name":"Beijing University of Technology, China"}]}],"member":"320","published-online":{"date-parts":[[2024,2,28]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_2_1_1_1","DOI":"10.1016\/j.neucom.2023.127063"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the International Conference on Learning Representations. ICLR. https:\/\/iclr.cc\/virtual\/2022\/poster\/6261","author":"Press Ofir","year":"2022","unstructured":"Ofir Press, Noah A. Smith, Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. In Proceedings of the International Conference on Learning Representations. ICLR. https:\/\/iclr.cc\/virtual\/2022\/poster\/6261"},{"unstructured":"Yutao Sun Li Dong Barun Patra Shuming Ma Shaohan Huang Alon Benhaim Vishrav Chaudhary Xia Song and Furu Wei. 2022. A length-extrapolatable transformer. ArXiv abs\/2212.10554(2022).","key":"e_1_3_2_1_3_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_4_1","DOI":"10.18653\/v1\/2023.acl-long.352"},{"key":"e_1_3_2_1_5_1","volume-title":"Nbce: Naive bayes-based context extension. https:\/\/kexue.fm\/archives\/9617","author":"Su Jianlin","year":"2023","unstructured":"Jianlin Su. 2023. Nbce: Naive bayes-based context extension. https:\/\/kexue.fm\/archives\/9617"},{"unstructured":"Chi Han Qifan Wang Wenhan Xiong Yu Chen Heng Ji Sinong Wang. 2023. LM-Infinite: Simple on-the-fly length generalization for large language models. ArXiv abs\/ 2308.16137(2023).","key":"e_1_3_2_1_6_1"},{"unstructured":"Guangxuan Xiao Yuandong Tian Beidi Chen Song Han Mike Lewis. 2023. Efficient Streaming Language Models with Attention Sinks. ArXiv abs\/2309.17453(2023).","key":"e_1_3_2_1_7_1"},{"unstructured":"Shouyuan Chen Sherman Wong Liangjian Chen and Yuandong Tian. 2023. Extending context window of large language models via positional interpolation. ArXiv abs\/2306.15595(2023).","key":"e_1_3_2_1_8_1"},{"key":"e_1_3_2_1_9_1","volume-title":"Giraffe: Adventures in expanding context lengths in llms. ArXiv abs\/2308.10882(2023).","author":"Pal Arka","year":"2023","unstructured":"Arka Pal, Deep Karkhanis, Manley Roberts, Samuel Dooley, Arvind Sundararajan, and Sid-dartha Naidu. 2023. Giraffe: Adventures in expanding context lengths in llms. ArXiv abs\/2308.10882(2023)."},{"unstructured":"LocalLLaMA. 2023. LocalLLaMA. Ntk-aware scaled rope allows llama models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation. URL https:\/\/www.reddit.com\/r\/LocalLLaMA\/comments\/14lz7j5\/ntkaware_scaled_rope_allows_llama_models_to_have.","key":"e_1_3_2_1_10_1"},{"unstructured":"LocalLLaMA. 2023. Dynamically scaled rope further increases performance of long context llama with zero fine-tuning. URL https:\/\/www.reddit.com\/r\/LocalLLaMA\/comments\/14mrgpr\/dynamically_scaled_rope_further_increases\/","key":"e_1_3_2_1_11_1"},{"key":"e_1_3_2_1_12_1","volume-title":"Code Llama: Open foundation models for code. ArXiv abs\/ 2308.12950.","author":"Rozi\u00e8re Baptiste","year":"2023","unstructured":"Baptiste Rozi\u00e8re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, 2023. Code Llama: Open foundation models for code. ArXiv abs\/ 2308.12950."},{"unstructured":"Wenhan Xiong Jingyu Liu Igor Molybog Hejia Zhang Prajjwal Bhargava Rui Hou Louis Martin Effective Long-Context Scaling of Foundation Models. ArXiv abs\/2309.16039.","key":"e_1_3_2_1_13_1"},{"key":"e_1_3_2_1_14_1","volume-title":"Proceedings of the 31st Conference on Neural Information Processing Systems. Red Hook, Curran Associates Inc, 6000-6010","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, N.S., Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,\u0141ukasz Kaiser. 2017. Attention Is All You Need. In Proceedings of the 31st Conference on Neural Information Processing Systems. Red Hook, Curran Associates Inc, 6000-6010. https:\/\/dl.acm.org\/doi\/10.5555\/3295222.3295349."}],"event":{"acronym":"MLNLP 2023","name":"MLNLP 2023: 2023 6th International Conference on Machine Learning and Natural Language Processing","location":"Sanya China"},"container-title":["Proceedings of the 2023 6th International Conference on Machine Learning and Natural Language Processing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639479.3639514","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639479.3639514","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T23:53:09Z","timestamp":1755906789000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639479.3639514"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,27]]},"references-count":14,"alternative-id":["10.1145\/3639479.3639514","10.1145\/3639479"],"URL":"https:\/\/doi.org\/10.1145\/3639479.3639514","relation":{},"subject":[],"published":{"date-parts":[[2023,12,27]]},"assertion":[{"value":"2024-02-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}