{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T05:43:02Z","timestamp":1777873382224,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":43,"publisher":"ACM","funder":[{"DOI":"10.13039\/501100006374","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["2348919"],"award-info":[{"award-number":["2348919"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,8,3]]},"DOI":"10.1145\/3711896.3737099","type":"proceedings-article","created":{"date-parts":[[2025,8,1]],"date-time":"2025-08-01T13:30:13Z","timestamp":1754055013000},"page":"451-462","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Rank It, Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Tasks"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-1687-8012","authenticated-orcid":false,"given":"Mohsen","family":"Dehghankar","sequence":"first","affiliation":[{"name":"University of Illinois Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5251-6186","authenticated-orcid":false,"given":"Abolfazl","family":"Asudeh","sequence":"additional","affiliation":[{"name":"University of Illinois Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,8,3]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang.","author":"Abdin Marah","year":"2024","unstructured":"Marah Abdin, Jyoti Aneja, Harkirat Behl, S\u00e9bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu, Cyril Zhang, and Yi Zhang. 2024. Phi-4 Technical Report. arXiv preprint arXiv:2412.08905(2024). https:\/\/arxiv.org\/abs\/2412.08905 Accessed: 2025-01-31."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.24432\/C5XW20"},{"key":"e_1_3_2_2_3_1","volume-title":"Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150(2020).","author":"Beltagy Iz","year":"2020","unstructured":"Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150(2020)."},{"key":"e_1_3_2_2_4_1","unstructured":"Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165(2020)."},{"key":"e_1_3_2_2_5_1","unstructured":"Howard Chen Ramakanth Pasunuru Jason Weston and Asli Celikyilmaz. 2023b. Walking down the memory maze: Beyond context limit through interactive reading. arXiv preprint arXiv:2310.05029(2023)."},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3522672"},{"key":"e_1_3_2_2_7_1","unstructured":"Shouyuan Chen Sherman Wong Liangjian Chen and Yuandong Tian. 2023c. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595(2023)."},{"key":"e_1_3_2_2_8_1","unstructured":"Rewon Child Scott Gray Alec Radford and Ilya Sutskever. 2019. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509(2019)."},{"key":"e_1_3_2_2_9_1","unstructured":"Mohsen Dehghankar and Abolfazl Asudeh. 2024. Rank It Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Tasks. arxiv:2412.00546 https:\/\/arxiv.org\/abs\/2412.00546"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"crossref","unstructured":"Daniel Deutsch Rotem Dror and Dan Roth. 2022. On the limitations of reference-free evaluations of generated text. arXiv preprint arXiv:2210.12563(2022).","DOI":"10.18653\/v1\/2022.emnlp-main.753"},{"key":"e_1_3_2_2_11_1","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783(2024)."},{"key":"e_1_3_2_2_12_1","volume-title":"15th IEEE International Conference on Knowledge Graphs(2024)","author":"Ebrahimi Sana","year":"2024","unstructured":"Sana Ebrahimi, Kaiwen Chen, Abolfazl Asudeh, Gautam Das, and Nick Koudas. 2024a. AXOLOTL: Fairness through Assisted Self-Debiasing of Large Language Model Outputs. 15th IEEE International Conference on Knowledge Graphs(2024)."},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-naacl.37"},{"key":"e_1_3_2_2_14_1","unstructured":"Paul Erdos Alfr\u00e9d R\u00e9nyi et al. 1960. On the evolution of random graphs. Publ. math. inst. hung. acad. sci Vol. 5 1 (1960) 17-60."},{"key":"e_1_3_2_2_15_1","unstructured":"Jianfei Gao Yangze Zhou Jincheng Zhou and Bruno Ribeiro. 2023. Double equivariance for inductive link prediction for both new nodes and new relation types. NeurIPS(2023)."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.736"},{"key":"e_1_3_2_2_17_1","first-page":"14982","article-title":"Found in the middle","volume":"2024","author":"Hsieh Cheng-Yu","year":"2024","unstructured":"Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, et al., 2024. Found in the middle: Calibrating Positional Attention Bias Improves Long Context Utilization. In Findings of the Association for Computational Linguistics ACL 2024. 14982-14995.","journal-title":"Calibrating Positional Attention Bias Improves Long Context Utilization. In Findings of the Association for Computational Linguistics ACL"},{"key":"e_1_3_2_2_18_1","volume-title":"Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.","author":"Jiang Albert Q","year":"2023","unstructured":"Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al., 2023. Mistral 7B. arXiv preprint arXiv:2310.06825(2023)."},{"key":"e_1_3_2_2_19_1","volume-title":"Open university learning analytics dataset. Scientific data","author":"Kuzilek Jakub","year":"2017","unstructured":"Jakub Kuzilek, Martin Hlosta, and Zdenek Zdrahal. 2017. Open university learning analytics dataset. Scientific data, Vol. 4, 1 (2017), 1-8."},{"key":"e_1_3_2_2_20_1","unstructured":"Rui Li Jianlin Su Chenxi Duan and Shunyi Zheng. 2020. Linear attention mechanism: An efficient attention for semantic segmentation. arXiv preprint arXiv:2007.14902(2020)."},{"key":"e_1_3_2_2_21_1","unstructured":"Yiming Lin Madelon Hulsebos Ruiying Ma Shreya Shankar Sepanta Zeigham Aditya G Parameswaran and Eugene Wu. 2024. Towards Accurate and Efficient Document Analytics with Large Language Models. arXiv preprint arXiv:2405.04674(2024)."},{"key":"e_1_3_2_2_22_1","volume-title":"Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano.","author":"Liu Chunwei","year":"2024","unstructured":"Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano. 2024b. A Declarative System for Optimizing AI Workloads. arXiv preprint arXiv:2405.14696(2024)."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00638"},{"key":"e_1_3_2_2_24_1","volume-title":"Chen Jie Yu, and Monica S Lam","author":"Liu Shicheng","year":"2023","unstructured":"Shicheng Liu, Jialiang Xu, Wesley Tjangnaka, Sina J Semnani, Chen Jie Yu, and Monica S Lam. 2023. SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models. arXiv preprint arXiv:2311.09818(2023)."},{"key":"e_1_3_2_2_25_1","unstructured":"Yusha Liu Yichong Xu Nihar B Shah and Aarti Singh. 2022. Integrating rankings into quantized scores in peer review. arXiv preprint arXiv:2204.03505(2022)."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_2_2_27_1","unstructured":"Arjun Panickssery Samuel R Bowman and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. arXiv preprint arXiv:2404.13076(2024)."},{"key":"e_1_3_2_2_28_1","volume-title":"Lotus: Enabling semantic queries with llms over tables of unstructured and structured data. arXiv preprint arXiv:2407.11418(2024).","author":"Patel Liana","year":"2024","unstructured":"Liana Patel, Siddharth Jha, Carlos Guestrin, and Matei Zaharia. 2024. Lotus: Enabling semantic queries with llms over tables of unstructured and structured data. arXiv preprint arXiv:2407.11418(2024)."},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","unstructured":"Erhard Rahm Hanna K\u00f6pcke and Andreas Thor. 2017. DBLP ACM Dataset. https:\/\/doi.org\/10.3886\/E100843V2","DOI":"10.3886\/E100843V2"},{"key":"e_1_3_2_2_30_1","unstructured":"Keita Saito Akifumi Wachi Koki Wataoka and Youhei Akimoto. 2023. Verbosity bias in preference labeling by large language models. arXiv preprint arXiv:2310.10076(2023)."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220088"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.2307\/2314570"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.2140\/pjm.1967.21.343"},{"key":"e_1_3_2_2_34_1","unstructured":"Rickard Stureborg Dimitris Alikaniotis and Yoshi Suhara. 2024. Large language models are inconsistent and biased evaluators. arXiv preprint arXiv:2405.01724(2024)."},{"key":"e_1_3_2_2_35_1","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Sun Zhiqing","year":"2024","unstructured":"Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2024. Principle-driven self-alignment of language models from scratch with minimal human supervision. Advances in Neural Information Processing Systems, Vol. 36 (2024)."},{"key":"e_1_3_2_2_36_1","volume-title":"Cassidy Hardin, Surya Bhupatiraju, L\u00e9onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram\u00e9, et al.","author":"Team Gemma","year":"2024","unstructured":"Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L\u00e9onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram\u00e9, et al., 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118(2024)."},{"key":"e_1_3_2_2_37_1","volume-title":"Aakanksha Chowdhery, and Denny Zhou.","author":"Wang Xuezhi","year":"2022","unstructured":"Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171(2022)."},{"key":"e_1_3_2_2_38_1","unstructured":"Koki Wataoka Tsubasa Takahashi and Ryokan Ri. 2024. Self-preference bias in llm-as-a-judge. arXiv preprint arXiv:2410.21819(2024)."},{"key":"e_1_3_2_2_39_1","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang Junzhe Wang Senjie Jin Enyu Zhou et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864(2023)."},{"key":"e_1_3_2_2_40_1","unstructured":"Sang Michael Xie Aditi Raghunathan Percy Liang and Tengyu Ma. 2021. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080(2021)."},{"key":"e_1_3_2_2_41_1","unstructured":"An Yang Baosong Yang Binyuan Hui Bo Zheng Bowen Yu Chang Zhou Chengpeng Li Chengyuan Li Dayiheng Liu Fei Huang et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671(2024)."},{"key":"e_1_3_2_2_42_1","volume-title":"Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al.","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al., 2020. Big bird: Transformers for longer sequences. Advances in neural information processing systems, Vol. 33 (2020), 17283-17297."},{"key":"e_1_3_2_2_43_1","unstructured":"Qihao Zhu Daya Guo Zhihong Shao Dejian Yang Peiyi Wang Runxin Xu Y Wu Yukun Li Huazuo Gao Shirong Ma et al. 2024. DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence. arXiv preprint arXiv:2406.11931(2024)."}],"event":{"name":"KDD '25: The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Toronto ON Canada","acronym":"KDD '25","sponsor":["SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGMOD ACM Special Interest Group on Management of Data"]},"container-title":["Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3711896.3737099","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T17:55:20Z","timestamp":1777571720000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3711896.3737099"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,3]]},"references-count":43,"alternative-id":["10.1145\/3711896.3737099","10.1145\/3711896"],"URL":"https:\/\/doi.org\/10.1145\/3711896.3737099","relation":{},"subject":[],"published":{"date-parts":[[2025,8,3]]},"assertion":[{"value":"2025-08-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}