{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T16:16:28Z","timestamp":1778948188886,"version":"3.51.4"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"name":"National Natural Science Foundation of China Youth Project","award":["62406162"],"award-info":[{"award-number":["62406162"]}]},{"name":"Beijing Natural Science Foundation","award":["L243006"],"award-info":[{"award-number":["L243006"]}]},{"name":"Institute for Guo Qiang, Tsinghua University","award":["2019GQB0003"],"award-info":[{"award-number":["2019GQB0003"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2026,5,31]]},"abstract":"<jats:p>\n                    Future event prediction (FEP) is a long-standing and crucial task in the world, as understanding the evolution of events enables early risk identification, informed decision-making, and strategic planning. Existing work typically treats event prediction as classification tasks and confines the outcomes of future events to a fixed scope, such as yes\/no questions, candidate set, and taxonomy, which is difficult to include all possible outcomes of future events. In this article, we introduce\n                    <jats:italic toggle=\"yes\">OpenEP<\/jats:italic>\n                    (an\n                    <jats:italic toggle=\"yes\">Open<\/jats:italic>\n                    -Ended F\n                    <jats:italic toggle=\"yes\">EP<\/jats:italic>\n                    task), which generates flexible and diverse predictions aligned with real-world scenarios. This is mainly reflected in two aspects: firstly, the predictive questions are diverse, covering different stages of event development and perspectives; secondly, the outcomes are flexible, without constraints on scope or format. To facilitate the study of this task, we construct\n                    <jats:italic toggle=\"yes\">OpenEPBench<\/jats:italic>\n                    , a dynamic dataset where events are annotated on the same day they occur. For question construction, we pose questions from seven perspectives, including time, location, event development, event outcome, event impact, event response, and other, to facilitate an in-depth analysis and understanding of the comprehensive evolution of events. For outcome construction, we collect free-form text containing the outcomes as ground truth to provide semantically complete and detail-enriched outcomes. Furthermore, we propose\n                    <jats:italic toggle=\"yes\">StkFEP<\/jats:italic>\n                    , a stakeholder-enhanced FEP framework, that incorporates event characteristics for open-ended settings. Our method extracts stakeholders involved in events to extend questions to gather diverse information. We also collect both contextually relevant events and historically analogous events to reveal potential evolutionary patterns. Experimental results indicate that accurately predicting future events in open-ended settings is challenging for existing LLMs. In addition, we thoroughly summarize the problems encountered in prediction, hoping to provide insights for future research. Code and data are available at:\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/ncepu-eai\/OpenEP\">https:\/\/github.com\/ncepu-eai\/OpenEP<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3802922","type":"journal-article","created":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T14:24:02Z","timestamp":1774448642000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["OpenEP: Open-Ended Future Event Prediction"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9044-2595","authenticated-orcid":false,"given":"Yong","family":"Guan","sequence":"first","affiliation":[{"name":"North China Electric Power University, Beijing, China and Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-7192-5790","authenticated-orcid":false,"given":"Hao","family":"Peng","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5727-143X","authenticated-orcid":false,"given":"Xiaozhi","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8907-3526","authenticated-orcid":false,"given":"Lei","family":"Hou","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6244-0664","authenticated-orcid":false,"given":"Juanzi","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,5,11]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compchemeng.2023.108502"},{"issue":"1","key":"e_1_3_3_3_2","first-page":"3249","article-title":"Spearman\u2019s correlation coefficient in statistical analysis","volume":"13","author":"Al-Hameed Khawla Ali Abd","year":"2022","unstructured":"Khawla Ali Abd Al-Hameed. 2022. Spearman\u2019s correlation coefficient in statistical analysis. International Journal of Nonlinear Analysis and Applications 13, 1 (2022), 3249\u20133255.","journal-title":"International Journal of Nonlinear Analysis and Applications"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-33-4389-4_8"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i11.26478"},{"key":"e_1_3_3_6_2","unstructured":"He Chang Chenchen Ye Zhulin Tao Jie Wu Zhengmao Yang Yunshan Ma Xianglin Huang and Tat-Seng Chua. 2024. A comprehensive evaluation of large language models on temporal event forecasting. arXiv:2407.11638. Retrieved from https:\/\/arxiv.org\/abs\/2407.11638"},{"key":"e_1_3_3_7_2","volume-title":"Proceedings of the ICLR","author":"Cheng Junyan","year":"2024","unstructured":"Junyan Cheng and Peter Chin. 2024. SocioDojo: Building lifelong analytical agents with real-world text and time series. In Proceedings of the ICLR."},{"key":"e_1_3_3_8_2","first-page":"325","volume-title":"Utilizing AI and Machine Learning in Financial Analysis","author":"Chhikara Harshdeep","year":"2025","unstructured":"Harshdeep Chhikara, Sumit Chhikara, and Lovelesh Gupta. 2025. Predictive analytics in finance: Leveraging AI and machine learning for investment strategies. In Utilizing AI and Machine Learning in Financial Analysis. Dina Darwish and Sanjeev Kumar (Eds.), IGI Global Scientific Publishing, 325\u2013336."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clsr.2023.105902"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajic.2024.09.012"},{"key":"e_1_3_3_11_2","first-page":"970","volume-title":"Proceedings of the ICML (Proceedings of Machine Learning Research)","volume":"70","author":"Dempsey Walter H.","year":"2017","unstructured":"Walter H. Dempsey, Alexander Moreno, Christy K. Scott, Michael L. Dennis, David H. Gustafson, Susan A. Murphy, and James M. Rehg. 2017. iSurvive: An interpretable, event-time prediction model for mHealth. In Proceedings of the ICML (Proceedings of Machine Learning Research), Vol. 70, 970\u2013979."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671466"},{"key":"e_1_3_3_13_2","unstructured":"Songgaojun Deng Huzefa Rangwala and Yue Ning. 2021. Causal knowledge guided societal event forecasting. arXiv:2112.05695. Retrieved from https:\/\/arxiv.org\/abs\/2112.05695"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM54844.2022.00019"},{"key":"e_1_3_3_15_2","unstructured":"Aohan Zeng Bin Xu Bowen Wang Chenhui Zhang Da Yin Diego Rojas Guanyu Feng Hanlin Zhao Hanyu Lai Hao Yu et al. 2024. ChatGLM: A family of large language models from GLM-130B to GLM-4 All Tools. arXiv:2406.12793. Retrieved from https:\/\/arxiv.org\/abs\/2406.12793"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589335.3651500"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.63317\/3bm5ymo5bbgx"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.637"},{"key":"e_1_3_3_19_2","unstructured":"Muhammad Usman Hadi Rizwan Qureshi Abbas Shah Muhammad Irfan Anas Zafar Muhammad Bilal Shaikh Naveed Akhtar Jia Wu and Seyedali Mirjalili. 2023. Large language models: A comprehensive survey of its applications challenges limitations and future prospects. Authorea Preprints 1 (2023) 1\u201326."},{"key":"e_1_3_3_20_2","doi-asserted-by":"crossref","unstructured":"Danny Halawi Fred Zhang Chen Yueh-Han and Jacob Steinhardt. 2024. Approaching human-level forecasting with language models. arXiv:2402.18563. Retrieved from https:\/\/arxiv.org\/abs\/2402.18563","DOI":"10.52202\/079017-1598"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM54844.2022.00027"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.3390\/fi16100365"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1093"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.futured-1.5"},{"key":"e_1_3_3_25_2","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier et al. 2023. Mistral 7B. arXiv:2310.06825. Retrieved from https:\/\/arxiv.org\/abs\/2310.06825"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.541"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3635059.3635104"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657901"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772751"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401947"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681593"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.422"},{"key":"e_1_3_3_33_2","unstructured":"Tianle Li Ge Zhang Quy Duc Do Xiang Yue and Wenhu Chen. 2024. Long-context LLMs struggle with long in-context learning. arXiv:2404.02060. Retrieved from https:\/\/arxiv.org\/abs\/2404.02060"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462963"},{"key":"e_1_3_3_35_2","first-page":"74","volume-title":"Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out, 74\u201381."},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2022.954559"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599285"},{"key":"e_1_3_3_38_2","unstructured":"Andrea Matarazzo and Riccardo Torlone. 2025. A survey on large language models with some insights on their capabilities and limitations. arXiv:2501.04040. Retrieved from https:\/\/arxiv.org\/abs\/2501.04040"},{"key":"e_1_3_3_39_2","unstructured":"Yaswanth Narsupalli Abhranil Chandra Sreevatsa Muppirala Manish Gupta and Pawan Goyal. 2024. Review-feedback-reason (ReFeR): A novel framework for NLG evaluation and reasoning. arXiv:2407.12877. Retrieved from https:\/\/arxiv.org\/abs\/2407.12877"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9639.1981.tb00422.x"},{"key":"e_1_3_3_41_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_3_42_2","unstructured":"Sarah Pratt Seth Blumberg Pietro Kreitlon Carolino and Meredith Ringel Morris. 2024. Can language models use forecasting strategies? arXiv:2406.04446. Retrieved from https:\/\/arxiv.org\/abs\/2406.04446"},{"key":"e_1_3_3_43_2","first-page":"70926","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Prystawski Ben","year":"2024","unstructured":"Ben Prystawski, Michael Li, and Noah Goodman. 2024. Why think step by step? Reasoning emerges from the locality of experience. In Advances in Neural Information Processing Systems, Vol. 36, 70926\u201370947."},{"issue":"1","key":"e_1_3_3_44_2","first-page":"67","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning and Research 21, 1 (2020), 67.","journal-title":"Journal of Machine Learning and Research"},{"key":"e_1_3_3_45_2","first-page":"361","article-title":"Predicting the future of predictive analytics: Imposing liability on AI technology in healthcare","volume":"24","author":"Raimondo Michael A.","year":"2020","unstructured":"Michael A. Raimondo. 2020. Predicting the future of predictive analytics: Imposing liability on AI technology in healthcare. Quinnipiac Health LJ 24 (2020), 361.","journal-title":"Quinnipiac Health LJ"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00322"},{"key":"e_1_3_3_48_2","unstructured":"Matthew Renze and Erhan Guven. 2024. Self-reflection in LLM Agents: Effects on problem-solving performance. arXiv:2405.06682. Retrieved from https:\/\/arxiv.org\/abs\/2405.06682"},{"key":"e_1_3_3_49_2","first-page":"552","volume-title":"Proceedings of the SIGSPATIAL","author":"Khan Rumi Shakila","year":"2018","unstructured":"Shakila Khan Rumi, Ke Deng, and Flora D. Salim. 2018. Theft prediction with individual risk factor of visitors. In Proceedings of the SIGSPATIAL, 552\u2013555."},{"key":"e_1_3_3_50_2","unstructured":"Parth Sarthi Salman Abdullah Aditi Tuli Shubh Khanna Anna Goldie and Christopher D. Manning. 2024. RAPTOR: Recursive abstractive processing for tree-organized retrieval. arXiv:2401.18059. Retrieved from https:\/\/arxiv.org\/abs\/2401.18059"},{"key":"e_1_3_3_51_2","unstructured":"Philipp Schoenegger and Peter S. Park. 2023. Large language model prediction capabilities: Evidence from a real-world forecasting tournament. arXiv:2310.13014. Retrieved from https:\/\/arxiv.org\/abs\/2310.13014"},{"key":"e_1_3_3_52_2","doi-asserted-by":"crossref","unstructured":"Philipp Schoenegger Indre Tuminauskaite Peter S. Park Rafael Valdece Sousa Bastos and Philip E. Tetlock. 2024. Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy. arXiv:2402.19379. Retrieved from https:\/\/arxiv.org\/abs\/2402.19379","DOI":"10.1126\/sciadv.adp1528"},{"key":"e_1_3_3_53_2","article-title":"Spearman\u2019s rank correlation coefficient","volume":"349","author":"Sedgwick Philip","year":"2014","unstructured":"Philip Sedgwick. 2014. Spearman\u2019s rank correlation coefficient. BMJ (Clinical Research ed.) 349 (2014), 502\u2013505.","journal-title":"BMJ (Clinical Research ed.)"},{"issue":"5","key":"e_1_3_3_54_2","first-page":"2064","article-title":"News event extraction using 5W1H approach & its analysis","volume":"4","author":"Sharma Smriti","year":"2013","unstructured":"Smriti Sharma, Rajesh Kumar, Pawan Bhadana, and Sumita Gupta. 2013. News event extraction using 5W1H approach & its analysis. International Journal of Scientific & Engineering Research 4, 5 (2013), 2064\u20132068.","journal-title":"International Journal of Scientific & Engineering Research"},{"key":"e_1_3_3_55_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288. Retrieved from https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781394234028.ch11"},{"key":"e_1_3_3_57_2","volume-title":"Proceedings of the ICLR","author":"Xiong Miao","year":"2024","unstructured":"Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs. In Proceedings of the ICLR."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i14.29544"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.41"},{"key":"e_1_3_3_60_2","unstructured":"Qi Yan Raihan Seraj Jiawei He Lili Meng and Tristan Sylvain. 2024. AutoCast++: Enhancing world event prediction with zero-shot ranking-based context retrieval. arXiv:2310.01880. Retrieved from https:\/\/arxiv.org\/abs\/2310.01880"},{"key":"e_1_3_3_61_2","unstructured":"Haoyan Yang Yixuan Wang Xingyin Xu Hanyuan Zhang and Yirong Bian. 2024. Can we trust LLMs? Mitigate overconfidence bias in LLMs through knowledge transfer. arXiv:2405.16856. Retrieved from https:\/\/arxiv.org\/abs\/2405.16856"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3358156"},{"key":"e_1_3_3_63_2","unstructured":"Chenchen Ye Ziniu Hu Yihe Deng Zijie Huang Mingyu Derek Ma Yanqiao Zhu and Wei Wang. 2024. MIRAI: Evaluating LLM Agents for Event Forecasting. arXiv:2407.01231. Retrieved from https:\/\/arxiv.org\/abs\/2407.01231"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/FSKD.2014.6980962"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645376"},{"key":"e_1_3_3_66_2","unstructured":"Wenqi Zhang Yongliang Shen Linjuan Wu Qiuying Peng Jun Wang Yueting Zhuang and Weiming Lu. 2024. Self-contrast: Better reflection through inconsistent solving perspectives. arXiv:2401.02009. Retrieved from https:\/\/arxiv.org\/abs\/2401.02009"},{"key":"e_1_3_3_67_2","doi-asserted-by":"crossref","unstructured":"Xiaoyu Zhang Yishan Li Jiayin Wang Bowen Sun Weizhi Ma Peijie Sun and Min Zhang. 2024. Large language models as evaluators for recommendation explanations. arXiv:2406.03248. Retrieved from https:\/\/arxiv.org\/abs\/2406.03248","DOI":"10.1145\/3640457.3688075"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450287"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i11.26645"},{"key":"e_1_3_3_70_2","first-page":"27293","volume-title":"Proceedings of the NeurIPS","volume":"35","author":"Zou Andy","year":"2022","unstructured":"Andy Zou, Tristan Xiao, Ryan Jia, Joe Kwon, Mantas Mazeika, Richard Li, Dawn Song, Jacob Steinhardt, Owain Evans, and Dan Hendrycks. 2022. Forecasting future world events with neural networks. In Proceedings of the NeurIPS, Vol. 35, 27293\u201327305."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3802922","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T15:44:02Z","timestamp":1778946242000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3802922"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,11]]},"references-count":69,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,5,31]]}},"alternative-id":["10.1145\/3802922"],"URL":"https:\/\/doi.org\/10.1145\/3802922","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,11]]},"assertion":[{"value":"2025-02-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}