{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T12:18:00Z","timestamp":1794917880980,"version":"4.9.1"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9]]},"abstract":"<jats:p>Evaluating LLM forecasting capabilities is constrained by a fundamental tension: prospective evaluation offers methodological rigor but prohibitive latency, while retrospective forecasting (RF)\u2014evaluating on already-resolved events\u2014faces rapidly shrinking clean evaluation data as SOTA models possess increasingly recent knowledge cutoffs. Simulated Ignorance (SI), prompting models to suppress pre-cutoff knowledge, has emerged as a potential solution. We provide the first systematic test of whether SI can approximate True Ignorance (TI). Across 470 competition-level questions and 9 models, we find that SI fails systematically: (1) cutoff instructions leave a 52% performance gap between SI and TI; (2) chain-of-thought reasoning fails to suppress prior knowledge, even when reasoning traces contain no explicit post-cutoff references; (3) reasoning-optimized models exhibit worse SI fidelity despite superior reasoning trace quality. These findings demonstrate that prompts cannot reliably \"rewind\" model knowledge. We conclude that RF on pre-cutoff events is methodologically flawed; we recommend against using SI-based retrospective setups to benchmark forecasting capabilities.<\/jats:p>","DOI":"10.24963\/ijcai.2026\/650","type":"proceedings-article","created":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T22:01:13Z","timestamp":1789596073000},"page":"5837-5845","source":"Crossref","is-referenced-by-count":0,"title":["Simulated Ignorance Fails: A Systematic Study of LLM Behaviors on Forecasting Problems Before Model Knowledge Cutoff"],"prefix":"10.24963","author":[{"given":"Zehan","family":"Li","sequence":"first","affiliation":[{"name":"University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuxuan","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"El Lahib","sequence":"additional","affiliation":[{"name":"University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying-Jieh","family":"Xia","sequence":"additional","affiliation":[{"name":"University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyu","family":"Pi","sequence":"additional","affiliation":[{"name":"University of California, San Diego"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Thirty-Fifth International Joint Conference on Artificial Intelligence {IJCAI-26}","theme":"Artificial Intelligence","location":"Bremen, Germany","acronym":"IJCAI-2026","number":"35","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2026,8,15]]},"end":{"date-parts":[[2026,8,21]]}},"container-title":["Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T11:31:11Z","timestamp":1794915071000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2026\/650"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2026,9]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2026\/650","relation":{},"subject":[],"published":{"date-parts":[[2026,9]]}}}