{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T04:30:58Z","timestamp":1793334658380,"version":"4.5.0"},"posted":{"date-parts":[[2026]]},"group-title":"SSRN","reference-count":0,"publisher":"Elsevier BV","license":[{"start":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T00:00:00Z","timestamp":1767225600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Studies that apply machine learning to financial forecasting often report that the importance of predictors changes over time and interpret this as evidence of structural change. We ask how much of such time variation can be distinguished from what models trained on noise would show, and how reliable the inference tools used for this purpose are. Using about 25 years of hourly bid and ask quotes for six currency pairs built from four currencies (EUR, USD, GBP, JPY), we train rollingwindow gradient-boosting classifiers of next-hour direction and summarise predictor importance with SHAP values aggregated into six groups. We compare the resulting series with those from up to 200 models trained on permuted labels per pair. Four results stand out. First, next-hour direction is weakly but persistently predictable (accuracy 51.2\u201352.0%), yet gross gains of 0.03\u20130.23 pips per trade are far below spreads. Second, part of the apparent predictability in the raw sample comes from the broker\u2019s quotes around the daily rollover (23:00\u201302:00 Athens time) and disappears when those hours are excluded. Third, normal-approximation p-values for trend and break tests are liberal (rejection rates of 7\u201310% at a 5% nominal level), whereas empirical permutation p-values have correct size but need large null ensembles; power is about 35% for a three-percentage-point shift in an importance share and about 80% for a shift of eight to nine points. Fourth, with family-wise error control, no change in importance is statistically significant; the most consistent candidate, a 6-point decline in the weight of recent own-pair returns in EUR\/USD alongside a fall in accuracy after 2015, has an adjusted p-value of 0.20 and weakens when only three independent exchange rates are used as features (p = 0.29); a logistic regression with the same inputs is about as accurate as the boosted trees. An extension to two further pairs, AUD\/JPY and NZD\/CAD (2015\u20132026, 200 permuted-label models each), gives accuracy of 50.9\u201351.2%, negative net results and no significant trend after adjustment. We offer an evaluation protocol and data-quality screens for researchers who interpret importance dynamics in financial forecasting. All results are preliminary (Section 7).&amp;nbsp;<\/jats:p>","DOI":"10.2139\/ssrn.7664638","type":"posted-content","created":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T03:40:15Z","timestamp":1793331615000},"source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking Instability in Machine-Learning Feature Importance: Hourly FX Forecasts, Permutation Nulls and Broker-Quote Artefacts, 2001-2026"],"prefix":"10.2139","author":[{"given":"George","family":"Petridis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"78","container-title":[],"original-title":[],"deposited":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T03:40:15Z","timestamp":1793331615000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ssrn.com\/abstract=7664638"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":0,"URL":"https:\/\/doi.org\/10.2139\/ssrn.7664638","relation":{},"subject":[],"published":{"date-parts":[[2026]]},"subtype":"preprint"}}