{"id":"dfb0eb52-57cb-4d67-9d53-8bca04232520","arxiv_id":"2506.06345","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DLinear beats LTSNet, Vanilla Transformer, and Time Series Transformer on BIST100 bank price forecasting, with R² up to 0.995.","lead":"This study compares four transformer-style forecasting models on daily prices of five Turkish bank stocks and two BIST indices, then uses SHAP and LIME to interpret which technical indicators drive predictions. The paper reports that the simple DLinear model gives the highest R² values, but the evaluation has serious methodological weaknesses that cast doubt on the headline numbers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported DLinear superiority is not supported as written: the paper defines only a train/test split, then says hyperparameters and sequence length were selected using MSE, RMSE, and MAPE, so the test set was likely used for model selection.","rationale":"The reader's weakest assumption lists three potential leaks in the evaluation pipeline; I isolate the missing validation set as the decisive one. Section 3.2.4 is explicit that sequence length and, for DLinear, all key hyperparameters were chosen by \"performance metrics,\" while only a single 80/20 train/test split is ever defined. DLinear receives the most detailed tuning narrative, so it is also the model most exposed to selection-on-test bias. Even if normalization were fixed and technical indicators were causal, the DLinear ranking could still be manufactured by this leak. The proposed check settles the point directly by separating model selection from evaluation. If the check passes, the paper would mainly need a reporting fix; as written, no valid out-of-sample comparison supports the central claim, so the reader's REJECT verdict remains appropriate and I do not change it.","tokens_in":22342,"tokens_out":3964,"duration_ms":45780,"concrete_test":"Run a strict chronological evaluation: fit the min-max scaler on the first 70% of each series, compute all technical indicators causally from past data only, reserve the next 10–15% as validation for sequence-length and hyperparameter selection, freeze all configurations, and evaluate exactly once on the final 15–20% test window. Add a naive last-value/random-walk forecast and an AR(1) baseline on the same test split. If DLinear no longer dominates all metrics or fails to beat the naive baselines, the headline claim is an artifact of test-set leakage; if DLinear still dominates under this protocol, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the Highlights and Section 5, \"DLinear consistently outperforms its counterparts across all performance metrics,\" rests entirely on Tables 3–6. Section 3.2.4 defines only an 80/20 train/test split (\"a fixed train-test split ratio of 80%–20%\") and then states that \"multiple values of sequence length are systematically tested for each model, and the most suitable value is selected based on performance metrics such as MSE, RMSE, and MAPE.\" For DLinear specifically, the text says the \"best results\" across epochs, batch size, learning rate, sequence length, and dropout were chosen by configurations that \"minimize error metrics.\" Nowhere in the methodology is a validation partition mentioned. On the most natural reading, the same 20% test window was used both to choose configurations and to produce the reported test metrics. That makes DLinear's advantage a selection artifact rather than an out-of-sample result: the model with the most extensive reported hyperparameter search is the one most likely to have been fitted to the test set. The absence of any naive baseline (last-value/random-walk or AR(1)) compounds this, since R² values of 0.98–0.99 on trending price levels are exactly what a persistence forecast would produce; without such a baseline even the absolute accuracy numbers are uncontextualised. This concern follows from the protocol as described and does not depend on imputing intent. If a validation set did exist but was simply omitted from the write-up, the paper must say so and must report test metrics obtained only after the final configuration was frozen.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes an empirical framework that combines four time-series forecasting models—DLinear, LSTNet (also written LTSNet), Vanilla Transformer, and Time Series Transformer—with a set of technical indicators to predict daily closing prices of five Turkish banks and two BIST indices. The authors report performance metrics (MSE, MAE, MAPE, RMSE, R²) for each model and use SHAP and LIME to interpret the DLinear model's predictions. The central claim, stated in the Highlights and Section 5, is that DLinear consistently outperforms the other architectures across all metrics.","tokens_in":22626,"tokens_out":5671,"duration_ms":52233,"significance":"If the evaluation were sound, the paper would offer a useful case study of simple linear decomposition models versus transformer architectures in an emerging market, and a concrete demonstration of how XAI can be attached to a time-series forecaster. The authors are transparent about the limitations of their dataset (Section 6.1). However, the significance is undercut by three unresolved methodological questions: whether the test set was used for model selection, whether normalization leaks test information, and whether any naive baseline would achieve similar R² values. These concerns directly affect the paper's central quantitative claim.","major_comments":[{"comment":"No validation set is described. The paper defines only an 80/20 train–test split and says that sequence length and hyperparameters were \"selected based on performance metrics such as MSE, RMSE, and MAPE\" and that DLinear's \"best results\" were determined by configurations that \"minimize error metrics.\" On the most natural reading, the same 20% test window was used both to choose configurations and to produce the reported test metrics. This makes the DLinear advantage in Section 5 a potential selection artifact rather than an out-of-sample result. The authors must introduce a proper validation split (or nested cross-validation) and report metrics from a single final evaluation on untouched data.","section":"§3.2.4"},{"comment":"The text states that \"all variables are normalized to the [0, 1] range using min-max normalization prior to training\" but does not state that the minimum and maximum values are computed exclusively on the training split. If the scaling parameters are computed on the full dataset, information from the test period propagates into the training inputs, and the reported R² values in Tables 3–6 are inflated. The authors must state the exact procedure and, if necessary, re-run the experiments with training-only scaling.","section":"§3.2.4"},{"comment":"The comparison contains no naive or statistical baseline, such as a persistence forecast or AR(1). Because the targets are trending price levels, a model that simply repeats the last observed value would also produce R² values in the 0.98–0.99 range. Without such a baseline, the claim in the Highlights and Section 5 that DLinear shows \"superior forecasting capability\" is not established; the authors should add at least a persistence and an ARIMA/AR baseline to the benchmark tables.","section":"§4, Tables 3–6"},{"comment":"Shuffling the training set once \"before the training process begins\" is incompatible with a causal time-series forecasting setup. If the model's input windows are constructed after shuffling, each window can contain observations that temporally follow the target, which is a form of lookahead leakage; if the windows are constructed before shuffling, the description is misleading. The authors need to clarify the exact order of windowing and shuffling and justify why shuffling is used for a temporal task.","section":"§3.2.4"},{"comment":"The reported numbers contain internal inconsistencies that undermine confidence in the results. For example, Table 3 lists GARAN's R² as 0.9955 and QNBTR's as 0.9847, but the text refers to \"the lowest R² of 0.984%\" for QNBTR; the paragraph on Table 4 claims GARAN has \"the highest R² of 0.995\", although Table 4 reports 0.9836 for GARAN; and the same paragraph attributes an MSE of 0.104 and RMSE of 0.3234 to ISCTR, which are actually the DLinear values from Table 3, not the LSTNet values. The model is also called LTSNet in Table 2 and LSTNet elsewhere. All of these need to be corrected and the tables re-verified.","section":"§4, Tables 3–6"}],"minor_comments":[{"comment":"The model name is written as \"LTSNet\" in the abstract and Table 2 but as \"LSTNet\" in Section 3.2.1 and Table 4; standardize the name throughout the manuscript.","section":"Abstract and Table 2"},{"comment":"The study claims to evaluate \"transformer models,\" but the SHAP and LIME analyses are applied only to DLinear; clarify that the XAI part concerns only the DLinear model.","section":"Figures 4–10"},{"comment":"Figure 3 is discussed before Figure 2 in the text; renumber either the figures or the references.","section":"§4"},{"comment":"The paper does not state the prediction horizon (single-step versus multi-step); specify whether the models output one day ahead or multiple days ahead.","section":"§3.2.4"},{"comment":"Figure 6 mentions \"MA_200\" and \"MA_300\" as influential features, but Table 1 does not include these indicators; either add them to Table 1 or correct the figure caption.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to have been assembled without a final consistency check: the model name spelling varies, numbers in the text do not match the tables, and the evaluation protocol is described ambiguously. The novelty is limited—applying existing models to BIST data—and the XAI component is only for DLinear. I would advise the editor that a major revision with a full re-analysis and careful proofreading is needed before this can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a routine application of known models to a new dataset, not a methodological advance. Second, the main result—DLinear beats four transformer variants—probably comes from a test-set selection artifact. Not because the authors intended anything shady, but because Section 3.2.4 never mentions a validation set, and it says hyperparameters and sequence length were selected using MSE, RMSE, and MAPE. On the natural reading, the same 20% test window was used to pick configurations and then to report metrics. That would make DLinear's superiority a selection effect.\n\nWhat it does well: it is a clean, readable case study on ten years of daily BIST100 bank data and two indices. Seven assets, four architectures, standard metrics, and SHAP/LIME applied consistently. The authors also state their hyperparameter choices per model, which is more than many papers do. The qualitative finding—that a simple linear decomposition model outperforms transformer variants—matches the earlier literature (e.g., Zeng et al. 2023), so the direction is plausible, even if this paper doesn't prove it.\n\nThe soft spots. The validation protocol is the load-bearing one, as above. There is also no naive baseline. R² of 0.98–0.99 on trending price levels is exactly what a last-value forecast would produce, so the absolute numbers are uncontextualised. Normalization is described as applied 'prior to training,' with no statement that min-max statistics are training-only; if they were fit on the full series, that's future leakage. And the training set is shuffled for a time series task, which breaks temporal ordering between samples. I'd call that a design mistake, though it may not change the relative ranking much.\n\nThere are also direct contradictions between the text and the tables. The LSTNet paragraph says AKBNK has R² 0.993 and GARAN 0.995, but Table 4 shows 0.9800 and 0.9836. The same paragraph attributes to LSTNet the MSE 0.104 and RMSE 0.323 for ISCTR, which are DLinear's numbers. These are easy to fix but currently undermine trust in the results section.\n\nWho should read it: anyone who wants a quick benchmark of simple versus transformer models on Turkish bank data, or an example of SHAP and LIME applied to DLinear. It is not a contribution to forecasting methodology.\n\nMy recommendation: this deserves peer review but not acceptance as is. A serious referee should require a naive baseline, a proper train/validation/test protocol, a statement on normalization statistics, and corrected tables. After that, it might be a publishable case study. So I would not desk-reject it, but I would not let it through unchanged.","headline":"A competent BIST100 benchmark whose DLinear-wins claim is undermined by a missing validation set and missing naive baseline; fixable, but not acceptable as is.","tokens_in":23207,"tokens_out":2959,"would_cite":false,"duration_ms":30910,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that on BIST100 bank stock data, the decomposition-based linear model DLinear forecasts daily prices more accurately than LSTNet, Vanilla Transformer, and Time Series Transformer, and that SHAP and LIME make the…","keywords":["stock price forecasting","time series transformers","DLinear","explainable AI","SHAP","LIME","BIST100","technical indicators"],"falsifier":"Re-run the identical experiment with min-max normalization fit only on the training split, a separate validation split for hyperparameter selection, and a fixed seeded DLinear configuration; if DLinear then no longer beats LSTNet, Vanilla Transformer, and TST on most series across all five metrics, the paper's main superiority claim is falsified.","tokens_in":22136,"feed_emoji":"📈","tokens_out":5489,"duration_ms":55427,"temperature":0.7,"pith_summary":"This paper tries to establish that a set of time series models, enriched with technical indicators and explained by SHAP and LIME, can forecast daily stock prices in an emerging market, and that among the four architectures tested the decomposition-based linear model DLinear is the most accurate. On daily data from five high-volume BIST100 banks plus the XBANK and XU100 indices from January 2015 to March 2025, DLinear reports $R^2$ values between 0.984 and 0.995 and MAPE below 4% for most series, beating LSTNet, Vanilla Transformer, and Time Series Transformer on every metric. The results matter because they suggest that a simple linear decomposition can outperform attention-heavy models on this forecasting task, and that pairing such models with SHAP and LIME can make price predictions legible enough to support financial literacy among individual investors.","feed_headline":"Simple DLinear model beats transformer rivals on BIST100","feed_subtitle":"Ten years of Turkish bank data: DLinear hits R² up to 0.995; SHAP and LIME show what drives each forecast.","key_machinery":"The load-bearing object is DLinear's trend\\u2013seasonality decomposition: the input series is smoothed with moving averages to separate a trend component from a residual seasonal component, each component is passed through its own linear layer, and the two outputs are summed. This is what lets a lightweight linear model capture the dominant temporal structure in bank stock prices without attention, and the study argues that it explains why DLinear beats the attention-based baselines. The interpretability machinery consists of SHAP, which assigns each feature a game-theoretic Shapley contribution averaged over the dataset, and LIME, which fits a local sparse surrogate around a single prediction; these are used to expose which technical indicators drive the forecasts.","core_discovery":"The central claim is that DLinear, a decomposition-based linear model, consistently outperforms the other three evaluated architectures across MSE, MAE, MAPE, RMSE, and $R^2$ on every stock and index in the dataset. The paper reports $R^2$ values from 0.984 to 0.995 for DLinear, with the strongest results on GARAN ($R^2=0.9955$, MAPE 2.34%) and the lowest percentage error on XU100 (MAPE 1.42%). It also claims that SHAP global explanations consistently rank short-lag RSI and MACD or volume features as the dominant drivers, while LIME explanations on the final day shift toward longer-horizon moving averages and volatility indicators. The study reads this global versus local divergence as evidence that the model is context-sensitive rather than contradictory.","pith_inferences":["We infer that a testable extension beyond the paper is to apply the same four-model comparison to other emerging-market bank panels; the expectation from these results is that DLinear's linear decomposition retains its edge whenever prices are dominated by strong trends, and loses it in regimes with regime shifts or non-stationarity.","The reported gap between SHAP and LIME suggests a practical design principle for investor-facing tools: show a global driver ranking and a local \\u2018why this prediction\\u2019 explanation together, because either alone gives an incomplete picture.","Because the dataset ends in March 2025 and includes high-inflation episodes, a natural stress test is to retrain on data containing a sudden crisis or policy shock and check whether DLinear's decomposition absorbs the break or whether its $R^2$ drops below the transformer models.","The paper's reliance on technical indicators only leaves room for adding macro and sentiment inputs; if those features carry independent signal, the RSI-dominant SHAP rankings would shift, changing the financial-literacy story from momentum-following to fundamental context."],"forward_implications":["DLinear gives a cheap, high-accuracy baseline for BIST100 bank forecasting, reaching near-99% $R^2$ at a fraction of the compute of attention models.","On this dataset, complex attention architectures buy little accuracy, so practitioners can prefer the simpler model and reserve transformers for settings with longer or more nonlinear dependencies.","SHAP's global emphasis on short-lag RSI and MACD suggests that momentum and volume indicators carry most forecasting signal for Turkish bank prices over this decade.","LIME's local emphasis on moving averages, Bollinger Bands, and Ichimoku components means explanations can change sharply by date, so forecasts should be accompanied by both global and local attribution.","The same 80/20 split with technical-indicator enrichment can be replicated for other BIST sectors or emerging markets, giving a direct benchmark for XAI-augmented forecasting."],"supporting_citations":[{"why":"Introduces DLinear, the decomposition-based linear model that the paper finds superior to all baselines.","marker":"[34]"},{"why":"Provides LSTNet, the hybrid convolutional-recurrent-autoregressive model used as one of the compared baselines.","marker":"[13]"},{"why":"Provides the Vanilla Transformer architecture used as a baseline.","marker":"[28]"},{"why":"Provides the Time Series Transformer (TST) architecture used as a baseline.","marker":"[35]"},{"why":"Supplies the SHAP framework used for global feature-attribution explanations.","marker":"[16]"},{"why":"Supplies the LIME method used for local, instance-level explanations.","marker":"[25]"}],"fun_headline_variants":["DLinear tops transformers in BIST100 forecast test","Simple linear model beats transformers on Turkish stocks","BIST100: DLinear edges out transformer rivals with R² 0.995","Explainable AI shows why DLinear wins on BIST100","DLinear's edge over transformers: interpretability and accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the evaluation is leak-free: min-max normalization statistics and hyperparameters such as sequence length are chosen without consulting the test period, so the reported test-set $R^2$ values and DLinear's margin are not inflated by information from the future.","fun_headline_variants_meta":{"raw":{"variants":["DLinear tops transformers in BIST100 forecast test","Simple linear model beats transformers on Turkish stocks","BIST100: DLinear edges out transformer rivals with R² 0.995","Explainable AI shows why DLinear wins on BIST100","DLinear's edge over transformers: interpretability and accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1395,"prompt_tokens":892,"completion_tokens":503,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":419}},"tokens_in":508,"tokens_out":503,"duration_ms":5458,"temperature":1.0,"reasoning_tokens":419,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:52:37.300796+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the identical experiment with min-max normalization fit only on the training split, a separate validation split for hyperparameter selection, and a fixed seeded DLinear configuration; if DLinear then no longer beats LSTNet, Vanilla Transformer, and TST on most series across all five metrics, the paper's main superiority claim is falsified.","supporting_citations":[{"cited_title":"Are transformers effective for time series forecasting?, in: Proceedings of the AAAI conference on artificial intelligence, pp","cited_arxiv_id":null,"evidence_quote":"Introduces DLinear, the decomposition-based linear model that the paper finds superior to all baselines."},{"cited_title":"Modeling long-and short-term temporal patterns with deep neural networks, in: The 41st international ACM SIGIR conference on research & development in information retrieval, pp","cited_arxiv_id":null,"evidence_quote":"Provides LSTNet, the hybrid convolutional-recurrent-autoregressive model used as one of the compared baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Time Series Transformer (TST) architecture used as a baseline."},{"cited_title":"A unified approach to interpreting model predictions","cited_arxiv_id":null,"evidence_quote":"Supplies the SHAP framework used for global feature-attribution explanations."}],"review_version":1}