{"id":"7f130c74-028a-490f-9ae9-9ffa16904609","arxiv_id":"2505.06892","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A new soft-sparsifying shapelet model with expert-based temporal learning reports top mean accuracy on the 128-dataset UCR time series benchmark.","lead":"SoftShape is a time series classifier that keeps every subsequence but weights each one by its learned importance, merging the least useful pieces into one pooled shape. It reports the highest average accuracy on 128 UCR datasets in its comparison and offers attention-based visual explanations of which signal regions matter.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SoftShape's SOTA claim rests on undertuned, non-standard baselines; ShapeConv and ModernTCN scoring ~0.77-0.79 suggest the comparison protocol, not the method, may drive the margin.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the SOTA claim is only as strong as the baseline comparison. The main table is the sole evidence for the headline claim, and the manuscript's own appendix shows the protocol is uniform but not tuned, with re-implemented ShapeConv and ModernTCN far below their published levels. This is not a disagreement with consensus; it is an internally evidenced risk that the comparison protocol, rather than the proposed method, drives the favorable ranking. The concern is concrete and testable by rerunning official implementations under the same splits. The architecture and ablations are coherent and the soft-sparsification idea is a genuine contribution, so the reader's CONDITIONAL verdict remains appropriate; no verdict change is needed.","tokens_in":46594,"tokens_out":8925,"duration_ms":95512,"concrete_test":"Rerun Table 1 on the same 60/20/20 splits with the official implementations and author-recommended hyperparameters for ShapeConv, ModernTCN, TSLANet, and LightTS (and, if feasible, HIVE-COTE 2.0), using 5 seeds and reporting mean and standard deviation; then recompute the average rank and the Wilcoxon test against SoftShape. If ShapeConv and ModernTCN recover published-level accuracy and the margin over TSLANet loses significance, the state-of-the-art claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the 19 baselines being strong and fairly tuned under the 60/20/20 merged-split protocol. This condition is the least secure. Appendix A.3 fixes the same optimizer (Adam, lr=0.001), batch size min(N/10,16), and early stopping for every deep baseline; no per-method tuning is reported. Table 9 shows re-implemented ShapeConv at 0.7688 and ModernTCN at 0.7938, with entry-level anomalies such as ShapeConv 0.317 on AllGestureWiimoteX and ModernTCN 0.350 on ElectricDevices while FCN/InceptionTime exceed 0.7/0.9 on the same rows; these are far below the levels at which those methods are published on UCR, indicating the comparison protocol, not method quality, may be driving the ranking. Meanwhile SoftShape's hyperparameters (eta=0.5 from Table 3, q=4 from Table 5, L=2 from Table 6, k=1, lambda=0.001) were selected on an 18-dataset subset of the same benchmark, and shape length m is chosen per dataset by validation. HIVE-COTE 2.0, the strongest prior UCR classifier, is absent, and LightTS/Shapeformer are evaluated only on the same 18 datasets. If the suspect baselines recover published-level accuracy under this protocol, SoftShape's 0.9334/2.72 margin over TSLANet (0.9205/3.68, p=1.06e-3) could shrink or disappear.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SoftShape, a deep-learning architecture for univariate time series classification. The model converts an input series into overlapping subsequence ('shape') embeddings via a 1D CNN, scores each embedding with a gated attention head, rescales the top-η fraction of shapes by their scores, and fuses all remaining low-scoring shapes into a single weighted-sum embedding ('soft shape sparsification'). The sparsified shapes then pass through a soft shape learning block that combines a mixture-of-experts router for intra-shape pattern learning with a shared Inception-style expert for inter-shape temporal modeling. Training is end-to-end with cross-entropy plus two MoE load-balancing losses. The authors evaluate on 128 UCR datasets under a merged 60/20/20 five-fold cross-validation protocol, reporting an average accuracy of 0.9334 and an average rank of 2.72 against 17 baselines, with Wilcoxon signed-rank p-values; they also provide ablations, hyperparameter studies, runtime measurements, and attention visualizations. The central claim is that SoftShape outperforms state-of-the-art TSC methods while remaining interpretable.","tokens_in":46906,"tokens_out":5820,"duration_ms":59314,"significance":"If the empirical claim is substantiated, the result is significant: it suggests that a shapelet-inspired architecture with soft sparsification and MoE-based intra- and inter-shape learning can outperform the strongest compared convolutional and Transformer baselines on a large benchmark while offering attention-based explanations. The model is clearly specified and the forward-pass mathematics are coherent. Strengths of the manuscript include the breadth of the main evaluation (128 UCR datasets), the ablation battery, the hyperparameter sensitivity analysis, the release of code, and the explicit reporting of Wilcoxon p-values. The main unresolved question is whether the comparison protocol is fair to the baselines and whether the reported margins would persist under better-tuned baselines and with variance information; the evidence currently presented is not sufficient to fully support the headline state-of-the-art claim.","major_comments":[{"comment":"The fairness of the baseline comparison under the merged 60/20/20 protocol is not established. The same optimizer, learning rate, batch size, and early-stopping rule are applied to all deep baselines without per-method hyperparameter tuning, and several re-implemented baselines score far below their published accuracy on the same UCR datasets: e.g., AllGestureWiimoteX ShapeConv 0.317 and ElectricDevices ModernTCN 0.350, with average accuracies of 0.7688 and 0.7938, respectively. The absence of HIVE-COTE 2.0, the strongest prior UCR classifier in recent bake-offs, further weakens the state-of-the-art claim. Because SoftShape's margin over the strongest compared baseline TSLANet is only 0.0129 in average accuracy, the conclusion that SoftShape 'outperforms state-of-the-art methods' is load-bearing on the quality of these baseline runs; if the baselines were undertuned, the margin could shrink or disappear. Please provide per-method hyperparameter tuning (or justification that the common setting is appropriate for each baseline), report published-level reference numbers on the official UCR splits, and add or justify the omission of HIVE-COTE 2.0.","section":"Appendix A.2-A.3, Table 9"},{"comment":"Several key hyperparameters of SoftShape (η=0.5, q=4, L=2, k=1, λ=0.001) are selected on an 18-dataset subset of the same 128-dataset benchmark, and the main results in Table 1 include those same 18 datasets. This makes the comparison partially in-sample for those datasets, since the model configuration was chosen after observing their test accuracy, and the reported Wilcoxon p-values do not account for this selection. To support a state-of-the-art claim, the authors should either fix all hyperparameters before evaluating on UCR, exclude the 18 development datasets from the main comparison, or use a separate validation benchmark. At minimum, the paper should explicitly state that the main results are not fully held-out with respect to hyperparameter selection.","section":"Section 5.3 and Tables 3, 5, 6, 13, 15-21"},{"comment":"The implementation description is internally contradictory about the number of seeds. The text states both 'we calculate the average test accuracy using five-fold test sets from a single seed' and 'Each experiment is conducted five times with five different random seeds'; no standard deviations or error bars are reported for any of the headline numbers in Table 1. Because the primary evidence is a small accuracy margin (0.9334 vs. 0.9205) and the claimed statistical significance is based on paired tests, the paper must clarify the seed protocol and report variance measures (e.g., per-dataset standard deviation across seeds or folds) for SoftShape and at least the strongest baselines. Without this, the reader cannot assess whether the reported difference is robust to seed variation.","section":"Appendix A.3, Implementation Details"}],"minor_comments":[{"comment":"The phrase 'improving model interop-erability' appears to be a typo for 'interpretability'.","section":"Section 2.2"},{"comment":"In Step 3, 'residul way' should be 'residual way'.","section":"Algorithm 1"},{"comment":"The summation index in the conjunctive pooling equation runs from i=0 to Num, which would produce Num+1 terms while the normalization factor is 1/Num; please check whether the intended range is i=1,...,Num or i=0,...,Num-1.","section":"Section 4.4, Eq. (13)"},{"comment":"The 'Win' column sums to more than the number of datasets because ties are apparently counted for multiple methods; the caption should state how ties are handled.","section":"Table 1"},{"comment":"The Wilcoxon p-values are reported for each baseline without correction for multiple comparisons across 17 baselines; please state whether any correction was applied or note that the p-values are uncorrected.","section":"Section 5.1"},{"comment":"The time series forecasting experiments (Tables 22 and 23) are not part of the paper's stated contributions and show SoftShape underperforming iTransformer; if kept, they should be better motivated and their limitations acknowledged, or they should be removed.","section":"Appendix B.5"},{"comment":"The tensor dimension annotations such as [B,D,5] and [5B,1,D] are not explained in the caption; adding a short explanation of the reshape operations would improve readability.","section":"Figure 2 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely fixable, but the central empirical claim needs substantial additional evidence: better-tuned baselines, explicit handling of the hyperparameter-selection overlap, variance reporting, and ideally a comparison with HIVE-COTE 2.0. I recommend major revision and a second review round."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The core mechanism—soft shape sparsification via gated attention, fusing low-scoring subsequences into one pooled shape—is genuinely new to the shapelet literature. Prior methods hard-select or discard. The MoE router for intra-shape and shared expert for inter-shape is a sensible pairing, and the ablations show each component carries weight. The math is clearly specified, code is released, and the 128-dataset evaluation is extensive, with Wilcoxon tests. That is real evidence.\n\nThe soft spots are in the empirical claim. The authors merge UCR train/test and re-split 60/20/20, which is defensible for validation, but then fix the same Adam lr=0.001, batch size, and early stopping for every deep baseline. Under that protocol ShapeConv and ModernTCN score 0.7688 and 0.7938, with entry-level oddities like 0.317 on AllGestureWiimoteX and 0.350 on ElectricDevices. Those numbers are far below what those methods publish on UCR, which suggests under-tuning rather than a level playing field. SoftShape's own hyperparameters (eta=0.5, q=4, L=2) were chosen on an 18-dataset subset of the same benchmark, so the comparison is not obviously apples-to-apples. HIVE-COTE 2.0, the strongest prior UCR classifier, is omitted for computational reasons, which is understandable but makes 'state-of-the-art' a stretch. There are also no error bars or variance numbers, and the appendix's seed description is ambiguous (single seed for five-fold test sets vs. five random seeds). The efficiency claim is fine for long sequences but they acknowledge short-sequence overhead, so I would not headline 'efficient.'\n\nNone of this kills the paper. The soft-sparsification idea is interesting enough to stand on its own, and the ablations give me confidence the mechanism is doing something. What needs work is the comparison: re-run under the standard UCR protocol with per-method tuning, include HC2 or at least a strong hybrid, and report variance. As written, I'd trust the ablations more than the Table 1 ranking.\n\nWho is this for? Researchers working on interpretable TSC and shapelet methods. It deserves a serious referee. I'd accept it for review and ask for major revisions on the evaluation. For a reading group, it's a good paper to discuss because it raises the right questions about benchmark protocols in TSC.","headline":"Soft sparsification of shapelets is a real idea, but the SOTA claim rests on a comparison protocol that flatters it.","tokens_in":47523,"tokens_out":2969,"would_cite":true,"duration_ms":30972,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Merging weak shapes, not deleting them, tops 128 UCR benchmarks","keywords":["time series classification","shapelets","soft sparsification","mixture of experts","interpretability","gated attention","UCR archive","deep learning"],"falsifier":"Two checks would settle the claim. First, replace the fused low-scoring shape embedding of Eq. (5) with a constant vector on the 18-dataset benchmark: if accuracy does not drop, the merge-retains-information mechanism is not doing the work claimed. Second, rerun the re-implemented baselines (TSLANet, ModernTCN, ShapeConv) on the original UCR train/test split with their published hyperparameters: if they recover their published accuracies, SoftShape's reported margin of 0.0129 in average accuracy has not survived contact with the standard protocol.","tokens_in":46301,"feed_emoji":"📈","tokens_out":12998,"duration_ms":108154,"temperature":0.7,"pith_summary":"The paper argues that the standard treatment of shapelets — extracting many candidate subsequences and then hard-filtering down to the few most discriminative ones — discards subsequences that could help classification. It proposes SoftShape, which makes sparsification soft: a gated attention head scores every subsequence by its classification contribution, the top-scoring ones are kept as weighted soft shapes, and the low-scoring ones are merged into a single fused shape rather than deleted. All subsequence information therefore survives the sparsification while the input the learner sees shrinks from $J$ shapes to $\\eta J + 1$. A dual pattern-learning block then reads the sparsified shapes: a mixture-of-experts router activates class-specific experts for intra-shape patterns, and a shared convolutional expert models inter-shape temporal dependencies. On 128 UCR datasets the model reports 0.9334 average accuracy and rank 2.72, ahead of the strongest compared deep baseline at 0.9205 and rank 3.68, and its attention scores double as per-subsequence explanations.","feed_headline":"Merging weak shapes, not deleting them, tops 128 UCR benchmarks","feed_subtitle":"Fusing low-scoring subsequences instead of dropping them gives 0.9334 average accuracy, ahead of every compared baseline.","key_machinery":"The load-bearing object is the soft shape sparsification of Eqs. (3)-(5). A parameter-shared gated attention head, $\\alpha(\\hat{S}^m_{n,p}) = \\sigma(W_2 \\tanh(W_1 \\hat{S}^m_{n,p} + b_1) + b_2)$, scores every subsequence embedding by its estimated classification contribution; the top-$\\eta$ proportion become scaled soft shapes $\\alpha \\hat{S}$, and the rest are merged into a single fused embedding $e^S_{n,\\mathrm{fused}} = \\sum_{p \\in E} \\alpha(\\hat{S}^m_{n,p}) \\hat{S}^m_{n,p}$. This merge-not-delete operation is what lets the model keep all subsequence information while shrinking its input from $J$ shapes to $\\eta J + 1$. The second mechanism is the soft shape learning block: a mixture-of-experts router with softmax gating and TOP-$k$ selection (here $k = 1$, Eqs. (6)-(9)) routes each soft shape to a class-specific expert for intra-shape pattern learning, while a shared Inception-style 1D convolutional expert on the reshaped sequence of shapes (Eq. (12)) captures inter-shape temporal dependencies. Importance and load-balancing losses (Eqs. (10)-(11)) keep the experts from collapsing onto a few favored routes.","core_discovery":"SoftShape's central claim is that sparsification of shapelet candidates should be soft, not hard. Instead of selecting the top shapes and discarding the rest — the move shared by classical shapelet transforms and recent deep shapelet layers — the model scales the top-$\\eta$ proportion of subsequence embeddings by their attention scores and fuses the remaining embeddings into a single weighted-sum shape appended to the sequence. Every subsequence therefore survives sparsification in some form, while the sequence shrinks from $J$ shapes to $\\eta J + 1$. On this sparsified input, a router activates exactly one class-specific expert per shape to capture intra-shape patterns, and a shared convolutional expert treats the shapes as a sequence to capture inter-shape temporal dependencies. The paper reports this design achieves average test accuracy 0.9334 and average rank 2.72 across 128 UCR datasets, ahead of the best compared deep baseline TSLANet (0.9205, rank 3.68) with a Wilcoxon p-value of $1.06 \\times 10^{-3}$, and that the learned attention scores highlight the same discriminative regions a human would pick out.","pith_inferences":["The fused shape is effectively a learned, class-contribution-weighted summary of a series' non-discriminative background; a testable extension would be learning several fused shapes per series instead of one, which might preserve more information at sparsity ratios above 50%.","The comparison set excludes the strongest published UCR classifiers such as HIVE-COTE 2.0 and baselines tuned to their published accuracies, so how large the real-world margin is under the standard UCR protocol remains open.","Read against Table 2, removing the inter-shape module costs more (0.9334 to 0.9022) than removing soft sparsification (to 0.9123), suggesting a substantial share of the reported gain may come from the shape-sequence CNN and MoE blocks rather than from the merge-not-delete mechanism alone; testing a plain patch-based model with the same learning blocks on the same protocol would size that share.","The forecasting results hint that soft sparsification could serve as a general tokenizer for time series; applying the same pipeline to anomaly detection or long-horizon forecasting, where rare local patterns carry the signal, would be a direct test."],"forward_implications":["A shapelet-based deep model can sparsify without losing information: merging low-scoring subsequences into a fused shape keeps their contribution while cutting the number of shapes the learner processes from $J$ to $\\eta J + 1$.","Interpretability comes from the same scores that drive sparsification: the multiple-instance-learning visualizations on Trace and Lightning2 show that high-scoring shapes are exactly the regions where the classes differ sharply.","Efficiency tracks the sparsity ratio: at $\\eta = 0.5$ the shared expert reads roughly half the shapes, and on long sequences SoftShape trains faster than the attention-based and transformer baselines reported.","Fusing up to 50% of shapes causes no significant accuracy drop while 90% does (Table 3), which directly supports the paper's claim that hard shapelet methods lose critical patterns by discarding too much.","The same model transfers to forecasting without hyperparameter adjustment, beating TS2Vec and TimesNet on four ETT datasets (Appendix B.5), so the soft-shape machinery is not specific to classification."],"supporting_citations":[{"why":"Defines shapelets as discriminative subsequences, the primitive the entire model is built on.","marker":"Ye & Keogh, 2009"},{"why":"Supplies the gated attention mechanism whose scores drive the soft sparsification.","marker":"Ilse et al., 2018"},{"why":"Supplies multiple instance learning for the interpretability visualizations and the conjunctive pooling training objective.","marker":"Early et al., 2024"},{"why":"Provides the importance and load-balancing losses that keep the MoE router from collapsing onto a few experts.","marker":"Shazeer et al., 2017"},{"why":"Provides the softmax TOP-k gating formulation the router follows in Eqs. (6)-(7).","marker":"Riquelme et al., 2021"},{"why":"The UCR archive is the 128-dataset benchmark the whole evaluation stands on.","marker":"Dau et al., 2019"},{"why":"InceptionTime is the basis of the shared inter-shape expert and a leading deep baseline SoftShape must beat.","marker":"Ismail Fawaz et al., 2020"},{"why":"Supplies the 60/20/20 merged five-fold protocol and nine of the reported baseline results.","marker":"Ma et al., 2024"},{"why":"The bake-off that motivates selecting InceptionTime, MR-H, and RDST as the strong non-deep baselines.","marker":"Middlehurst et al., 2024"},{"why":"Provides the Wilcoxon signed-rank test and critical difference diagram used for the statistical significance claims.","marker":"Demšar, 2006"}],"fun_headline_variants":["Soft shapes beat hard selection on 128 UCR","SoftShape: merge weak shapes, not drop them, tops UCR","Fusing low-score subsequences lifts time-series accuracy","Soft sparsification keeps all shapes, gains on UCR","Router + shared expert: soft shapes win 128 UCR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation rests on 19 baselines, several re-implemented or borrowed from another study, all trained under the paper's own 60/20/20 re-split of the UCR data; if those baselines were tuned below their published best, the reported accuracy edge would shrink or vanish.","fun_headline_variants_meta":{"raw":{"variants":["Soft shapes beat hard selection on 128 UCR","SoftShape: merge weak shapes, not drop them, tops UCR","Fusing low-score subsequences lifts time-series accuracy","Soft sparsification keeps all shapes, gains on UCR","Router + shared expert: soft shapes win 128 UCR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1547,"prompt_tokens":992,"completion_tokens":555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":471}},"tokens_in":608,"tokens_out":555,"duration_ms":5364,"temperature":1.0,"reasoning_tokens":471,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:30:02.927581+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two checks would settle the claim. First, replace the fused low-scoring shape embedding of Eq. (5) with a constant vector on the 18-dataset benchmark: if accuracy does not drop, the merge-retains-information mechanism is not doing the work claimed. Second, rerun the re-implemented baselines (TSLANet, ModernTCN, ShapeConv) on the original UCR train/test split with their published hyperparameters: if they recover their published accuracies, SoftShape's reported margin of 0.0129 in average accuracy has not survived contact with the standard protocol.","supporting_citations":[],"review_version":1}