{"id":"547267ce-2299-4374-9933-bbedbd1c46bd","arxiv_id":"2507.11436","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Symmetric activation functions Tanh and Abs outperform ReLU in fNIRS deep learning classification, with a controlled parameter sweep supporting the role of symmetry.","lead":"This paper compares seven activation functions across four deep learning architectures for classifying brain signals measured with fNIRS, and finds that symmetric functions like Tanh and Abs often beat the default ReLU. The result could help researchers building brain-computer interfaces and cognitive-state decoders choose better building blocks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MAF symmetry experiment is internally inconsistent: MAF(α=-1) should reproduce Absolute and MAF(α=0) should reproduce ReLU, yet AbsoluteNet test accuracy differs by 8 and 20 percentage points between the two tables, so the controlled comparison is not controlled.","rationale":"The reader's weakest assumption concerns the biological plausibility of symmetric activations; my concern is more direct and internal to the reported experiments. The centerpiece of the symmetry argument is the MAF experiment, but it fails a basic sanity check. Since MAF with α=-1 is algebraically identical to |x| and α=0 to ReLU, the two tables should agree under the stated identical pipeline. The discrepancies (8 and 20 percentage points in AbsoluteNet) cannot be explained by CV-fold standard deviations because the same folds and architecture should produce the same results; they must stem from different random seeds, different fold splits, or implementation differences. This means the MAF numbers do not provide a controlled comparison, and the conclusion that symmetry is beneficial rests on uncontrolled variation. The other tables lack significance tests and code, so one cannot tell whether the Tanh/Abs advantages are larger than seed noise. I would not reject the underlying hypothesis, but the current manuscript's evidence is unreliable; a reanalysis with fixed seeds, matched folds, and paired tests is necessary before the central claim can be accepted.","tokens_in":9806,"tokens_out":6562,"duration_ms":79232,"concrete_test":"Re-run AbsoluteNet and MDNN with MAF(α=-1), MAF(α=0), plain Absolute, and plain ReLU under the same five fixed seeds and report per-seed test accuracies; verify that MAF(α=-1) exactly matches Absolute and MAF(α=0) exactly matches ReLU. If the match is not within numerical tolerance, the MAF comparison is invalid. Also compute a paired permutation test across CV folds for the Tanh/Abs vs ReLU comparisons to establish whether the headline differences are above seed noise.","verdict_should_be":"REJECT","load_bearing_attack":"Section IV and Table VII present the Modified Absolute Function as a controlled test of symmetry. By Eq. (8), MAF(x)=x for x≥0 and αx for x<0; hence α=-1 is exactly Abs(x) and α=0 is exactly ReLU(x). The manuscript reports AbsoluteNet test accuracy of 85.70±2.75 for α=-1 (Table VII) versus 77.73±4.73 for Absolute (Table IV), and 55.94±8.44 for α=0 (Table VII) versus 75.76±2.92 for ReLU (Table IV). MDNN shows similar mismatches (α=-1: 76.53 vs 78.92; α=0: 72.17 vs 73.96). If the pipeline is the same, identical activation functions should give the same results; the observed differences imply uncontrolled seed or optimization variance, or inconsistent implementations. Therefore the focused analysis does not isolate symmetry, and the 20-point drop attributed to ReLU in AbsoluteNet is not a measured effect. Table VII also contains an apparent typo (MDNN α=2: 64.95±64.95), further reducing confidence in the table's reliability.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical benchmark of seven activation functions (ReLU, ELU, Swish, Sigmoid, Tanh, Square, Absolute) in four deep architectures (fNIRSNet, AbsoluteNet, MDNN, ShallowConvNet) for binary classification of fNIRS data recorded during an auditory oddball task. All models are trained on the same dataset (9 subjects, 918 trials per class) with a unified preprocessing and training protocol, and results are reported as mean ± standard deviation over 5-fold cross-validation. The paper claims that symmetric activation functions such as Tanh and Abs(x) can outperform ReLU in several architectures, and introduces a Modified Absolute Function (MAF) with varying negative-side scaling to isolate the effect of symmetry. The conclusions emphasize that activation-function choice should be aligned with the bidirectional nature of fNIRS signals.","tokens_in":1543,"tokens_out":1679,"duration_ms":64296,"significance":"If the findings are reliable, the paper offers a practically useful guideline for fNIRS deep learning: symmetric activations like Tanh and Absolute may be better default choices than ReLU in domain-specific architectures. The study is commendable for using a standardized pipeline, consistent hyperparameters, and a domain-specific activation family (MAF) to probe symmetry. The paper also includes an explicit limitations section acknowledging the single-dataset scope. However, the evidence base is currently thin: a single small dataset, no statistical significance testing, and an internally inconsistent MAF comparison that undermines the paper's most direct controlled test. Because the central claim is empirical and benchmark-oriented, the value of the paper depends on the soundness of the comparison, which is not yet established.","major_comments":[{"comment":"The MAF experiment is internally inconsistent. By Eq. (8), MAF(x)=x for x≥0 and MAF(x)=αx for x<0, so α=-1 is exactly Abs(x) and α=0 is exactly ReLU(x). Yet the reported test accuracies for these identical functions differ substantially between the main tables and Table VII. For AbsoluteNet, Table VII reports α=-1 as 85.70±2.75 while Table IV reports Abs as 77.73±4.73, and α=0 as 55.94±8.44 while ReLU is 75.76±2.92. For MDNN, α=-1 is 76.53±2.84 versus Abs 78.92±2.56, and α=0 is 72.17±5.57 versus ReLU 73.96±4.13. Since the functions are mathematically identical, the differences imply uncontrolled variation in the experimental pipeline (seeds, initialization, early stopping, or preprocessing) between the two sets of runs. As presented, the MAF analysis does not isolate symmetry; it reintroduces the very confounds it is designed to control. This is load-bearing because the conclusion that the MAF analysis 'confirmed that symmetrical activations like Absolute outperform ReLU' rests on this comparison. The authors must either reconcile the results with identical conditions or remove the MAF-based symmetry claim.","section":"Section IV, Tables IV, V, VII"},{"comment":"The MDNN α=2 row reports a test accuracy of 64.95 with a standard deviation of 64.95. A standard deviation equal to the mean is implausible for an accuracy measure and strongly suggests a typographical error (e.g., a misplaced decimal or a mis-copied value). This error, combined with the inconsistencies described above, reduces confidence in the reliability of Table VII and should be corrected before the paper can be considered publishable.","section":"Table VII"},{"comment":"The study uses a single dataset of 9 subjects and 5-fold cross-validation, but performs no statistical significance tests to support the ranking of activation functions. Many of the reported differences are within one standard deviation; for example, MDNN Tanh (74.94±3.27) versus ReLU (73.96±4.13) is a difference of less than 1 percentage point, and in ShallowConvNet ReLU (66.56±2.74) actually exceeds Absolute (64.98±4.25). Even larger apparent gaps, such as AbsoluteNet Tanh (86.33±2.10) versus ReLU (75.76±2.92), would be more convincing with a paired test over the five folds or across subjects. Without such tests, the central claim that symmetric activations 'can outperform' ReLU is not statistically established, and the generalization beyond this dataset is speculative. The authors should add appropriate significance tests or explicitly temper the conclusions to the measured folds.","section":"Section III.B and Section IV"}],"minor_comments":[{"comment":"The sentence 'nonlinearity, low signal-to-noise ratio (SNR), and signal variability poses significant challenges' has a subject-verb agreement error: 'pose' should be used with the plural subject. Please correct the verb.","section":"Abstract"},{"comment":"The text refers to 'tables III–IV' when results actually span Tables III through VII. This cross-reference is incorrect and should be updated.","section":"Section IV, first paragraph"},{"comment":"The paragraph states 'Table IV presents a controlled evaluation of the Modified Absolute Function,' but the MAF results are in Table VII, not Table IV. This is a cross-reference error that should be fixed alongside the data-consistency issues above.","section":"Section IV, MAF paragraph"},{"comment":"The header cell 'Simoid' is a typo for 'Sigmoid.' In the same table, 'Square' is listed as non-smooth, but the square function is smooth; the table appears to conflate smoothness of the function with properties of its gradient at zero. This should be clarified.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The MAF inconsistency is the most serious issue: identical activations yield very different results in Tables IV/V and Table VII, which suggests either an uncontrolled pipeline change or an error in one of the tables. I would ask the authors to rerun the MAF experiments under exactly the same conditions as the main benchmark, or to clearly explain the discrepancy. The lack of statistical tests is also a barrier for a journal-level publication. The paper is otherwise within scope for a venue interested in applied deep learning for neuroimaging, but the empirical claims need stronger support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the good news: this is the first systematic comparison of activation functions for fNIRS classification across multiple architectures, and the main benchmark in Tables III–VI is run carefully—same preprocessing, same optimizer and learning rate, same 5-fold CV, softmax output. The finding that Tanh matches or beats ReLU on all four networks is plausible and worth knowing. The authors also list their limitations honestly (single auditory dataset, homogeneous activation replacement) and don't inflate clinical claims.\n\nThe problem is the MAF experiment, which is the paper's key evidence for the symmetry hypothesis. By Eq. (8), MAF(α=-1) is exactly Abs(x) and MAF(α=0) is exactly ReLU(x). Yet Table VII reports AbsoluteNet test accuracies of 85.70±2.75 for α=-1 versus 77.73±4.73 for Absolute in Table IV, and 55.94±8.44 for α=0 versus 75.76±2.92 for ReLU. MDNN shows the same pattern. These gaps are far larger than the reported stds, and the paper says the pipeline is identical, so the controlled comparison is not controlled. Either the implementation changed or the seeds/optimization were not held fixed. That means the MAF numbers cannot isolate symmetry. Table VII also has a clear typo (MDNN α=2: 64.95±64.95), which suggests the table wasn't checked. Without the MAF result, the symmetry claim rests on the main tables, where the evidence is more mixed: Absolute does not beat ReLU in ShallowConvNet, and in fNIRSNet the difference is within noise.\n\nOther soft spots: no statistical significance tests, so many pairwise differences sit inside one std; the dataset is small (9 subjects, 918 trials per class); no code is released. The conclusion that Absolute \"consistently ranked among top-performing\" is overbroad for the same reason.\n\nNet: the benchmark part is a useful data point, but the paper's central argument—symmetry helps—is not supported by the evidence as presented. I would send this to a serious referee, because the question is relevant and the main tables are publishable after fixing the MAF analysis, adding significance tests, and softening the claims. As a reviewer, I'd ask for a major revision. I would not cite it in its current form.","headline":"The MAF symmetry experiment fails its own control, so the paper's central claim about symmetric activations is unsupported despite a useful benchmark.","tokens_in":10526,"tokens_out":5102,"would_cite":false,"duration_ms":55495,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On fNIRS classification, symmetric activations like Tanh and Abs(x) outperform ReLU in most deep architectures, with gains depending on the network.","keywords":["functional near-infrared spectroscopy (fNIRS)","fNIRS classification","activation functions","symmetric activation","Tanh","Absolute function","convolutional neural network","auditory oddball"],"falsifier":"Retrain the same four networks on a second fNIRS dataset, such as motor imagery or cognitive workload, and sweep $\\alpha$ in the Modified Absolute Function exactly as this paper does; if $\\alpha=0$ ($\\mathrm{ReLU}$) matches or beats $\\alpha=-1$ ($\\mathrm{Abs}$) there, the claim that symmetric activations are inherently better for fNIRS fails to generalize. Alternatively, on the current dataset, remove the sign structure of the inputs (for instance by taking per-channel absolute values before the network); if $\\mathrm{Tanh}$ and $\\mathrm{Abs}$ then lose their edge over $\\mathrm{ReLU}$, the polarity-preservation mechanism is confirmed.","tokens_in":9632,"feed_emoji":"🧠","tokens_out":9283,"duration_ms":92405,"temperature":0.7,"pith_summary":"Functional near-infrared spectroscopy (fNIRS) records brain hemodynamic responses that are noisy, low in signal-to-noise ratio, and highly variable, yet deep-learning classifiers for fNIRS usually default to ReLU activations without evidence that this suits the signal. This paper compares seven activation functions across four convolutional architectures — fNIRSNet, AbsoluteNet, MDNN, and ShallowConvNet — on a single auditory oddball dataset, holding preprocessing, optimizer, batch size, and training schedule fixed. Its central finding is that symmetric activation functions, especially $\\mathrm{Tanh}$ and the absolute-value function $\\mathrm{Abs}(x)$, reach higher test accuracy than $\\mathrm{ReLU}$ in most of the networks tested. A controlled variant called the Modified Absolute Function interpolates between $\\mathrm{ReLU}$ and $\\mathrm{Abs}$ and shows that keeping the negative side of neuron pre-activations is what drives the gain. If this holds, fNIRS practitioners can improve classification accuracy by choosing activations matched to the bidirectional nature of hemodynamic signals instead of defaulting to $\\mathrm{ReLU}$.","feed_headline":"Tanh and Abs beat ReLU for fNIRS brain-signal classification","feed_subtitle":"Swapping ReLU for Tanh or Abs raises test accuracy across four deep architectures on auditory fNIRS data.","key_machinery":"The central device is activation-function symmetry, meaning evenness, $f(x)=f(-x)$, implemented as the Modified Absolute Function $\\mathrm{MAF}(x)=x$ for $x\\ge 0$ and $\\mathrm{MAF}(x)=\\alpha x$ for $x<0$. This one-parameter family contains $\\mathrm{ReLU}$ ($\\alpha=0$) and $\\mathrm{Abs}$ ($\\alpha=-1$) as special cases, so sweeping $\\alpha$ under a fixed architecture isolates what preserving or discarding the negative pre-activations contributes, independently of network depth or training setup. The four networks supply the test beds: two dual-branch spatio-temporal CNNs (fNIRSNet, AbsoluteNet), a deeper single-branch CNN (MDNN), and a shallow temporal-spatial baseline (ShallowConvNet).","core_discovery":"The paper's central claim is that activation-function choice materially changes fNIRS classification accuracy, and that symmetric functions are the better default in this domain. Across fNIRSNet, AbsoluteNet, and MDNN, $\\mathrm{Tanh}$ and/or $\\mathrm{Abs}$ deliver the highest test accuracy — up to $86.33\\%$ for $\\mathrm{Tanh}$ on AbsoluteNet versus $75.76\\%$ for $\\mathrm{ReLU}$, and $78.92\\%$ for $\\mathrm{Abs}$ on MDNN versus $73.96\\%$ for $\\mathrm{ReLU}$ — while $\\mathrm{ReLU}$ is outperformed in most cases and the Square function is consistently the weakest standalone activation. The MAF experiment, which scales negative inputs by $\\alpha \\in \\{-2,-1,0,2\\}$, isolates symmetry as the mechanism: $\\alpha=-1$ (Abs) and $\\alpha=-2$ beat $\\alpha=0$ (ReLU) on both networks tested, and $\\alpha=2$ degrades performance, in one case to chance level. The authors conclude that fNIRS signals carry meaningful information in both positive and negative polarities around the mean, and that symmetric activations preserve that information.","pith_inferences":["Beyond the paper's data, the polarity explanation predicts that any preprocessing that destroys the sign structure of the hemodynamic response — for example, aggressive detrending or per-channel absolute-value scaling — should erase the $\\mathrm{Tanh}$/$\\mathrm{Abs}$ advantage over $\\mathrm{ReLU}$; this is directly testable on the same dataset.","If symmetry is the active ingredient, the result may transfer to other zero-centered, bidirectional neurophysiological signals such as EEG or MEG, but the single-dataset design leaves that open.","A learnable MAF, with $\\alpha$ trained per layer or per channel, would let a network decide how much negative information to preserve and could outperform both fixed $\\mathrm{ReLU}$ and fixed $\\mathrm{Abs}$; the paper's $\\alpha$ sweep supplies the baseline for such a variant.","Because only nine subjects contributed the data, the observed gaps may partly reflect subject-specific response shapes; an across-subject replication with more participants is the most direct way to confirm that symmetry itself — not subject idiosyncrasy — drives the result."],"forward_implications":["Choosing $\\mathrm{Tanh}$ or $\\mathrm{Abs}$ over $\\mathrm{ReLU}$ raises test accuracy by several points in the deeper fNIRS-specific networks: AbsoluteNet goes from $75.76\\%$ with $\\mathrm{ReLU}$ to $86.33\\%$ with $\\mathrm{Tanh}$, and MDNN from $73.96\\%$ to $78.92\\%$ with $\\mathrm{Abs}$.","The benefit of symmetric activations shrinks in the shallow baseline, so the gain appears to scale with network depth and complexity rather than being a universal constant.","Discarding negative pre-activations, as $\\mathrm{ReLU}$ does, limits representational power for fNIRS signals that vary in both directions around a baseline.","The Square function, despite appearing in some domain-specific designs as part of a mixture, performs poorly as a standalone activation and is not recommended alone.","A heterogeneous strategy — different activation functions in different layers or network branches — is the natural next step the paper identifies but does not test."],"supporting_citations":[{"why":"Supplies the fNIRSNet architecture, one of the two dual-branch spatio-temporal CNNs tested.","marker":"[16]"},{"why":"Supplies the AbsoluteNet architecture, the other dual-branch network and the one where Tanh reaches its top accuracy.","marker":"[17]"},{"why":"Supplies the MDNN architecture, the deeper single-branch network where Abs achieves the highest test accuracy.","marker":"[15]"},{"why":"Supplies the ShallowConvNet baseline, the shallow network used to gauge depth dependence.","marker":"[18]"},{"why":"Supplies the Swish activation function, one of the seven functions compared in the study.","marker":"[19]"},{"why":"Supplies the auditory oddball dataset on which all models are trained and tested.","marker":"[20]"}],"fun_headline_variants":["Symmetric activations beat ReLU for fNIRS brain signals","Tanh and Abs outperform ReLU in fNIRS deep learning","Symmetric activations boost fNIRS classification accuracy","Activation symmetry matters for fNIRS deep nets","fNIRS classification: Tanh and Abs beat ReLU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fNIRS signals carry class-relevant information in both positive and negative pre-activation values, so a symmetric activation that keeps both polarities produces the accuracy gains; if that polarity information is absent or specific to this auditory task, the advantage over $\\mathrm{ReLU}$ will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["Symmetric activations beat ReLU for fNIRS brain signals","Tanh and Abs outperform ReLU in fNIRS deep learning","Symmetric activations boost fNIRS classification accuracy","Activation symmetry matters for fNIRS deep nets","fNIRS classification: Tanh and Abs beat ReLU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00056,"raw_usage":{"total_tokens":2708,"prompt_tokens":1038,"completion_tokens":1670,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":1586}},"tokens_in":654,"tokens_out":1670,"duration_ms":13049,"temperature":1.0,"reasoning_tokens":1586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:07:57.586411+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same four networks on a second fNIRS dataset, such as motor imagery or cognitive workload, and sweep $\\alpha$ in the Modified Absolute Function exactly as this paper does; if $\\alpha=0$ ($\\mathrm{ReLU}$) matches or beats $\\alpha=-1$ ($\\mathrm{Abs}$) there, the claim that symmetric activations are inherently better for fNIRS fails to generalize. Alternatively, on the current dataset, remove the sign structure of the inputs (for instance by taking per-channel absolute values before the network); if $\\mathrm{Tanh}$ and $\\mathrm{Abs}$ then lose their edge over $\\mathrm{ReLU}$, the polarity-preservation mechanism is confirmed.","supporting_citations":[{"cited_title":"Adeli, J","cited_arxiv_id":null,"evidence_quote":"Supplies the AbsoluteNet architecture, the other dual-branch network and the one where Tanh reaches its top accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MDNN architecture, the deeper single-branch network where Abs achieves the highest test accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ShallowConvNet baseline, the shallow network used to gauge depth dependence."},{"cited_title":"Elfwing, E","cited_arxiv_id":null,"evidence_quote":"Supplies the Swish activation function, one of the seven functions compared in the study."},{"cited_title":"McLinden, S","cited_arxiv_id":null,"evidence_quote":"Supplies the auditory oddball dataset on which all models are trained and tested."}],"review_version":1}