Pith. sign in

REVIEW 23 cited by

Mish: A Self Regularized Non-Monotonic Activation Function

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1908.08681 v3 pith:QQZ22ZJ6 submitted 2019-08-23 cs.LG cs.CVcs.NEstat.ML

Diganta Misra This is my paper
classification cs.LGcs.CVcs.NEstat.ML
keywords mishactivationfunctionsarchitecturesbenchmarksfunctionimagenet-1kms-coco
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

We propose $\textit{Mish}$, a novel self-regularized non-monotonic activation function which can be mathematically defined as: $f(x)=x\tanh(softplus(x))$. As activation functions play a crucial role in the performance and training dynamics in neural networks, we validated experimentally on several well-known benchmarks against the best combinations of architectures and activation functions. We also observe that data augmentation techniques have a favorable effect on benchmarks like ImageNet-1k and MS-COCO across multiple architectures. For example, Mish outperformed Leaky ReLU on YOLOv4 with a CSP-DarkNet-53 backbone on average precision ($AP_{50}^{val}$) by 2.1$\%$ in MS-COCO object detection and ReLU on ResNet-50 on ImageNet-1k in Top-1 accuracy by $\approx$1$\%$ while keeping all other network parameters and hyperparameters constant. Furthermore, we explore the mathematical formulation of Mish in relation with the Swish family of functions and propose an intuitive understanding on how the first derivative behavior may be acting as a regularizer helping the optimization of deep neural networks. Code is publicly available at https://github.com/digantamisra98/Mish.

Discussion (0). Sign in to comment.

Forward citations

Cited by 23 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Silent Impact: Tracking Tennis Shots from the Passive Arm

    cs.HC 2025-07 conditional novelty 7.0 of 10

    Tennis shots can be detected and classified from IMU data on the passive arm alone, with accuracy close to dominant-arm sensors and lower perceived burden.

  2. Compress-Align-Detect: onboard change detection from unregistered images

    cs.CV 2025-07 conditional novelty 7.0 of 10

    A single neural network performs compression, co-registration, and change detection onboard a satellite, achieving F1 up to about 70% at low bitrates on simulated unregistered image pairs.

  3. Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Action chunking in robotic behavioral cloning works mainly because it acts as a delayed-prediction policy and an implicit ensemble, not because of temporal consistency or horizon reduction.

  4. UniRank: Benchmarking Ranking Models for Unified Sequential Modeling and Feature Interaction

    cs.IR 2026-07 conditional novelty 6.0 of 10

    UniRank is an open benchmark that standardizes chronological autoregressive supervision, multi-task evaluation, and capacity controls for 15 unified ranking models on five large datasets.

  5. Rethinking Neural Nonlinearity as Gating

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Common activations and softmax are instances of a single input-conditioned Threshold Gating primitive with few branches, enabling lossless conversion and a unified analog implementation path.

  6. LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian process

    stat.ML 2025-07 conditional novelty 6.0 of 10

    A hybrid model coupling a Gaussian process latent field with a neural operator provides uncertainty estimates for forward and inverse PDE problems with noisy data.

  7. Latent-attention Based Transformer for Near ML Polar Decoding in Short-code Regime

    eess.SP 2025-07 conditional novelty 6.0 of 10

    A code-aware latent-attention transformer matches near-ML error rates for short polar codes only after per-configuration fine-tuning.

  8. EAT: QoS-Aware Edge-Collaborative AIGC Task Scheduling via Attention-Guided Diffusion Reinforcement Learning

    cs.DC 2025-07 conditional novelty 6.0 of 10

    EAT, an attention-guided diffusion reinforcement learning scheduler for collaborative edge AIGC, reduces Stable Diffusion inference latency by 56-74% versus baselines while keeping CLIP quality nearly unchanged.

  9. Transformers Learn Faster with Semantic Focus

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Input-dependent top-k sparse attention makes small transformers converge faster and generalize as well as full attention, while input-agnostic sparsity does not, and the effect is tied to reduced dispersion of attenti...

  10. Meteosat Third Generation imagery improves CNN-based SSI retrieval

    physics.ao-ph 2026-07 conditional novelty 5.5 of 10

    Higher-resolution MTG/FCI imagery cuts CNN SSI RMSE under overcast and cloudy skies versus MSG-only, with ~20% overall skill over SARAH-3, but not under clear skies.

  11. Relative Value Learning

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A critic that learns antisymmetric value differences ∆(s_i,s_j)=V(s_i)−V(s_j) has a provably contracting Bellman operator and an unbiased advantage estimator, and PPO with this critic matches standard PPO on Atari.

  12. Measurement of the branching ratio of the $K^{+}\rightarrow\pi^{+}\nu\bar{\nu}$ decay

    hep-ex 2026-07 accept novelty 5.0 of 10

    NA62 measures Br(K⁺→π⁺νν̄) = (9.6±1.9)×10⁻¹¹ from 2016–2024 data, a 20% relative-precision measurement consistent with the Standard Model.

  13. Path optimization method for the sign problem: Insights from random matrix models

    hep-lat 2026-07 conditional novelty 5.0 of 10

    Path optimization improves the average phase factor in the Stephanov model at high chemical potential but not at low chemical potential or in the chiral random matrix model, pointing to the global sign problem as the ...

  14. Learning-based Physics-Constrained Neural Kernel for Sound Field Estimation With Source-Position-Dependent Directional Weighting

    cs.SD 2026-07 conditional novelty 5.0 of 10

    A source-position-dependent implicit neural representation for the directional weighting in a physics-constrained kernel enables sound field estimation that generalizes to unseen source positions without per-snapshot ...

  15. The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network

    cs.LG 2025-08 reject novelty 5.0 of 10

    The paper demonstrates non-monotonic move-policy dynamics in a chess transformer, but its abstract claims a causal safety-prior override result that never appears in the body.

  16. FLUX-Makeup: High-Fidelity, Identity-Consistent, and Robust Makeup Transfer via Diffusion Transformer

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    This document pairs a makeup-transfer abstract with a mismatched body about activation functions, leaving the claimed method unverifiable.

  17. Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos

    cs.CV 2025-07 conditional novelty 5.0 of 10

    MTID improves action sequence planning in instructional videos by adding a learnable latent interpolation module, a masked projection, and a task-adaptive proximity loss to a DDIM-based diffusion model.

  18. Distributional Soft Actor-Critic with Diffusion Policy

    cs.LG 2025-07 reject novelty 5.0 of 10

    DSAC-D couples a diffusion-based value distribution network with a diffusion policy, but its headline state-of-the-art claim is contradicted by its own benchmark table.

  19. Reinforcement Learning: From Algorithms To Foundation Models

    cs.AI 2026-07 conditional novelty 3.0 of 10

    A dissertation uniting the author's published results: non-exploitable Nash-DQN policies and the FightLadder benchmark for games, plus diffusion/consistency-model world models for RL — a compilation rather than new results.

  20. Hybrid activation functions for deep neural networks: S3 and S4 -- a novel approach to gradient flow optimization

    cs.LG 2025-07 reject novelty 3.0 of 10

    A new parameterized activation function, S4, that blends sigmoid and softsign through a smooth sigmoid-weighted transition is claimed to improve accuracy and convergence on small neural network benchmarks.

  21. Tangma: A Tanh-Guided Activation Function with Learnable Parameters

    cs.NE 2025-07 conditional novelty 3.0 of 10

    Tangma, defined as x times tanh(x plus a learned shift) plus a learned linear term, reaches 99.09% validation accuracy on MNIST and 78.15% on CIFAR-10 after 10 epochs, slightly beating ReLU, Swish, and GELU in these s...

  22. SoftReMish: A Novel Activation Function for Enhanced Convolutional Neural Networks for Visual Recognition Performance

    cs.CV 2025-07 reject novelty 2.0 of 10

    SoftReMish is presented as new but reduces to Mish with rescaled input and output, and its MNIST evidence is not self-consistent.

  23. YOLOv1 to YOLOv11: A Comprehensive Survey of Real-Time Object Detection Innovations and Challenges

    cs.CV 2025-08 unverdicted

    A survey of YOLO object detectors from version 1 to version 11 that compiles architectures, benchmarks, and applications, with several factual inconsistencies.

Pith tools