Functional Attention replaces pairwise softmax attention with structured linear operators inspired by geometric functional maps to produce compact, resolution-invariant representations for operator learning.
hub
On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning
14 Pith papers cite this work. Polarity classification is still indexing.
14
Pith papers citing it
abstract
In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax function is the monotone gradient map of the log-sum-exp function. By exploiting this connection, we show that the inverse temperature parameter determines the Lipschitz and co-coercivity properties of the softmax function. We then demonstrate the usefulness of these properties through an application in game-theoretic reinforcement learning.
hub tools
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 14roles
background 1polarities
background 1representative citing papers
For finite-dimensional affine logit systems the sharp dimension-free stability threshold is β‖ΠWΠ‖_{T→T}<2, extending the certified regime beyond classical conservative bounds.
Establishes last-iterate convergence rates for on-policy Q-learning under minimal irreducibility assumptions, with sample complexity O(1/ξ²) matching off-policy up to exploration factors.
ConfSMoE adds expert-opinion imputation and detaches softmax routing scores to ground-truth task confidence to relieve expert collapse in SMoE without extra load-balance losses, evaluated on four real-world datasets.
Bayesian softmax-gated mixture-of-experts models achieve posterior contraction for density estimation and parameter recovery using Voronoi losses, plus two strategies for choosing the number of experts.
Policy composition emerges from variational free energy minimization through a convergent gradient flow with a soft-competitive recurrent neural implementation.
Derives algorithm-dependent generalization bounds for neural nets using multilevel entropic regularization and proposes a Metropolis-simulated multi-scale Gibbs training procedure tested on a two-layer net for MNIST.
SC-DN establishes a global first-order stationary point per round and solves a mixed-integer signomial program to optimize four control variables for VFL, yielding better classification performance and lower resource use than greedy baselines on image and multi-modal data.
Common ID estimators fail to track the true intrinsic dimension of neural representations and are instead driven by other factors.
Decentralized Q-learning agents reach an Empirical Evidence Equilibrium in weakly coupled dynamic environments.
InGSL reduces edge redundancy in existing graph structure learning methods by adding a mutual-information-guided diversity term, delivering better results with fewer edges across six tested frameworks.
Unified ODE convergence analysis for smooth Q-learning variants via p-norm Lyapunov functions, valid even when the Bellman operator is not a contraction.
SCGFM creates transferable graph representations by aligning heterogeneous topologies to shared learnable geometric bases via Gromov-Wasserstein distances and re-encoding features accordingly.
QAOA ansatz with finite layers can capture any bitstring distribution and solves the Fair Cut Cover problem with provable and empirical advantages over classical approximations on certain graphs.
citing papers explorer
-
Functional Attention: From Pairwise Affinities to Functional Correspondences
Functional Attention replaces pairwise softmax attention with structured linear operators inspired by geometric functional maps to produce compact, resolution-invariant representations for operator learning.
-
Sharp Spectral Thresholds for Logit Fixed Points
For finite-dimensional affine logit systems the sharp dimension-free stability threshold is β‖ΠWΠ‖_{T→T}<2, extending the certified regime beyond classical conservative bounds.
-
On Bayesian Softmax-Gated Mixture-of-Experts Models
Bayesian softmax-gated mixture-of-experts models achieve posterior contraction for density estimation and parameter recovery using Voronoi losses, plus two strategies for choosing the number of experts.
-
Optimizing Server Placement for Vertical Federated Learning in Dynamic Edge/Fog Networks
SC-DN establishes a global first-order stationary point per round and solves a mixed-integer signomial program to optimize four control variables for VFL, yielding better classification performance and lower resource use than greedy baselines on image and multi-modal data.
-
Rethinking Intrinsic Dimension Estimation in Neural Representations
Common ID estimators fail to track the true intrinsic dimension of neural representations and are instead driven by other factors.
-
Learning Empirical Evidence Equilibria under Weak Environmental Coupling
Decentralized Q-learning agents reach an Empirical Evidence Equilibrium in weakly coupled dynamic environments.
-
Informative Graph Structure Learning
InGSL reduces edge redundancy in existing graph structure learning methods by adding a mutual-information-guided diversity term, delivering better results with fewer edges across six tested frameworks.
-
Structure-Centric Graph Foundation Model via Geometric Bases
SCGFM creates transferable graph representations by aligning heterogeneous topologies to shared learnable geometric bases via Gromov-Wasserstein distances and re-encoding features accordingly.
-
Learning Cut Distributions with Quantum Optimization
QAOA ansatz with finite layers can capture any bitstring distribution and solves the Fair Cut Cover problem with provable and empirical advantages over classical approximations on certain graphs.