A Set-Transformer architecture with self-attention encodes Pauli-string correlations, optimizes via commutation objective, and finds symmetries with near-deterministic success on physical models like Ising and Toric code.
hub
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
36 Pith papers cite this work. Polarity classification is still indexing.
36
Pith papers citing it
abstract
The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution. After refactoring, the gradients of the loss propagated by the chain rule through the graph are low variance unbiased estimators of the gradients of the expected loss. While many continuous random variables have such reparameterizations, discrete random variables lack useful reparameterizations due to the discontinuous nature of discrete states. In this work we introduce Concrete random variables---continuous relaxations of discrete random variables. The Concrete distribution is a new family of distributions with closed form densities and a simple reparameterization. Whenever a discrete stochastic node of a computation graph can be refactored into a one-hot bit representation that is treated continuously, Concrete stochastic nodes can be used with automatic differentiation to produce low-variance biased gradients of objectives (including objectives that depend on the log-probability of latent stochastic nodes) on the corresponding discrete graph. We demonstrate the effectiveness of Concrete relaxations on density estimation and structured prediction tasks using neural networks.
hub tools
citation-role summary
background 1
method 1
citation-polarity summary
verdicts
UNVERDICTED 36representative citing papers
INSHAPE discovers instance-specific non-overlapping shapelets, models their temporal dependencies, and aggregates them bottom-up into population-level prototypes for improved accuracy and interpretability in time-series classification.
FLDD learns non-Markovian marginal and posterior distributions for the forward process so a factorized reverse process can match the target better and produce higher-quality samples in fewer steps.
HPO enables unbiased policy optimization in hybrid action spaces by mixing differentiable simulation gradients with score-function estimates, outperforming PPO as continuous dimensions increase.
In-context symbolic regression methods improve robustness of symbolic formula recovery from KANs, cutting median OFAT test MSE by up to 99.8 percent across hyperparameter sweeps.
Non-Euclidean distance variants of harmonic loss improve accuracy, gradient stability, and energy efficiency over cross-entropy and Euclidean harmonic loss in vision backbones and large language models.
MirrorCheck detects adversarial attacks on VLMs via T2I regeneration for semantic consistency checks, using stochastic model selection and one-time perturbations for robustness against adaptive attacks.
LeapTS reformulates forecasting as adaptive multi-horizon scheduling via hierarchical control and NCDEs, delivering at least 7.4% better performance and 2.6-5.3x faster inference than Transformer baselines while adapting to non-stationary dynamics.
DTSemNet gives an exact, invertible neural-network encoding of hard oblique decision trees that supports direct gradient training for both classification and regression without probabilistic softening or quantized estimators.
AS-LoRA adaptively chooses which LoRA factor to update per layer and round using a curvature-aware second-order score, eliminating reconstruction error floors and improving performance in DP federated learning.
ARCH is a hierarchical flow-based generative model that enables tractable conditional intensity computation and arbitrary conditioning for spatiotemporal event distributions.
LumiMotion improves albedo estimation and scene relighting in dynamic scenes by leveraging motion to separate lighting effects from surface appearance in a dynamic 2D Gaussian Splatting representation.
LMFT enables state-of-the-art performance in video unsupervised domain adaptation by focusing on motion-rich tokens and reducing computational overhead.
FedLAB organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context via federated semantic barycenter pre-training, improving performance by up to 7.53% on benchmarks while enabling semantic traceability.
The generalization advantage of SGD over random sampling diminishes with growing training set size in binary networks, as measured by joint density of states over train and test accuracy.
A binary-bottleneck model discovers object and action symbols from multi-modal effect predictions on random interactions, then uses discrete planning on predicted trajectories for tabletop repositioning and stacking with few-shot generalization to novel objects.
Bayesian shrinkage priors on factor models produce sparse substitute confounders that support consistent regression-adjusted causal estimates under latent variable identification assumptions.
HPME proposes hard-perturbation mixup explainer grounded in generalized Graph Information Bottleneck to extract discrete subgraphs and generate in-distribution explanations that outperform soft-mask approaches on synthetic and real datasets.
Entropy-adaptive Gumbel-Sinkhorn formulation for unsupervised permutation learning that modulates temperature per assignment to address non-uniform uncertainty.
SNMPP builds a product-form neural influence kernel from a signed interaction network over event classes and a delay-aware monotonic temporal network to enable explicit discovery of inter-event relationships alongside strong prediction.
GSQ uses Gumbel-Softmax to optimize scalar quantization grids for LLMs, closing most of the accuracy gap to vector methods like QTIP at 2-3 bits per parameter while using symmetric scalar grids compatible with existing kernels.
A differentiable logic programming approach optimizes continuous gate switches to discover and adapt quantum circuits while satisfying user-defined logical axioms.
LILogicNet trains compact logic-gate networks with learnable sparse connectivity via Top-K selection, reaching 98.45% MNIST accuracy with 8k gates and 60.98% CIFAR-10 accuracy with 256k gates while using far fewer gates than prior logic models.
A two-stage framework learns a world graph of pivotal states task-agnostically via joint training of a latent model and curiosity-driven policy, then uses the graph to accelerate hierarchical RL on maze tasks.
citing papers explorer
-
Attention-based optimizer for symmetry finding
A Set-Transformer architecture with self-attention encodes Pauli-string correlations, optimizes via commutation objective, and finds symmetries with near-deterministic success on physical models like Ising and Toric code.
-
Differentiable Logical Programming for Quantum Circuit Discovery and Optimization
A differentiable logic programming approach optimizes continuous gate switches to discover and adapt quantum circuits while satisfying user-defined logical axioms.