TIDE enables the first cross-architecture distillation of dLLMs, improving a 0.6B student by 1.53 average points over baselines when trained from 8B dense and 16B MoE teachers.
Title resolution pending
319 Pith papers cite this work. Polarity classification is still indexing.
319
Pith papers citing it
citation-role summary
background 3
method 1
citation-polarity summary
representative citing papers
JumpLoRA uses JumpReLU gating to induce adaptive sparsity in LoRA blocks, achieving dynamic parameter isolation that prevents task interference and improves continual learning performance over IncLoRA and ELLA.
LLM judges exhibit up to 9.8 percentage point leniency bias from stakes signaling in prompts, acting implicitly without mentioning it in chain-of-thought.
InfiniteScienceGym procedurally generates unbounded scientific repositories with exact ground-truth QA pairs to benchmark LLMs on data reasoning, abstention, and tool use without static datasets.
EnsembleCert and ScaLabelCert enable tighter and exact certificates for neural network robustness against label-flipping attacks by leveraging white-box information and neural tangent kernel equivalence.
Steered LLM activations are non-surjective: under practical assumptions, they lie outside the set of states reachable from any discrete prompt.
AgentSocialBench demonstrates that privacy preservation is fundamentally harder in human-centered agentic social networks than in single-agent cases due to cross-domain coordination pressures and an abstraction paradox where privacy instructions increase discussion of sensitive information.
MiCP is the first conformal prediction method for multi-turn LLM pipelines that allocates per-turn error budgets to enable adaptive stopping with an overall coverage guarantee, shown to reduce turns and cost on RAG and ReAct benchmarks.
The paper proves W[1]-hardness parameterized by dimension d for positivity, zonotope containment, max approximation, and L_p-Lipschitz constants in 2- and 3-layer ReLU networks, showing enumeration methods are optimal under ETH.
RLCracker is a reinforcement learning attack that erases LLM watermarks at 98.5% success rate with minimal data and generalizes across ten schemes and multiple model sizes.
ErrorRadar is a new benchmark of 2,500 multimodal K-12 math problems for MLLM error step identification and categorization, where GPT-4o trails human experts by ~10%.
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
A noisy top-k gated mixture-of-experts layer between LSTMs scales neural networks to 137B parameters with sub-linear compute, beating SOTA on language modeling and machine translation.
A first-order stochastic optimizer that maintains bias-corrected exponential moving averages of the gradient and its square, dividing the former by the square root of the latter to set per-parameter step sizes.
Physics foundation models show regime-specific performance biases on a benchmark with 8 dynamics and 25 test regimes, indicating they are not universal generalists.
AutoSP automates sequence parallelism and long-context activation checkpointing via compilation, enabling up to 2.7x longer training contexts on NVIDIA hardware with negligible throughput loss.
VLM judges exhibit task-dependent uncertainty in their scores, with conformal prediction revealing wide intervals for complex tasks and a decoupling between good ranking performance and poor absolute scoring reliability.
C2C is a new testbed where LM agents negotiate differently from humans and targeted prompting raises their win rate from 22.2% to 32.7% across 1,100+ games.
XGRAG uses graph perturbations to quantify component contributions in GraphRAG and achieves 14.81% better explanation quality than text-based baselines on QA datasets, with correlations to graph centrality.
GraphPlanner augments multi-agent LLM routing with a heterogeneous graph memory and RL-optimized MDP workflow generation, delivering up to 9.3% higher accuracy and over 99% lower GPU cost than prior routers while supporting zero-shot generalization.
MMEB-V3 benchmark shows omni-modality embedding models fail to enforce instruction-specified modality constraints and exhibit asymmetric, query-biased retrieval.
A new SFT framework for MoE models combines bias-driven sparsification with gated condenser experts to retain long-tailed expert information, outperforming DenseMixer and ESFT by over 2.5% on math reasoning and commonsense QA benchmarks.
Abstract-CoT lets models reason with short discrete latent token sequences from a reserved vocabulary, using warm-up training and RL to match verbal CoT performance with up to 11.6x fewer tokens.
Humans show broad weak directional confusions while DNNs show sparse strong collapses; these structures shift rate-distortion geometry differently and reveal divergent inductive biases.
citing papers explorer
-
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
JumpLoRA uses JumpReLU gating to induce adaptive sparsity in LoRA blocks, achieving dynamic parameter isolation that prevents task interference and improves continual learning performance over IncLoRA and ELLA.
-
Exact Certification of Neural Networks and Partition Aggregation Ensembles against Label Poisoning
EnsembleCert and ScaLabelCert enable tighter and exact certificates for neural network robustness against label-flipping attacks by leveraging white-box information and neural tangent kernel equivalence.
-
Score-Based Generative Modeling through Stochastic Differential Equations
Introduces an SDE-based framework for score-based generative modeling that unifies prior methods, enables predictor-corrector sampling and neural ODE likelihoods, and achieves SOTA unconditional image generation on CIFAR-10.
-
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
A noisy top-k gated mixture-of-experts layer between LSTMs scales neural networks to 137B parameters with sub-linear compute, beating SOTA on language modeling and machine translation.
-
Adam: A Method for Stochastic Optimization
A first-order stochastic optimizer that maintains bias-corrected exponential moving averages of the gradient and its square, dividing the former by the square root of the latter to set per-parameter step sizes.
-
Do Physics Foundation Models Learn Generalizable Physics? A Bias-Aware Benchmark Across Physical Regimes and Distribution Shifts
Physics foundation models show regime-specific performance biases on a benchmark with 8 dynamics and 25 test regimes, indicating they are not universal generalists.
-
AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism
AutoSP automates sequence parallelism and long-context activation checkpointing via compilation, enabling up to 2.7x longer training contexts on NVIDIA hardware with negligible throughput loss.
-
VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation
VLM judges exhibit task-dependent uncertainty in their scores, with conformal prediction revealing wide intervals for complex tasks and a decoupling between good ranking performance and poor absolute scoring reliability.
-
Preserving Long-Tailed Expert Information in Mixture-of-Experts Tuning
A new SFT framework for MoE models combines bias-driven sparsification with gated condenser experts to retain long-tailed expert information, outperforming DenseMixer and ESFT by over 2.5% on math reasoning and commonsense QA benchmarks.
-
Randomized Antipodal Search Done Right for Data Pareto Improvement of LLM Unlearning
RASLIK uses randomized antipodal search on linearized influence kernels to achieve data Pareto improvement in LLM unlearning, outperforming baselines with sublinear complexity and double gains in quality and efficiency.
-
Reinforcement Learning via Value Gradient Flow
VGF solves behavior-regularized RL by transporting particles from a reference distribution to the value-induced optimal policy via discrete value-guided gradient flow.
-
The Verification Tax: Fundamental Limits of AI Auditing in the Rare-Error Regime
The minimax rate for estimating calibration error is Theta((L epsilon/m)^{1/3}), creating a verification tax that makes auditing harder as models improve.
-
$S^3$: Stratified Scaling Search for Test-Time in Diffusion Language Models
S³ is a verifier-guided stratified search over denoising trajectories that reallocates inference compute to improve output quality from fixed diffusion language models on reasoning benchmarks.
-
From Curiosity to Caution: Mitigating Reward Hacking for Best-of-N with Pessimism
Caution mitigates reward hacking in Best-of-N sampling by penalizing prediction errors from an error model as signals of uncertainty, with empirical improvements and provable gains over standard BoN in a linear setting.
-
QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation
QuanBench+ is a new multi-framework benchmark showing LLMs reach 43-60% Pass@1 on quantum code tasks across three libraries, rising to 67-83% with error-feedback repair, yet performance remains strongly framework-dependent.
-
OXtal: An All-Atom Diffusion Model for Organic Crystal Structure Prediction
OXtal recovers experimental organic crystal structures with conformer RMSD below 0.5 Å and over 80% packing similarity using a lattice-free diffusion model trained on 600K structures.
-
Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models
F2D2 jointly distills sampling and likelihood computation in flow-based models by adding a divergence head to a few-step flow map, achieving accurate log-likelihoods at 2-10 NFEs while preserving sample quality.
-
Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
Malicious nodes in decentralized GRPO can poison models with up to 100% success in 50 iterations on math and coding tasks, but logit probability checks and LLM judges filter most poisoned completions.
-
Score-based Membership Inference on Diffusion Models
Presents SimA, a score-based single-query membership inference attack for diffusion models and LDMs that uses denoiser output norm to reveal training set proximity and outperforms multi-query baselines on eight datasets.
-
ZeroSiam: An Efficient Asymmetry for Test-Time Entropy Optimization without Collapse
ZeroSiam is an asymmetric architecture using a learnable predictor and stop-gradient that prevents collapse in test-time entropy minimization while also regularizing biased signals for improved performance.
-
Causal Time Series Generation via Diffusion Models
CaTSG is a unified diffusion model for causal time series generation that handles observational, interventional, and counterfactual tasks via backdoor adjustment and abduction-action-prediction.
-
Explicit and Effectively Symmetric Schemes for Neural SDEs on Lie Groups
Introduces the first explicit near-reversible integrator for neural SDEs on Lie groups by extending EES schemes with Bazavov's commutator-free lift, achieving better stability and up to 10x memory reduction on manifold benchmarks.
-
On the Convergence of Muon and Beyond
Muon-MVR2 attains the optimal anytime convergence rate of ~O(T^{-1/3}) in stochastic non-convex settings under horizon-free schedules.
-
Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
Prefix-RFT blends SFT and RFT via prefix sampling from demonstrations to outperform standalone SFT, RFT, and mixed-policy baselines on math reasoning problems.
-
Physiome-ODE: A Benchmark for Irregularly Sampled Multivariate Time Series Forecasting Based on Biological ODEs
Physiome-ODE is a new benchmark consisting of 50 IMTS datasets derived from biological ODEs that shows ODE-based forecasting models performing better and differentiating more meaningfully than on the existing four datasets.
-
Power-Softmax: Towards Secure LLM Inference over Encrypted Data
Power-Softmax is a new HE-compatible attention variant that permits training and inference of billion-parameter polynomial LLMs with performance matching standard transformers.
-
Towards Generalized Certified Robustness with Multi-Norm Training
CURE is the first multi-norm certified training method that improves union robustness across l_p norms and unseen perturbations on MNIST, CIFAR-10 and TinyImagenet.
-
Near-Optimal Policy Identification in Robust Constrained Markov Decision Processes via Epigraph Form
Presents the first algorithm to identify an ε-optimal policy in robust constrained MDPs via epigraph form and bisection search with Õ(ε^{-4}) robust policy evaluations.
-
Learning to Forget: Continual Learning with Adaptive Weight Decay
FADE adapts per-parameter weight decay rates online via approximate meta-gradient descent to improve controlled forgetting over fixed decay in online tracking and streaming classification.
-
NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
NORACL dynamically grows network capacity via neurogenesis-inspired signals to achieve oracle-level continual learning performance without pre-specifying architecture size.
-
Reparameterization through Coverings and Topological Weight Priors
Reparameterization through coverings makes the KL term tractable in VAEs whose latent manifolds have non-trivial topology, demonstrated on a Klein bottle latent space.
-
Process Supervision of Confidence Margin for Calibrated LLM Reasoning
RLCM trains LLMs with a margin-enhanced process reward that widens the gap between correct and incorrect reasoning steps, improving calibration on math, code, logic, and science tasks without hurting accuracy.
-
Spend Less, Fit Better: Budget-Efficient Scaling Law Fitting via Active Experiment Selection
An uncertainty-aware sequential selection algorithm fits scaling laws to near-full accuracy using only about 10% of the total experimental training budget across diverse benchmarks.
-
Estimating Tail Risks in Language Model Output Distributions
Importance sampling via activation-steered unsafe proposal models estimates rare harmful-output probabilities in language models with 10-20x fewer samples than brute-force Monte Carlo.
-
OT on the Map: Quantifying Domain Shifts in Geographic Space
GeoSpOT applies optimal transport to longitude-latitude data to quantify geospatial domain shifts and predict cross-region model transfer performance.
-
Faster LLM Inference via Sequential Monte Carlo
SMC-SD replaces rejection sampling with particle resampling in speculative decoding to deliver 2.36x speedup over standard SD and 5.2x over autoregressive decoding while staying within 3% of target accuracy.
-
SCATR: Simple Calibrated Test-Time Ranking
SCATR calibrates a simple scorer from base-model hidden representations on limited data to improve Best-of-N response selection, delivering up to 9% gains over heuristics with orders-of-magnitude less compute than fine-tuning or PRMs.
-
ProtoTTA: Prototype-Guided Test-Time Adaptation
ProtoTTA is a test-time adaptation framework for prototype models that uses intermediate prototype signals and entropy minimization to improve robustness and semantic focus under distribution shifts.
-
From $P(y|x)$ to $P(y)$: Investigating Reinforcement Learning in Pre-train Space
PreRL applies reward-driven updates to P(y) in pre-train space, uses Negative Sample Reinforcement to prune bad reasoning paths and boost reflection, and combines with standard RL in Dual Space RL to outperform baselines on reasoning tasks.
-
Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascades
CTD trains a lightweight DV probe to predict escalation benefits and calibrates its threshold via multiple hypothesis testing on held-out data to deliver finite-sample guarantees on delegation rate while outperforming uncertainty-based cascades on safety tasks.
-
Learning to Adapt: In-Context Learning Beyond Stationarity
Gated linear attention enables lower training and test errors in non-stationary in-context learning by adaptively modulating past inputs through a learnable recency bias under an autoregressive model of task evolution.
-
Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents
Skill-SD turns an agent's completed trajectories into dynamic natural-language skills that condition only the teacher in self-distillation, yielding 14-42% gains over RL and OPSD baselines on multi-turn agent benchmarks.
-
CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
CodeQuant unifies learnable rotation smoothing with cluster-centroid absorption of outliers to reduce quantization error in low-precision MoE models, reporting up to 4.15x speedup and higher accuracy than prior PTQ methods.
-
Latent Instruction Representation Alignment: defending against jailbreaks, backdoors and undesired knowledge in LLMs
LIRA aligns latent instruction representations in LLMs to defend against jailbreaks, backdoors, and undesired knowledge, blocking over 99% of PEZ attacks and achieving optimal WMDP forgetting.
-
OASIS: Online Activation Subspace Learning for Memory-Efficient Training
OASIS tracks an evolving low-dimensional activation subspace to project activations, gradients, and optimizer states, cutting peak memory up to 2x versus full fine-tuning while matching performance on finetuning and pretraining tasks.
-
ExecTune: Effective Steering of Black-Box LLMs with Guide Models
ExecTune trains guide models via acceptance sampling, supervised fine-tuning, and structure-aware RL to boost executability of strategies for black-box LLMs, yielding up to 9.2% higher accuracy and 22.4% lower cost on math and code tasks.
-
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
Bit-by-Bit achieves stable 2-bit quantization of Llama models via block-wise progressive training and outlier channel splitting, reporting only 2.25 WikiText2 PPL degradation versus full precision while outperforming prior QAT baselines.
-
The Depth Ceiling: On the Limits of Large Language Models in Discovering Latent Planning
LLMs discover latent planning strategies up to five steps during training and execute them up to eight steps at test time, with larger models reaching seven under few-shot prompting, revealing a dissociation between discovery and execution.
-
How Reasoning Evolves from Post-Training Data: An Empirical Study Using Chess
Training language models on single best-move predictions in chess leads to strong but unfaithful reasoning after RL, while multi-move trajectories produce faithful reasoning with similar performance and stability.
-
Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
Scaling Decision Pre-Trained Transformer with Flow Matching on hundreds of tasks yields an agent with improved generalization in in-context reinforcement learning.