CanViT is the first task- and policy-agnostic AVFM pretrained via passive-to-active dense latent distillation on 13.2M scenes and 1B random glimpses, achieving 38.5% ADE20K mIoU in one glimpse and 84.5% ImageNet-1k top-1 after fine-tuning.
super hub Mixed citations
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Mixed citation behavior. Most common role is background (62%).
130
Pith papers citing it
Background
62% of classified citations
abstract
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
hub tools
citation-role summary
background 8
method 3
baseline 1
other 1
citation-polarity summary
claims ledger
- abstract In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
authors
co-cited works
representative citing papers
Mamba is a linear-time sequence model using input-dependent selective SSMs that achieves SOTA results across modalities and matches twice-larger Transformers on language modeling with 5x higher inference throughput.
CPF-GCD enforces low-rank compositional structure on vision backbone features via spatial primitive fields so that novel categories emerge as new activation patterns over a shared vocabulary of reusable visual primitives.
Presents UKHD, the first historical offline Urdu handwritten text lines dataset from Katib materials, and benchmarks CRNN-based models with CNN-BGRU-CTC showing lowest CER and WER.
LongSpike integrates fractional-order state-space modeling into spiking neural networks, enabling better long-sequence performance than prior SNNs on LRA, WikiText-103, and Speech Commands benchmarks while retaining sparse computation.
CoMetaPNS combines meta-learned neural surrogates with a continual Bayesian Gaussian Mixture Model to adapt cardiac electrophysiology simulations to new data while avoiding catastrophic forgetting.
AdaState replaces the static first-frame KV anchor with an evolving hidden latent that the model denoises alongside content, treating time as relative to enable recurrence and richer dynamics in streaming video generation.
LC-Flow introduces a continuous local recurrent network for learning sparse optical flow and confidence directly from event streams, with confidence-guided aggregation reaching new SOTA on MVSEC.
Nested-GPT is an autoregressive Transformer surrogate that generates variable-multiplicity parton showers while enforcing ordered Markovian branching and matches reference Monte Carlo results for leading-log non-global logarithm resummation in the large-Nc limit.
A two-stage contrastive teacher-student framework learns and then projects latent dynamics onto port-Hamiltonian submanifolds from partial observations.
TokAlign++ learns token alignments between LLM vocabularies from monolingual representations to enable faster adaptation, better text compression, and effective token-level distillation across 15 languages with minimal steps.
PRISM-VQ integrates vector-quantized latent factors with financial priors and a structure-conditioned mixture-of-experts to deliver improved cross-sectional stock return predictions and portfolio performance on CSI 300 and S&P 500.
The What-Where Transformer achieves explicit what-where separation in a ViT-style backbone via concurrent token and attention-map streams, yielding emergent object discovery from attention maps and better weakly-supervised localization.
TailedTS supplies 24.69 billion Wikipedia page-view records as a public benchmark for heavy-tailed time series forecasting and periodicity analysis, revealing weaker periodic structure in high-traffic pages.
TCRTransBench provides a new benchmark with bidirectional TCR-peptide generation tasks, a large validated dataset, and metrics to evaluate neural models for immunological sequence modeling.
NEO is a probabilistic neural model that induces compositional programs as a learned Language of Thought from non-textual observations and executes them via a shared transition model to enable explanation-driven generalization.
CLOVER augments value decomposition with a GNN mixer whose weights depend on the realized wireless communication graph, proving permutation invariance, monotonicity, and greater expressiveness than QMIX while showing gains on Predator-Prey and Lumberjacks under p-CSMA channels.
Damped harmonic oscillators with closed-form solutions model keys, values, and queries in continuous attention for irregular time series, preserving universal approximation while being orders of magnitude faster than prior NODE-based methods.
NeuroKalman mitigates state drift in vision-language UAV navigation by using memory-augmented Kalman filtering where attention retrieves historical anchors to correct predictions without gradient updates.
ExDoS uses expert-guided dual-focus distillation between source semantic graphs and bytecode control-flow graphs plus a dual-attention network to improve smart contract vulnerability detection, reporting 3-6% F1 gains over baselines.
Unsupervised GNN model learns local updates for approximate MaxIS on dynamic graphs, achieving competitive ratios on 200-1000 node instances and 1.00-1.18x larger solutions than other unsupervised models when generalizing to 100x larger graphs.
Transformers and SSMs are unified through structured state space duality, producing a 2-8X faster Mamba-2 model that remains competitive with Transformers.
Griffin hybrid model matches Llama-2 performance while trained on over 6 times fewer tokens and offers lower inference latency with higher throughput.
Characterizes an estimation-prediction tradeoff in binary logistic models for causal probabilistic temporal graphs and proposes a framework to jointly evaluate temporal link prediction with causal parameter recovery via Cramér-Rao bounds.
citing papers explorer
-
MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
MiniMax-M1 is a 456B parameter hybrid-attention MoE model trained with CISPO RL that achieves performance comparable or superior to DeepSeek-R1 and Qwen3-235B on reasoning and software engineering tasks while training in three weeks on 512 GPUs.
-
Logo-LLM: Local and Global Modeling with Large Language Models for Time Series Forecasting
Logo-LLM improves time series forecasting by pulling local dynamics from shallow LLM layers and global trends from deeper layers, then aligning them via new Local-Mixer and Global-Mixer modules.
-
DeePen: Penetration Testing for Audio Deepfake Detection
DeePen demonstrates that both production and academic audio deepfake detectors can be reliably deceived by simple signal processing attacks such as time-stretching or echo addition, with some attacks resistible via retraining and others remaining effective.
-
Non-invasive electromyographic speech neuroprosthesis: a geometric perspective
Direct sequence-to-sequence EMG-to-text conversion for silent articulation using a geometric representation of high-dimensional signals, without audio targets or time-alignment.
-
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
DPOP is a new loss function that prevents DPO from lowering preferred response likelihoods and outperforms standard DPO on diverse datasets, MT-Bench, and enables Smaug-72B to exceed 80% on the Open LLM Leaderboard.
-
R-Transformer: Recurrent Neural Network Enhanced Transformer
R-Transformer integrates RNNs with multi-head attention to model local and global sequence dependencies without position embeddings and reports large-margin gains over prior methods on diverse tasks.
-
A Bi-directional Transformer for Musical Chord Recognition
A bi-directional Transformer achieves competitive chord recognition by using self-attention to capture long-term dependencies in audio in a single training phase.
-
Improving Description-based Person Re-identification by Multi-granularity Image-text Alignments
The MIA model with GC, RGA, and BFM modules achieves state-of-the-art performance on the CUHK-PEDES dataset for description-based person re-identification.
-
Learning to Distributedly Estimate under Partially Known Dynamics: A Covariance-Agnostic Neural Kalman Consensus Filter
CA-NKCF is a hybrid neural-Kalman consensus filter for distributed state estimation that operates without noise covariance knowledge and shows robustness to model misspecification in linear, chaotic, and wireless scenarios.
-
Interpretable Kolmogorov-Arnold Network with Feature-Isolated Temporal Attention Mechanism for Electricity Load Forecasting
LoadKAN combines feature-isolated temporal attention with KAN to produce competitive load forecasts on three U.S. markets and enables quantitative analysis of non-linear mobility-load relationships via learned activation functions.
-
Graph Grounded Cross Attention Transformer Neural Network for Structurally Constrained Full Event Sequence Generation in Predictive Process Monitoring
GGATN combines graph grounding with transformer self- and cross-attention to generate full event sequences, timestamps, length, and attributes in a single pass followed by Viterbi-style constrained decoding, outperforming prompted LLM baselines on six logs with zero hallucinated activities.
-
Predicting Cognitive Load from Speech and Interaction Dynamics in Dyadic Conversations
A GRU encoder using static acoustic, dynamic, and interaction features from 53 dyads predicts cognitive load related to time pressure, mental work, effort, and performance, with turn-taking linked to temporal demand and imbalanced participation to mental demand.
-
KinematicRL: A Sim-to-Real Reinforcement Learning Framework For Social Navigation With Kinodynamic Feasibility
KinematicRL is a sim-to-real RL framework for social navigation using second-order control inputs, iLQR pretraining, and cluster-based 2D LiDAR tracking to produce kinodynamically feasible policies deployable on real robots with minimal modifications.
-
Boosting ECG Classification Performance by Pre-training with Synthesized Data
Pre-training ten DNN architectures on knowledge-driven synthetic ECGs generated via Gaussian PQRST wave composition improves classification of AF, AFLT, PVC, and WPW, with largest gain of 33.2% for AFLT and stronger benefits on smaller real datasets.
-
Reconstructing and forecasting disease trajectories of patients with Alzheimer's disease using routine data in resource-constrained settings
GNOVA reconstructs and forecasts CDR-SB and MMSE scores with MAEs of 1.35 and 2.28 on 1727 ADNI patients over 10 years using only routine visit data, enabling interpolation, extrapolation, and uncertainty estimates.
-
EvoCSFL: Surrogate-Assisted Evolutionary Client Selection for Efficient and Robust Federated Learning
EvoCSFL combines candidate generation, a multi-objective metric, surrogate approximation, and evolutionary search to optimize client subsets in federated learning, reporting faster convergence and lower energy on image classification tasks.
-
HoT-SSM:Higher-order Temporal Knowledge Graph Reasoning with State Space Models for Health Care
HoT-SSM combines hypergraph construction from domain knowledge with a dynamic state space model to jointly capture higher-order clinical interactions and long-range temporal dependencies, yielding improved predictions on MIMIC-III and MIMIC-IV.
-
RePercENT: Scaling Disentangled Representation Learning Beyond Two Modalities
RePercENT introduces a plug-and-play self-supervised framework for scalable pairwise disentangled representation learning across more than two modalities using pre-extracted embeddings and a joint optimization objective with theoretical optimality guarantees.
-
3D Temporal Analysis for Autism Spectrum Disorder Screening During Attention Tasks
A 3D temporal framework extracts head pose and facial features via DECA from VR tasks, then uses GRU classifiers to achieve 84.6% accuracy distinguishing ASD from TD children, outperforming 2D baselines.
-
Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions
GTR introduces a bounded non-monotonic Gaussian trust region and Mixture Gaussian Anchor to enable effective behavior transitions in non-stationary RL where standard PPO fails.
-
Physics-Guided Recurrent State-Space Neural Networks for Multi-Step Prediction
PG-RSSNN adds recurrent structures to physics-guided neural networks to enable stable multi-step prediction that beats both physics-only and black-box models even with partial physics and limited data.
-
CHAM-net: A Contrastive Hierarchical Adaptive Meta-network for Robust Global Methane Flux Prediction
CHAM-net is a contrastive hierarchical adaptive meta-network that conditions predictions on historical site data to outperform baselines on methane flux tasks with nRMSE down to 0.43.
-
Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
A decoupled estimator combining gated dynamics learning and recursive Kalman filtering improves robustness of pre-trained MARL policies under stale observations and message loss.
-
DeGRe: Dense-supervised Generative Reranking for Recommendation
DeGRe decouples offline exploration via a lookahead evaluator using beam search and cumulative regression to distill dense supervision into an online generator that approximates optimal reranking sequences with greedy decoding.
-
Atoms of Thought: Universal EEG Representation Learning with Microstates
Microstate tokenizer from clustered EEG signals provides universal representations that outperform traditional time- and frequency-domain features across sleep staging, emotion recognition, and motor imagery tasks.
-
Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning
An RL-based outer-loop quadrotor controller augmented with an online Residual Dynamics Predictor for disturbance estimation and a data-efficient sim-to-real calibration bridge.
-
Spatiotemporal decoupled physics-informed Stone-Weierstrass neural operator for long-time prediction of time-dependent parametric PDEs
A spatiotemporally decoupled physics-informed Stone-Weierstrass neural operator for stable long-time prediction of time-dependent parametric PDEs.
-
Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging
Randomly initialized Transformers act as adaptive sequence smoothers for sleep staging via a Random Attention Prior Kernel, with gains mainly from inductive bias rather than training.
-
MDN: Parallelizing Stepwise Momentum for Delta Linear Attention
MDN parallelizes stepwise momentum for delta linear attention using geometric reordering and dynamical systems analysis, yielding performance gains over Mamba2 and GDN on 400M and 1.3B models.
-
Neural Co-state Policies: Structuring Hidden States in Recurrent Reinforcement Learning
Recurrent RL policies can have their hidden states aligned with PMP co-states through a derived loss, yielding robust performance on partially observable control tasks.
-
ReMedi: Reasoner for Medical Clinical Prediction
ReMedi boosts LLM performance on EHR clinical predictions by up to 19.9% F1 through ground-truth-guided rationale regeneration and fine-tuning.
-
HOI-aware Adaptive Network for Weakly-supervised Action Segmentation
AdaAct employs a HOI encoder and two-branch hypernetwork to adaptively adjust temporal encoding parameters based on video-level human-object interactions for improved weakly-supervised action segmentation.
-
STK-Adapter: Incorporating Evolving Graph and Event Chain for Temporal Knowledge Graph Extrapolation
STK-Adapter adds Spatial-Temporal MoE, Event-Aware MoE, and Cross-Modality Alignment MoE to integrate evolving TKG graphs and event chains into LLMs, reducing information loss and improving extrapolation performance over prior methods.
-
Gated Memory Policy
GMP selectively activates and represents memory via a gate and lightweight cross-attention, yielding 30.1% higher success on non-Markovian robotic tasks while staying competitive on Markovian ones.
-
Efficient and Effective Internal Memory Retrieval for LLM-Based Healthcare Prediction
K2K framework enables internal memory retrieval in LLMs for healthcare outcome prediction, achieving state-of-the-art results on four benchmarks.
-
Adaptive Learned State Estimation based on KalmanNet
AM-KNet adds sensor-specific modules, hypernetwork conditioning on target type and pose, and Joseph-form covariance estimation to KalmanNet, yielding better accuracy and stability than base KalmanNet on nuScenes and View-of-Delft data.
-
Probabilistic Hysteresis Factor Prediction for Electric Vehicle Batteries with Graphite Anodes Containing Silicon
A data-driven probabilistic approach predicts the hysteresis factor for silicon-graphite anode batteries in electric vehicles, with tests for generalization across vehicle models.
-
Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation
HaNoRec dynamically weights harder preference samples and applies Gaussian perturbations to output distributions to improve multimodal LLM performance on sequential recommendation tasks.
-
WaveletInception Networks for on-board Vibration-Based Infrastructure Health Monitoring
The WaveletInception-BiGRU network uses learnable wavelet packet transforms, 1D Inception-ResNet modules, and BiGRU layers to generate high-resolution, spatially mapped health profiles from variable-speed vibration data, outperforming prior methods on track stiffness and transition zone tasks.
-
From Time-series Generation, Model Selection to Transfer Learning: A Comparative Review of Pixel-wise Approaches for Large-scale Crop Mapping
A comparative review with experiments identifying optimal preprocessing, models, and transfer strategies for large-scale pixel-wise crop mapping using Landsat 8 data across five sites.
-
What Matters in Building Vision-Language-Action Models for Generalist Robots
Systematic tests of VLM backbones, policy architectures, and cross-embodiment data yield RoboVLMs that set new SOTA on robot manipulation benchmarks while requiring few manual designs.
-
Click-Through Rate Prediction with the User Memory Network
MA-DNN augments DNNs with per-user memory vectors capturing likes and dislikes to exploit historical behavior for CTR prediction while remaining simpler than RNNs.
-
Latent Multi-Criteria Ratings for Recommendations
Uses variational autoencoders on user reviews to generate latent multi-criteria ratings that outperform baselines on multiple datasets.
-
Attention Is All You Need
Pith review generated a malformed one-line summary.
-
Improving Patient Subtyping on Longitudinal Data using Representations from Mamba-based Architecture
Self-supervised Mamba model learns EHR representations that improve patient subtyping on longitudinal data compared to baselines.
-
Risk Stratification for ICU Delirium using Pervasive Ambient Sensing Information
Ambient sound and light data from ICU rooms predict delirium with AUC 0.80 using convolutional neural networks, with sound as the dominant predictor, on data from 309 patients across 9 ICUs.
-
Time-Series Foundation Model Embeddings for Remaining Useful Life Estimation
Frozen Chronos-2 TSFM embeddings plus a lightweight regression head outperform standard baselines for RUL estimation on two industrial sensor datasets.
-
Learning and Adaptation in Wire Arc Additive Manufacturing Bead Geometry Control
RNN predictive control with layer-wise adaptation improves bead height and width consistency in WAAM experiments over constant-input and static-model baselines.
-
Enhancing BiGRU with a KAN Block for Legal Document Classification and Summarization
KAN-enhanced BiGRU reaches 67.96% accuracy and 0.65 F1 on legal document classification plus ROUGE-1/2/L of 0.38/0.23/0.31 on summarization, with ablation attributing a 10.62-point accuracy gain to the KAN block.
-
Bridging Classification and Reconstruction: Cooperative Time Series Anomaly Detection
CoAD unifies outlier exposure classification and masked autoencoder reconstruction in a cooperative loop to detect subtle and prolonged time series anomalies.