REVIEW 41 cited by
The Forward-Forward Algorithm: Some Preliminary Investigations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
The Forward-Forward Algorithm: Some Preliminary Investigations
read the original abstract
The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.
Forward citations
Cited by 41 Pith papers
-
Adaptive directional gradients for parameterised quantum circuits
Forward gradient framework for PQCs unifies SPSA and parameter-shift as limits, introduces QUIVER adaptive optimizer with closed-form measurement allocation, and demonstrates efficient training of 60-qubit circuits on...
-
Training Non-Differentiable Networks via Optimal Transport
PolyStep optimizes non-differentiable networks via forward-only polytope evaluations and optimal-transport barycentric updates, reaching 93.4% accuracy on hard-LIF spiking networks while outperforming gradient-free baselines.
-
Forget, Anticipate and Adapt: Test Time Training for Long Videos
FFN enables efficient TTT for long videos by operating on three frames and using a surprise-based adaptive window, shown on a new dataset of up to 3-hour videos for segmentation and classification tasks.
-
Embodied Explainability and Ontological Obstacles: Why We Struggle to Explain the Answers of Large Language Models (LLMs)
An argument paper reframes LLM explainability as an embodied, situated practice based on Dourish and enactivist cognition, identifying ontological obstacles in internal explanations and advocating affordance-based designs.
-
Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Training
DTG-FF reaches 91.8% on CIFAR-10 and 49.4% on ImageNet-100 224x224 but BP baselines beat it by 2.4-5.93 pp with gaps widening by class count on real data while reversing the synthetic trend.
-
FFR: Forward-Forward Learning for Regression
FFR adapts Forward-Forward learning to regression via ordinal competitive goodness, stratified ladder layers, and hierarchical uncertainty-aware prediction, recovering 98.6% of backpropagation accuracy with substantia...
-
Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
SBD unifies broadcast credit assignment for losses including cross-entropy via output-score orthogonality to hidden activations and reports performance gains on CIFAR-10 and Tiny ImageNet.
-
NeuroTrain: Surveying Local Learning Rules for Spiking Neural Networks with an Open Benchmarking Framework
A taxonomy of SNN training algorithms is presented with the release of NeuroTrain, an open benchmarking framework for reproducible comparisons across datasets and architectures.
-
ArrowFlow: Hierarchical Machine Learning in the Space of Permutations
ArrowFlow demonstrates that competitive classification is possible using a hierarchical architecture of ranking filters in permutation space, with a single polynomial-degree parameter controlling robustness-accuracy t...
-
Selectivity and Shape in the Design of Forward-Forward Goodness Functions
Shape- and peak-sensitive goodness functions for Forward-Forward deliver up to 72pp gains over sum-of-squares, reaching 98.2% on MNIST and 89% on Fashion-MNIST.
-
ArrowFlow: Hierarchical Machine Learning in the Space of Permutations
A hierarchical ranking-filter architecture learns classifiers entirely in permutation space via Spearman's footrule and non-gradient updates, claiming competitive accuracy without core floating-point parameters.
-
Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions
Sparse Bayesian-network factorization plus sparsity-aware regression yields polynomial TV rates for high-dimensional mixed-type distribution estimation, beating classical histogram rates under sparsity.
-
Shunting Inhibition and Dendritic Branching Shape Local Credit Assignment
Exact dendritic gradients factor into local eligibility times path-transported compartment error, and shunting inhibition can improve restricted-feedback local learning by reshaping that error field.
-
Forget, Anticipate and Adapt: Test Time Training for Long Videos
FFN performs TTT on multi-hour videos by restricting updates to three frames and using a surprise metric for adaptive window sizing, plus a new EpicTours dataset.
-
Forget, Anticipate and Adapt: Test Time Training for Long Videos
FFN performs efficient test-time training on multi-hour videos by forgetting the exiting frame, anticipating the next, and adapting only when a surprise metric exceeds a dynamic threshold.
-
TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems
TL++ recovers centralized mini-batch gradients via virtual batches in split learning and adds secret sharing for cut-layer tensors, achieving 91.41% accuracy on CIFAR-10 with 13x lower communication than full-model sync.
-
Decentralised AI Training and Inference with BlockTrain
BlockTrain partitions models into block-local diffusion objectives that train near end-to-end WikiText quality with one-block worker memory, real WAN transport, and one-sweep distributed serving.
-
What Accuracy and Gradient Cosine Miss: Evaluating Feedback Alignment via Scale Stability, Reference Validity, and Depth Utility
Standard accuracy and aggregate cosine metrics for feedback alignment are insufficient because they miss measurement degeneracy in terminal layers and aggregation collapse across depths; a new protocol using scale sta...
-
Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment
Learnable channel-class assignment and adaptive layer weighting allow forward-only CNNs to reach new state-of-the-art results among FF models on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
-
Intrinsic Computational Functionalism: From Observer-Relative Maps to Observer-Independent Structures
Intrinsic computational functionalism uses system-intrinsic instantiation (C1) and causal-dynamical organisation under intervention (C2) to identify observer-independent computational structures for consciousness via ...
-
Is Backpropagation Optimal? When Synthetic Gradients Improve Sample Efficiency
Synthetic gradients can achieve arbitrarily lower gradient-estimation MSE than backpropagation under characterized conditions in a new vectorized feedback framework for computational graphs.
-
HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm
HCL-FF augments the Forward-Forward algorithm with hierarchical learning and contrastive objectives to reach new state-of-the-art accuracies among FF methods on CIFAR-10 (+5.46%), CIFAR-100 (+17.00%), and Tiny-ImageNe...
-
Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data
Layerwise self-supervised local rules learn the hierarchical structure of the Random Hierarchy Model as data-efficiently as supervised backpropagation, while direct feedback approximations fail due to missing masking ...
-
Adaptive Multi-Scale Goodness Aggregation for Forward-Forward Learning
AMSGA extends Forward-Forward learning via multi-scale goodness aggregation, curriculum-guided hard negative mining, and adaptive thresholds, reporting up to 1.5% accuracy gains on MNIST and Fashion-MNIST.
-
Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant
Cumulative-goodness Forward-Forward networks exhibit layer free-riding where discrimination gradients decay exponentially with prior positive margins; per-block, hardness-gated, and depth-scaled remedies yield 4-45x b...
-
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
FAAST analytically compiles labeled examples into fast weights via a single forward pass, matching backprop adaptation performance with over 90% less time and up to 95% less memory than memory-based methods.
-
A Synthesizable RTL Implementation of Predictive Coding Networks
A complete RTL substrate executes discrete-time predictive coding dynamics directly in hardware with fixed local rules and adjacent-layer communication only.
-
Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
A framework combining stochastic zeroth-order optimization and dynamic low-rank surrogate modeling with an implicit projector-splitting integrator enables end-to-end training of hybrid neural networks containing black...
-
MemFlow: A Lightweight Forward Memorizing Framework for Quick Domain Adaptive Feature Mapping
MemFlow uses forward-only memorization via randomly connected neurons and spiking signals on a frozen backbone for unsupervised domain adaptation, claiming up to 10% gains at under 1% of traditional compute costs.
-
Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks
Single-parameter Monte Carlo mutation-selection trains deep networks and a simple Transformer on MNIST and Tiny Shakespeare without backpropagation.
-
Decentralised AI Training and Inference with BlockTrain
BlockTrain partitions models into blocks trained on local objectives, reaching CE 1.359 on WikiText within 0.04 of end-to-end baseline while enabling distributed training and inference over TCP for up to 75B-parameter models.
-
Breaking chains with trees: Deep learning with $\mathcal{O}(\log N)$ parallel time complexity
HBLL decomposes DNNs into hierarchical blocks trained with local variational objectives to achieve O(log N) parallel training time without end-to-end backpropagation.
-
Some[Body] Must Receive That Pain for Agent Accountability
AI agents lack the persistent identity and feedback mechanisms needed for consequence reception, requiring new architectures or continued human accountability.
-
Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpowered
Zeroth-order optimization is underexplored rather than underpowered in deep learning, with limitations stemming from full-space designs that can be addressed via subspace, spectral, and systems-aware approaches.
-
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
FAAST performs test-time supervised adaptation by analytically deriving fast weights from examples in one forward pass, matching backprop performance with over 90% less adaptation time and up to 95% memory savings ver...
-
Physical Foundation Models: Fixed hardware implementations of large-scale neural networks
Physical Foundation Models are fixed physical hardware realizations of foundation-scale neural networks that compute via inherent material dynamics, potentially delivering orders-of-magnitude gains in energy efficienc...
-
LightTune: Lightweight Forward-Only Online Fine-Tuning with Applications to Link Adaptation
LightTune introduces a lightweight forward-only online fine-tuning framework that reduces BLER prediction error by up to 48.8% and boosts throughput by 15.5% in 6G link adaptation.
-
Introducing Echo Networks for Computational Neuroevolution
Echo Networks are recurrent networks defined by a single connection matrix with no layers, enabling matrix-based mutation and recombination in neuroevolution, and demonstrated on ECG signal classification.
-
LightTune: Lightweight Forward-Only Online Fine-Tuning with Applications to Link Adaptation
LightTune is a backpropagation-free online fine-tuning framework that reduces BLER prediction error by up to 48.8% and improves throughput by 15.5% in 6G link adaptation.
-
Cultivating Machine Intelligence: The OMEGA Shift from Top-Down Optimization to Autopoietic Cognitive Ecologies
The paper introduces the RECLAIM framework and OMEGA shift as a transition from top-down optimization to autopoietic cognitive ecologies for cultivating machine intelligence.
-
Grid-SD2E: A General Grid-Feedback in a System for Cognitive Learning
Proposes Grid-SD2E, a theoretical grid-feedback cognitive learning system combining grid-cell inspiration with Bayesian reasoning for self-reinforcing interaction.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.