Integration of Contrastive Predictive Coding and Spiking Neural Networks

Aykut G\"orkem Gelen; Emirhan Bilgi\c{c}; Nam{\i}k Berk Yalab{\i}k; Neslihan Serap \c{S}eng\"or; Rahmi Elibol; Yavuz Selim \.I\c{s}ler

REVIEW 3 major objections 5 minor 11 references

Integration of Contrastive Predictive Coding and Spiking Neural Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An SNN trained for digit classification can serve as the encoder in contrastive predictive coding, reaching 96.8 percent accuracy on a sequential MNIST task.

desk verdict Modest but legitimate integration result (CPC + SNN encoders) whose reported accuracies are undercut by underspecified sequence construction and no held-out test set; worth refereeing. read the letter →

arxiv 2506.09194 v1 pith:CGIG2IPA submitted 2025-06-10 eess.SP cs.AI

Emirhan Bilgiç , Neslihan Serap Şengör , Nam{i}k Berk Yalab{i}k , Yavuz Selim İşler , Aykut Görkem Gelen , Rahmi Elibol This is my paper · ORCID

classification eess.SPcs.AI

keywords contrastivepredictivecodingspikingneuralnetworksself-supervisedlearningspike-timing-dependentplasticityleakyintegrate-and-fireMNISTsequenceclassificationneuromorphiccomputing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that contrastive predictive coding (CPC) can be driven by a spiking neural network (SNN) instead of a conventional convolutional encoder, and that the SNN does not need to be trained for encoding to work in this role. The authors pair MNIST digits into short sequences, train a GRU to predict the SNN's spike-count vector for the next digits, and classify each sequence as positive (sequential) or negative (non-sequential). With an SNN previously trained for digit classification, the integrated model reaches 80.3 percent validation accuracy; with an SNN autoencoder trained for encoding, it reaches 96.8 percent; with random encoding it stays near chance (55.6 percent). The significance is that a biologically plausible spike-based encoder can be combined with CPC's contrastive learning, and that the encoding capacity of SNNs trained for other tasks is real. The result positions CPC-plus-SNN as a route toward energy-efficient, neuromorphic predictive coding.

What carries the argument

The load-bearing object is the SNN encoder: a leaky integrate-and-fire (LIF) network trained with spike-timing-dependent plasticity that converts each 784-pixel MNIST image into a 400-dimensional vector of per-neuron spike counts over a $\Delta t = 350$ ms window. That spike-count vector replaces the convolutional encoder in CPC; a GRU reads four consecutive encoded vectors, produces a context vector, and a dense network predicts a future encoding vector. Prediction quality is scored by dot-product similarity (Equation 5) averaged over time steps, passed through a sigmoid, and trained with binary cross-entropy (Equation 7). The SNN-Autoencoder variant additionally provides an encoder explicitly trained for reconstruction, giving the comparison between task-trained and encoding-trained spike encoders.

What would settle it

Shuffle the digits within each positive test sequence while keeping the same multiset of digits and rerun the trained model: if validation accuracy stays near the reported values, the model is recognizing digit content rather than temporal order. A stronger control is to assign positive and negative labels randomly, independent of sequence contents; the method should collapse to chance if the task actually requires sequence structure.

Watch

Extended reading notes

Core claim

The central claim is that CPC and SNNs can be integrated, with the SNN acting as the encoder that converts each input image into a 400-dimensional spike-count vector, and the GRU-based autoregressive network predicting future encoding vectors from past ones. Positive examples (sequences of digits in natural ascending order) are trained to maximize the dot product between predicted and actual encoding vectors, while negative examples (non-sequential orderings) are trained to minimize it. Under this scheme, an SNN trained solely for MNIST classification, using LIF neurons and STDP, produces encodings good enough to support 80.3 percent accuracy in distinguishing sequential from non-sequential digit sequences, and an SNN autoencoder produces 96.8 percent. The authors take the near-chance performance of random encoding as evidence that the learned SNN encodings carry the structure that CPC needs. The paper concludes that SNNs trained for classification can also function as an encoding mechanism when paired with CPC.

Load-bearing premise

The results would only be meaningful if positive and negative digit sequences differ in temporal order rather than in some simpler property, because the paper specifies that samples are "paired sequentially (and non-sequentially for negative examples)" but does not give the exact sampling procedure.

Editorial extensions

If this is right

A classification-trained SNN can be reused as a representation encoder for self-supervised contrastive learning, avoiding the need for a separate training phase dedicated to encoding.
The gap between SNN-Autoencoder (96.8 percent) and SNN-Classifier (80.3 percent) quantifies how much encoding-specific training improves the representations that CPC can exploit.
Random encoding staying near chance (55.6 percent) confirms that the spike-count encodings, not the CPC architecture alone, are carrying the predictive signal.
Scaling from 2500 to 5000 training samples does not improve validation accuracy but leads to earlier convergence, suggesting the sequence task is easy at small scale.
The same architecture is, per the authors, ready to be tested on other time-dependent datasets such as human action recognition and on neuromorphic hardware.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors leave implicit is to test the same CPC-plus-SNN pipeline on real temporal streams like video frames or sensor readings, where sequential order is semantically meaningful rather than constructed by digit pairing.
Because the discriminator is a linear dot product on the predicted and encoded vectors, the method effectively performs a linear probe on CPC representations; evaluating downstream classification with a learned probe would separate representation quality from the hand-picked similarity measure.
The ambiguity in sequence construction invites a control experiment: if the reported accuracy survives shuffling the order of digits within sequences, the model is learning digit co-occurrence statistics rather than temporal structure, which would change the interpretation of the biological-plausibility claim.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, the genuinely new thing: this is a straightforward swap of a CNN encoder for a spiking encoder inside a CPC-style contrastive loop, and it asks whether an SNN trained for classification with STDP can serve as the encoder. That specific test is not in the cited literature, as far as the paper reports. The experiments are internally consistent, and the random-encoding control (~56%) is a reasonable sanity check: if the spiking encoders were carrying nothing, the control should match them. The code is on GitHub, which helps reproducibility.

The results themselves point one direction: SNN-Autoencoder (0.97 validation accuracy) beats SNN-Classifier (0.80), which beats random (0.56). So there is evidence that classification-trained spiking features still contain usable structure. That is the paper's honest core, and it is a modest but real finding.

The soft spots are real and proportionate to the paper's claims. The biggest one is the underspecified construction of positive and negative sequences. The text says only that MNIST samples are 'paired sequentially' for positives and 'non-sequentially' for negatives; Figure 2 shows one positive example. If negatives are arbitrary random digit sequences, the classifier can solve the task by recognizing which digits appear, not by learning temporal order. The random baseline does not settle this, because random vectors destroy digit identity, so the accuracy drop is exactly what a digit-identity shortcut predicts. The authors may have a cleaner protocol in code, but the paper must state it.

Two further gaps: they report max validation accuracy, not test accuracy, which is optimistic when early stopping is tied to validation; and there is no conventional CPC baseline (a CNN encoder) to frame the SNN results. The missing baseline matters less, because the question is biological plausibility, but the missing test set makes the headline numbers unreliable as performance claims.

Citation pattern is unremarkable: the relevant CPC, STDP, and predictive-coding references are there. No obvious self-citation problem.

Bottom line: this is a small, honest integration experiment that deserves a proper referee, not a desk rejection. A referee should ask for the sequence-generation details, a held-out test set, and ideally a shuffle-based negative construction so the task requires order. With open code, those are easy fixes. I would bring it to a reading group mainly to discuss the experimental-design pitfalls.

Referee Report

3 major / 5 minor

Summary. The paper proposes combining Contrastive Predictive Coding (CPC) with Spiking Neural Networks (SNNs) by replacing the CPC encoder with two kinds of SNN encoders: an SNN trained for MNIST classification, and an SNN autoencoder trained for reconstruction. A GRU autoregressive predictor and a dense predictor produce a context vector that is compared with future encoded vectors via dot-product similarity, and the model is trained with binary cross-entropy to classify sequences as positive (sequential) or negative (non-sequential). Experiments on two MNIST subsets report that the SNN-autoencoder encoding reaches 0.9683 validation accuracy on MNIST-2500, the SNN-classifier encoding reaches 0.8033, and random encoding reaches 0.5558. The authors conclude that CPC and SNNs can be effectively integrated and that an SNN trained for classification can serve as an encoder.

Significance. If the sequence-construction protocol is valid and the accuracy contrast is meaningful, the paper provides a useful first demonstration of a CPC-SNN integration. The random-encoding control is a good internal validity check: it shows that the accuracy difference is not purely an artifact of CPC training. The claim that a classification-trained SNN can be repurposed as a frozen encoder is also directly tested by the comparison between SNN-Classifier and SNN-Autoencoder. The paper is honest about the small scale of the study and provides a GitHub link for code. The significance is nevertheless limited by the lack of a held-out test set, the absence of a non-spiking CPC baseline, and most importantly by the underspecified construction of positive and negative sequences, which is the load-bearing premise of the entire experimental contrast.

major comments (3)

[Section I and Figure 2] The exact construction of positive and negative sequences is not specified. The text says only that MNIST samples are "paired sequentially (and non-sequentially for negative examples)", and Figure 2 gives one positive example ([1,2,3,4] -> [5,6,7,8]) without stating the sampling rule. If negative examples are generated by drawing arbitrary random digits, the label may be predictable from the set of digits appearing in the eight frames rather than from temporal order, and the reported accuracies could reflect digit-identity recognition rather than predictive coding. The authors must specify the full data-generation procedure, including how positive and negative sequences are drawn, and should provide a control experiment in which positive and negative sequences contain the same multiset of digits and differ only in temporal ordering. This is required to support the central claim.
[Section III, Table I] Table I reports maximum validation accuracy over epochs, and early stopping is itself based on validation accuracy. Reporting the maximum over the training curve is an optimistic estimate of generalization, and the absence of a held-out test set means the reported numbers cannot be interpreted as unbiased performance. The authors should report test-set accuracy at the early-stopped epoch (or averaged over a fixed schedule), along with the standard deviations promised in Section III. Without this, the quantitative contrast in Table I is not a reliable basis for the paper's conclusions.
[Section II and Section III] The paper claims that CPC and SNNs can be effectively integrated, but it does not compare against a standard CPC baseline with a non-spiking encoder (e.g., the CNN encoder from the original CPC formulation) or against a CPC model using raw pixels. Such a baseline would clarify whether the observed accuracy is attributable to the CPC mechanism, to the frozen SNN encoders, or to the dataset construction. The random-encoding control is useful but only partially addresses this question, because it changes the encoder distribution rather than isolating the effect of the spiking encoder. Adding one or two standard baselines would substantially strengthen the central claim.

minor comments (5)

[Section II] Equation (1) gives the simulation duration as 350 ms, while the text immediately after Equation (2) states Delta_t = 0.35 seconds; the units are consistent but the presentation should be unified.
[Section II-D] The heading "Similarity Calculation (Dot Product and Average)" appears twice: once for Section II-C content about the autoencoder and once for the actual dot-product comparison. The first occurrence should be corrected.
[Section IV] The conclusions refer to "VND trained specifically for a classification task", but the abbreviation VND is not defined anywhere in the paper; presumably it refers to the SNN, and the text should say so.
[Figure 2] The figure caption says "the second example is another positive sample", but the visual content of the second example is not described; adding the digit sequence of the second positive sample and the negative sample would make the figure self-contained.
[Section III] The text reports results as means +/- standard deviations and mentions shaded regions in Figures 3-5, but Table I lists only mean values without standard deviations; adding the standard deviations to Table I would align the table with the stated reporting convention.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported accuracy is an empirical comparison with frozen encoders and a random-encoding control.

full rationale

The paper's central claim is an empirical accuracy result (Table I), not a derivation. The CPC network is trained with binary cross-entropy on positive/negative sequence labels (Eq. 7); the SNN encoders are pre-trained and frozen (Sections II-A and II-B), and the random-encoding condition (Figure 5) is an external control. The dot-product similarity (Eq. 5) and the BCE loss are the standard CPC objective, not a disguised reuse of the labels. No equation reduces to another by construction; no fitted parameter is renamed as a prediction; and no load-bearing self-citation appears, because references [1]-[3] are external prior work (Rao and Ballard; Oord et al.; Diehl and Cook). The main manuscript-level limitation is that the positive/negative sequence sampling is underspecified ('paired sequentially (and non-sequentially for negative examples)', Section I), which raises a potential confound about whether the classifier solves the task via digit identity rather than temporal predictive coding. That is a validity risk about what the model learns, not a circularity in the derivation chain, because the paper does not define the sequence labels in terms of the encoder's output and does not claim a prediction that is mathematically forced by its inputs.

Assumptions & free parameters 11 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are introduced; the parameters above are choices of existing model components. The most consequential assumptions are the validity of the sequential MNIST task and the adequacy of the pretrained spiking encoders.

free parameters (11)

Scaling factor k for Poisson firing rate = Adaptive, incremented by 1 when total spikes < 5
Controls the firing rate in Equation (1); chosen by hand and adjusted per input to meet the minimum spike condition, affecting the encoding vectors.
Minimum spike threshold Smin = 5 spikes
Used in Equation (2) to trigger k adaptation; chosen by hand.
Simulation duration Delta t = 0.35 s
Time window over which spike counts are accumulated in Equation (1).
LIF leakage factor beta = Not specified
Membrane potential leakage in Equation (3) for the SNN autoencoder; value not given in the text.
Spike threshold V_thresh = Not specified
Threshold for spike emission in Equation (4); not provided.
GRU hidden size = 256
Context vector dimension shown in Figure 2; not justified.
Adam learning rate = 1e-4
Optimizer hyperparameter for CPC training; standard but chosen.
Batch size = 32 positive and 32 negative
Training batch composition in Section III.
Early stopping patience = 10 epochs
Stops training if validation accuracy does not improve for 10 epochs.
Learning rate halving patience = 3 epochs
Halves learning rate if validation loss does not improve for 3 epochs.
Sequence length T = Not explicitly stated, Figure 2 shows 8 time steps
The mean similarity in Equation (6) averages over T time steps; T is not given in the text.

assumptions (5)

domain assumption LIF neuron model in Equations (3)-(4) describes the spiking encoder dynamics.
The SNN-Autoencoder uses leaky integrate-and-fire dynamics without biological verification; this is a modeling choice.
domain assumption The STDP-trained SNN from Diehl and Cook [3] can serve as a useful MNIST encoder.
The paper relies on this pretrained model's spike-count vectors without reporting its classification accuracy or verifying its encoding quality independently.
domain assumption CPC-style prediction with dot-product similarity and a 0.5 sigmoid threshold is a valid decision rule for sequence order.
The decision rule in Section II-D assumes normalized dot products above 0.5 indicate positive sequences.
ad hoc to paper The MNIST sequential pairing creates a task that requires temporal prediction rather than trivial class statistics.
The exact construction of positive and negative sequences is not specified, and the validity of the entire experiment depends on this assumption.
domain assumption Frozen pretrained encoders can be combined with a trainable GRU without adaptation.
The encoder weights are frozen, so success depends on the features being directly usable by the GRU.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Integration of Contrastive Predictive Coding and Spiking Neural Networks." pith.science (2026). https://pith.science/paper/CGIG2IPA

@misc{pith2026250609194,
  author       = {Pith},
  title        = {Pith review of: Integration of Contrastive Predictive Coding and Spiking Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CGIG2IPA}},
  note         = {Machine review of arXiv:2506.09194}
}

read the original abstract

This study examines the integration of Contrastive Predictive Coding (CPC) with Spiking Neural Networks (SNN). While CPC learns the predictive structure of data to generate meaningful representations, SNN mimics the computational processes of biological neural systems over time. In this study, the goal is to develop a predictive coding model with greater biological plausibility by processing inputs and outputs in a spike-based system. The proposed model was tested on the MNIST dataset and achieved a high classification rate in distinguishing positive sequential samples from non-sequential negative samples. The study demonstrates that CPC can be effectively combined with SNN, showing that an SNN trained for classification tasks can also function as an encoding mechanism. Project codes and detailed results can be accessed on our GitHub page: https://github.com/vnd-ogrenme/ongorusel-kodlama/tree/main/CPC_SNN

Figures

Figures reproduced from arXiv: 2506.09194 by the authors.

**Figure 3.** Validation accuracy graph of the CPC network using [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

**Figure 4.** Validation accuracy graph of the CPC network using [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Validation accuracy graph of the model with random [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 10 canonical work pages

[1]

P., & Ballard, D

Rao, R. P., & Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1), 79-87

work page 1999
[2]

Oord, A. V. D., Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

arXiv 2018
[3]

U., & Cook, M

Diehl, P. U., & Cook, M. (2015). Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in computational neuroscience, 9, 99

work page 2015
[4]

M., Versari, L., Fischbacher, T., & Alakuijala, J

Comşa, I. M., Versari, L., Fischbacher, T., & Alakuijala, J. (2021). Spiking autoencoders with temporal coding. Frontiers in neuroscience, 15, 712667

work page 2021
[5]

V., Xue, H., & Salim, F

Deldari, S., Smith, D. V., Xue, H., & Salim, F. D. (2021, April). Time series change point detection with self-supervised contrastive predictive coding. In Proceedings of the web conference 2021 (pp. 3124-3135)

work page 2021
[6]

Lucas, S., & Portillo, E. (2024). Methodology based on spiking neural networks for univariate time-series forecasting. Neural Networks, 173, 106171

work page 2024
[7]

Predictive Coding with Spiking Neural Networks: a Survey

N'dri, A. W., Gebhardt, W., Teulière, C., Zeldenrust, F., Rao, R. P., Triesch, J., & Ororbia, A. (2024). Predictive Coding with Spiking Neural Networks: a Survey. arXiv preprint arXiv:2409.05386

work page Pith review arXiv 2024
[8]

F., Bohte, S

Lee, K., Dora, S., Mejias, J. F., Bohte, S. M., & Pennartz, C. M. (2024). Predictive coding with spiking neurons and feedforward gist signaling. Frontiers in Computational Neuroscience, 18, 1338280

work page 2024

Show all 11 references

[9]

K., & Denève, S

Boerlin, M., Machens, C. K., & Denève, S. (2013). Predictive coding of dynamical variables in balanced spiking networks. PLoS computational biology, 9(11), e1003258

2013
[10]

Auge, D., Hille, J., Mueller, E., & Knoll, A. (2021). A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters, 53(6), 4693-4710

2021
[11]

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcomp...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

P., & Ballard, D

Rao, R. P., & Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1), 79-87

work page 1999

[2] [2]

Oord, A. V. D., Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

arXiv 2018

[3] [3]

U., & Cook, M

Diehl, P. U., & Cook, M. (2015). Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in computational neuroscience, 9, 99

work page 2015

[4] [4]

M., Versari, L., Fischbacher, T., & Alakuijala, J

Comşa, I. M., Versari, L., Fischbacher, T., & Alakuijala, J. (2021). Spiking autoencoders with temporal coding. Frontiers in neuroscience, 15, 712667

work page 2021

[5] [5]

V., Xue, H., & Salim, F

Deldari, S., Smith, D. V., Xue, H., & Salim, F. D. (2021, April). Time series change point detection with self-supervised contrastive predictive coding. In Proceedings of the web conference 2021 (pp. 3124-3135)

work page 2021

[6] [6]

Lucas, S., & Portillo, E. (2024). Methodology based on spiking neural networks for univariate time-series forecasting. Neural Networks, 173, 106171

work page 2024

[7] [7]

Predictive Coding with Spiking Neural Networks: a Survey

N'dri, A. W., Gebhardt, W., Teulière, C., Zeldenrust, F., Rao, R. P., Triesch, J., & Ororbia, A. (2024). Predictive Coding with Spiking Neural Networks: a Survey. arXiv preprint arXiv:2409.05386

work page Pith review arXiv 2024

[8] [8]

F., Bohte, S

Lee, K., Dora, S., Mejias, J. F., Bohte, S. M., & Pennartz, C. M. (2024). Predictive coding with spiking neurons and feedforward gist signaling. Frontiers in Computational Neuroscience, 18, 1338280

work page 2024