ProSper -- A Python Library for Probabilistic Sparse Coding with Non-Standard Priors and Superpositions

Abdul-Saboor Sheikh; Georgios Exarchakis; Jakob Drefs; J\"org Bornschein; J\"org L\"ucke; Marc Henniges; Zhenwen Dai

REVIEW 3 major objections 5 minor 20 references

ProSper -- A Python Library for Probabilistic Sparse Coding with Non-Standard Priors and Superpositions

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ProSper is a Python library that unifies six probabilistic sparse-coding algorithms—binary, ternary, discrete, maximal-causes, maximum-magnitude-causes, and spike-and-slab—under a shared variational-EM and parallelization framework.

desk verdict A cleanly written software library paper whose central claim—that the library exists and scales—is currently unverifiable because the preprint links no code and shows no measurements. read the letter →

arxiv 1908.06843 v1 pith:GBM4FAN5 submitted 2019-08-01 eess.SP cs.LGstat.ML

Georgios Exarchakis , Jörg Bornschein , Abdul-Saboor Sheikh , Zhenwen Dai , Marc Henniges , Jakob Drefs , Jörg Lücke This is my paper · ORCID

classification eess.SPcs.LGstat.ML

keywords Pythonparallelcomputingsoftwarelibraryexpectation-maximizationsparsecodingfeaturelearninglatentvariablemodelsvariationalapproximations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ProSper aims to make probabilistic sparse coding with non-standard priors and superposition rules practical in a single Python library. The paper claims that six algorithms—binary, ternary, discrete, maximal-causes, maximum-magnitude-causes, and spike-and-slab sparse coding—can share one interface based on a generative model, truncated-posterior EM, and MPI parallelization, and that this scales to hundreds of CPUs and tens of millions of data values. A sympathetic reader would care because standard dictionary-learning libraries cover mostly linear L1/L2 or NMF objectives, whereas these models handle non-linear combinations such as max and learn prior and noise parameters from data. If the claimed scalability holds, the library would let researchers apply occlusion-style or discrete-component models at scales previously reserved for linear sparse coding.

What carries the argument

The load-bearing object is the truncated posterior approximation: rather than summing over all latent configurations, each EM step retains only the most probable subset of configurations, so inference stays tractable with hundreds of dictionary elements. Model identity is fixed by three choices: the prior over $\vec s$, the noise distribution on $\vec y$, and the link function $\vec f(\Theta,\vec s)$, which is linear ($W\vec s$) for BSC, TSC, DSC, and GSC, and a max or magnitude-max operation for MCA and MMCA. A common Annealing/Model/EM class structure lets these choices be combined, annealed, and parallelized uniformly across all algorithms.

What would settle it

Take the library's BSC and GSC implementations on a standard natural-image patch dataset, compute the final data likelihood or compare learned dictionaries against the published results for the same models, and run a weak-scaling experiment from 1 to several hundred CPUs; if the likelihoods or speedups fall far short of the cited numbers, the paper's central functionality and scalability claims are not supported.

Watch

Extended reading notes

Core claim

On its own terms, the paper's contribution is a unified software architecture for probabilistic dictionary learning. All included models are instances of one generative specification: a prior on latent causes $p(\vec s|\Theta)$, a noise model $p(\vec y|\vec s,\Theta)$ parameterized by a link function $\vec f(\Theta,\vec s)$, and truncated-posterior EM for parameter estimation. The library instantiates this specification for binary, ternary, discrete, max-causes, maximum-magnitude-causes, and spike-and-slab sparse coding, and it reports that the resulting implementations can learn dictionaries from data with tens of millions of numbers using hundreds of CPUs in parallel. The paper claims that this combination—non-linear superpositions, inferred prior and noise parameters, rich approximate posteriors, and parallel scalability—is what distinguishes ProSper from standard sparse-coding libraries.

Load-bearing premise

The central claim rests on the assumption that the implementations in the library are faithful to the cited algorithms and that the advertised scaling to hundreds of CPUs holds in this particular code, with this version providing no benchmark results or code to check that.

Editorial extensions

If this is right

Researchers can build and train generative sparse-coding models whose causes combine through max or other non-linear rules, not only through a linear sum.
Sparsity level and noise variance are estimated from data as part of EM, so users do not need to hand-tune these parameters as in standard L1 sparse coding.
Because every algorithm is exposed through the same Model/Annealing/EM interface, a dataset or annealing schedule can be reused across BSC, TSC, DSC, MCA, MMCA, and GSC.
Parallel execution over many CPUs and machines is available for all six algorithms, which the paper says supports hundreds of dictionary elements and tens of millions of data points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the advertised scaling holds, the max-superposition models MCA and MMCA, which are usually considered computationally expensive, become practical on image and audio datasets with tens of millions of samples, making occlusion-style generative models a testable alternative to linear $\ell^1$ sparse coding.
The abstract Model/Annealing/EM separation suggests that a new prior or link function could be added without touching the EM loop; a concrete next step would be to implement structured or factorial priors, such as group sparsity, and reuse the same truncation and MPI machinery.
Because the paper gives no benchmarks, the strongest independent check is to reproduce published likelihood curves or learned dictionaries from the original method papers using the library's implementations; that would separate the software-architecture claim from the scalability claim.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a software paper describing ProSper, a Python library that unifies several probabilistic sparse coding algorithms (BSC, TSC, DSC, MCA, MMCA, GSC, plus GMM and PMM) under a common interface. The manuscript itself is clear, the three-object design (Model, Annealing, EM) is sensible, and the usage example is concrete enough to see how a user would train a model. The authors are honest that the algorithms come from prior publications; the new thing is the artifact, not the learning rules. For someone who works with these models, a single library with a consistent API and MPI parallelization would genuinely lower the barrier to using non-standard priors and non-linear superpositions.

The soft spots are exactly where the reader put them, and they are not minor. There is no code repository link, no installation instructions, no unit tests, and no empirical results. The abstract claims typical runs use hundreds of CPUs and tens of millions of floating-point numbers, but nothing in the paper measures runtime, speedup, or convergence. The comparative statement that no other library is "as optimized for probabilistic sparse coding as ProSper" is unsupported without benchmarks against SPAMS, scikit-learn, or libDAI. Because the value of a software library is its executable artifact, this preprint currently describes a library the reader cannot inspect or run. That is a load-bearing gap: the scalability and correctness claims rest on an unprovided implementation.

I would not call this a fatal flaw in the idea. The underlying algorithms are published in respected venues, and a unified implementation is plausibly useful. But as a standalone paper, it is more of a documentation draft than a verified contribution. The authors need to link the code, provide installation and basic smoke-test scripts, and ideally report scaling curves on at least one dataset. Without that, a referee cannot check whether the implementations match the cited truncated EM procedures or whether the MPI parallelization actually works across nodes.

For you: if you work on dictionary learning or latent variable models, this paper is worth a quick read for the model taxonomy and the interface design, but I would not cite it until the artifact is public and verified. For peer review, I would send it out only after the authors add the missing code and empirical checks; the topic is legitimate and the writing is competent, so it deserves a chance to become a real software contribution.

Referee Report

3 major / 5 minor

Summary. The paper presents ProSper, a Python library for probabilistic dictionary learning with non-standard priors and superposition models. It describes the generative framework, lists eight implemented models (BSC, TSC, DSC, GSC, MCA, MMCA, GMM, PMM), outlines a three-object design (Annealing, Model, EM), gives a short usage example, and positions the library against related software. The algorithms are based on truncated expectation maximization and MPI parallelization, with claimed scalability to hundreds of CPUs.

Significance. If the library is real and performs as claimed, it would fill a practical gap by providing a unified, extensible implementation of sparse coding models with non-linear superpositions and flexible priors, complementing standard deterministic libraries. The paper's strengths are its clear architectural description and the careful enumeration of models with their references. However, the manuscript provides no code, no benchmarks, and no empirical validation, so the central claims about the library's existence, correctness, and scalability are currently unsubstantiated.

major comments (3)

[Section 3 (User Interface) and Abstract] The central claim of the paper is that ProSper is a usable, scalable library, but the manuscript provides no artifact: there is no code repository URL, installation instructions, version identifier, or test suite. The usage example in Section 3 cannot be executed or independently checked, and the implementation of the cited algorithms cannot be verified. For a software contribution, the executable artifact is the primary evidence, so its absence leaves the existence and correctness of the library unverified.
[Abstract and Section 1 (Introduction)] The scalability claim (hundreds of CPUs, tens of millions of floating-point numbers, hundreds of dictionary elements) is stated without any benchmark, profiling data, or experimental section in this manuscript. The citations to earlier papers (e.g., Guiraud et al., 2018; Sheikh et al., 2019) support the underlying algorithms in their original implementations, but they do not establish that this specific Python library achieves those scales. Without runtime or scaling results for ProSper itself, this claim is unsupported.
[Section 4 (Related Software Libraries)] The statement that 'None of the libraries is as optimized for probabilistic sparse coding as ProSper' is a comparative claim with no empirical basis. No benchmarks against SPAMS, scikit-learn, libDAI, or Libra are provided, and the term 'optimized' is not defined. This unsubstantiated superlative overstates the contribution relative to the evidence presented.

minor comments (5)

[Section 2 (Learning Algorithm and Data Models)] The sentence 'All algorithms use expectation maximization for parameter optimization and truncated posteriors as efficient approximation' is inaccurate for GMM and PMM, which are standard EM models without truncated posteriors; the sentence should be restricted to the sparse coding models.
[Section 3 (User Interface and Documentation)] The instantiation example 'BSC ET(D, H, Hprime, gamma)' uses 'ET' without defining the abbreviation; please expand it (presumably 'expectation truncation') or explain it in the text.
[Section 4 (Related Software Libraries)] For reproducibility, URLs for the cited software packages (SPAMS, mlpack, scikit-learn, libDAI, Libra) should be included in the reference list.
[Abstract] The phrase 'data with tens of millions of floating-point numbers' is awkward; it likely refers to dataset size, but the intended meaning should be clarified (e.g., number of data points or total data elements).
[Table 1] The prior model for TSC is listed as 'categorical/zero-mean'; specifying the exact distribution (e.g., a three-state categorical with zero mean) would be clearer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ProSper is a software paper that implements and cites previously derived algorithms; it makes no new mathematical derivation whose conclusion is presupposed.

full rationale

The paper does not derive a new algorithm or result. Equations (1) and (2) merely define the generic generative form p(s|Theta) and p(y|s,Theta) used across the library, and all model-specific inference and learning details are delegated to external publications listed in Table 2, such as Luecke and Eggert (2010), Exarchakis and Luecke (2017), and Sheikh et al. (2014). These are independent, published derivations that do not depend on the present paper. The self-citations are normal references to the group's own prior work and are not used to justify a prediction inside this manuscript: the library implements those cited algorithms rather than using the citations to close a derivation loop. The scalability claim ('Typical large-scale runs of the algorithms can use hundreds of CPUs') is supported only by citations to earlier papers, not by benchmarks in this preprint; the absence of a code link and of experiments makes the library's correctness and scalability unverified, but that is a reproducibility and verifiability concern, not circularity. There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors, no ansatz smuggled in via citation, and no known result relabeled as a new organization. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper relies on the generative model structure and the truncated posterior approximation from prior work. It also assumes the implementations are faithful to the cited algorithm descriptions. No new free parameters or invented scientific entities are introduced.

assumptions (3)

domain assumption All models follow the generative process defined by prior p(s|Theta) and noise model p(y|s,Theta).
Section 2 states that all contained models are based on this generative process, but the paper does not justify why this covers all intended use cases.
domain assumption The truncated posterior approximation (Luecke and Eggert, 2010) is a valid and sufficient approximation for efficient learning in all implemented models.
Section 2 invokes this approximation without new justification or analysis of its error bounds in the library context.
domain assumption The implementations are faithful to the referenced algorithm descriptions.
The paper does not provide tests or independent verification that the library code matches the mathematical specifications in the cited papers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ProSper -- A Python Library for Probabilistic Sparse Coding with Non-Standard Priors and Superpositions." pith.science (2026). https://pith.science/paper/GBM4FAN5

@misc{pith2026190806843,
  author       = {Pith},
  title        = {Pith review of: ProSper -- A Python Library for Probabilistic Sparse Coding with Non-Standard Priors and Superpositions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GBM4FAN5}},
  note         = {Machine review of arXiv:1908.06843}
}

read the original abstract

ProSper is a python library containing probabilistic algorithms to learn dictionaries. Given a set of data points, the implemented algorithms seek to learn the elementary components that have generated the data. The library widens the scope of dictionary learning approaches beyond implementations of standard approaches such as ICA, NMF or standard L1 sparse coding. The implemented algorithms are especially well-suited in cases when data consist of components that combine non-linearly and/or for data requiring flexible prior distributions. Furthermore, the implemented algorithms go beyond standard approaches by inferring prior and noise parameters of the data, and they provide rich a-posteriori approximations for inference. The library is designed to be extendable and it currently includes: Binary Sparse Coding (BSC), Ternary Sparse Coding (TSC), Discrete Sparse Coding (DSC), Maximal Causes Analysis (MCA), Maximum Magnitude Causes Analysis (MMCA), and Gaussian Sparse Coding (GSC, a recent spike-and-slab sparse coding approach). The algorithms are scalable due to a combination of variational approximations and parallelization. Implementations of all algorithms allow for parallel execution on multiple CPUs and multiple machines for medium to large-scale applications. Typical large-scale runs of the algorithms can use hundreds of CPUs to learn hundreds of dictionary elements from data with tens of millions of floating-point numbers such that models with several hundred thousand parameters can be optimized. The library is designed to have minimal dependencies and to be easy to use. It targets users of dictionary learning algorithms and Machine Learning researchers.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 17 canonical work pages

[1]

org Bornschein, Marc Henniges, and J\

J\"org Bornschein, Marc Henniges, and J\"org L\"ucke. Are V1 receptive fields shaped by low-level visual occlusions? A comparative study. PLOS Computational Biology, 9 0 (6): 0 e1003062, 2013

work page 2013
[2]

Curtin, Marcus Edel, Mikhail Lozhnikov, Yannis Mentekidis, Sumedh Ghaisas, and Shangtong Zhang

Ryan R. Curtin, Marcus Edel, Mikhail Lozhnikov, Yannis Mentekidis, Sumedh Ghaisas, and Shangtong Zhang. mlpack 3: a fast, flexible machine learning library. Journal of Open Source Software, 3: 0 726, 2018. doi:10.21105/joss.00726. URL https://doi.org/10.21105/joss.00726

doi:10.21105/joss.00726 2018
[3]

o rg L \

Zhenwen Dai, Georgios Exarchakis, and J \"o rg L \"u cke. What are the invariant occlusive components of image patches? a probabilistic generative approach. In Advances in Neural Information Processing Systems, pages 243--251, 2013

work page 2013
[4]

Exarchakis, M

G. Exarchakis, M. Henniges, J. Eggert, and J. L\"ucke. Ternary sparse coding. In Proceedings LVA/ICA, LNCS. Springer, 2012. in press

work page 2012
[5]

o rg L \

Georgios Exarchakis and J \"o rg L \"u cke. Discrete sparse coding. Neural computation, 29 0 (11): 0 2979--3013, 2017

work page 2017
[6]

o rg L\

Enrico Guiraud, Jakob Drefs, and J\" o rg L\" u cke. Evolutionary expectation maximization. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '18, pages 442--449, New York, NY, USA, 2018. ACM. ISBN 978-1-4503-5618-3. doi:10.1145/3205455.3205588. URL http://doi.acm.org/10.1145/3205455.3205588

arXiv 2018
[7]

Henniges, G

M. Henniges, G. Puertas, J. Bornschein, J. Eggert, and J. L\"ucke. Binary sparse coding. In Proceedings LVA/ICA, LNCS 6365, pages 450--57. Springer, 2010

work page 2010
[8]

The libra toolkit for probabilistic models

Daniel Lowd and Amirmohammad Rooshenas. The libra toolkit for probabilistic models. Journal of Machine Learning Research, 16: 0 2459--2463, 2015

work page 2015

Show all 20 references

[9]

L\"ucke and J

J. L\"ucke and J. Eggert. Expectation truncation and the benefits of preselection in training generative models. JMLR, 11: 0 2855--900, 2010

2010
[10]

L\"ucke and M

J. L\"ucke and M. Sahani. Maximal causes for non-linear component extraction. JMLR, 9: 0 1227--67, 2008

2008
[11]

L \"u cke and A.-S

J. L \"u cke and A.-S. Sheikh. Closed-form EM for sparse coding and its application to source separation. In LVA/ICA, LNCS, pages 213--221. Springer, 2012

2012
[12]

Online learning for matrix factorization and sparse coding

J Mairal, F Bach, J Ponce, and G Sapiro. Online learning for matrix factorization and sparse coding . JMLR, 11, 2010. URL http://portal.acm.org/citation.cfm?id=1756008

2010
[13]

Online dictionary learning for sparse coding

Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online dictionary learning for sparse coding. In ICML, page 87, 2009

2009
[14]

JM. Mooij. libdai: A free and open source c++ library for discrete approximate inference in graphical models. Journal of Machine Learning Research, 11: 0 2169--2173, August 2010

2010
[15]

Nickisch

H. Nickisch. glm-ie: The generalised linear models inference and estimation toolbox. Journal of Machine Learning Research, 13: 0 1699--1703, May 2012

2012
[16]

Olshausen and D

B. Olshausen and D. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381: 0 607--9, 1996

1996
[17]

Pedregosa, G

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in P ython. Journal of Machine Learnin...

2011
[18]

Puertas, J

G. Puertas, J. Bornschein, and J. L\" u cke. The maximal causes of natural scenes are edge filters. In Advances in Neural Information Processing Systems, volume 23, pages 1939--1947. 2010

1939
[19]

o rg L \

Abdul-Saboor Sheikh, Jacquelyn A. Shelton, and J \"o rg L \"u cke. A truncated em approach for spike-and-slab sparse coding. JMLR, 15: 0 2653--2687, 2014

2014
[20]

o rg L \

Abdul-Saboor Sheikh, Nicol S. Harper, Jakob Drefs, Yosef Singer, Zhenwen Dai, Richard E. Turner, and J \"o rg L \"u cke. Strfs in primary auditory cortex emerge from masking-based statistics of natural sounds. PLOS Computational Biology, 15 0 (1): 0 e1006595, 01 2019. doi:10.1...

2019 doi

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

org Bornschein, Marc Henniges, and J\

J\"org Bornschein, Marc Henniges, and J\"org L\"ucke. Are V1 receptive fields shaped by low-level visual occlusions? A comparative study. PLOS Computational Biology, 9 0 (6): 0 e1003062, 2013

work page 2013

[2] [2]

Curtin, Marcus Edel, Mikhail Lozhnikov, Yannis Mentekidis, Sumedh Ghaisas, and Shangtong Zhang

Ryan R. Curtin, Marcus Edel, Mikhail Lozhnikov, Yannis Mentekidis, Sumedh Ghaisas, and Shangtong Zhang. mlpack 3: a fast, flexible machine learning library. Journal of Open Source Software, 3: 0 726, 2018. doi:10.21105/joss.00726. URL https://doi.org/10.21105/joss.00726

doi:10.21105/joss.00726 2018

[3] [3]

o rg L \

Zhenwen Dai, Georgios Exarchakis, and J \"o rg L \"u cke. What are the invariant occlusive components of image patches? a probabilistic generative approach. In Advances in Neural Information Processing Systems, pages 243--251, 2013

work page 2013

[4] [4]

Exarchakis, M

G. Exarchakis, M. Henniges, J. Eggert, and J. L\"ucke. Ternary sparse coding. In Proceedings LVA/ICA, LNCS. Springer, 2012. in press

work page 2012

[5] [5]

o rg L \

Georgios Exarchakis and J \"o rg L \"u cke. Discrete sparse coding. Neural computation, 29 0 (11): 0 2979--3013, 2017

work page 2017

[6] [6]

o rg L\

Enrico Guiraud, Jakob Drefs, and J\" o rg L\" u cke. Evolutionary expectation maximization. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '18, pages 442--449, New York, NY, USA, 2018. ACM. ISBN 978-1-4503-5618-3. doi:10.1145/3205455.3205588. URL http://doi.acm.org/10.1145/3205455.3205588

arXiv 2018

[7] [7]

Henniges, G

M. Henniges, G. Puertas, J. Bornschein, J. Eggert, and J. L\"ucke. Binary sparse coding. In Proceedings LVA/ICA, LNCS 6365, pages 450--57. Springer, 2010

work page 2010

[8] [8]

The libra toolkit for probabilistic models

Daniel Lowd and Amirmohammad Rooshenas. The libra toolkit for probabilistic models. Journal of Machine Learning Research, 16: 0 2459--2463, 2015

work page 2015