REVIEW 4 major objections 4 minor 26 references
Parametric Neural Amp Modeling with Active Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read PANAMA claims that the most informative amp settings are the ones where an ensemble of models disagrees most, and that finding them by gradient ascent beats random sampling at 64 recorded points.
desk verdict A clean late-breaking demo that makes a real case for active learning over amp knob settings, but the single-run evidence and unvalidated disagreement proxy keep it at 'promising' rather than 'proven'. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the cross-model disagreement $D_G(x,g)$, computed from $M$ independently trained copies of the model evaluated at a fixed input signal $x$ and a candidate knob vector $g$. It serves as a differentiable proxy for uncertainty: gradients are propagated back from $D_G$ to $g$, an optimizer climbs them to find local maxima, and the distinct maxima (usually 4 to 5 per round after clustering) become the next settings to record. This turns the question of which setting to label into a continuous optimization problem.
What would settle it
A direct control: gather 64 settings by maximizing disagreement and another 64 by minimizing it, using the same 10 starting points, ensemble size, and training budget; if the low-disagreement settings match or beat the high-disagreement settings on the same validation set, then disagreement is not the useful signal. A second control would run the same loop with a different fixed input signal for selection, to test whether the gains depend on that particular 3-minute clip.
Extended reading notes
Core claim
The paper proposes PANAMA, a parametric neural amp modeler whose acquisition step is driven by ensemble disagreement. The claim is that the best next amp settings to record are found by maximizing the cross-model disagreement $D_G(x,g) = \frac{1}{M} \operatorname{tr}(\operatorname{Var}_i[f^{(i)}_G(x,g)])$ over the continuous space of knob vectors, using gradient ascent, with the input signal held fixed. The reported evidence is that at 64 recorded settings the active-learning model reaches $3.4 \times 10^{-4}$ validation mean-squared error, versus $8.6 \times 10^{-4}$ for uniform random sampling and $9 \times 10^{-4}$ for $\beta$-distributed sampling, and that the chosen settings cluster near the extremes of the knob range.
Load-bearing premise
The load-bearing premise is that when the ensemble members disagree strongly at an unrecorded amp setting, recording that setting will actually reduce the model's error; the paper takes disagreement as a proxy for information gain without directly testing that connection.
Editorial extensions
If this is right
- At a 64-point budget, active selection reaches roughly 2.5 times lower validation error than uniform random sampling, showing that the acquisition policy, not just the model architecture, determines data efficiency.
- The selected settings concentrate near knob extremes, indicating that extreme settings carry information that interpolation from interior data cannot provide.
- The beta-distribution heuristic that also favors extremes performs no better than random, so the gain is not simply sampling corners; the disagreement signal matters.
- If the procedure scales to more knobs, the exponential growth of configurations becomes less of a blocker, since each active-learning round adds only a handful of settings.
Reading between the lines
- Beyond the paper, the same acquisition loop should transfer to other parametric audio effects, such as distortion pedals, compressors, or synthesizer parameters, where labels are expensive and the control space is continuous.
- Because the chosen settings depend on the fixed 3-minute input signal, an untested extension would optimize over short alternative input excerpts or jointly over input and settings.
- A practical extension is to stop adding datapoints when the top disagreement values stop decreasing, turning the fixed 64-point budget into an automatic stopping rule; the paper does not evaluate this stopping criterion.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PANAMA, an active-learning framework for training parametric guitar amplifier models. The model is a WaveNet-style network conditioned on a six-dimensional amp-knob setting vector, and the training data consists of input-signal/amp-setting pairs whose labels are recorded wet signals. The acquisition function is the average cross-model output variance over an ensemble of independently trained models, and candidate settings are found by gradient ascent on this disagreement measure over the continuous knob space. The authors report results on an amp-sim setup: starting from 10 random points and acquiring up to 64 total points, PANAMA achieves a validation MSE of 3.4e-04, compared with 8.6e-04 for uniform random sampling and 9e-04 for a beta-distribution heuristic. The paper also releases code and provides a histogram of the acquired setting values, which cluster near 0 and 1.
Significance. If the central claim is valid, PANAMA would address a genuine practical bottleneck in parametric amp modeling: reducing the number of physical recording configurations needed to train a controllable model. The contribution includes an open-source implementation, a differentiable acquisition procedure over a continuous setting space, and a direct comparison against random and heuristic sampling. These are real strengths. However, the empirical evidence is currently very thin: the headline result rests on a single table with one MSE value per method, no repeated seeds or error bars, and no independent validation of the disagreement proxy. The significance of the paper is therefore conditional on substantially stronger experimental support.
major comments (4)
- [Table 1] The entire load-bearing claim that active learning improves over random sampling rests on a single MSE value per method, with no reported variance, no number of seeds, and no statistical comparison. Because model training involves random initialization and data shuffling, the observed 8.6e-04 vs. 3.4e-04 gap could be within run-to-run variability. The authors should report mean and standard deviation over at least 5–10 independent runs for each sampling method, and ideally paired comparisons using the same initialization conditions.
- [Section 2, Eq. (1)] The acquisition function D_G(x,g) is introduced as a proxy for model uncertainty, but the paper never validates that maximizing D_G actually reduces future validation error. The sentence "we cannot compute the uncertainty directly and opt to use model disagreement... as a proxy" is an assertion, not a demonstrated property. A concrete test would be to compare settings chosen by D_G against settings chosen by simpler baselines (e.g., farthest-point sampling, random extremes, or D_G averaged over multiple input signals) and to measure the correlation between D_G values and the realized error reduction after labeling. Without such validation, the observed improvement could be an artifact of high-gain/high-sensitivity regions producing large output variance without providing generalizable information.
- [Section 3, experimental setup] All active-learning acquisitions are performed with a single fixed 3-minute input signal taken from NAM, while validation uses 30 minutes of multi-genre guitar audio. The paper does not address whether settings selected as informative for this one signal remain informative for other input signals. The authors should repeat the acquisition procedure with at least two or three different fixed input signals (e.g., clean, distorted, different genres) and verify that the improvement over random sampling is consistent. Otherwise the reported gain may be specific to the chosen excitation signal.
- [Section 3, active-learning hyperparameters] Several choices that are likely to affect the result are not ablated: ensemble size M=4, the number of Adam restarts (10), the initial dataset size (10), the total budget (64), and the clustering procedure used to deduplicate local optima. In particular, the ensemble size directly controls the quality of the disagreement estimate, and the number of restarts controls how well the continuous optimization explores the acquisition surface. The paper should include a sensitivity analysis over at least ensemble size and number of restarts to show that the method does not rely on a fragile configuration.
minor comments (4)
- [Section 2, Eq. (1)] The prose after the equation says "we take the cross-model variances of the output signals in element and then average them into one scalar"; this is missing the word "element-wise" and should clarify that the trace is taken over the output channels/features and then averaged over time samples.
- [Figure 3] The caption does not state whether the histogram aggregates the 64 acquired g-vectors or all intermediate acquisitions, nor does it note that the flattened component values are pooled across the six knobs. Please specify this in the caption.
- [Section 3] The sentence "For active learning, We use a model ensemble of size 4" has an unnecessary capital 'W'; also, the Adam optimizer hyperparameters (learning rate, number of steps) are not provided, which hinders reproducibility.
- [References] The code URL is mentioned in the text but not included in the reference list; for a late-breaking/demo paper this is acceptable, but the authors should at least provide the full repository URL and a version or commit hash for archival reproducibility.
Circularity Check
No circularity: the acquisition function is computed from ensemble disagreement, and the central claim is evaluated on held-out random amp settings with an external dataset.
full rationale
The paper's claimed derivation chain is self-contained. The active learning target is D_G(x,g) = (1/M) tr(Var_i[f^(i)_G(x,g)]), which is computed purely from the outputs of independently trained ensemble members on an unlabeled datapoint (fixed input signal x and candidate settings g). The label for a datapoint is the wet signal recorded from an amp sim, and the final comparison in Table 1 is against a validation set of roughly 1,000 randomly sampled amp settings with 30 minutes of external guitar audio from the IDMT-SMT-GUITAR dataset. No parameter is fitted to the validation targets, and no equation in the paper identifies maximizing D with minimizing validation MSE by construction. The beta-distribution heuristic is motivated by a histogram of the gathered g-vectors, but the paper uses α=β=0.5 rather than the fitted values α=0.5396, β=0.4122, and the heuristic performs worse than random, so the comparison is not forced. The central concern raised by a skeptical reader—that ensemble disagreement may not be a validated proxy for label informativeness—is a correctness or robustness issue about the acquisition function, not a circularity: the outcome is empirically testable and the paper reports a non-trivial result. There are no load-bearing self-citations: references to NAM, WaveNet, and IDMT-SMT-GUITAR are external prior work. Overall, the derivation reduces neither by definition nor by fitted-parameter renaming, so the circularity score is 0.
Assumptions & free parameters
free parameters (6)
- Ensemble size M =
4
- Initial datapoints =
10
- Adam restarts =
10
- Total datapoint budget =
64
- Training epochs =
50
- Beta heuristic parameters =
alpha=beta=0.5
assumptions (4)
- domain assumption Cross-model disagreement is a valid proxy for informativeness
- domain assumption The fixed input signal x is representative for selecting knob settings
- domain assumption A feed-forward WaveNet-like architecture can represent the parametric amp transform
- standard math Validation settings are sampled i.i.d. and MSE is a meaningful metric
Cite this review
Pith. "Pith review of Parametric Neural Amp Modeling with Active Learning." pith.science (2026). https://pith.science/paper/G4EHGLEN
@misc{pith2026250702109,
author = {Pith},
title = {Pith review of: Parametric Neural Amp Modeling with Active Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/G4EHGLEN}},
note = {Machine review of arXiv:2507.02109}
}
read the original abstract
We introduce PANAMA, an active learning framework for the training of end-to-end parametric guitar amp models using a WaveNet-like architecture. With \model, one can create a virtual amp by recording samples that are determined by an active learning strategy to use a minimum amount of datapoints (i.e., amp knob settings). We show that gradient-based optimization algorithms can be used to determine the optimal datapoints to sample, and that the approach helps under a constrained number of samples.
Reference graph
Works this paper leans on
-
[1]
Parametric Neural Amp Modeling with Active Learning
INTRODUCTION In recent years, data-driven guitar amp modeling has be- come increasing popular. Such approaches treat an amp as a blackbox, and simply learn the transform which the amp applies to the raw guitar signal in an end-to-end fash- ion. The trained models can then be integrated as plugins into Digital Audio Workstations (DAWs), or deployed on mode...
work page 2025
-
[2]
METHODOLOGY Model Architecture.We adapt the WaveNet [21] architec- ture to be used in a feed-forward manner, similar to prior works. WaveNet was originally developed as an autore- gressive generative model, which processes audio using a stack of dilated convolutional layers and predicts a categor- ical distribution over the next sample to be generated. We...
work page Pith review arXiv 2025
-
[3]
This in- cludes the following parameters: Gain, Bass, Mid, Treble, Master, Presence
EXPERIMENTAL EV ALUA TION For our experiments, we choosedim(g) = 6. This in- cludes the following parameters: Gain, Bass, Mid, Treble, Master, Presence. The Master knob is included, as it dif- fers from a standard volume knob. The fixed input signal is taken from NAM [20] and roughly 3 minutes long. We ob- tain the ground truth signals from an amp sim. Al...
-
[4]
Deep Learning for Tube Amplifier Emulation
E.-P. Damskägg, L. Juvela, E. Thuillier, and V . Välimäki, “Deep learning for tube am- plifier emulation,” 2019. [Online]. Available: https://arxiv.org/abs/1811.00334
work page Pith review arXiv 2019
-
[5]
Real- time modeling of audio distortion circuits with deep learning,
E.-P. Damskägg, L. Juvela, and V . Välimäki, “Real- time modeling of audio distortion circuits with deep learning,” 2019. [Online]. Available: https: //api.semanticscholar.org/CorpusID:196205025
work page 2019
-
[6]
Deep learning for black-box modeling of audio effects,
M. A. Martínez Ramírez, E. Benetos, and J. D. Reiss, “Deep learning for black-box modeling of audio effects,”Applied Sciences, vol. 10, no. 2,
-
[7]
A vacuum- tube guitar amplifier model using a recurrent neural network,
J. M. Covert and D. L. Livingston, “A vacuum- tube guitar amplifier model using a recurrent neural network,”2013 Proceedings of IEEE Southeastcon, pp. 1–5, 2013. [Online]. Available: https://api. semanticscholar.org/CorpusID:38784328
work page 2013
-
[9]
Efficient neural networks for real-time modeling of analog dynamic range compression,
C. J. Steinmetz and J. D. Reiss, “Efficient neural networks for real-time modeling of analog dynamic range compression,” 2022. [Online]. Available: https: //arxiv.org/abs/2102.06200
arXiv 2022
Show all 26 references
-
[10]
Modelling black-box audio effects with time-varying feature modulation,
M. Comunità, C. Steinmetz, H. Phan, and J. D. Reiss, “Modelling black-box audio effects with time-varying feature modulation,”ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2022. [Online]. Available: https://api.s...
2023
-
[11]
A general-purpose deep learning approach to model time- varying audio effects,
M. A. M. Ramírez, E. Benetos, and J. D. Reiss, “A general-purpose deep learning approach to model time- varying audio effects,”ArXiv, vol. abs/1905.06148,
1905 arXiv
-
[12]
A vacuum-tube guitar amplifier model using long/short-term memory net- works,
Z. Zhang, E. Olbrych, J. Bruchalski, T. J. Mc- Cormick, and D. L. Livingston, “A vacuum-tube guitar amplifier model using long/short-term memory net- works,”SoutheastCon 2018, pp. 1–5, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID: 52935728
2018
-
[13]
Real time emulation of parametric guitar tube amplifier with long short term memory neural network,
T. Schmitz and J.-J. Embrechts, “Real time emulation of parametric guitar tube amplifier with long short term memory neural network,” 2018. [Online]. Available: https://arxiv.org/abs/1804.07145
2018 arXiv
-
[14]
Nonlinear modeling of the guitar signal chain enabling its real-time emulation,
T. Schmitz, “Nonlinear modeling of the guitar signal chain enabling its real-time emulation,” 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID: 201147276
2019
-
[15]
Pruning deep neural network models of guitar distortion effects,
D. Sudholt, A. Wright, C. Erkut, and V . Välimäki, “Pruning deep neural network models of guitar distortion effects,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 256–264, 2023. [Online]. Available: https://api. semanticscholar.org/CorpusID:253694043
2023
-
[16]
Sample rate independent recurrent neural networks for audio effects processing,
A. Carson, A. Wright, J. Chowdhury, V . Välimäki, and S. Bilbao, “Sample rate independent recurrent neural networks for audio effects processing,”ArXiv, vol. abs/2406.06293, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:270371338
2024 arXiv
-
[17]
Real- time black-box modelling with recurrent neural net- works,
A. Wright, E.-P. Damskägg, and V . Välimäki, “Real- time black-box modelling with recurrent neural net- works,” 09 2019
2019
-
[18]
Neural modelling of periodically modulated time-varying effects,
A. Wright and V . Välimäki, “Neural modelling of periodically modulated time-varying effects,” 2020. [Online]. Available: https://api.semanticscholar.org/ CorpusID:226306209
2020
-
[19]
Neural modeling of phaser and flanging effects,
A. Wright and V . Välimäki, “Neural modeling of phaser and flanging effects,”Journal of the Audio Engineering Society, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:236505785
2021
-
[20]
Atkinson
S. Atkinson. (2025) Neural amp modeller. Ac- cessed: 2025-06-13. [Online]. Available: https: //www.neuralampmodeler.com/
2025
-
[21]
Wavenet: A generative model for raw audio,
A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” 2016. [Online]. Available: https://arxiv.org/abs/1609.03499
2016 arXiv
-
[22]
Hyper recurrent neural network: Condition mechanisms for black-box audio effect modeling,
Y .-T. Yeh, W.-Y . Hsiao, and Y .-H. Yang, “Hyper recurrent neural network: Condition mechanisms for black-box audio effect modeling,”ArXiv, vol. abs/2408.04829, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:271843407
2024 arXiv
-
[23]
Real-time guitar amplifier em- ulation with deep learning,
A. Wright, E.-P. Damskägg, L. Juvela, and V . Välimäki, “Real-time guitar amplifier em- ulation with deep learning,”Applied Sciences, vol. 10, no. 3, 2020. [Online]. Available: https://www.mdpi.com/2076-3417/10/3/766
2020
-
[24]
End-to-end amp modeling: From data to controllable guitar amplifier models,
L. Juvela, E.-P. Damskägg, A. Peussa, J. Mäkinen, T. Sherson, S. I. Mimilakis, and A. Gotsopoulos, “End-to-end amp modeling: From data to controllable guitar amplifier models,” 2024. [Online]. Available: https://arxiv.org/abs/2403.08559
2024 arXiv
-
[27]
IDMT- SMT-Guitar dataset,
C. Kehling, A. Männchen, and A. Eppler, “IDMT- SMT-Guitar dataset,” 2023. [Online]. Available: https: //doi.org/10.5281/zenodo.7544110
2023 doi
-
[2019]
Available: https://api.semanticscholar
[Online]. Available: https://api.semanticscholar. org/CorpusID:155093236
-
[2020]
Available: https://www.mdpi.com/ 2076-3417/10/2/638
[Online]. Available: https://www.mdpi.com/ 2076-3417/10/2/638
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.