Pith. sign in

REVIEW 2 major objections 5 minor 20 references

Determinantal point process sampling for bioacoustic active learning

T0 review · 2 major / 5 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DPP-driven active learning cuts bioacoustic annotation costs

desk verdict Solid challenge submission combining DPP batch selection with annealed uncertainty-novelty for bioacoustic active learning. The method works and the ablation is informative, but the key ablation has a gap that weakens the central attribution claim. read the letter →

arxiv 2607.06063 v1 pith:RZU7VSXO submitted 2026-07-07 cs.SD cs.LG

Hugo Magaldi , Gabriel Dubus This is my paper · ORCID
classification cs.SDcs.LG
keywords activelearningdeterminantalpointprocessbioacousticsbatchselectionclassimbalanceannotationefficiencyembeddingnovelty
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CARE-DPP, an active learning method for bioacoustic classification that reduces annotation effort by intelligently selecting which audio samples to label. The core mechanism is a determinantal point process (DPP), a mathematical object that selects diverse, non-redundant subsets of data points. By combining DPP-based batch diversification with an annealed schedule that shifts from geometric coverage early to classifier uncertainty late, the method achieves a mean AULC of 0.50 across four datasets, outperforming the CoreSet baseline of 0.46. The central claim is that enforcing batch-level diversity via DPP is the single most important factor for improving label efficiency in this setting.

What carries the argument

Determinantal point process (DPP) batch selection: a quality-weighted kernel that greedily maximizes the log-determinant of selected sample embeddings, rewarding high individual quality while penalizing redundancy within the batch.

What would settle it

If the unspecified fallback used when DPP is removed is a degenerate or poorly-tuned selection method, the attributed gain to DPP may be overstated; a properly tuned alternative batch diversity method might close the gap.

Watch

Extended reading notes

Core claim

The key finding is that using a determinantal point process to select diverse, non-redundant annotation batches yields the largest performance gains in a bioacoustic active learning loop, more so than class-balanced uncertainty, annealed novelty, or exploration strategies. Removing DPP selection drops mean AULC from 0.50 to 0.46, with the steepest losses on datasets with heterogeneous embedding spaces (HSN and UHH).

Load-bearing premise

The claim that DPP is the largest contributor rests on a single ablation configuration where DPP is replaced by an unspecified fallback, without reporting statistical significance across the five repeats or testing sensitivity to the fallback choice.

Editorial extensions

If this is right

  • DPP-based batch selection could be directly transferred to other active learning domains with high embedding heterogeneity, such as medical imaging or remote sensing.
  • The finding that adaptive batch sizing (small early, large late) matters suggests that frequent model updates early in training are critical when initial predictions are unreliable.
  • The dataset-dependent effects of annealing and class balancing indicate that a one-size-fits-all active learning strategy may be suboptimal across bioacoustic domains with different label densities and class distributions.
  • The decreasing random exploration fraction provides a template for balancing exploitation of model scores with exploration of the feature space in other pool-based active learning settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This technical report presents CARE-DPP, a batch active-learning acquisition method for the BioDCASE 2026 challenge. The method combines class-balanced multilabel uncertainty, cosine embedding novelty with annealed weights, a candidate pool mixing top-quality and random exploration samples, and DPP-based greedy batch selection via pivoted Cholesky. An adaptive acquisition schedule uses smaller batches early and larger batches later. Evaluated on four bioacoustic datasets (three BirdSet subsets and ATBFL) over five repeats, CARE-DPP achieves a mean development AULC of 0.50 versus 0.46 for the official CoreSet baseline. An ablation study attributes the largest gains to DPP batch selection and the adaptive acquisition schedule.

Significance. The method is clearly and completely specified, including all equations, hyperparameter schedules, and Algorithm 1, making it reproducible. The use of DPP for batch active learning in bioacoustics is a reasonable and well-motivated application of an established technique. The evaluation against four official baselines on four datasets provides useful evidence. The per-dataset ablation breakdown (Table 3) is informative and goes beyond reporting only aggregate means. The submission includes reproducible code and five-repeat result exports, which is commendable.

major comments (2)
  1. §4.2, Table 3, 'No DPP batch selection' row: The paper does not specify what selection mechanism replaces DPP in this ablation. This is load-bearing for the central claim that 'DPP batch diversification is the largest contributor.' If the fallback is random sampling from the candidate pool, the ablation conflates DPP's diversity benefit with the benefit of using quality scores for selection at all. A fairer fallback would be greedy top-k selection by quality score q_i from the same candidate pool, which would isolate the diversity contribution of DPP from its quality-weighting role. The paper should specify the fallback and, ideally, include this stronger baseline.
  2. §4.2, Table 3: No standard deviations or significance tests are reported across the five repeats, despite the paper stating that five independent repeats were conducted (§4) and that submitted YAML files contain per-cycle means and standard deviations. The mean AULC drop attributed to removing DPP (0.5017 to 0.4639) is concentrated on HSN (0.608 to 0.541) and UHH (0.4335 to 0.3697), while ATBFL barely moves (0.4652 to 0.4634) and POW drops moderately (0.5002 to 0.4813). Without variance estimates, it is unclear whether the 0.04 mean difference is robust or driven by one or two datasets. Adding standard deviations (or at minimum, per-dataset repeat-level variance) would substantially strengthen the ablation claims.
minor comments (5)
  1. §3, paragraph below Eq. (6): The text states weights were 'fixed after a coarse validation sweep' but does not specify the validation split used, the range of values explored, or the selection criterion. A brief note on which dataset(s) the sweep was performed on would help assess generalization risk.
  2. Table 1: The 'Labels/sample' column is useful but the caption could clarify whether these are averages over the development pool or the full dataset.
  3. §3.3, Eq. (7): The candidate pool size formula M_t = min{|U_t|, max(30B_t, 1500)} could benefit from a brief justification for the specific constants (30, 1500), especially since the pool size directly affects DPP computational cost and selection quality.
  4. Figure 1 is referenced but not visible in the provided text; ensure it clearly labels all ablation variants and uses consistent ordering with Table 3.
  5. §5, final paragraph: The mention of 'additional candidate-pool variants were explored but did not yield robust improvements' without reporting them is acceptable for a technical report but could be briefly footnoted with the variants tried, to save future researchers from repeating the same exploration.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. Both major points are well-taken and addressable in revision. We will (1) specify the DPP ablation fallback and add a greedy top-k baseline, and (2) add standard deviations across the five repeats to the ablation table. We agree with both points and will revise accordingly.

read point-by-point responses

  1. Referee: The paper does not specify what selection mechanism replaces DPP in the 'No DPP batch selection' ablation. If the fallback is random sampling from the candidate pool, the ablation conflates DPP's diversity benefit with the benefit of using quality scores for selection at all. A fairer fallback would be greedy top-k selection by quality score q_i from the same candidate pool.

    Authors: The referee is correct on both counts. The current ablation uses random sampling from the candidate pool as the fallback, and we agree this does not isolate DPP's diversity contribution from its quality-weighting role. This is a genuine gap in the ablation design. We will (1) explicitly state the fallback mechanism in the revised Table 3 caption and §4.2, and (2) add a new ablation row using greedy top-k selection by quality score q_i from the same candidate pool, which isolates the diversity contribution of DPP from its quality-weighting role. This stronger baseline will allow a fairer attribution of the gains to batch-level non-redundancy specifically. revision: yes

  2. Referee: No standard deviations or significance tests are reported across the five repeats in Table 3, despite the paper stating five independent repeats were conducted. Without variance estimates, it is unclear whether the 0.04 mean difference attributed to removing DPP is robust or driven by one or two datasets.

    Authors: The referee is right that variance estimates are necessary to assess the robustness of the ablation differences, and we have no reason not to report them. The per-repeat data is already available in the submitted YAML exports. We will add standard deviations across the five repeats to Table 3 for every cell (both per-dataset and mean columns). We will also add a brief note in §4.2 discussing the per-dataset variability pattern the referee identifies, particularly the contrast between the large DPP-removal effects on HSN and UHH versus the negligible effect on ATBFL. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected: the method is defined independently of the evaluation metric, all citations are external, and results are empirical measurements against external baselines.

full rationale

The paper presents CARE-DPP as a combination of standard components (class-balanced entropy, cosine novelty, DPP batch selection, adaptive scheduling) and evaluates it empirically against external baselines provided by the BioDCASE challenge organizers. The DPP formulation (Eqs. 7-9) follows the standard Kulesza & Taskar (2012) and Bıyık et al. (2019) framework, both external references. No self-citations appear in the reference list. The hyperparameters (annealing weights in Eq. 5, exploration fractions in Eq. 8, batch sizes in Eq. 10) are stated as chosen from 'simple design principles' and 'a coarse validation sweep' — the paper does not claim these are derived from first principles or predicted theoretically. The AULC results in Tables 2-3 are empirical measurements, not analytical predictions. The ablation in Table 3 removes components and reports resulting AULC changes; while the skeptic correctly notes that the 'No DPP' fallback is unspecified and lacks significance tests, this is an experimental-rigor concern (correctness risk), not circularity — the ablation results are not forced by construction. No step in the derivation chain reduces to its own inputs by definition or by self-citation.

Assumptions & free parameters 12 free parameters · 3 assumptions · 0 invented entities

The method introduces no new entities (particles, forces, dimensions). It uses standard mathematical tools (DPP, entropy, cosine similarity) applied to existing embeddings (Perch v2). The free parameters are all hyperparameters chosen by design principles and a coarse validation sweep, not fitted to the target result.

free parameters (12)
  • Uncertainty blend weight = 0.75
    Chosen to bias toward underrepresented classes; fixed after coarse validation sweep (Eq. 3).
  • Novelty blend weight = 0.25
    Complement to uncertainty blend; fixed after coarse validation sweep (Eq. 3).
  • Annealing slope = 0.40
    Controls rate of shift from novelty to uncertainty over budget (Eq. 5).
  • Novelty base weight = 0.65
    Initial novelty weight before annealing (Eq. 5).
  • Uncertainty base weight = 0.25
    Initial uncertainty weight before annealing (Eq. 5).
  • Quality floor = 0.05
    Prevents low-scored candidates from being completely suppressed in DPP (Section 3.3).
  • DPP regularization = 1e-6
    Numerical stability term for log-determinant selection (Section 3.3).
  • Exploration fraction schedule = 0.40, 0.25, 0.15
    Three-tier random exploration proportion decreasing with labeled set size (Eq. 8).
  • Acquisition batch schedule = 25, 50, 75
    Three-tier batch size increasing with labeled set size (Eq. 10).
  • Candidate pool size factor = 30
    Multiplier for batch size in candidate pool construction (Eq. 7).
  • Candidate pool minimum = 1500
    Floor for candidate pool size (Eq. 7).
  • Class weight exponent = -0.5
    Exponent for inverse square root class frequency weighting (Eq. 2).
assumptions (3)
  • domain assumption Perch v2 embeddings provide a meaningful feature space for measuring novelty and diversity.
    The entire method relies on embedding-space cosine novelty and DPP diversity. If embeddings are not semantically structured, novelty and diversity are uninformative. Section 2.
  • standard math Greedy maximization of log det(K_St + 1e-6 I) is a sufficient approximation to exact DPP sampling.
    The paper uses pivoted Cholesky for greedy selection instead of exact DPP sampling. This is a standard approximation. Section 3.3.
  • domain assumption The fixed BaseAL training pipeline (10 epochs, lr=1e-3) is sufficient for the classifier to provide informative uncertainty scores.
    The method relies on classifier uncertainty. If the training pipeline is inadequate, uncertainty scores are noise. Section 2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Determinantal point process sampling for bioacoustic active learning." pith.science (2026). https://pith.science/paper/RZU7VSXO

@misc{pith2026260706063,
  author       = {Pith},
  title        = {Pith review of: Determinantal point process sampling for bioacoustic active learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZU7VSXO}},
  note         = {Machine review of arXiv:2607.06063}
}
read the original abstract

Eco-acoustic monitoring generates vast volumes of audio data, making active learning a promising approach for reducing annotation effort while efficiently training reliable biodiversity classifiers. This report presents CARE-DPP, a batch active-learning acquisition method submitted to BioDCASE Active Learning for Bioacoustics 2026 challenge. The method combines class-balanced predictive uncertainty with embedding-space novelty, while a determinantal point process (DPP) objective selects a high-quality and non-redundant acquisition batch. The uncertainty-novelty balance is annealed over the annotation budget: early cycles emphasize geometric coverage, whereas later cycles increasingly exploit classifier uncertainty. To mitigate unreliable early scores, the DPP candidate pool mixes top-quality candidates with a decreasing proportion of random exploration. An adaptive acquisition schedule uses smaller batches early and larger batches later. Evaluated over five repeats on the BirdSet HSN, POW and UHH subsets and on ATBFL, CARE-DPP obtains a mean development AULC of 0.50 for macro mAP, compared with 0.46 for the official CoreSet baseline. Ablations identify DPP batch diversification and the adaptive acquisition schedule as the largest contributors.

Figures

Figures reproduced from arXiv: 2607.06063 by the authors.

Figure 1
Figure 1. Mean AULC of CARE-DPP and component ablations. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

  1. [1]

    INTRODUCTION Passive acoustic monitoring can produce far more recordings than experts can annotate. Pool-based active learning aims at address- ing this imbalance by repeatedly selecting a small set of unlabeled samples whose annotation is expected to improve a model most efficiently [1, 2]. The BioDCASE Active Learning for Bioacous- tics 2026 task standa...

    work page 2026

  2. [2]

    TASK AND DA TA The BaseAL loop starts from a randomly initialized multilabel clas- sification head. At each cycle, the sampler receives current pre- dictions, Perch v2 embeddings, and the currently labeled indices; selected labels are revealed by an oracle and the head is retrained. Perch v2 is a multi-taxa bioacoustic representation model designed for tr...

    work page

  3. [3]

    Determinantal point process sampling for bioacoustic active learning

    METHOD LetL t andU t be the labeled and unlabeled sets at cyclet, and letB t be the requested acquisition size. Each sample has a nor- malized embeddingx i and multilabel probabilitiesp ic for classes c= 1, . . . , C. The scalar weights and thresholds used below were chosen from simple design principles - favoring coverage early, un- arXiv:2607.06063v1 [c...

    work page Pith review arXiv 2026

  4. [4]

    Submitted Y AML files contain the per-cycle means and standard deviations

    EXPERIMENTS AND RESULTS All results use the same maximum budget of 500, the fixed task learning rate and training epochs, five independent repeats, and AULC as the ranking metric. Submitted Y AML files contain the per-cycle means and standard deviations. 4.1. Main results Table 2 compares CARE-DPP with official aggregate baselines. The proposed sampler re...

    work page arXiv 2026

  5. [5]

    Removing it reduces mean AULC from 0.50 to 0.46, with particularly large losses on HSN and UHH (Table 3)

    DISCUSSION The ablation study shows that DPP batch selection is the most important component of CARE-DPP. Removing it reduces mean AULC from 0.50 to 0.46, with particularly large losses on HSN and UHH (Table 3). This is likely because these datasets contain more heterogeneous or sparsely covered embedding regions, so selecting a non-redundant batch is esp...

    work page

  6. [6]

    The method reaches a mean development AULC of 0.50 across ter- restrial and marine bioacoustic datasets

    CONCLUSION We introduced CARE-DPP, a general active-learning sampler com- bining class-balanced uncertainty, annealed embedding novelty, strong early candidate exploration, and DPP batch diversification. The method reaches a mean development AULC of 0.50 across ter- restrial and marine bioacoustic datasets. Ablations show that DPP selection and adaptive a...

    work page

  7. [7]

    ACKNOWLEDGMENT The author thanks the BioDCASE 2026 Task 4 organizers for pro- viding the BaseAL framework and curated datasets

    work page 2026

  8. [8]

    A survey of deep active learning,

    P. Ren, Y . Xiao, X. Chang, P.-Y . Huang, Z. Li, B. B. Gupta, X. Chen, and X. Wang, “A survey of deep active learning,” ACM Computing Surveys, vol. 54, no. 9, pp. 1–40, 2022

    work page 2022

Show all 20 references
  1. [9]

    Computational bioacoustics with deep learning: A review and roadmap,

    D. Stowell, “Computational bioacoustics with deep learning: A review and roadmap,”PeerJ, vol. 10, p. e13152, 2022

    2022

  2. [10]

    BioDCASE 2026 Task 4: Active Learning for Bioacoustics,

    BioDCASE Task 4 Coordinators, “BioDCASE 2026 Task 4: Active Learning for Bioacoustics,” Online challenge description, 2026, accessed: 2026-06-05. [Online]. Available: https://biodcase.github.io/challenge2026/task4

    2026

  3. [11]

    BaseAL: Active learning baseline,

    B. McEwen and S. Zhang, “BaseAL: Active learning baseline,” 2026. [Online]. Available: https://doi.org/10.5281/ zenodo.18467564

    2026

  4. [12]

    Active learning for convolutional neural networks: A core-set approach,

    O. Sener and S. Savarese, “Active learning for convolutional neural networks: A core-set approach,” inInternational Con- ference on Learning Representations, 2018

    2018

  5. [13]

    The coincidence approach to stochastic point pro- cesses,

    O. Macchi, “The coincidence approach to stochastic point pro- cesses,”Advances in Applied Probability, vol. 7, no. 1, pp. 83–122, 1975

    1975

  6. [14]

    Determinantal point processes for machine learning,

    A. Kulesza and B. Taskar, “Determinantal point processes for machine learning,”F oundations and Trends in Machine Learning, vol. 5, no. 2–3, pp. 123–286, 2012

    2012

  7. [15]

    Batch active learning using determinantal point processes,

    E. Bıyık, K. Wang, N. Anari, and D. Sadigh, “Batch active learning using determinantal point processes,”arXiv preprint arXiv:1906.07975, 2019

    1906 arXiv

  8. [16]

    Perch 2.0: The bittern lesson for bioacoustics,

    B. van Merri ¨enboer, V . Dumoulin, J. Hamer, L. Harrell, A. Burns, and T. Denton, “Perch 2.0: The bittern lesson for bioacoustics,”arXiv preprint arXiv:2508.04665, 2025. Evaluation and Benchmarking in Automated Bioacoustics 2026 Challenge

    2025

  9. [17]

    BirdSet: A large-scale dataset for audio classifi- cation in avian bioacoustics,

    L. Rauch, R. Schwinger, M. Wirth, R. Heinrich, D. Huseljic, M. Herde, J. Lange, S. Kahl, B. Sick, S. Tomforde, and C. Scholz, “BirdSet: A large-scale dataset for audio classifi- cation in avian bioacoustics,” inInternational Conference on Learning Representations, 2025

    2025

  10. [18]

    BioDCASE 2026 Task 4: BirdSet Dataset,

    L. Rauch, M. Herde, and B. McEwen, “BioDCASE 2026 Task 4: BirdSet Dataset,” Mar. 2026. [Online]. Available: https://doi.org/10.5281/zenodo.19191602

    2026 doi

  11. [19]

    BioD- CASE 2026 Task 4: ATBFL Dataset,

    R. Kurinchi-Vendhan, S. Zhang, and B. McEwen, “BioD- CASE 2026 Task 4: ATBFL Dataset,” 2026. [Online]. Available: https://doi.org/10.5281/zenodo.19133111

    2026 doi

  12. [20]

    On the low-rank approximation by the pivoted cholesky decomposition,

    H. Harbrecht, M. Peters, and R. Schneider, “On the low-rank approximation by the pivoted cholesky decomposition,”Ap- plied Numerical Mathematics, vol. 62, no. 4, pp. 428–440, 2012

    2012

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.