Pith. sign in

REVIEW 4 cited by

Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1903.06059 v2 pith:ISOJZDKW submitted 2019-03-14 cs.LG stat.ML

Wouter Kool , Herke van Hoof , Max Welling This is my paper · ORCID
classification cs.LGstat.ML
keywords replacementwithoutsamplingsequencestrickbeamdistributionmodel
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

The well-known Gumbel-Max trick for sampling from a categorical distribution can be extended to sample $k$ elements without replacement. We show how to implicitly apply this 'Gumbel-Top-$k$' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Search. Even for exponentially large domains, the number of model evaluations grows only linear in $k$ and the maximum sampled sequence length. The algorithm creates a theoretical connection between sampling and (deterministic) beam search and can be used as a principled intermediate alternative. In a translation task, the proposed method compares favourably against alternatives to obtain diverse yet good quality translations. We show that sequences sampled without replacement can be used to construct low-variance estimators for expected sentence-level BLEU score and model entropy.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rank-Conditioned Sample Reuse for the Plackett--Luce Best-of-$K$ Objective

    cs.LG 2026-07 accept novelty 6.5 of 10

    Rank-conditioned Horvitz–Thompson reuses all C(n,K) subsets of one Gumbel-Top-n pool for unbiased Plackett–Luce best-of-K value and score-function gradient, with an exact Max-specific DP collapse to a 1-D integral.

  2. An Iterative Dual-Channel Neural Quantum State Algorithm for Selected Configuration Interaction

    physics.chem-ph 2026-06 unverdicted novelty 6.0 of 10

    HI-NQS uses a dual-channel autoregressive Transformer NQS inside an iterative sample-diagonalize-update loop to reach chemical accuracy on small molecules and nitrogen active spaces with better determinant scaling than CIPSI.

  3. Looking elsewhere: improving variational Monte Carlo gradients by importance sampling

    quant-ph 2025-07 conditional novelty 6.0 of 10

    Adaptively tuned overdispersed importance sampling, q_alpha proportional to |psi|^alpha, cuts the Monte Carlo sample count needed to converge neural quantum states, especially for peaked molecular wavefunctions.

  4. Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models

    cs.SE 2025-05 conditional novelty 5.0 of 10

    Variational Prefix Tuning adds a CVAE with sampled latent prefixes to pre-trained code summarization models, improving the accuracy and diversity of generated summary sets.

Pith tools