Pith. sign in

REVIEW 3 major objections 4 minor 18 references

On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read The paper establishes that the Gibbs decoder has super-polynomial worst-case mixing time for Reed–Muller codes over the binary symmetric channel.

desk verdict A genuine worst-case slow-mixing result for Gibbs decoding of RM codes, but Theorem 3 as stated overclaims; the core bound survives with rate-dependent constants. read the letter →

arxiv 2505.00974 v1 pith:JYYJXCAC submitted 2025-05-02 cs.IT math.IT

classification cs.ITmath.IT MSC 94B3560J10
keywords Reed–MullercodesGibbsdecodermixingtimeMarkovchainMonteCarloposteriorsamplingbottleneckratiobinarysymmetricchannelsuper-polynomialcomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a negative worst-case complexity result for a natural Markov-chain decoder. It shows that for Reed–Muller codes over a binary symmetric channel, the Gibbs decoder can require super-polynomial time to converge to the posterior distribution, even when the received word is conditionally typical for some codeword. The obstruction is a carefully chosen received sequence that makes the all-zero message a bottleneck: the message has moderate posterior probability, but every single-bit flip sharply increases its Hamming distance to the received word, so the sampler is very unlikely to escape. This matters because posterior sampling is known to achieve capacity for Reed-Muller codes, but a decoder that mixes too slowly cannot realize that performance in the worst case. The result leaves open whether average-case mixing or modified Gibbs variants are fast.

What carries the argument

The proof is carried by two mechanisms. The Gibbs chain on the message space has stationary distribution $\mu(m)\propto \theta^{d_H(mG,y)}$ with $\theta=p/(1-p)<1$, so posterior mass is exponentially concentrated near codewords close to $y$; a step flips one uniformly chosen message bit with probability $\mu(m\oplus i)/(\mu(m)+\mu(m\oplus i))$. The bottleneck ratio (conductance) of a set $S$ is $\Phi(S)=\sum_{s_1\in S,s_2\notin S}\mu(s_1)P(s_1\to s_2)/\mu(S)$, and the standard bound $T_{\rm mix}\ge 1/(4\Phi(P))$ lets the proof use the single state $S=\{0\}$: its conductance is just $1-P(0\to 0)\le (1/k)\sum_i \theta^{\delta_i}$, where $\delta_i=d_H(e_iG,y)-d_H(0,y)$. The constructed polynomial $f_y$ makes $\delta_i\ge 2^{m-r-q+1}$ for every generator row, and the normal approximation to the binomial distribution converts the rate condition $R_j\to R$ into $r=m/2+c\sqrt{m}/2$, which turns $2^{m-r-q}$ into $\sqrt{n}\,\exp(-\sqrt{\log n})$ and hence, through the bottleneck lemma, into the mixing-time lower bound $\exp(\Omega(\sqrt{n}\,\exp(-\sqrt{\log n})))$.

What would settle it

Take the sequence $RM(0,m)$ with any $p<1/2$: the message space has only two states, the Gibbs transition matrix is $2\times 2$, and direct diagonalization gives $T_{\rm mix}=O(1)$ independent of $m$, contradicting the claimed super-polynomial lower bound. This pins the missing hypothesis as $R>0$ (equivalently $r\ge 1$).

Watch

Extended reading notes

Core claim

The central claim is that, for any crossover probability $p<1/2$ and any rate $R<1-H(p)$, every sequence of Reed–Muller codes with unbounded length and rate approaching $R$ admits received words for which the Gibbs decoder mixes in time $\Omega(\exp(\sqrt{n}\cdot \exp(-\sqrt{\log n})))$, which is super-polynomial in the blocklength $n=2^m$. The proof constructs the received word as the evaluation of the polynomial $f_y(z_1,\dots,z_m)=(\prod_{i=1}^{q-1}z_i+1)(z_m+1)$ with $q=\lceil\log_2(1/p)\rceil$. For this word, the all-zero message has posterior probability at most $1/2$, while flipping any one message bit changes the Hamming distance to $y$ by at least $2^{m-r-q+1}$; the Gibbs chain must cross this exponentially small escape probability, and the bottleneck-ratio bound converts it into the super-polynomial mixing-time lower bound. In the paper's own terms, this is a worst-case lower bound on the complexity of Gibbs decoding, not a statement about average received words.

Load-bearing premise

The proof assumes the normal approximation that turns the code rate $R$ into the code order $r$ is accurate enough, without quantifying the error, and the approximation fails at $R=0$, where the stated theorem would be false; the claim therefore implicitly requires $R>0$ (or $r\ge 1$).

Editorial extensions

If this is right

  • For every $p<1/2$ and every rate below capacity, some conditionally typical received word makes the Gibbs chain take $\Omega(\exp(\sqrt{n}\,\exp(-\sqrt{\log n})))$ steps to mix, so no universal polynomial worst-case guarantee is possible for this decoder.
  • The bad received words are not arbitrary adversarial noise; each lies in the conditional typical set for a codeword, so the slow mixing can occur on outputs the channel would actually produce.
  • Because the only property used is an energy gap under single-bit flips, any MCMC decoder whose updates resample one message bit from the posterior will inherit the same exponential mixing lower bound for the same constructed words.
  • The result does not settle average-case complexity: the constructed received word may have vanishing probability, and the paper explicitly leaves open the possibility that typical received sequences mix quickly on average.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The bottleneck construction is likely not special to Reed–Muller structure: any linear code whose generator rows have weights growing like $2^{m-r}$ and that contains a codeword at distance $2^{m-q}$ from a suitable received word would yield the same lower bound through the same bottleneck argument.
  • A direct check at rate zero shows the theorem needs an $R>0$ (or $r\ge 1$) hypothesis; replacing the normal approximation with an exact binomial tail bound would make the statement airtight for positive rates and quantify the constant in the exponent.
  • Empirically, the bound may be loose: running the Gibbs chain on the constructed received word at moderate blocklengths would show whether the bottleneck at the all-zero message is as severe as the exponent suggests, and whether annealed or block-update variants escape it.
  • If average-case mixing were later shown to be polynomial, then combining that with the posterior-sampling equivalence lemma would produce a polynomial-time capacity-achieving posterior-sampling decoder for Reed–Muller codes; the present worst-case result would not contradict it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper studies the mixing time of the Gibbs decoder (Glauber dynamics on the message space) for Reed–Muller codes over the binary symmetric channel. The main result, Theorem 3, claims that for any crossover probability p < 1/2 and any asymptotic rate R < 1 - H(p), there exists a sequence of conditionally typical received sequences for which the mixing time of the Gibbs decoder is Ω(exp(√n · exp(−√log n))), i.e., super-polynomial. The proof combines a conductance-based bottleneck lemma (Lemma 3) with an explicit construction of a received vector y (Lemma 4) such that the all-zero message has posterior mass at most 1/2 and every single-bit flip increases the Hamming distance to y by at least 2^{m−r−q+1}, where q = ⌈log_2(1/p)⌉. The final asymptotic step converts r ≈ m/2 + c√m/2 into a lower bound on the mixing time using the normal approximation to the binomial distribution.

Significance. If the main theorem were correct as stated, it would be a notable worst-case lower bound: it would show that a natural MCMC decoder for RM codes can require super-polynomial time to converge to the posterior, in contrast to the capacity-achieving MAP decoding of RM codes. The construction in Lemma 4 is explicit and the conductance argument is self-contained. However, as written the theorem is not proven: the proof drops a rate-dependent exponent, and the statement is false for R = 0. The qualitative super-polynomial conclusion appears salvageable after restricting the rate range and carrying the rate-dependent constants, but the paper in its current form does not establish the stated uniform bound.

major comments (3)
  1. [Theorem 3, proof] The passage from the bound δ_i ≥ 2^{m−r−q+1} to the claimed log T_mix = Ω(√n exp(−√log n)) silently drops the dependence on c = Φ^{-1}(R). Substituting r = m/2 + (c/2)√m into the Lemmas 3–4 lower bound gives log T_mix = Ω(2^{m/2 − (c/2)√m − q + 1}) = Ω(√n · 2^{−q+1} · exp(−(c√ln 2 / 2)√log n)). The claimed uniform bound holds only when c√ln 2 / 2 ≤ 1, i.e., R ≤ Φ(2/√ln 2) ≈ 0.9918. For p small enough that 1 − H(p) > 0.9918, the theorem's assumptions allow rates in (0.9918, 1 − H(p)) for which the proven bound is exp(√n · exp(−C√log n)) with C > 1, which is not Ω(exp(√n · exp(−√log n))) because the ratio of the exponents tends to zero. Please either restrict the theorem to the regime c√ln 2 / 2 ≤ 1 or state the rate-dependent bound; the latter still yields the advertised super-polynomiality for every fixed R < 1 − H(p).
  2. [Theorem 3 (statement), Lemma 4] The theorem claims any rate R < 1 − H(p), which includes R = 0. For the sequence RM(0,m) (r = 0, k = 1), the message space has two states and the Gibbs chain has P(0→1) = µ(1), P(1→0) = µ(0); after one step the distribution is exactly the stationary distribution, so T_mix = 1, contradicting the claimed lower bound. The proof fails at Lemma 4(2): the codeword c = Eval(z_m + 1) used for the comparison message u is not in RM(0,m), so the posterior comparison µ(u) ≥ µ(0) is unavailable; indeed µ(0) > 1/2 in that case. The theorem and Lemma 4 must exclude r = 0 (or state a separate treatment).
  3. [§IV, Eq. (2)] Equation (2) treats the normal approximation to the binomial CDF as an exact identity and inverts it to obtain r = m/2 + (c/2)√m. The paper does not quantify the approximation error. Since the exponent of the mixing-time lower bound depends on r through √n · 2^{−r}, an unquantified O(1) error in r would change the constant in front of √log n, so the asymptotic conclusion is not rigorous as written. A Berry–Esseen bound (or a direct large-deviation estimate) should be invoked to justify the expansion for the fixed-rate sequences under consideration.
minor comments (4)
  1. [Lemma 4 statement] The statement says "there exists a non-zero vector y ∈ F_m^2"; this should read y ∈ F_2^{2^m} (or F_2^n).
  2. [Lemma 4 statement (3)] The notation "d/(2q−1)" is missing an exponent; it should be d/2^{q−1}.
  3. [Theorem 3 proof] The bound is written as δ_i ≥ 2^{m−r−q}, whereas Lemma 4 gives 2^{m−r−q+1}; the exponent is off by one. This does not change the asymptotics but should be corrected.
  4. [Theorem 3 proof] The formula "k = (m ≤ r)" in the proof of Theorem 3 is typeset incorrectly; it should be the binomial sum ∑_{i=0}^r \binom{m}{i}.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the mixing-time lower bound follows from standard conductance estimates and an explicit code/channel-output construction, with no fitted parameter renamed as a prediction.

full rationale

The paper's central claim is a worst-case lower bound on the Gibbs decoder's mixing time for Reed--Muller codes. The derivation chain is self-contained: Lemma 3 reduces slow mixing to a bottleneck ratio bound, using the standard Levin--Peres theorem Tmix >= 1/(4Phi(P)); Lemma 4 explicitly constructs the received word y = Eval(f_y) and computes the required quantities wt(y), dH(c,y), and wt(y+g_i)-wt(y) exactly from the polynomial structure. No step takes as input the conclusion it claims to derive, and no fitted parameter is later called a prediction. The cited Liu--Cuff--Verdú lemma on posterior sampling appears only as motivation for why sampling from the posterior is relevant; the proof of Theorem 3 does not depend on it. There are no load-bearing self-citations: the references to Huang's thesis and Huang--Kim are contextual comparisons, not justifications of the main theorem. The asymptotic step in Theorem 3 does use the binomial CLT to express r in terms of R, which is a legitimate (though approximate) input-to-parameter relation, not a circular reuse of the claimed bound. Separate concerns about the theorem as stated — the silent dropping of the rate-dependent coefficient c = Φ^{-1}(R) and the q factor, and the R=0 counterexample — are correctness issues, not circularity under the requested taxonomy. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard Markov chain theorems (existence of stationary distribution and the conductance lower bound) and on an asymptotic normal approximation for the binomial CDF. No free parameters are fitted; p and R are inputs to the theorem. No new entities are postulated. The R=0 edge case is a boundary condition the theorem statement should explicitly exclude.

assumptions (4)
  • standard math Ergodic finite Markov chains converge to a unique stationary distribution (Theorem 1 in the paper).
    Used to justify that the Gibbs sampler has a well-defined stationary posterior and converges; from Levin and Peres, Markov Chains and Mixing Times.
  • standard math Bottleneck ratio bound: Tmix ≥ 1/(4Φ(P)) (Theorem 2 in the paper).
    The main lower-bound tool; from Levin and Peres, Theorem 7.4.
  • domain assumption Binomial CDF asymptotic normal approximation: Pr[Bin(m,0.5) ≤ r] ≈ Φ((r-m/2)/√(m/4)).
    Used in Proof of Theorem 3 to relate code rate R to order r; the paper does not state error bounds, and the approximation's validity for all R in (0,1) is assumed.
  • domain assumption For any rate R in (0,1), there exists a sequence of RM codes RM(r_j,m_j) with m_j→∞ and rate → R.
    Needed to apply the theorem to arbitrary R; standard but not proven in the paper. For R=0 the construction may fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes." pith.science (2026). https://pith.science/paper/JYYJXCAC

@misc{pith2026250500974,
  author       = {Pith},
  title        = {Pith review of: On the Worst-Case Complexity of Gibbs Decoding for Reed--Muller Codes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JYYJXCAC}},
  note         = {Machine review of arXiv:2505.00974}
}
read the original abstract

Reed--Muller (RM) codes are known to achieve capacity on binary symmetric channels (BSC) under the Maximum a Posteriori (MAP) decoder. However, it remains an open problem to design a capacity achieving polynomial-time RM decoder. Due to a lemma by Liu, Cuff, and Verd\'u, it can be shown that decoding by sampling from the posterior distribution is also capacity-achieving for RM codes over BSC. The Gibbs decoder is one such Markov Chain Monte Carlo (MCMC) based method, which samples from the posterior distribution by flipping message bits according to the posterior, and can be modified to give other MCMC decoding methods. In this paper, we analyze the mixing time of the Gibbs decoder for RM codes. Our analysis reveals that the Gibbs decoder can exhibit slow mixing for certain carefully constructed sequences. This slow mixing implies that, in the worst-case scenario, the decoder requires super-polynomial time to converge to the desired posterior distribution.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

  1. [1]

    Application of boolean algebra to switchi ng circuit design and to error detection,

    D. E. Muller, “Application of boolean algebra to switchi ng circuit design and to error detection,” Transactions of the I.R.E. Professional Group on Electronic Computers , vol. EC-3, no. 3, pp. 6–12, 1954

  2. [2]

    A class of multiple-error-correcting codes an d the decoding scheme,

    I. Reed, “A class of multiple-error-correcting codes an d the decoding scheme,” Transactions of the IRE Professional Group on Information Theory, vol. 4, no. 4, pp. 38–49, 1954

  3. [3]

    Reed–muller codes achieve capacity on erasure ch annels,

    S. Kudekar, S. Kumar, M. Mondelli, H. D. Pfister, E. ¸ Sa¸ so ˇglu, and R. L. Urbanke, “Reed–muller codes achieve capacity on erasure ch annels,” IEEE Transactions on Information Theory , vol. 63, no. 7, pp. 4298– 4316, 2017

  4. [4]

    Reed–muller codes on bms chan nels achieve vanishing bit-error probability for all rates belo w capacity,

    G. Reeves and H. D. Pfister, “Reed–muller codes on bms chan nels achieve vanishing bit-error probability for all rates belo w capacity,” IEEE Transactions on Information Theory , vol. 70, no. 2, pp. 920–949, 2024

  5. [5]

    A proof that Reed-Muller codes achieve Shannon capacity on symmetric channels

    E. Abbe and C. Sandon, “A proof that reed-muller codes ach ieve shannon capacity on symmetric channels,” 2023. [Online]. A vailable: https://arxiv.org/abs/2304.02509

  6. [6]

    A serial orthogonal decoder,

    R. R. Green, “A serial orthogonal decoder,” in JPL Space Programs Summary, vol. 37, 1966, pp. 247–253

  7. [7]

    Recursive projection-aggregation de coding of reed- muller codes,

    M. Y e and E. Abbe, “Recursive projection-aggregation de coding of reed- muller codes,” in 2019 IEEE International Symposium on Information Theory (ISIT) , 2019, pp. 2064–2068

  8. [8]

    An Analysis of RPA Decoding of Reed-Muller Codes Over the BSC

    V . A. Rameshwar and V . Lalitha, “An upper bound on the erro r probability of rpa decoding of reed-muller codes over the bs c,” 2024. [Online]. Available: https://arxiv.org/abs/2412.08129

Show all 18 references
  1. [9]

    Recursive decoding and its performance for lo w-rate reed- muller codes,

    I. Dumer, “Recursive decoding and its performance for lo w-rate reed- muller codes,” IEEE Transactions on Information Theory , vol. 50, no. 5, pp. 811–823, 2004

  2. [10]

    Decoding of second order reed-muller code s with a large number of errors,

    B. Sakkour, “Decoding of second order reed-muller code s with a large number of errors,” in IEEE Information Theory W orkshop, 2005. , 2005, pp. 3 pp.–

  3. [11]

    Comparing the bit-map and block-map decoding thresholds o f reed- muller codes on bms channels,

    S. Kudekar, S. Kumar, M. Mondelli, H. D. Pfister, and R. Ur bankez, “Comparing the bit-map and block-map decoding thresholds o f reed- muller codes on bms channels,” in 2016 IEEE International Symposium on Information Theory (ISIT) . IEEE Press, 2016, p. 1755–1759. [Online]. Av...

  4. [12]

    On α -decodability and α -likelihood decoder,

    J. Liu, P . Cuff, and S. V erdú, “On α -decodability and α -likelihood decoder,” in 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2017, pp. 118–124

  5. [13]

    Monte carlo decoding of LDPC codes,

    R. M. Neal, “Monte carlo decoding of LDPC codes,” 2001, t alk given at ICTP Workshop on Statistical Physics and Capacity-Approaching Codes

  6. [14]

    Estimating th e weight enu- merators of reed-muller codes via sampling,

    S. Jain, V . A. Rameshwar, and N. Kashyap, “Estimating th e weight enu- merators of reed-muller codes via sampling,” in 2024 IEEE International Symposium on Information Theory (ISIT) , 2024, pp. 280–285

  7. [15]

    Markov chain monte carlo methods for dete ction and decoding,

    J.-T. Huang, “Markov chain monte carlo methods for dete ction and decoding,” PhD thesis, University of California San Diego, 2023

  8. [16]

    D. A. Levin and Y . Peres, Markov chains and mixing times . American Mathematical Soc., 2017, vol. 107

  9. [17]

    Parallel monte carlo markov chain decoding of linear codes,

    J.-T. Huang and Y .-H. Kim, “Parallel monte carlo markov chain decoding of linear codes,” in 2023 IEEE International Symposium on Information Theory (ISIT) , 2023, pp. 2051–2056

  10. [18]

    MCMC decoding of LDPC codes with BP preprocessing,

    ——, “MCMC decoding of LDPC codes with BP preprocessing, ” in GLOBECOM 2020 - 2020 IEEE Global Communications Conference , 2020, pp. 1–5

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.