{"id":"526a59ef-63e1-4be5-b5c9-8bcdd1c2354c","arxiv_id":"2505.12404","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Performing residual quantization with hyperbolic operations and distance instead of Euclidean ones yields discrete multitoken representations that improve downstream hypernym generation and recommendation.","lead":"Hyperbolic Residual Quantization (HRQ) replaces the Euclidean steps of residual quantization with hyperbolic geometry to create discrete tokens for hierarchical data. The paper reports gains of up to 20% over Euclidean residual quantization on WordNet hypernym modeling, and smaller gains on recommendation tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Poincaré curvature c is never reported; since every hyperbolic operation and distance (Eq. 1) depends on c, the claimed HRQ advantage is not reproducible and its attribution to hyperbolic geometry is underdetermined.","rationale":"The reader's weakest assumption—that the unreported curvature c undermines the causal attribution to hyperbolic geometry—is the most load-bearing issue I can find. The paper's entire argument is that hyperbolic operations provide a better inductive bias for latent hierarchies; c is the parameter that sets the strength of that bias and appears in every formula (Eq. 1, Möbius addition/subtraction, exp/log maps). Appendix C gives implementation details for both experiments but no c value and no sweep. If c was fixed to a default, it should be stated; if it was selected by validation, that selection is an unacknowledged hyperparameter that could explain part of the gain. The concrete sweep over c would settle this. I also note the Euclidean codebook loss in Algorithm 1 as a compounding problem, but I do not make it the primary attack because the comparison could still be meaningful under a Euclidean codebook regularizer; the missing c is the condition that directly gates the headline claim. The reader's conditional verdict remains appropriate: the ideas are plausible and the gains are consistent across all reported configurations, but a few concrete revisions (report c, add sensitivity analysis, correct the pseudocode) are needed before the claim can be accepted.","tokens_in":16309,"tokens_out":9353,"duration_ms":100219,"concrete_test":"Re-run the WordNet hierarchy-modeling experiment for c ∈ {0.01, 0.1, 1, 10} (or re-implement from Algorithm 1 if no code is released), keeping all other hyperparameters fixed, and report Recall@10 for HRQ vs RQ at each c. If HRQ's advantage is positive across the whole range, the missing c is a reporting issue; if it shrinks, reverses, or is non-monotonic at the unreported value, the headline gain is not robust to the unspecified geometric choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 defines the Poincaré ball P_n^c and Eq. (1) as a function of curvature c; Section 3 and Algorithm 1 use exp_c, log_c, ⊕_c, ⊖_c, and d_Pc throughout. Yet Appendix C, the only implementation-details section, never states the value of c used in the WordNet or recommendation experiments, and no sensitivity analysis is reported. This matters because c controls the inductive bias that is the paper's entire causal story: as c→0, the Möbius operations and hyperbolic distance converge to Euclidean ones, so HRQ collapses to RQ; as c grows, points are pushed toward the boundary and the geometry becomes strongly scale-dependent. Without knowing c, the reader cannot tell whether the consistent 8-22% gains reflect hyperbolic geometry as claimed, or an unreported (possibly dataset-specific) choice of c that happens to favor HRQ. A related internal inconsistency compounds this: Algorithm 1's codebook loss l_cmt is written with Euclidean squared norms (||sg[r_i^C]−e_i^C||^2) rather than the hyperbolic distance Eq. (1) that the text says is used for quantization. If the implemented loss is Euclidean, the method is not 'full hyperbolic' as claimed; if it is hyperbolic, Algorithm 1 is wrong. Either way, the central comparison is incompletely specified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Hyperbolic Residual Quantization (HRQ), a modification of RQ-VAE in which the latent space is the Poincaré ball, codebook selection uses hyperbolic distance, residuals are computed with Möbius subtraction, and reconstruction uses Möbius addition. The authors evaluate HRQ on WordNet hypernym modeling and on recommendation tasks (Amazon Beauty/Toys/Sports, MovieLens), reporting consistent gains over Euclidean RQ multitokens, with up to 20% relative improvement in hierarchy modeling. Appendix D offers a structural explanation based on reduced norm variation in hyperbolic space.","tokens_in":16610,"tokens_out":7002,"duration_ms":65771,"significance":"If the empirical claims hold, the paper fills a clear gap: discrete residual quantization in hyperbolic space has been mostly unexplored, and the multitoken representations it produces could benefit generative retrieval and interpretability. The method is simple, and the experimental design of keeping the downstream model fixed while changing only the tokenization is appropriate for isolating the effect of hyperbolic geometry. The paper is not circular: the reported gains are measured against an external Euclidean RQ baseline. However, the contribution's value is contingent on resolving the specification gaps below; the current manuscript does not support a definitive causal attribution to hyperbolic geometry.","major_comments":[{"comment":"Algorithm 1's codebook loss l_cmt is written with Euclidean squared norms, ||sg[r_i^C]−e_i^C||^2 + α||r_i^C−sg[e_i^C]||^2, while §3.1 states that codebook selection uses the hyperbolic distance d_Pc of Eq. (1) and residual computation uses Möbius subtraction. If the implemented loss is Euclidean, the codebook is not trained with the hyperbolic metric and the 'full hyperbolic' claim in §3.1 is not supported; if the implemented loss is hyperbolic, Algorithm 1 is incorrect. This mismatch is load-bearing because the codebook geometry is exactly what the paper credits for the gains. Please state explicitly which loss is used, and if it is Euclidean, provide the hyperbolic version or an argument that the Euclidean norm on Poincaré coordinates is a sufficient training signal.","section":"§3.1 and Algorithm 1"},{"comment":"The Poincaré curvature c is never stated for any experiment. Eq. (1), the exponential/logarithmic maps, the Möbius operations, and Algorithm 1 all depend on c; as c→0 the hyperbolic operations converge to Euclidean ones, so without c the reader cannot tell whether HRQ's reported gains reflect hyperbolic geometry or an unreported, possibly dataset-specific choice of c. Appendix C gives learning rates, epochs, and architecture but no c value and no sensitivity analysis. In addition, Eq. (1) omits the standard 1/sqrt(c) prefactor of the Poincaré distance; while this does not change nearest-neighbor selection, it changes the scale of distances used in the §4.1 contrastive loss and should be corrected or explicitly justified. Report c for every experiment and add at least a small sweep (e.g., c in {0.01, 0.1, 1}) on the WordNet task to show that the claimed advantage is not an artifact of a single curvature.","section":"§2.1, §3, Appendix C"},{"comment":"The statistical evidence is incompletely reported. Table 1 has no error bars, number of seeds, or significance tests, although 8–22% relative gains are claimed; Table 2 reports only mean±std over 8 runs, and several HRQ improvements (e.g., AR SaO NDCG@10, −1.4%; MovieLens NDCG@10, +2.7%) are within one standard deviation of the RQ baseline. Please report standard deviations or confidence intervals for the hierarchy-modeling task and run paired significance tests across repeated tokenization and downstream training to support the 'consistently outperform' claim.","section":"§4.1, Tables 1 and 2"}],"minor_comments":[{"comment":"The residual update ri+1_C ← ri_C ⊖_c e^{i−1}_c should use e^i_C rather than e^{i−1}_c, and Algorithm 1 should list c as an explicit input since all operations depend on it.","section":"Algorithm 1"},{"comment":"The sentence 'In section 5. Finally, in Section 6 we summarize our findings' is incomplete; Section 5 is not introduced.","section":"§1"},{"comment":"There are several typos: 'Specficially' (§4.1), 'joinlty' (§4.1), 'taoxnomy' (§4.1), 'equiped' (Appendix C), 'parantheses' (Table 2 caption), and 'concanted' (Appendix C).","section":"Throughout"},{"comment":"The text says the (H)RQ-VAE learning rate is chosen using the validation split of the downstream task; please clarify whether this selection happens before or after the downstream model is trained, since using downstream validation to select the tokenizer may affect the cleanliness of the comparison.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear X,\n\nQuick take: this paper is worth a real referee slot, but it needs revision before I'd trust the numbers. The contribution is what the title says: residual quantization moved into the Poincaré ball, with Möbius subtraction for residuals, hyperbolic distance for codebook lookup, and Möbius addition for reconstruction. HyperVQ is single-level and RQ-VAE is Euclidean, so the combination is new, and the inductive-bias story is genuinely motivated.\n\nI'm fairly convinced by the WordNet hypernym experiment: Table 1 shows 8-22% Recall@10 gains over RQ across all codebook sizes and dimensions, despite identical downstream models. That's a lot of configurations with no obvious confound. The recommendation results are more modest but mostly positive.\n\nThe soft spots are real, and they cluster around specification. First, the Poincaré curvature c is never reported anywhere, including Appendix C. Every operation and Eq. (1) depends on c. As c→0, HRQ collapses to Euclidean RQ; without knowing c, I can't reproduce the experiments or attribute the gain to hyperbolic geometry rather than an unreported, possibly dataset-specific choice of c. That's a genuine gap, not a nit. Second, Algorithm 1 defines the codebook loss l_cmt with Euclidean squared norms, even though the text says quantization is hyperbolic. If the code actually uses Eq. (1) for the loss, the pseudocode is wrong; if it uses Euclidean norms, the model is not \"full hyperbolic\" as claimed. Either way, the manuscript has an internal contradiction. I'd also want error bars for Table 1 (the gain is large enough to survive, but still), and the hierarchy-discovery improvements are small enough that some negative cells (NDCG@10 on Sports/Outdoors) need explanation. No code or data is released, and the MovieLens descriptions depend on a proprietary LLM prompt, so independent replication is currently impossible.\n\nAm I skeptical of the central claim? Not really. The method is structurally sound, the math checks out, and the results are consistent across many configurations. The c omission is fixable with a sensitivity analysis; the loss inconsistency is fixable with a corrected algorithm and an honest statement of what was optimized. I'd classify this as major revision, not reject.\n\nAudience: people working on tokenization, generative retrieval, or hyperbolic representation learning. It deserves a serious referee, and if I were the editor I'd send it out.","headline":"HRQ is a plausible incremental extension of RQ-VAE to hyperbolic space, but the paper as written hides the curvature and uses a Euclidean codebook loss, so the main comparison isn't reproducible.","tokens_in":17111,"tokens_out":2442,"would_cite":true,"duration_ms":25934,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that moving residual quantization into hyperbolic space produces discrete multitoken representations that preserve latent hierarchies better than Euclidean residual quantization, with up to 20% gains on a noun hypernym…","keywords":["residual quantization","hyperbolic space","Poincaré ball","discrete representation learning","multitoken","hierarchical data","RQ-VAE","latent hierarchy"],"falsifier":"Train HRQ and RQ on the noun hypernym task while sweeping the Poincaré curvature $c$ across a wide range (e.g., $0.1$ to $10$) and report Recall@10 for each $c$; if no choice of $c$ beats Euclidean RQ, or if the gain appears only in a narrow window, the reported advantage depends on a favorable fixed constant rather than on hyperbolic geometry itself.","tokens_in":16108,"feed_emoji":"🌲","tokens_out":15001,"duration_ms":127679,"temperature":0.7,"pith_summary":"This paper proposes Hyperbolic Residual Quantization (HRQ), which moves residual quantization into the Poincaré ball, a model of hyperbolic space. In HRQ, the encoder, the residual difference, the reconstruction sum, and the nearest-codebook search all use hyperbolic operations instead of Euclidean ones. The intended effect is to give discrete multitoken representations an inductive bias toward hierarchical branching, so that tokens naturally group by shared ancestors. On a noun hypernym prediction task, HRQ tokens improve Recall@10 by up to 20% over Euclidean RQ tokens, and on recommender benchmarks with no hierarchy supervision HRQ-VAE tokens are also better than RQ-VAE tokens. The takeaway is that the geometry of the quantization space, not just the training loss, controls how well discrete codes capture latent hierarchy.","feed_headline":"Hyperbolic residual quantization beats Euclidean by up to 20%","feed_subtitle":"Discrete tokens learned in curved space beat Euclidean tokens on hypernym prediction and recommender tasks.","key_machinery":"The central object is the Poincaré ball model of hyperbolic space with curvature $c$, a ball-shaped manifold whose volume grows exponentially with distance from the origin. The mechanism is the replacement of Euclidean subtraction, addition, and distance in residual quantization by Möbius subtraction $x\\ominus_c y = x\\oplus_c(-y)$, Möbius addition $x\\oplus_c y$, and the hyperbolic distance $d_{\\mathcal{P}_c}(u,v)=\\operatorname{arcosh}(1+2c\\|u-v\\|^2/((1-c\\|u\\|^2)(1-c\\|v\\|^2)))$, which keeps every quantization step inside the manifold.","core_discovery":"HRQ-VAE replaces each component of residual quantization with a hyperbolic analogue: encoder and decoder are hyperbolic neural networks in the Poincaré ball, residuals are computed by Möbius subtraction, the quantized reconstruction is built by Möbius addition, and the nearest codebook vector is selected by hyperbolic distance. The training objective remains the standard autoencoder plus codebook loss of RQ-VAE, but optimization runs with a Riemannian optimizer. The paper's claim is that this geometric change gives the model an inductive bias toward hierarchical branching, so the resulting multitokens are more semantically organized and more useful to a downstream transformer, even when no hierarchy label is available during training.","pith_inferences":["An untested extension suggested by the paper's structure is per-dataset curvature selection: if the Poincaré curvature $c$ is tuned, gains on deeper hierarchies could exceed the reported 20%.","The same hyperbolic residual mechanism could plausibly transfer to image and audio RQ-VAEs, which the paper explicitly leaves out of scope.","If HRQ's radial token structure is interpretable, shared prefix tokens should align with shared ancestors; counting token sharing across known hyponym pairs would test that directly.","A non-hierarchical control dataset would clarify whether the geometry helps specifically through hierarchies or through a generic regularization effect."],"forward_implications":["HRQ multitokens improve Recall@10 for hypernym generation over RQ multitokens by up to about 20%, across the tested codebook sizes, token lengths, and embedding dimensions.","On hierarchy discovery, HRQ-VAE multitokens beat RQ-VAE multitokens on next-item prediction metrics without any hierarchy supervision, on both product and movie datasets.","The downstream transformer architecture and training are identical for both token types in the hierarchy modeling experiments, so the performance gap is attributed to the tokens themselves.","HRQ produces a latent space with lower norm variability than RQ, which the paper interprets as a more tree-like spanning structure for quantization.","Because the training objective is unchanged, HRQ can be adopted wherever RQ-VAE is used, provided the data have latent hierarchies."],"supporting_citations":[{"why":"Grounds the core premise: tree-like structures embed into hyperbolic space with low distortion, motivating the geometric inductive bias.","marker":"[22]"},{"why":"Supplies the noun taxonomy evaluation setup and the evidence that hyperbolic embeddings beat Euclidean ones on hierarchy modeling.","marker":"[42]"},{"why":"Provides the Poincaré ball definitions and hyperbolic neural network layers that HRQ-VAE's encoder and decoder are built from.","marker":"[16]"},{"why":"Defines residual quantization in an autoencoder (RQ-VAE), the base method HRQ modifies.","marker":"[32]"},{"why":"Introduces the residual-quantization multitoken process in audio, the structure HRQ makes hyperbolic.","marker":"[60]"},{"why":"The closest prior hyperbolic vector-quantization method; HRQ differs by quantizing residuals and using hyperbolic distance instead of hyperbolic multinomial regression.","marker":"[20]"},{"why":"Provides the generative-retrieval recommender setup used to evaluate whether HRQ multitokens help a downstream transformer.","marker":"[46]"},{"why":"The fixed pretrained text encoder that maps item descriptions to dense embeddings in the hierarchy-discovery experiments.","marker":"[50]"},{"why":"Supplies the product review datasets with item descriptions used in the hierarchy-discovery comparison.","marker":"[37]"},{"why":"Supplies the movie rating dataset for the second hierarchy-discovery comparison.","marker":"[24]"}],"fun_headline_variants":["Hyperbolic geometry adds 20% to residual quantization","Discrete tokens in curved space beat Euclidean by 20%","Curved codebooks: hyperbolic RQ tops Euclidean by 20%","Residual quantization goes hyperbolic for 20% gain","Hyperbolic RQ: 20% better at capturing hierarchies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that hyperbolic geometry is the right inductive bias for the latent hierarchies in these datasets, and that bias is controlled by the curvature $c$, which the paper never tunes or reports, so if the fixed $c$ mismatches the branching structure the claimed improvements could shrink or reverse.","fun_headline_variants_meta":{"raw":{"variants":["Hyperbolic geometry adds 20% to residual quantization","Discrete tokens in curved space beat Euclidean by 20%","Curved codebooks: hyperbolic RQ tops Euclidean by 20%","Residual quantization goes hyperbolic for 20% gain","Hyperbolic RQ: 20% better at capturing hierarchies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000291,"raw_usage":{"total_tokens":1695,"prompt_tokens":934,"completion_tokens":761,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":677}},"tokens_in":550,"tokens_out":761,"duration_ms":6970,"temperature":1.0,"reasoning_tokens":677,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:34:00.551428+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train HRQ and RQ on the noun hypernym task while sweeping the Poincaré curvature $c$ across a wide range (e.g., $0.1$ to $10$) and report Recall@10 for each $c$; if no choice of $c$ beats Euclidean RQ, or if the gain appears only in a narrow window, the reported advantage depends on a favorable fixed constant rather than on hyperbolic geometry itself.","supporting_citations":[{"cited_title":"Hyperbolic groups.Essays in Group Theory, 1987","cited_arxiv_id":null,"evidence_quote":"Grounds the core premise: tree-like structures embed into hyperbolic space with low distortion, motivating the geometric inductive bias."},{"cited_title":"Poincaré embeddings for learning hierarchical represen- tations.Advances in Neural Information Processing Systems, 2017","cited_arxiv_id":null,"evidence_quote":"Supplies the noun taxonomy evaluation setup and the evidence that hyperbolic embeddings beat Euclidean ones on hierarchy modeling."},{"cited_title":"Soundstream: An end-to-end neural audio codec","cited_arxiv_id":null,"evidence_quote":"Introduces the residual-quantization multitoken process in audio, the structure HRQ makes hyperbolic."},{"cited_title":"Tran, Jonah Samost, Maciej Kula, Ed H","cited_arxiv_id":null,"evidence_quote":"Provides the generative-retrieval recommender setup used to evaluate whether HRQ multitokens help a downstream transformer."},{"cited_title":"Mpnet: Masked and permuted pre-training for language understanding.Advances in Neural Information Processing Systems, 2020","cited_arxiv_id":null,"evidence_quote":"The fixed pretrained text encoder that maps item descriptions to dense embeddings in the hierarchy-discovery experiments."},{"cited_title":"Image-based recommendations on styles and substitutes","cited_arxiv_id":null,"evidence_quote":"Supplies the product review datasets with item descriptions used in the hierarchy-discovery comparison."},{"cited_title":"The movielens datasets: History and context.ACM Transactions on Interactive Intelligent Systems (TIIS), (4), 2015","cited_arxiv_id":null,"evidence_quote":"Supplies the movie rating dataset for the second hierarchy-discovery comparison."}],"review_version":1}