{"id":"1985f12f-6a15-44b1-809f-c9695962b39a","arxiv_id":"2601.23156","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An observation-only pipeline that combines optimal-transport skill segmentation (ASOT) with Sequitur grammar induction produces reusable skill hierarchies that accelerate reinforcement learning in Craftax and Minecraft.","lead":"The paper presents HiSD, a pipeline that segments unlabeled game-play videos into skill labels, then compresses the labels into a hierarchy using grammar rules — no action labels or rewards needed. The authors report better segmentation and hierarchy quality than two baselines in Craftax and Minecraft, plus faster downstream reinforcement learning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hierarchy-quality claim rests on a self-consistency check: the 'ground truth' hierarchy is produced by the paper's own modified Sequitur on clean labels, so 'semantically meaningful' is never validated externally; Table 2's Minecraft-All numbers also contradict 'consistently'.","rationale":"The reader's weakest assumption already identifies the circular hierarchy ground truth, and that is also the most load-bearing issue for the paper's headline claim. If Sequitur's compression is not the true task decomposition, then all hierarchy metrics (unique trees, depth, size, branching) and the 'perfect recovery' statements only measure how well ASOT labels reproduce the grammar's own deterministic output. In contrast, the RL results in Figures 5-6 are genuine external evidence of utility, and the segmentation metrics in Table 1 are standard comparisons against inventory-delta labels, so the paper is not without support. The right outcome is the same CONDITIONAL verdict: the central semantic claim needs an external benchmark before it can be accepted as stated. I would not escalate to REJECT because the method is clearly described, the pipeline is deterministic and reproducible in principle, and the downstream RL transfer result provides a positive signal that is independent of the hierarchy metrics. The proposed test is feasible and would either resolve the concern or force the authors to soften the claim from 'semantically meaningful' to 'self-consistent'.","tokens_in":23240,"tokens_out":8088,"duration_ms":96966,"concrete_test":"Build an independent ground-truth hierarchy for Craftax Stone Pickaxe (Static and Random) from the known task structure: collect wood → craft workbench → craft wooden pickaxe → mine stone → craft stone pickaxe (the prerequisite graph in Table 5 and Appendix A.1.1). Run HiSD with ≥10 seeds. For each discovered internal node, compute precision/recall/F1 of its skill-span against the corresponding subgoal interval defined by inventory-delta boundaries; compare against a control grammar induced on within-episode shuffled labels. If HiSD's F1 is near chance or no better than the shuffle control, the 'semantically meaningful hierarchies' claim is unsupported; if F1 is high and passes the control, the concern is resolved. Repeat on Minecraft Mapped with human-annotated subgoal boundaries.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §5.4 the ground-truth structure for every hierarchy metric is defined as 'generated by running our modified Sequitur on clean ground-truth labels.' HiSD's output is therefore compared against the same deterministic grammar applied to the same segmentation labels. 'Perfect recovery' (App. D.3, Fig. 12) is a self-consistency check of the ASOT labels under that compression, not evidence that the induced non-terminals correspond to true task subgoals. This matters because the abstract's central claim is that HiSD produces 'more structured and semantically meaningful hierarchies'. No independent semantic target (human-annotated subgoal decomposition, planner-generated HTN, or externally validated benchmark) is used anywhere, and the paper's own Minecraft-All row of Table 2 shows the consistency claim fails precisely where the environment is noisy (HiSD: 500 unique trees, mean size 283.25 vs ground truth 293/22.69; §6.2.2 concedes the grammar cannot find a consistent structure there). Without an external target, the hierarchy metrics can only show self-consistency under label noise, not semantic meaningfulness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Hierarchical Skill Discovery (HiSD), a two-stage unsupervised pipeline: ASOT-based temporal action segmentation converts raw observation features into discrete skill labels, then a modified Sequitur grammar induction (with a boundary-token fix) compresses concatenated skill sequences into a hierarchical CFG. The method is evaluated on Craftax and Minecraft (VPT) datasets, comparing segmentation quality (Table 1) and tree-structure metrics (Table 2) against CompILE and OMPN, and showing a downstream RL proof-of-concept in which HiSD-derived hierarchies improve PPO sample efficiency. The manuscript claims that HiSD 'consistently produces more structured and semantically meaningful hierarchies' than baselines and that the discovered hierarchies accelerate and stabilise downstream RL.","tokens_in":23419,"tokens_out":7361,"duration_ms":74452,"significance":"If validated, HiSD would provide one of the first fully observation-only pipelines for multi-level skill discovery in high-dimensional pixel domains, with the practical advantage of not needing actions, rewards, or interaction during discovery. The paper deserves credit for decoupling structure discovery from policy execution, evaluating on the full MineRL action space, and including a downstream RL demonstration; the segmentation results are anchored to externally derived inventory-delta labels, which is meaningful external grounding. However, the central hierarchy-quality claims currently lack external semantic validation, and the quantitative results in the noisiest environment contradict the 'consistently' qualifier. The core method is plausible but the current claims outrun the evidence.","major_comments":[{"comment":"The 'ground truth structure' for hierarchy metrics is defined as 'generated by running our modified Sequitur on clean ground-truth labels' (§5.4). Consequently, every hierarchy-quality claim—unique tree counts, depth, size, branching, and the 'perfectly recovers' statement in App. D.3 (Fig. 12)—is a comparison of the pipeline's output against the same deterministic grammar applied to the same (clean) labels. This is a self-consistency check of the ASOT stage under label noise, not evidence that the induced non-terminals correspond to true subgoals. The abstract's central claim of 'semantically meaningful hierarchies' therefore rests on an unvalidated premise: that Sequitur's compression of the clean skill sequence is the correct semantic task decomposition. No independent semantic target (human-annotated subgoal structure, planner-generated HTN, or external benchmark) is used. Please add","section":"§5.4, Table 2, App. D.3"},{"comment":"Table 2's Minecraft-All row shows HiSD producing 500 unique trees (vs 293 for the ground-truth), mean tree size 283.25 (vs 22.69) and max branching 28.73 (vs 2.25). This directly contradicts the abstract's claim that HiSD 'consistently produces more structured and semantically meaningful hierarchies' and the §6.1.2 claim that HiSD 'matches or closely approximates the ground truth.' The text in §6.2.2 concedes that noise 'prevent[s] our deterministic grammar from discovering a consistent underlying structure.' The Mapped row is also far from the ground truth (500 vs 151 unique trees, size 83.85 vs 21.36). Thus, the hierarchy-quality advantage over baselines is not consistent across the evaluated settings. A revision should either restrict the claim to the simpler/low-noise tasks or provide evidence that the inflated trees are still semantically meaningful despite their size.","section":"Table 2, §6.2.2"},{"comment":"The downstream RL comparison is not fully controlled. In App. C, OMPN hierarchies are not given learned low-level skill policies: 'we do not learn a BC model or a PU model for the OMPN hierarchies, only for the segmented skills OMPN produces.' OMPN leaf nodes are sequences of primitive actions, whereas HiSD's options are instantiated with BC-trained intra-option policies and PU-learned initiation/termination gates. Figure 5 therefore compares HiSD options (representation + learned controllers) with OMPN hierarchies (representation only). The observed RL improvement may be due to the added low-level controllers rather than the hierarchy structure. Please add an ablation where all methods share the same option-grounding mechanism (e.g., BC on the relevant segment labels) or explicitly state that the RL result is a combined proof of concept, not a hierarchy-quality comparison.","section":"§7, Appendix C"}],"minor_comments":[{"comment":"F1 and mIoU under the 'Full' (global Hungarian) matching scheme need precise definitions: how are segment-level F1 and mIoU computed after global alignment? Please specify the matching and scoring procedure.","section":"§5.3"},{"comment":"Qualitative hierarchy analysis uses the single best-performing run per framework (selected via mIoU), which is a form of cherry-picking; the visual 'perfect recovery' examples in App. D.3 may not be representative. Please report how often the best-run hierarchy is typical, or provide aggregate visualizations.","section":"§6, first paragraph"},{"comment":"Typo: 'while HiSD still finds no unique trees' should read '500 unique trees' (Table 2 shows HiSD Mapped: 500 unique trees).","section":"§6.2.2"},{"comment":"The reported n-frames values for Craftax (60, 135, 110, 205) exceed the average episode lengths in Table 4 (14.9–33.0 steps). Please clarify what n-frames controls and how these values are applied to trajectories shorter than n-frames.","section":"Table 19"},{"comment":"In the RL evaluation, why is CompILE omitted from Figures 5 and 6 despite being a baseline in Table 1? Please state the reason (e.g., memory or training issues) for completeness.","section":"Appendix C, §7"}],"recommendation":"major_revision","confidential_remarks":"The paper combines existing components (ASOT + Sequitur) with a useful boundary-token fix; the conceptual novelty is modest, and the main contribution is the empirical evaluation. The self-referential hierarchy ground truth is the most substantive issue; without an external semantic target, the 'semantically meaningful' language in the abstract is not supported. The editor may wish to require an external hierarchy benchmark or a re-framed abstract before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the segmentation results are the real contribution. The paper's inventory-delta ground truth is an external, programmatic label set, and HiSD beats CompILE and OMPN on most mIoU/F1 numbers even when the baselines are given the ground-truth sub-task order. The downstream RL experiment (Fig. 5) is a genuine held-out test: the HiSD hierarchy learns faster than flat skills, and that evidence is independent of the grammar-quality metrics.\n\nWhat's new is the combination: ASOT for observation-only temporal segmentation, then boundary-token Sequitur for compression into a grammar. That is a simple, clearly described pipeline, and the Minecraft experiments on the full unmodified game with VPT data are nontrivial. The paper is also honest about its limitations: K sensitivity, unreliable termination PU models in Minecraft, pre-extracted features.\n\nThe soft spot is the hierarchy evaluation. Section 5.4 defines the ground-truth hierarchy as 'generated by running our modified Sequitur on clean ground-truth labels.' So 'perfect recovery' in Fig. 12 means the algorithm agrees with itself on clean vs noisy labels. It does not show the non-terminals correspond to any real subgoals. The abstract's 'semantically meaningful hierarchies' is therefore not supported by the hierarchy metrics; only the segmentation metrics carry semantic weight. On top of that, Table 2's Minecraft-All row directly contradicts 'consistently': HiSD produces 500 unique trees vs truth's 293, and mean size 283.25 vs 22.69. The paper concedes this in §6.2.2 but the abstract still says 'consistently.'\n\nTwo smaller but real issues: baselines in Minecraft got manually set hyperparameters and a 2385-action discretization, while HiSD got an Optuna sweep; Table 2 reports no seed variance. And the paper cites Lange & Faisal 2019 (grammar-based skill discovery from behavior sequences) but never compares to it.\n\nNone of this sinks the central idea. The pipeline is reproducible, the segmentation evaluation is externally grounded, and the held-out RL result is genuinely supportive. The fixes are clear: externalize the hierarchy ground truth, report variance, match tuning effort, and soften the consistency claim.\n\nI would send this to peer review. The reviewers should ask for the hierarchy evaluation to be redone or reframed, but the combination is worth engaging with.","headline":"The segmentation results are the real contribution; the hierarchy metrics are a self-consistency check, so the 'semantically meaningful' claims should be read with caution.","tokens_in":24084,"tokens_out":2589,"would_cite":true,"duration_ms":28964,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An observation-only pipeline — temporal segmentation plus grammar induction — can extract reusable skills and compositional hierarchies from unlabelled video, beating action-supervised baselines and accelerating downstream RL.","keywords":["hierarchical skill discovery","skill segmentation","temporal action segmentation","grammar induction","Sequitur","unsupervised learning","reinforcement learning","options"],"falsifier":"Check whether HiSD's internal grammar nodes align with independently defined subgoal points — the first frame after each crafted item appears, or human-annotated milestone frames — better than chance-level boundaries. If the non-terminals do not coincide with externally anchored subgoals, the 'semantically meaningful hierarchy' claim fails. A second check: re-run the downstream RL experiment with behaviour-cloning-from-observation instead of action-labelled cloning; if the hierarchy's advantage disappears, the observation-only claim holds only for discovery, not learning.","tokens_in":22953,"feed_emoji":"🧩","tokens_out":13908,"duration_ms":134308,"temperature":0.7,"pith_summary":"This paper claims that hierarchical behaviour structure can be extracted from raw observations alone — no action labels, rewards, or human annotation. The proposed method, HiSD, segments pixel-level trajectories into discrete skills by temporally coherent optimal-transport clustering, collapses contiguous same-skill runs into symbols, and then runs a linear-time grammar-induction algorithm over all episodes to compress recurring subroutines into a multi-level hierarchy. The authors report that HiSD's hierarchies are more consistent and reusable than those of the baselines, even though those baselines were given the true sub-task ordering, and that the recovered hierarchy — not just the flat skills — supplies the inductive bias that lets a reinforcement-learning agent solve a sparse-reward crafting task in roughly 30,000 steps where primitive-action policies fail. A careful reader would care because the method's inputs are just observations, so if the claim holds it points a path to learning reusable task structure from the large amounts of unlabelled video that already exist.","feed_headline":"Learn skill hierarchies from unlabeled video alone","feed_subtitle":"A two-stage grammar pipeline beats action-supervised baselines in Craftax and Minecraft and speeds up downstream RL.","key_machinery":"The central mechanism is the pairing of ASOT — unsupervised temporal action segmentation posed as unbalanced optimal transport with a fused Gromov-Wasserstein temporal-regularity term, which labels frames while preventing label flicker — with a modified Sequitur grammar induction. Sequitur greedily turns repeated adjacent-symbol pairs into non-terminal rules while maintaining two invariants (digram uniqueness and rule utility), and its derivation tree is the hierarchy. The modification that carries the argument is the boundary token: episodes are concatenated with a unique marker that is forbidden from entering any production rule, so subroutines never span episode boundaries, and a non-term","core_discovery":"HiSD claims a two-stage, fully unsupervised pipeline can recover skills and their composition from unlabelled trajectories. Temporally regularised optimal transport labels each frame as one of K visually coherent skills; same-skill runs become atomic symbols. The linear-time grammar inducer Sequitur compresses repeated symbol pairs into non-terminals, so the induced grammar's leaves are skills and its internal nodes are recurring subroutines. The paper claims this grammar tracks the intended decomposition in controlled tasks, beats action-supervised baselines on most segmentation metrics (Craftax, Minecraft), and that the hierarchy, not flat skills, gives a masked-PPO agent the inductive bia","pith_inferences":["Editorial inference: the paper's hierarchy evaluation is anchored to a ground truth it defines as its own modified Sequitur run on clean labels (§5.4; §D.3 reports 'perfect recovery' of that object). The hierarchy metrics therefore establish self-consistency under label noise; validating the 'semantically meaningful' claim requires an independent anchor, such as human- or reward-defined subgoal st","Editorial inference: the Table 2 results in the noisiest Minecraft setting — one unique tree per episode and a 283-node average tree against a 22.7-node truth — suggest the pipeline's validity is strongest where segmentation is clean. A testable extension is to make the grammar stage probabilistic or to feed it segmentation confidence, which might resist cascading symbol errors.","Editorial inference: the paper's own downstream experiment still assumes action labels for behaviour cloning (§7). A fuller test of the observation-only thesis is to close that loop with behaviour-cloning-from-observation, so that both discovery and control run without actions.","Editorial inference: the boundary-token modification is a transferable recipe — keep grammar induction from merging across sequence boundaries — that should generalize to other concatenated-sequence compression tasks, such as extracting recurring routines from logged telemetry or documents."],"forward_implications":["Unlabelled demonstration video becomes a usable input for structure discovery: skills and hierarchies can be mined from observations alone, skipping the expensive step of collecting action labels or rewards.","Hierarchy matters for learning, not just description: in both evaluation domains the hierarchical option agent outperformed the flat skills-only agent, implying compositional structure helps temporal credit assignment.","The discovered skills are object-centric and reusable: visually distinct approaches to the same resource are grouped into a single skill, which is what makes cross-episode reuse and compact grammars possible.","The pipeline is cheap to run: the paper reports training and inference on consumer-grade GPUs with 6–10 GB of VRAM, in contrast to the roughly 24 GB the baselines required, lowering the barrier for scaling structure discovery."],"fun_headline_variants":["Unsupervised grammar learns skill hierarchies from raw video","Grammar induction from unlabeled video beats action-supervised baselines","Segment frames, induce grammar: unsupervised skill hierarchies","Unsupervised method discovers reusable skills from unlabeled video","No labels needed: grammar builds skill hierarchy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that Sequitur's compression of clean skill labels is the correct task decomposition: the paper's 'ground truth' hierarchy is its own modified Sequitur run on clean labels, so if that grammar does not encode the true sub-goal structure, the hierarchy results demonstrate self-consistency, not semantic correctness.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised grammar learns skill hierarchies from raw video","Grammar induction from unlabeled video beats action-supervised baselines","Segment frames, induce grammar: unsupervised skill hierarchies","Unsupervised method discovers reusable skills from unlabeled video","No labels needed: grammar builds skill hierarchy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001163,"raw_usage":{"total_tokens":4603,"prompt_tokens":649,"completion_tokens":3954,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":393,"completion_tokens_details":{"reasoning_tokens":3878}},"tokens_in":393,"tokens_out":3954,"duration_ms":31016,"temperature":1.0,"reasoning_tokens":3878,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T06:13:08.483130+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Check whether HiSD's internal grammar nodes align with independently defined subgoal points — the first frame after each crafted item appears, or human-annotated milestone frames — better than chance-level boundaries. If the non-terminals do not coincide with externally anchored subgoals, the 'semantically meaningful hierarchy' claim fails. A second check: re-run the downstream RL experiment with behaviour-cloning-from-observation instead of action-labelled cloning; if the hierarchy's advantage disappears, the observation-only claim holds only for discovery, not learning.","supporting_citations":[],"review_version":1}