{"id":"e90102a2-7931-4200-9581-896fb4ea99ad","arxiv_id":"2507.14426","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CRAFT combines ConceptNet or LLM-derived affordance priors with CLIP visual similarity in an iterative energy-based re-ranking loop, achieving the best non-oracle accuracy on a functional affordance grounding benchmark.","lead":"CRAFT is a neuro-symbolic system that decides which object in an image supports an action like 'cut' by combining commonsense priors from ConceptNet or an LLM with CLIP image-text similarity. The paper reports that adding iterative energy-based re-ranking improves accuracy over prior-only baselines and shows reasoning traces for interpretability.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The best-non-oracle accuracy claim rests on a 1.13-point top-1 gain that may be within sampling noise, and the iterative reweighting that supposedly produces it is never ablated or swept over lambda.","rationale":"The reader's CONDITIONAL verdict is essentially right, but the condition should be anchored more sharply. For the abstract claim 'CRAFT enhances accuracy' to hold, the comparison that matters is CRAFT+GPT4o versus GPT-4o. The reported top-1 gap is only 1.13 points (46.43% vs 45.30%). Over 5,000 randomized episodes, an independent-sample normal approximation puts the standard error of that difference near 1.0 point, so the gap is about one standard error; without paired per-verb statistics, multiple seeds, or bootstrap confidence intervals, the 'best non-oracle' conclusion is not established. This is load-bearing because it is the paper's headline quantitative claim, and the qualitative interpretability argument cannot carry it alone. I partially agree with the reader that the iterative update is under-justified; that is the second half of the same problem. The update is the novel mechanism, yet the paper never reports a lambda sweep or a no-iteration (lambda=0) ablation, so even if the gap is real, we cannot attribute it to the reasoning loop. That said, the paper does use a public dataset, reports oracle upper bounds, and shows directionally consistent gains across three priors (ConceptNet, Gemini, GPT-4o), which is genuine supporting evidence. The right outcome is not rejection: the method is plausible and the fix (confidence intervals, seeds, ablation) is straightforward. The reader's conditional acceptance should stand, with these conditions made explicit.","tokens_in":5867,"tokens_out":9359,"duration_ms":116813,"concrete_test":"Run a seeded evaluation of CRAFT+GPT4o with lambda in {0, 0.5, 1, 2, 5} over 100 episodes per verb, 50 verbs, and 10 episode-seeds, reporting paired bootstrap 95% confidence intervals for the top-1/MRR/nDCG differences versus GPT-4o. If the lambda=0 condition matches the reported 46.43% top-1, or if the confidence interval for the CRAFT+GPT4o versus GPT-4o difference includes zero, the central accuracy and iterative-reasoning claims are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CRAFT's central empirical claim is that it 'enhances accuracy' over prior-only baselines. The headline evidence is CRAFT+GPT4o at 46.43% top-1 versus GPT-4o at 45.30% (Table 1). The paper reports 100 randomized episodes per verb across 50 verbs, so n is on the order of 5,000 episodes, but no seeds, confidence intervals, or per-verb paired statistics are given. Under an independence approximation, the standard error of this 1.13-point difference is about 1.0 percentage point, putting the gain near one standard error; the paper provides no basis to rule out sampling noise. This matters because the abstract's 'enhances accuracy' claim rests on exactly this comparison. Additionally, the proposed iterative update phi^{(t+1)}(o,v) proportional to phi^{(t)}(o,v) * exp(lambda * cos_sim(o, x_t)) in Section 2 is the stated novelty, yet the paper never reports a lambda sensitivity sweep or a no-iteration / lambda=0 ablation. Without that, even a real gain cannot be attributed to the energy-based reasoning loop rather than to a single static fusion of prior and CLIP similarity. The interpretability claim is supported only by qualitative traces, so the quantitative comparison is the load-bearing part of the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CRAFT, a neuro-symbolic framework for functional affordance grounding. Given a verb query and a set of unlabeled candidate images, CRAFT builds an affordance graph from ConceptNet or LLM-generated candidates, scores image-label pairs by CLIP cosine similarity, and iteratively updates prior scores with an exponential reweighting rule. The framework is evaluated on the Nguyen et al. benchmark in single- and multi-label settings, reporting top-1 accuracy, MRR, and nDCG. The authors claim that CRAFT+GPT-4o achieves the best non-oracle accuracy and that the method improves interpretability through traceable reasoning paths.","tokens_in":6154,"tokens_out":4902,"duration_ms":55472,"significance":"The problem is well motivated, and the proposed integration of structured commonsense priors with CLIP similarity is simple and plausible. If the accuracy gains are statistically reliable, the framework would be a useful contribution to label-free affordance grounding, with the notable strength of producing inspectable reasoning traces on an external benchmark. However, the current evidence does not yet establish the central claims: the headline single-label gain is small and unaccompanied by significance testing, the iterative reweighting loop is not ablated, and the interpretability claim is supported only by qualitative examples.","major_comments":[{"comment":"The headline single-label improvement of CRAFT+GPT-4o (46.43%) over GPT-4o (45.30%) is 1.13 percentage points across 5,000 episodes (50 verbs × 100 episodes). Under an independence approximation, the standard error of this difference is about 1.0 percentage point, so the gain is within sampling noise. The paper reports no seeds, confidence intervals, per-verb paired tests, or error bars; since the abstract's claim of enhanced accuracy rests on this comparison, the authors should provide significance testing or at least per-verb variance statistics.","section":"Section 3, Table 1"},{"comment":"The update rule φ^{(t+1)}(o,v) ∝ φ^{(t)}(o,v) · exp(λ·s(o,x_t)) introduces a free parameter λ that controls the influence of visual feedback, yet the paper reports no λ value, no sensitivity sweep, and no λ=0 ablation. Without such an ablation, the observed gains cannot be attributed to the iterative energy-based reasoning loop rather than to a single static fusion of the prior and CLIP similarity.","section":"Section 2, Iterative Reasoning and Re-ranking"},{"comment":"The statement that 'CRAFT+Gemini and CRAFT+ConceptNet also show consistent gains over their prior-only counterparts' is not supported by Table 1: for CRAFT+Gemini, multi-label MRR and nDCG are 56.50% and 59.30%, which are below Gemini alone (57.50% and 59.70%). In addition, no ConceptNet prior-only baseline is reported, so the claimed improvement of CRAFT+ConceptNet over its prior cannot be verified. The authors should correct this inconsistency and add the missing baseline.","section":"Section 3, Table 1 and text"},{"comment":"The claim of improved interpretability is supported only by qualitative traces of a few correct and incorrect reasoning paths. There is no comparison with the interpretability of baseline models, no quantitative measure such as a human evaluation or faithfulness metric, and no user study. The second half of the central claim—'improving interpretability'—is therefore currently unsubstantiated beyond anecdotal illustration.","section":"Appendix A"}],"minor_comments":[{"comment":"The table header contains a stray space in 'F unctional Affordance Performance'; please fix the typo.","section":"Table 1"},{"comment":"The ALGO single-label accuracy is given as 42.52% in Table 1 but as 42.46% in the text; these should be reconciled.","section":"Section 3"},{"comment":"The exact prompts used for GPT-4o and Gemini are not provided, and the convergence criterion and maximum number of iterations for the reweighting loop are not specified; these details are needed for reproducibility.","section":"Section 2, Implementation Details"},{"comment":"The distractor experiment is described in the text, but the figure caption should state the axis labels and the precise definition of 'number of distractors' (5 to 20 candidates per episode).","section":"Figure 2"},{"comment":"The baseline 'Afford-CLIP' is attributed to Radford et al. (2021), but that reference describes CLIP, not an affordance-specific model; please clarify whether Afford-CLIP is a new model introduced here or an existing one with a proper citation.","section":"Section 3, Baselines"}],"recommendation":"major_revision","confidential_remarks":"The paper is not circular in the narrow sense: the evaluation uses an external benchmark, and the method does not exploit ground-truth labels. My main concern is that the empirical evidence is thinner than the abstract suggests, and one of the paper's own 'consistent gains' statements is contradicted by Table 1 for CRAFT+Gemini in the multi-label setting. I would encourage the editor to request the missing significance tests, the λ ablation, and the ConceptNet prior-only baseline before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this one. First, the core idea—use ConceptNet or an LLM to generate candidate object concepts for a verb, score images with CLIP, then iteratively reweight the candidates by exponentiating the similarity to the current top image—is a reasonable neuro-symbolic integration and, as far as I can tell, not in the cited prior work. Second, the empirical case is weaker than the abstract suggests. The headline 'enhances accuracy' rests on a 1.13-point top-1 improvement over GPT-4o (46.43 vs 45.30). With roughly 5,000 episodes, that difference sits around one standard error. No confidence intervals, no significance test, no per-verb paired stats.\n\nWhat deserves credit: the evaluation uses an external benchmark (Nguyen et al., 2020), so there's no circularity. The gains are consistent across three different priors—ConceptNet, Gemini, GPT-4o—which makes the method look robust even if each individual delta is small. The multi-label nDCG improvement (67.0 vs 60.4) is more substantive. And the appendix's reasoning traces and ego-graphs are a genuine interpretability contribution, not just a figure.\n\nThe soft spots, in order of importance. (1) The iterative reweighting is the novelty, but the paper gives no ablation without it, and no sweep over the temperature λ. So the observed gains can't be attributed to the energy-based loop; a static fusion of prior and CLIP similarity might do just as well. (2) The contribution list says 'we introduce a rigorous benchmark,' but the experiments run on Nguyen et al.'s dataset. That's a factual inconsistency that has to be fixed. (3) Missing error bars, which matters given the small headline delta. (4) Minor: no code release mentioned, and the text and Table 1 disagree on ALGO's accuracy (42.46 vs 42.52). These are all fixable in revision.\n\nOverall, the method is plausible, clearly written, and the interpretability angle is a plus. The weaknesses are in the evidence, not the logic. I'd send it to review but require the ablations and error bars before accepting. It's a solid workshop-level paper that could become a useful reference if the authors tighten the empirical claims.","headline":"Plausible neuro-symbolic assembly for affordance grounding, but the headline top-1 gain is near sampling noise and the iterative reweighting is never ablated.","tokens_in":6658,"tokens_out":3773,"would_cite":false,"duration_ms":44011,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CRAFT is a neuro-symbolic framework that grounds functional affordances by fusing commonsense priors with visual evidence through an energy-based iterative loop.","keywords":["functional affordance","neuro-symbolic reasoning","affordance grounding","vision-language models","energy-based reasoning","ConceptNet","CLIP","interpretability"],"falsifier":"Run CRAFT on the same benchmark with $\\lambda$ set to values from 0 to 10 and see whether the reported gains persist; if accuracy peaks at one value and collapses nearby, or if setting $\\lambda=0$ (no iterative update) already matches the final numbers, the iterative reweighting is not the source of the improvement.","tokens_in":5680,"feed_emoji":"🧠","tokens_out":6101,"duration_ms":573158,"temperature":0.7,"pith_summary":"CRAFT is a neuro-symbolic framework that identifies which objects in a scene afford a given action such as “cut”, without relying on object labels. It combines structured commonsense priors from ConceptNet or large language models with visual evidence from CLIP, then refines the candidate list through an energy-based iterative re-ranking loop. The paper argues that this integration makes affordance grounding more accurate and more interpretable than prior-only or purely learned baselines. Its headline result is that CRAFT with GPT-4o priors reaches 46.43% top-1 accuracy and 60.80% MRR / 67.00% nDCG, the best non-oracle performance on the benchmark.","feed_headline":"CRAFT plus GPT-4o sets best non-oracle affordance score","feed_subtitle":"Combining ConceptNet and LLM priors with CLIP evidence and iterative re-ranking finds objects that afford a verb.","key_machinery":"The central object is an affordance graph $G_v$ built from ConceptNet by traversing relations such as UsedFor and CapableOf from a verb to object candidates, with each object assigned a prior score $\\phi(o,v)$. The grounding energy is $E(v,x_i) = -\\max_{o \\in G_v}[\\phi(o,v) \\cdot s(o,x_i)]$, where $s(o,x_i)=\\cos(g(o), f(x_i))$ is the cosine similarity between a CLIP text embedding of the object label and a CLIP image embedding of the candidate image. The iterative step updates priors as $\\phi^{(t+1)}(o,v) \\propto \\phi^{(t)}(o,v) \\cdot \\exp(\\lambda \\cdot s(o,x_t))$, with $x_t$ the current top-ranked image, so visual evidence gradually sharpens the symbolic prior.","core_discovery":"The paper's central claim is that functional affordance grounding—matching a verb like “cut” to the image that supports it—is best treated as neuro-symbolic inference rather than pure visual similarity. On a benchmark of verb-object affordance labels, combining symbolic candidates with CLIP image-text similarity and an iterative reweighting step consistently beats each prior alone: CRAFT with GPT-4o is the best non-oracle method in single-label accuracy and multi-label ranking, and CRAFT also improves over its ConceptNet and Gemini priors. The authors further claim the framework is interpretable because each prediction carries a traceable reasoning path through the affordance graph, allowing post-hoc inspection of both correct and erroneous decisions.","pith_inferences":["The iterative reweighting rule resembles a temperature-scaled ranking feedback; if the adjustment strength $\\lambda$ is not tuned per verb, hard verbs with diffuse ConceptNet graphs may over-amplify the first visual match. Varying $\\lambda$ and measuring accuracy would test this directly.","The energy formulation could be extended to region proposals or bounding boxes, allowing the same reasoning loop to pick out a specific object within an image rather than a whole image.","A natural extension is to use the affordance graph's ego-graph structure as a prior over sequences of tool use, which would connect CRAFT to action planning and robot manipulation.","The benchmark uses validation images from a 216-category ImageNet-derived dataset; moving to cluttered real-world scenes with distractors that share visual similarity with the affordant object would stress the method more than the current episode setup."],"forward_implications":["If CRAFT's central claim is right, affordance grounding can be improved without retraining or labels by dressing up symbolic priors with one CLIP pass and a short iterative loop.","The same pipeline should transfer to any verb for which ConceptNet or an LLM can propose object candidates, including verbs unseen during training.","Because the final ranking depends on the prior, CRAFT's gains are largest when the prior is strong (GPT-4o) but still positive when the prior is noisy (ConceptNet, Gemini).","The per-prediction reasoning traces give an operator a concrete path to inspect when the model fails, which is what the authors mean by interpretability."],"supporting_citations":[{"why":"Supplies ConceptNet as the source of structured commonsense relations used to build the affordance graph.","marker":"(Liu and Singh, 2004)"},{"why":"Provides ConceptNet 5.5 and NumberBatch, used for ranking candidate object concepts by similarity to the query verb.","marker":"(Speer et al., 2017)"},{"why":"CLIP supplies the visual and text encoders for cosine similarity and also defines the Afford-CLIP baseline.","marker":"(Radford et al., 2021)"},{"why":"Provides the verb-object affordance dataset, the episode construction, and the supervised ResNet-RNN baseline.","marker":"(Nguyen et al., 2020)"},{"why":"Gemini is both a prior-only baseline and the LLM prior source for the CRAFT+Gemini variant.","marker":"(Anil et al., 2023)"},{"why":"GPT-4o is both a prior-only baseline and the LLM prior source for the CRAFT+GPT-4o variant.","marker":"(Hurst et al., 2024)"},{"why":"ALGO serves as a prior-only baseline that relies on language-based or symbolic affordance candidates.","marker":"(Kundu et al., 2024)"}],"fun_headline_variants":["CRAFT with GPT-4o leads non-oracle affordance grounding","Neuro-symbolic CRAFT beats pure CLIP for affordance grounding","CRAFT: interpretable affordance grounding through graph reasoning","Best non-oracle affordance score via CRAFT plus GPT-4o","CRAFT merges concepts and vision for transparent affordance grounding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that exponentially boosting the current top-ranked image's matched concepts, with a fixed adjustment strength $\\lambda$, drives the ranking toward the true affordance; the paper does not justify the choice of $\\lambda$ or report how sensitive the gains are to it.","fun_headline_variants_meta":{"raw":{"variants":["CRAFT with GPT-4o leads non-oracle affordance grounding","Neuro-symbolic CRAFT beats pure CLIP for affordance grounding","CRAFT: interpretable affordance grounding through graph reasoning","Best non-oracle affordance score via CRAFT plus GPT-4o","CRAFT merges concepts and vision for transparent affordance grounding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1214,"prompt_tokens":780,"completion_tokens":434,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":396,"completion_tokens_details":{"reasoning_tokens":342}},"tokens_in":396,"tokens_out":434,"duration_ms":4883,"temperature":1.0,"reasoning_tokens":342,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:56:03.945425+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CRAFT on the same benchmark with $\\lambda$ set to values from 0 to 10 and see whether the reported gains persist; if accuracy peaks at one value and collapses nearby, or if setting $\\lambda=0$ (no iterative update) already matches the final numbers, the iterative reweighting is not the source of the improvement.","supporting_citations":[{"cited_title":"Conceptnet—a practical commonsense reasoning tool-kit","cited_arxiv_id":null,"evidence_quote":"Supplies ConceptNet as the source of structured commonsense relations used to build the affordance graph."},{"cited_title":"Conceptnet 5.5: An open multilingual graph of general knowledge","cited_arxiv_id":null,"evidence_quote":"Provides ConceptNet 5.5 and NumberBatch, used for ranking candidate object concepts by similarity to the query verb."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"CLIP supplies the visual and text encoders for cosine similarity and also defines the Afford-CLIP baseline."},{"cited_title":"Robot object retrieval with contextual natural language queries","cited_arxiv_id":null,"evidence_quote":"Provides the verb-object affordance dataset, the episode construction, and the supervised ResNet-RNN baseline."},{"cited_title":"Discovering novel actions from open world egocentric videos with object-grounded visual commonsense reasoning","cited_arxiv_id":null,"evidence_quote":"ALGO serves as a prior-only baseline that relies on language-based or symbolic affordance candidates."}],"review_version":1}