{"id":"18f76e71-415f-4e40-b058-8a8dc15d7fab","arxiv_id":"2608.01927","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"An LLM-driven partial dependency graph retriever improves repository-level code generation Pass@1 by 25.63% and 59.73% over RAG baselines on CoderEval and DevEval, with a 7.4x speedup over static graph construction.","lead":"This paper presents DyCoder, a retrieval method that uses an LLM to build a partial dependency graph on demand and walk it hop by hop to find context for repository-level code generation. The authors report that it beats existing retrieval-augmented generation baselines on two code generation benchmarks while running an order of magnitude faster than static graph-based methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 60-operation accuracy check in Sec. 6.1 does not establish that LLM dependency expansion reliably preserves the dependency chains DyRetriever needs; a missed callee at any hop can remove the only route to essential context.","rationale":"The reader's weakest_assumption is the same spot I would stress: reliability of LLM expansion. I agree with the conditional verdict. This concern is load-bearing because the entire novelty of DyRetriever over RepoScope/GraphCoder is the on-demand, LLM-constructed partial graph; the entry-point selection and final top-k selection are secondary. The paper's own Section 6.1 limits the evidence to 60 hand-inspected operations and does not provide per-task reachability analysis. A missed call edge is not detectable by the name-matching post-processor, so the method can silently lose a dependency that a static graph would have found. However, I do not regard this as fatal: the DevEval gains over RepoScope are consistent, and the ablation shows multi-hop reasoning contributes, which suggests the mechanism is doing real work. The verification step I propose would settle whether incomplete LLM expansion actually limits correctness. Other issues (maxHop tuning on CoderEval, adapted baselines, reporting of relative improvements) are real but secondary because the DevEval comparison and the 7.4x efficiency claim provide independent support. Final recommendation remains conditional: accept if the expansion recall and oracle-substitution check confirm that graph construction is not a causal bottleneck. Since the reader already reached a conditional verdict, no adjustment is needed.","tokens_in":20485,"tokens_out":7927,"duration_ms":91557,"concrete_test":"Collect all Expand Step invocations from the CoderEval and DevEval runs. For each visited function, build a ground-truth callee/lazy-import set using a static resolver (PyCG + Tree-sitter with manual inspection for dynamic calls). Compute (i) per-expansion recall of the LLM's neighbor set (before post-processing) and (ii) the fraction of tasks in which any function on the gold dependency path to a required dependency is absent from DyRetriever's candidate set. Then rerun DyCoder on a random sample of tasks replacing only the LLM-expansion neighbor set with the static-oracle set, keeping entry-point selection, visit order, final top-k selection, and generation fixed. If oracle substitution raises Pass@1 materially, the 60-case check in Sec. 6.1 is insufficient and the expand step is a causal bottleneck; if Pass@1 is unchanged, the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (that LLM-driven partial dependency graph retrieval can replace static global graph RAG) rests on the Expand Step (Algorithm 1, Line 15): after selecting a function, the LLM must identify all callee and lazy-import dependencies from the function body plus file imports. Post-processing only discards hallucinated names that do not match any repository function; it cannot recover a callee the LLM omitted. Section 6.1 validates this step with 60 manually inspected expand operations and reports 93.22% 'completely accurate' expansion, but this is the wrong unit of analysis for a multi-hop process. If ~7% of expanded functions have at least one missing/wrong edge and a task traverses up to 10 hops, a substantial fraction of tasks will experience at least one faulty expansion; one missed edge can sever the only path from an entry point to an essential dependency. The 60 cases also lack a documented sampling protocol and an error breakdown distinguishing missed edges from fabricated ones—a distinction that matters because the current post-processing handles only fabrication. Reporting that LLM accuracy is comparable to static tools (77.4%–99.2%) does not answer whether the errors occur on load-bearing edges. This is the single most load-bearing concern because the novelty claim is not 'entry points are useful' but 'partial graph expansion via LLM is both effective and efficient'; if expansion recall is poor, the method's graph component is unvalidated even if the full pipeline still benefits from similarity retrieval and entry-point selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DyRetriever, an LLM-driven context-retrieval method for repository-level code generation. Instead of building a static global dependency graph, DyRetriever selects entry-point functions from a repository structure and then performs multi-hop traversal over a partial dependency graph, using the LLM to identify callees (Algorithm 1, Expand Step) and to judge whether a function is useful for the target function. The retrieved functions are combined with similarity-based retrieval results to form DyCoder. The authors evaluate DyCoder on CoderEval and DevEval with three LLMs, reporting relative Pass@1 gains of 25.63% and 59.73% over existing RAG baselines and a 7.4x speedup over the static-graph baseline RepoScope. Ablations show that both the similarity-based component and DyRetriever contribute, and RQ4 shows that DyRetriever can be combined with BM25, RLCoder, and AlignCoder.","tokens_in":20833,"tokens_out":5565,"duration_ms":61190,"significance":"If the findings hold, DyRetriever is a practical contribution: it avoids the construction and maintenance cost of global repository graphs, adapts to different context types without hand-crafted rules, and improves generation accuracy consistently across models and benchmarks. The paper's strengths include the multi-model and multi-benchmark evaluation, the clean ablation design (Table 3), the generalizability experiments (Table 4), and the explicit discussion of limitations in Sections 6.1 and 6.2. The replication package and official repository are also valuable. However, the central mechanism—LLM-based expansion of partial dependency graphs—is validated with only a small manual spot-check of 60 expand operations, and the maximum-hop hyperparameter is tuned on one of the two evaluation benchmarks. These issues do not invalidate the end-to-end results, but they are load-bearing for the paper's specific novelty claim that LLM-driven partial graph expansion can replace static global graph RAG.","major_comments":[{"comment":"The validity of the Expand Step is the load-bearing premise of the paper, but the evidence provided is not commensurate. The manual check covers only 60 expand operations, with no stated sampling protocol (which tasks, models, hops, or repositories), and reports only a binary 'completely accurate' rate of 93.22%. The post-processing described in Section 3.1.2 can discard hallucinated names, but it cannot recover a callee that the LLM omitted; a single missed edge on a multi-hop path can remove the only route to an essential dependency. With maxHop=10, even a small per-expansion miss rate can affect a substantial fraction of tasks. The paper should report an error breakdown distinguishing missed from fabricated edges, per-hop and path-level recall/precision, a documented sampling procedure for the 60 operations, and an analysis of whether errors occur on load-bearing edges (e.g., by compa","section":"Section 6.1 / Algorithm 1, Line 15"},{"comment":"The maximum hop hyperparameter is selected by 'preliminary experiments' on CoderEval with values in [1,20], and CoderEval is also one of the two benchmark datasets used for the headline results. This is effectively tuning on the test set. No sensitivity analysis is reported for DevEval, where the claimed relative improvement is largest (59.73%). The paper should report Pass@1 and token consumption as functions of maxHop on both benchmarks, or on a held-out development split, to demonstrate that the chosen value does not inflate the reported gains.","section":"Section 4.4"},{"comment":"The treatment of the 209 DevEval instances that cannot be executed in the authors' environment is transparent, but the effect on the reported comparison may still be material. 209/1825 is about 11.5% of the benchmark; if these instances are concentrated in particular repositories or have particular dependency structures, the relative improvements and rankings could be biased. The paper should characterize these 209 instances (e.g., reasons for failure, distribution across repositories) and provide a sensitivity analysis that recomputes Pass@1 after excluding them or using an alternative evaluation, to confirm the DevEval results are robust.","section":"Section 4.1"}],"minor_comments":[{"comment":"The 60 expand operations are not tied to any specific LLM, benchmark, or sampling protocol. If they were collected from a particular configuration (e.g., Qwen3-Coder on CoderEval), that should be stated explicitly so readers can judge generalizability. The comparison with static-analysis accuracies (77.4%–99.2%) cites [42] but does not clarify whether the accuracy definitions are directly comparable.","section":"Section 6.1"},{"comment":"The parser used to extract functions from the related-file list is not specified. Please name the tool/version (e.g., tree-sitter, ast) and include it in the replication package, since function extraction is a prerequisite for both the entry-point stage and the similarity-based retrieval.","section":"Section 3.1.1"},{"comment":"The indentation post-processing for generated code is mentioned but not described. Please provide details or examples, or point to the exact implementation in the replication package, so that readers can assess its effect on the reported Pass@1.","section":"Section 3.3"},{"comment":"The 'Total imp.' column mixes a multiplier with token counts (e.g., '1.00×13.09k'), and the footnote formatting is awkward. Please clean up the table and clarify whether the dollar costs are based on local Qwen3-Coder deployment or on API pricing for the other models.","section":"Table 2"},{"comment":"There is a typo in the sentence 'Existing studies have [13, 29] demonstrated'—the word 'have' appears without its complement. Also, the abstract's '7.4x faster' claim is based on a single configuration (CoderEval with Qwen3-Coder-30B); please state this qualification where the speedup is advertised.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"This is a strong empirical paper with a clear practical contribution. The major concerns are all addressable with additional experiments and clearer reporting: strengthen the validation of the dependency-expansion step (Section 6.1), provide hyperparameter sensitivity on both benchmarks (Section 4.4), and analyze the 209-devEval-instance issue (Section 4.1). I do not see a fundamental flaw in the method or the end-to-end evaluation, so revision rather than rejection is appropriate. No concerns about citation patterns or scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a solid, genuinely new retrieval method, and the central empirical claim holds up, but the paper under-tests the one step its entire architecture leans on. DyRetriever replaces the static global dependency graph with an LLM that picks entry points and expands a partial graph on demand, validating each hop semantically. I have not seen that in repo-level code generation RAG, and it is well motivated by the cost and maintenance problems of global graphs. The experiments are serious: three LLMs, two benchmarks, consistent Pass@1 gains over a reasonable baseline set, clean ablations, and a credible 7.4x speedup over RepoScope. The replication package is available, which is a plus.\n\nThe soft spots are fixable rather than fatal. Section 6.1 validates the load-bearing expand step with 60 manually checked operations and reports 'completely accurate' per-function counts. That is the wrong unit. A per-edge recall with a documented sampling protocol and a breakdown of missed vs. fabricated edges would tell us whether errors land on load-bearing paths. The stress-test concern is valid: a ~7% per-expansion error rate across up to 10 hops means many tasks will hit a bad edge, and a missed callee cannot be recovered by post-processing, which only drops hallucinated names. The fact that the full pipeline still works suggests either the errors are not critical or the similarity path compensates, but we need the edge-level data to know. Also, maxHop=10 is tuned on CoderEval and then reported on that same benchmark; the authors disclose this, but it is data fitting and the DevEval numbers are where an independent confirmation would matter. The adapted RLCoder/AlignCoder baselines may be disadvantaged, though reusing the official artifacts is the right thing to do.\n\nNet: the paper deserves a serious referee. I would ask for an edge-level accuracy study and a better hyperparameter protocol, but I would not desk-reject it. I'd cite it.","headline":"Genuinely new on-demand partial dependency graph retrieval with consistent gains; the load-bearing expand step is under-validated (60 cases, no error breakdown), but the paper still deserves serious review.","tokens_in":21308,"tokens_out":3172,"would_cite":true,"duration_ms":35608,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Repository-level code generation can skip static global graphs: build dependency context on demand with an LLM, improving Pass@1 by up to 59.7% at 7.4x lower build time.","keywords":["repository-level code generation","retrieval-augmented generation","partial dependency graph","multi-hop reasoning","LLM-based code retrieval","Pass@1","dependency graph construction","context retrieval"],"falsifier":"Take a set of target functions whose full ground-truth dependency sets are known; run the Expand step for each hop and count whether the LLM's identified callees cover every real dependency (especially same-file callees not appearing in import statements). If coverage falls well below the reported 93% on a few hundred functions, or if a deliberately seeded dependency with a non-descriptive name is systematically missed, the method's core mechanism fails. Simpler: audit more than 60 expand operations and check whether missed edges correspond to Pass@1 failures.","tokens_in":20398,"feed_emoji":"⚡","tokens_out":4471,"duration_ms":43341,"temperature":0.7,"pith_summary":"The paper claims that the standard recipe for repository-level code generation—retrieve similar snippets, optionally traverse a pre-built static dependency graph—misses the context that matters: the specific functions the target actually calls or depends on. It proposes DyRetriever, which uses an LLM to imitate how a developer explores a codebase: pick likely entry-point functions, then hop from function to function, at each step asking the LLM whether the next dependency is worth keeping. The graph is built partially, on demand, and discarded after use, avoiding the hours-long static parse of large repositories. Integrated with similarity retrieval into DyCoder, the method reports Pass@1 improvements of 25.63% on CoderEval and 59.73% on DevEval over RAG baselines, and a 7.4x speedup over static-graph construction. The point is that retrieval rules can be replaced by the LLM's semantic judgment, and that dependency context, not just similarity, is what unlocks correct generation.","feed_headline":"LLM-built dependency graphs beat static RAG by up to 60%","feed_subtitle":"Repository-scale code generation gets faster and more accurate by building graphs on demand instead of parsing the whole repo.","key_machinery":"Partial dependency graph with LLM-driven multi-hop expansion. Named DyRetriever, the mechanism replaces a repository-wide static graph with a small, on-demand graph grown hop by hop. The three-step loop—select, visit, expand—uses the LLM both to choose which candidate function to inspect next and to identify callee dependencies from the function's source plus its file's import statements; a post-processing step matches names against the repository to remove hallucinated or out-of-repo functions. The graph exists only during retrieval, which is what avoids global parse and maintenance cost.","core_discovery":"DyCoder's central claim is that a partial dependency graph, built dynamically by an LLM and discarded after retrieval, is both more effective and more efficient than the global static dependency graphs used by prior graph-based RAG. DyRetriever first asks the LLM to select a small set of entry-point functions from a tree-structured view of the repository, then runs a bounded multi-hop traversal: select a candidate, visit it, extract the file's import statements, ask the LLM to name the functions it calls or lazily imports, and add those as new candidates. A name-matching post-process drops fabricated or standard-library dependencies. After up to ten hops the traversal trajectory is given to","pith_inferences":["If the LLM's dependency-spotting accuracy generalizes beyond the 60 manually inspected expansion operations, the same on-demand traversal could be applied to other repository-scale tasks such as bug localization, feature addition, or refactoring, where the relevant code is also defined by dependency chains rather than textual similarity.","The reported accuracy of 93.22% is measured on a small sample; a larger audit or an adversarially constructed repository with unusual naming, dynamic dispatch, or re-exports would show how much of the method's gain depends on the specific LLM's ability to read callees.","Token consumption scales with the hop budget (2–2.4x baseline), so the optimal max-hop setting likely trades accuracy against cost; the paper's choice of 10 is empirical and may shift for longer functions or larger repositories.","The approach as presented is Python-only; statically typed languages with explicit type relations may make dependency identification easier, which would be a direct testable extension of the method's core assumption."],"forward_implications":["Repository-level RAG no longer needs a pre-built global dependency graph; on-demand partial graphs can carry the dependencies that similarity misses.","Hand-crafted retrieval rules (e.g., import-based traversal or similarity thresholds) can be replaced by an LLM's per-hop helpfulness judgment, making retrieval adapt to in-file and cross-file dependencies alike.","The retrieved context is complementary to sparse and dense similarity retrieval; DyRetriever improved BM25, RLCoder, and AlignCoder by 7.5%–30.99% Pass@1 when combined.","The 7.4x speedup over static graph construction means the approach can serve large repositories where static parsing takes hours, and avoids rebuilding the graph as code evolves.","Because the method is training-free and plug-and-play, it can be layered onto existing retrievers without retraining cost."],"fun_headline_variants":["On-demand LLM-built graphs beat static RAG by up to 60%","Partial graph retrieval: 7x faster, up to 60% better codegen","DyCoder: dynamic dependency graphs outperform static RAG","LLM-crafted partial graphs outpace static retrieval in codegen","Discard after use: why partial graphs win for repo codegen"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes the LLM can reliably name all callee dependencies of a function from its source plus the file's import statements, without a static parser; this is spot-checked on only 60 expansions (93.22% accuracy), and a single missed or fabricated edge can break the retrieval chain.","fun_headline_variants_meta":{"raw":{"variants":["On-demand LLM-built graphs beat static RAG by up to 60%","Partial graph retrieval: 7x faster, up to 60% better codegen","DyCoder: dynamic dependency graphs outperform static RAG","LLM-crafted partial graphs outpace static retrieval in codegen","Discard after use: why partial graphs win for repo codegen"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000769,"raw_usage":{"total_tokens":3300,"prompt_tokens":855,"completion_tokens":2445,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":2349}},"tokens_in":599,"tokens_out":2445,"duration_ms":19000,"temperature":1.0,"reasoning_tokens":2349,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:04:46.314756+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of target functions whose full ground-truth dependency sets are known; run the Expand step for each hop and count whether the LLM's identified callees cover every real dependency (especially same-file callees not appearing in import statements). If coverage falls well below the reported 93% on a few hundred functions, or if a deliberately seeded dependency with a non-descriptive name is systematically missed, the method's core mechanism fails. Simpler: audit more than 60 expand operations and check whether missed edges correspond to Pass@1 failures.","supporting_citations":[],"review_version":1}