{"id":"3d3e2ab3-fb88-4195-9a62-5c8e2b24fb0e","arxiv_id":"2501.14114","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LeCoPCR augments prior-case-retrieval queries with legal concepts generated from case facts, using DPP-based weak supervision to extract those concepts from reasoning sections.","lead":"Legal case retrieval usually matches facts against prior cases without modeling why a precedent was cited. This paper generates legal concepts from case facts and adds them to the query, reporting modest recall gains on European Court of Human Rights cases.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gains in Table 1 may reflect generic query expansion rather than the legal-concept mechanism; no control or significance testing distinguishes these.","rationale":"The reader's weakest_assumption focuses on whether the extracted/generated concepts approximate true legal concepts; my concern is adjacent but more specifically about experimental identification: the current design cannot attribute the observed gains to the legal-concept mechanism because no non-concept query-expansion control is included and no significance testing is reported. This is load-bearing because the strongest_claim is an empirical claim about Recall@k gains, and those gains could be equally produced by any query expansion that adds reasoning-vocabulary terms. I agree with the reader that the oracle gap is large and suggests generation quality is a bottleneck; however, I would sharpen that observation: the oracle concepts are drawn from the same reasoning section that contains the citation labels, so the gap is partly an information-theoretic limit (facts alone may not determine the legal concepts the court will later use) rather than purely a model-quality issue. The paper does have genuine strengths: the DPP weak-supervision formulation is coherent, the ablations show meaningful differences among extraction methods, and the hybrid training (HT) is a reasonable response to exposure bias. These support a conditional acceptance, not rejection. However, the absence of significance tests and expansion controls means the headline result is not yet robustly established, so the CONDITIONAL verdict remains appropriate.","tokens_in":9891,"tokens_out":8268,"duration_ms":85157,"concrete_test":"Run a controlled query-expansion ablation on the test set: for each query, create (1) the LongT5 legal concepts as in LeCoPCR, (2) an equal-length RM3 or doc2query expansion, (3) a LongT5-generated factual summary with no legal-concept training, and (4) random noun phrases sampled from the training corpus. Use each as query augmentation with BM25 and Longformer, and compute paired bootstrap 95% confidence intervals over the 3,200 test queries for Recall@50/100/500/1000 and MAP. If the LeCoPCR advantage over all three controls is within the CI of zero on any primary metric, the claim that legal concepts specifically drive the gains is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that model-generated legal concepts improve PCR recall (Table 1). The most load-bearing unsecured link is that the Recall@k gains are caused by 'legal concepts' as such, rather than by generic query expansion or by unmodeled lexical overlap. LeCoPCR is evaluated only against no-augmentation baselines (BM25, Longformer); no control adds the same number of generated tokens (e.g., a fact summary, RM3 expansion terms, or random noun phrases) to the query. Because the generator is trained to emit noun phrases from the reasoning section, the augmented query will share surface vocabulary with the reasoning of relevant prior cases; a generic expansion model could produce the same overlap without instantiating the claimed 'semantic intent.' Additionally, no significance tests or confidence intervals are reported; the BM25 gains are small (e.g., +0.68 Recall@50, +0.24 MAP) and could be within query-level or run-to-run variance for the 3.2k test queries. The large oracle gap (BM25 R50: 31.26 vs. 22.52) may indicate label leakage rather than headroom for fact-based generation, because oracle concepts are extracted from the query's own reasoning using proximity to citation markers, i.e., from the label-bearing section the fact-only system is supposed to predict. Thus, neither the specific mechanism nor the magnitude of the effect is established by the current experimental design.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LeCoPCR, a two-stage pipeline for prior case retrieval on the ECtHR-PCR dataset. A weak-supervision stage extracts key legal concepts from the reasoning sections of existing cases via noun-phrase chunking, a Determinantal Point Process (DPP) selection step, a masked-document relevance score, and citation-proximity regularization; a LongT5 seq2seq model is then trained to generate such concepts from case facts alone. At inference, the generated concepts are concatenated with the fact description and fed to BM25 or a Longformer bi-encoder retriever. Experiments report Recall@k and MAP on a chronological split, oracle ablations over concept extraction methods, and a hybrid training variant designed to make the dense retriever robust to noisy generated concepts. The central claim is that explicitly generating legal concepts improves prior case retrieval over strong no-augmentation baselines.","tokens_in":10156,"tokens_out":6518,"duration_ms":59168,"significance":"If the attribution claim were established, LeCoPCR would be a practical, model-agnostic query-expansion method for legal retrieval, with a useful weak-supervision recipe for concept extraction. Strengths of the paper include the chronologically split evaluation that respects temporal realism, the oracle ablations that compare several concept extractors, and the hybrid training idea that addresses exposure bias in query augmentation. The paper is also candid about limitations such as exact-match evaluation of generated concepts, the absence of a human study, and the lack of a re-ranking stage. However, the experiments as presented do not yet rule out the possibility that the gains come from generic query expansion or uncontrolled lexical overlap, and no significance testing is provided; the central contribution is therefore plausible but not yet firmly established.","major_comments":[{"comment":"No confidence intervals or significance tests are reported anywhere, and the actual gains are small: for BM25, +0.68 Recall@50 and +0.24 MAP; for Longformer, +1.24 Recall@50. With 3.2k test queries and stochastic concept generation plus stochastic dense training, these differences could be within run-to-run or query-level variance. Report query-level bootstrap confidence intervals and paired significance tests for all tables, and state the number of training runs and seed variance.","section":"Section 3.1, Table 1"},{"comment":"The experimental design lacks a generic query-expansion control. The augmented query is the fact description plus generated noun phrases, so any expansion method that adds reasoning-related vocabulary could improve both lexical and neural retrieval without instantiating the claimed legal-concept mechanism. Add controls matched in token count, such as RM3 expansion terms, doc2query pseudo-relevant terms, noun phrases extracted from the facts, or a factual summary generated by the same LongT5 backbone. Without such controls, the Recall@k gains in Table 1 cannot be attributed to legal-concept semantics rather than to generic vocabulary overlap.","section":"Section 2.2 and Table 1"},{"comment":"The oracle concepts are extracted from the query's own reasoning section, which is the label-bearing section unavailable at inference time. The large gap between LeCoPCR and the oracle (BM25 Recall@50 22.52 vs. 31.26) is interpreted as headroom for better concept generation, but part of that gap may reflect access to citation labels rather than concept quality. Please quantify this effect, for example by evaluating oracle concepts extracted from facts only, or by removing the citation-proximity regularization and measuring how much of the oracle gain depends on proximity to citation markers.","section":"Section 2.1 and Tables 1-2"},{"comment":"The DPP ablation is not matched for selection size: DPP, MDERank, KeyBERT, TextRank, and TF-IDF likely return different numbers of concepts per query, and the number of augmentation tokens strongly affects BM25 performance. Report the average concept count for each method and either fix the number of concepts across methods or show performance as a function of concept budget. Without this, the claim that DPP selection itself is superior is confounded by the number of expansion terms.","section":"Section 3.1, Table 2"}],"minor_comments":[{"comment":"There are typographical errors: 'relavance' in the Abstract and 'required to to establish precedence' in Section 2.","section":"Abstract and Section 2"},{"comment":"The table caption reads 'RK, HT indicate Recall@k and Hybrid training respectively'; this should be 'R@k' and 'HT', and the entry '26.47' is visually merged with 'LeCoPCR-HT', making the row difficult to read.","section":"Table 1"},{"comment":"'Word Cov.' and 'Concept Cov.' are not formally defined; please specify the exact-match or fuzzy-match procedure and the denominator used for coverage.","section":"Table 3"},{"comment":"The two case-study URLs are appended to case names without final punctuation, and there are line-break artifacts in case names such as 'MOLDOV A1' and 'MAGY AR'; please clean these.","section":"Section 3.2"},{"comment":"The hybrid training setup is only described in one sentence in Section 2.2; please specify how many noisy concept samples are used per query, from which documents they are sampled, and whether they replace or supplement the golden extracted concepts.","section":"Appendix A"},{"comment":"No code or data release link is mentioned; if available, including a repository would strengthen reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope. The main risk is overclaiming the legal-concept mechanism from an uncontrolled query-expansion experiment; the requested controls, significance testing, and concept-budget matching should be feasible within the existing pipeline. The self-citation of the ECtHR-PCR dataset is appropriate, and the limitations section is honest about the remaining evaluation gaps."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jeremy — quick take on LeCoPCR (arXiv:2501.14114). The paper is a solid, honest piece of incremental engineering: it augments ECtHR prior case retrieval queries with legal concepts generated from case facts, using DPP-based weak supervision to extract silver concepts from reasoning sections. The combination of DPP selection, seq2seq generation, and query augmentation for PCR is new; the oracle ablations against TextRank, KeyBERT, and MDERank show DPP helps when concepts are perfect. Credit where due: the split is chronological, the ablations are informative, and the limitations section is unusually candid about the recall-only setup, dataset biases, and need for human evaluation.\n\nThe soft spots are real but not fatal. First, the reported gains are small — BM25 Recall@50 goes from 21.84 to 22.52, Longformer to 25.22 — and there are no significance tests or error bars anywhere. On a 3.2k-query test set that gap is plausibly within variance. Second, and more load-bearing, there is no control that adds non-concept tokens to the query. The stress-test note is right: the generator emits noun phrases from reasoning, so the augmented query shares surface vocabulary with prior case reasoning; a generic expansion model (RM3, or even random noun phrases) could produce the same overlap. The paper therefore doesn't yet establish that 'legal concepts' as such, rather than query expansion in general, drive the gains. Third, the oracle gap (31.26 vs 22.52 for BM25) shows the generator is a bottleneck, and the hybrid training (LeCoPCR-HT) helps more than the clean LeCoPCR, which suggests noise robustness matters as much as concept quality. Minor: no code or data release, which for a method paper is a transparency issue.\n\nThe math in the DPP section is standard and correctly presented; the citation pattern is appropriate, including self-citation of the dataset paper, which is legitimate. The paper does not oversell — the conclusion and limitations acknowledge the modest improvement and the need for reranking.\n\nWho's this for? People building legal case retrieval pipelines, especially on ECtHR. It's a reasonable workshop-or-short-paper level contribution, not a major advance. I'd send it to review — the oracle ablations and the hybrid training idea are worth refereeing seriously — but I'd require significance testing and a generic expansion control before acceptance. I wouldn't cite it myself in the next year unless I was directly working on this dataset; it's an incremental data point.","headline":"Solid incremental method for legal concept-guided query expansion, but the mechanism isn't separated from generic expansion and the gains lack significance testing.","tokens_in":10705,"tokens_out":1760,"would_cite":false,"duration_ms":15105,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding model-generated legal concepts to query facts improves prior case retrieval, with Recall@k and MAP gains on the ECtHR-PCR benchmark for both BM25 and Longformer retrievers.","keywords":["prior case retrieval","legal concepts","query expansion","determinantal point process","weak supervision","legal information retrieval","European Court of Human Rights","sequence-to-sequence generation"],"falsifier":"Replace the generated concepts appended to each query with an equal number of random noun phrases drawn from the same facts; if Recall@k stays within noise of the un-augmented baselines, the concept signal is not what drives the paper's gains.","tokens_in":9664,"feed_emoji":"⚖️","tokens_out":7560,"duration_ms":62593,"temperature":0.7,"pith_summary":"The paper tries to show that prior case retrieval improves when the query is augmented with legal concepts, short noun phrases naming the principles behind a potential citation, generated from the facts of an undecided case. Because no gold concepts exist, the authors build silver labels by extracting noun phrases from the reasoning sections of already-decided cases, selecting them with a determinantal point process that rewards both relevance and diversity, and then train a text-to-text model to produce such concepts from facts alone. On the ECtHR-PCR benchmark, appending generated concepts raises Recall@k and MAP for both a lexical retriever and a neural retriever, while appending oracle concepts from the reasoning section yields much larger gains. The result matters because it shows that making the semantic intent explicit, rather than leaving the retriever to infer it, is a workable direction for legal retrieval, with concept generation quality as the main remaining bottleneck.","feed_headline":"Legal concepts boost prior-case retrieval in human-rights law","feed_subtitle":"Augmenting query facts with model-generated legal terms lifts Recall@k for both lexical and neural retrievers.","key_machinery":"The load-bearing object is the determinantal point process (DPP) used for weak-supervised concept selection, with an L-ensemble kernel $L_{ij}=q_i s_{ij} q_j$ where $q_i$ is a concept's relevance score and $s_{ij}$ the cosine similarity between concept embeddings. Because $\\det(L_Y)$ for a subset measures the squared volume spanned by the concept vectors, a selected set simultaneously favors high-relevance concepts and penalizes pairwise similar ones; greedy MAP inference then chooses concepts one at a time. Relevance itself is computed by masking a candidate concept from its source paragraph, taking the complement of the cosine similarity between the masked and original paragraphs, and multiplying by a position weight $\\rho_i = \\mathrm{softmax}(e^{1/k})$ that grows as the concept sits closer to a citation marker. This machinery defines the silver concept labels that train the generator and serve as the oracle upper bound in the paper's experiments.","core_discovery":"LeCoPCR's central claim is that a small set of legal concepts can mediate between a case's facts and the prior cases it should cite, and that generating these concepts explicitly and adding them to the query improves retrieval recall. The authors support the claim on the ECtHR-PCR dataset by first extracting candidate concepts as noun phrases from reasoning sections, scoring their relevance through masked-paragraph similarity and proximity to citation markers, selecting a balanced subset with a determinantal point process, and then training a long-context sequence-to-sequence model to generate such concepts from the facts portion alone. At inference the generated concepts are concatenated with the facts and passed to a retriever that is otherwise unchanged; the paper reports consistent Recall@k and MAP improvements for both BM25 and Longformer, with a further gain from training the dense retriever on noisy as well as clean concepts.","pith_inferences":["Editorial extension: the facts-to-concepts training recipe should transfer to other legal corpora that separate facts from reasoning, such as those of other human-rights tribunals, provided the concept extractor and generator are retrained on local case language.","Editorial extension: the paper's coverage-style evaluation likely understates retrieval value; a generated concept that overlaps a reasoning section only partially can still be the phrase that matches the precedent's holding, so utility should be measured by downstream citation quality.","Editorial extension: because a single query can cite different precedents for different reasons, a per-citation concept generation variant could close part of the gap between generated and oracle concepts that the paper reports."],"forward_implications":["If the claim is right, retrieval systems can be improved without manually annotated legal concepts: the silver labels come from existing case reasoning and a DPP-based filter.","Because the augmentation step is retriever-agnostic, the same generated concepts should improve other lexical and dense retrievers beyond the two tested.","The large oracle-versus-generated gap implies that better concept generators, or better ways of denoising generated concepts, should translate directly into further Recall@k gains.","Hybrid training with noisy concepts is a concrete recipe for making dense retrievers robust to imperfect query expansion, not just a trick for this dataset."],"supporting_citations":[{"why":"supplies the ECtHR-PCR dataset with facts-only queries and cited cases as relevance labels, defining the task and evaluation.","marker":"Santosh et al. (2024)"},{"why":"provides the DPP L-ensemble decomposition into quality and similarity terms that the concept selector is built on.","marker":"Kulesza et al. (2012)"},{"why":"supplies the masked document-to-document relevance computation used to score candidate concepts.","marker":"Zhang et al. (2022)"},{"why":"provides the legal-domain embeddings used for concept similarity and masked-paragraph relevance.","marker":"Chalkidis et al. (2020)"},{"why":"provides the long-context sequence-to-sequence model trained to generate legal concepts.","marker":"Guo et al. (2022)"},{"why":"provides the long-document transformer used as the dense retriever evaluated with and without concept augmentation.","marker":"Beltagy et al. (2020)"},{"why":"provides the BM25 lexical retriever baseline that receives the same augmented queries.","marker":"Robertson et al. (2004)"}],"fun_headline_variants":["LeCoPCR: legal concepts improve ECtHR prior-case retrieval","Weak supervision yields legal concepts that lift retrieval","DPP-selected concepts sharpen precedent search","Legal concept injection raises Recall@k for prior cases","Adding model-generated concepts to queries improves retrieval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the legal concepts that make a prior case relevant are adequately represented by noun phrases from the reasoning section, that masked-paragraph relevance and closeness to citation markers can rank those phrases by importance, and that a model can regenerate enough of them from the facts of a case alone.","fun_headline_variants_meta":{"raw":{"variants":["LeCoPCR: legal concepts improve ECtHR prior-case retrieval","Weak supervision yields legal concepts that lift retrieval","DPP-selected concepts sharpen precedent search","Legal concept injection raises Recall@k for prior cases","Adding model-generated concepts to queries improves retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001132,"raw_usage":{"total_tokens":4655,"prompt_tokens":848,"completion_tokens":3807,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":3734}},"tokens_in":464,"tokens_out":3807,"duration_ms":27410,"temperature":1.0,"reasoning_tokens":3734,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:20:56.115653+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the generated concepts appended to each query with an equal number of random noun phrases drawn from the same facts; if Recall@k stays within noise of the un-augmented baselines, the concept signal is not what drives the paper's gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the masked document-to-document relevance computation used to score candidate concepts."}],"review_version":1}