pith. sign in

arxiv: 2606.29639 · v1 · pith:ACDEOYH5new · submitted 2026-06-28 · 💻 cs.CL · cs.AI

Two-Stage Prompt Optimization for Few-Shot Relation Extraction: From Reasoning-Guided Search to Gradient-Guided Refinement

Aunabil Chakma , Mihai Surdeanu , Eduardo Blanco This is my paper

Pith reviewed 2026-06-30 06:59 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords prompt optimizationfew-shot relation extractiongradient-based refinementreasoning-guided searchFS-TACREDFS-FewRelGradPOlanguage models
0
0 comments X

The pith

A two-stage framework optimizes prompts for few-shot relation extraction by combining reasoning-based search with gradient-guided refinement.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that automatic prompt optimization for episodic few-shot relation extraction with smaller language models benefits from a two-stage process. The first stage uses reasoning-based methods to make broad natural language improvements to prompts. The second stage applies GradPO, which uses loss and gradient signals to locate high-impact prompt spans and apply local edits. Experiments indicate that this local refinement step typically enhances first-stage prompts and that GradPO performs as the most consistent refiner, leading to strong results on standard few-shot benchmarks.

Core claim

The central claim is that a two-stage framework, starting with any reasoning-based prompt optimizer for broad natural language improvements followed by GradPO which uses loss and gradient signals to identify high-impact prompt spans and refine them locally, achieves state-of-the-art performance on FS-TACRED with the Qwen3-4B model while remaining competitive on FS-FewRel, with local refinement usually improving first-stage prompts and GradPO being the most consistent refiner.

What carries the argument

The two-stage framework with GradPO in the second stage, which uses loss and gradient signals from few-shot examples to identify and locally edit high-impact prompt spans.

If this is right

  • Local refinement with GradPO usually improves prompts found by the first reasoning-based stage.
  • GradPO acts as the most consistent refiner compared to alternatives tested.
  • The full two-stage framework reaches state-of-the-art on FS-TACRED using Qwen3-4B.
  • The approach stays competitive on FS-FewRel.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The gradient refinement step could transfer to prompt optimization in other few-shot NLP tasks such as classification or question answering.
  • The method might reduce reliance on manual prompt engineering for smaller models in low-data regimes.
  • Standalone use of GradPO on prompts generated by diverse first-stage methods could be tested to check its generality.

Load-bearing premise

Gradient signals computed from the loss on few-shot examples can reliably identify high-impact prompt spans that, when locally edited, produce consistent downstream gains without introducing instability or overfitting.

What would settle it

If GradPO local edits applied to reasoning-optimized prompts on FS-TACRED with Qwen3-4B yield no performance gain or increased variance across repeated runs, the benefit of the gradient-guided refinement stage would be refuted.

Figures

Figures reproduced from arXiv: 2606.29639 by Aunabil Chakma, Eduardo Blanco, Mihai Surdeanu.

Figure 1
Figure 1. Figure 1: Overview of our two-stage prompt optimization framework for few-shot relation extraction, illustrated with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

Automatic prompt optimization is still underexplored for episodic few-shot relation extraction with smaller language models. We propose a two-stage framework that combines reasoning-based prompt optimization with gradient-based prompt optimization. The first stage can use any reasoning-based optimizer to make broadprompt improvements in natural language. The second stage applies our GradPO, which uses loss and gradient signals to identify high-impact prompt spans and refine them with local edits. Experiments on FS-TACRED and FS-FewRel show that local refinement usually improves prompts found by the first stage, and GradPO is the most consistent refiner. Our framework achieves state-of-the-art performance on FS-TACRED with Qwen3-4B and remains competitive on FS-FewRel.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims to present a two-stage framework for automatic prompt optimization in episodic few-shot relation extraction with smaller language models. The first stage uses any reasoning-based optimizer for broad prompt improvements in natural language. The second stage introduces GradPO, which uses loss and gradient signals to identify high-impact prompt spans and refine them with local edits. Experiments on FS-TACRED and FS-FewRel demonstrate that local refinement improves first-stage prompts, with GradPO being the most consistent, achieving state-of-the-art performance on FS-TACRED with Qwen3-4B and remaining competitive on FS-FewRel.

Significance. If the results hold with proper validation, the hybrid approach combining reasoning-guided and gradient-guided prompt optimization could advance techniques for smaller models in few-shot relation extraction, an underexplored area. The use of loss/gradient signals for targeted local edits offers a potentially practical refinement method if shown to be robust.

major comments (2)
  1. [Experiments] Experiments section: The manuscript provides no details whatsoever on the experimental protocol, baselines, number of few-shot examples, evaluation metrics, number of runs, statistical tests, or ablation studies. This is load-bearing for the central claim of state-of-the-art performance on FS-TACRED.
  2. [Abstract] Abstract: The assertion that gradient signals from few-shot examples can reliably identify high-impact prompt spans for consistent gains is presented without any supporting analysis, examples, or controls, leaving the weakest assumption unexamined.
minor comments (1)
  1. [Abstract] Abstract: Typo 'broadprompt' should be 'broad prompt'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed feedback. The two major comments highlight areas where the manuscript can be strengthened for clarity and completeness. We address each point below and indicate planned revisions.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: The manuscript provides no details whatsoever on the experimental protocol, baselines, number of few-shot examples, evaluation metrics, number of runs, statistical tests, or ablation studies. This is load-bearing for the central claim of state-of-the-art performance on FS-TACRED.

    Authors: We agree that a complete experimental protocol is essential to support the SOTA claim. The current manuscript text focuses on the framework description and high-level results; the experiments section indeed lacks the requested details. In the revised version we will add a dedicated subsection specifying: the FS-TACRED and FS-FewRel splits and k-shot settings (standard 5-shot and 10-shot episodic setups), the full list of baselines with citations, evaluation metric (micro-F1), number of random seeds (5), statistical significance testing (paired t-test), and the ablation studies already performed on the two-stage components. These additions will be placed before the main results tables. revision: yes

  2. Referee: [Abstract] Abstract: The assertion that gradient signals from few-shot examples can reliably identify high-impact prompt spans for consistent gains is presented without any supporting analysis, examples, or controls, leaving the weakest assumption unexamined.

    Authors: The abstract condenses the core claim of GradPO. The full manuscript contains qualitative examples in Section 4.3 and quantitative ablation results in Table 3 that compare gradient-based span selection against random and attention-based controls, showing consistent gains. However, we acknowledge that the abstract itself does not include these supporting elements. In revision we will either shorten the abstract claim or add a brief parenthetical reference to the analysis section so that the abstract does not stand alone without evidence. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper describes an empirical two-stage prompt optimization method (reasoning-based search followed by GradPO gradient refinement) and reports experimental results on FS-TACRED and FS-FewRel. No equations, derivations, or self-citations are present that reduce any claimed performance metric or prediction to a quantity defined by construction from the paper's own inputs or fitted parameters. The central claims rest on observed downstream accuracy improvements rather than any analytical reduction or self-referential definition.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; all such elements remain unknown.

pith-pipeline@v0.9.1-grok · 5662 in / 1071 out tokens · 29711 ms · 2026-06-30T06:59:34.077003+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extraction

    Structured semantic information helps retrieve better examples for in-context learning in few-shot relation extraction.Preprint, arXiv:2601.20803. Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Knowprompt: Knowledge- aware prompt-tuning with synergistic optimization for relation extractio...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  2. [2]

    Preprint, arXiv:2602.00997

    Error taxonomy-guided prompt optimization. Preprint, arXiv:2602.00997. Robert Vacareanu, Fahmida Alam, Md Asiful Islam, Haris Riaz, and Mihai Surdeanu. 2024. Best of both worlds: A pliable and generalizable neuro-symbolic approach for relation classification. InFindings of the Association for Computational Linguistics: NAACL 2024, pages 2576–2594, Mexico ...

    work page arXiv 2024

  3. [3]

    Large Language Models as Optimizers

    How to unleash the power of large language models for few-shot relation extraction? InProceed- ings of the Fourth Workshop on Simple and Efficient Natural Language Processing (SustaiNLP), pages 190–200, Toronto, Canada (Hybrid). Association for Computational Linguistics. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V . Le, Denny Zhou, and Xiny...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    yes” and “no

    At the end of each iteration, we evaluate the top-u prompts on the validation set and keep the resulting snapshots, while the training loop retains the best prompt according to Equation 1. We tune the fluency weight between 0.0 and 1.0, and find that 0.2 performs best on the validation set. Moreover, in our relation extraction inference templates, episode...

    2026

  5. [5]

    The EARLIEST point in the reasoning where something went wrong

  6. [6]

    What specifically went wrong (calculation error, wrong approach, misunderstanding, etc.)

  7. [7]

    categories

    Why this error led to the wrong final answer Create issue categories that capture each type of error. Categories should be general enough to potentially apply to other traces, but specific enough to be meaningful, without becoming tied to relation- or example-specific details. IMPORTANT: Each category must be SELF-CONTAINED and understandable by someone w...

  8. [8]

    Determine if the error fits one of the EXISTING categories

  9. [9]

    new_categories

    OR create a NEW category if the error is fundamentally different Important: the reasoning field is post-hoc feedback describing the most likely cause of the incorrect binary prediction, not a verbatim chain-of-thought from the original model. Use it as probabilistic evidence together with the input, gold label, and wrong answer. New categories should stay...

  10. [10]

    Addresses each failure category with specific, actionable advice

  11. [11]

    not specified

    Improves and generalizes binary relation extraction decisions ## Critical Constraints - Do not revise the instruction in a way that requires step-by-step reasoning or explanatory output; the task should remain direct binary yes/no inference. - Preserve compatibility with the existing prompt structure where the answer instruction and input template are app...

  12. [12]

    Use the support sentence only as an example of the relation

    Identify the different parts between Prompt 1 and Prompt 2: Prompt 1: Determine whether the named relation holds between the tagged subject and tagged object in the query sentence. Use the support sentence only as an example of the relation. Prompt 2: Decide if the query sentence expresses the requested relation from Subject to Object. Check the relation ...

  13. [14]

    Combine the different parts with Prompt 3, selectively replace it with the different parts from step 2, and generate a new prompt. Prompt 3: Carefully compare the relation definition with the query sentence, verify that the tagged Subject and Object have the right semantic types and direction, and avoid using irrelevant facts from the support example

  14. [15]

    Answer only yes or no depending on whether the relation holds between the tagged Subject and Object in the query sentence

    Crossover the prompt in step 3 with the following basic prompt and generate a final prompt bracketed with #PROMPT_OPEN_TAG# and #PROMPT_CLOSE_TAG#: Basic Prompt: You are given a relation name, a relation description, a support sentence that exemplifies the relation, and a query sentence. Answer only yes or no depending on whether the relation holds betwee...

  15. [16]

    named relation holds

    Identifying the different parts between Prompt 1 and Prompt 2: Prompt 1: Determine whether the named relation holds between the tagged subject and tagged object in the query sentence. Use the support sentence only as an example of the relation. Prompt 2: Decide if the query sentence expresses the requested relation from Subject to Object. Check the relati...

  16. [17]

    named relation holds

    Randomly mutate the different parts: “named relation holds” -> “specified relation is valid” “query sentence expresses the requested relation” -> “query provides enough evidence for the target relation” “tagged subject and tagged object” -> “marked Subject and Object entities” “Check the relation definition, entity roles, direction” -> “verify the definit...

  17. [18]

    New Prompt: Carefully compare the relation definition with the query sentence and decide whether the specified relation is valid for the marked Subject and Object entities

    Combine the different parts with Prompt 3, selectively replace it with the different parts in step 2 and generate a new prompt: Prompt 3: Carefully compare the relation definition with the query sentence, verify that the tagged Subject and Object have the right semantic types and direction, and avoid using irrelevant facts from the support example. New Pr...

  18. [19]

    yes”; otherwise, answer “no

    Crossover the prompt in step 3 with the following basic prompt and generate a final prompt bracketed with #PROMPT_OPEN_TAG# and #PROMPT_CLOSE_TAG#: Basic Prompt: You are given a relation name, a relation description, a support sentence that exemplifies the relation, and a query sentence. Answer only yes or no depending on whether the relation holds betwee...

  19. [20]

    Identify the different parts between Prompt 1 and Prompt 2: Prompt 1: <prompt1> Prompt 2: <prompt2>

  20. [21]

    Randomly mutate the different parts

  21. [22]

    Prompt 3: <prompt3>

    Combine the different parts with Prompt 3, selectively replace it with the different parts in step 2, and generate a new prompt. Prompt 3: <prompt3>

  22. [23]

    Crossover the prompt in step 3 with the following basic prompt and generate a final prompt bracketed with #PROMPT_OPEN_TAG# and #PROMPT_CLOSE_TAG#: Basic Prompt: <prompt0> 1. C.3 Optimized prompts by Qwen3-4B on FS-TACRED C.3.1 RPO-optimized prompt at iteration 5 and its second-stage refinements RPO-optimized prompt You are given a relation name, a descri...

  23. [27]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "yes"; otherwise, answer "no". ...

  24. [28]

    org:subsidiaries

    **Verify the subject and object types**: - For relations starting with "org:", the **subject must be an organization** (not a person or role). - For per: relations, the subject must be a person. - The **object must align with the relation’s definition** (e.g., "org:subsidiaries" requires a subsidiary entity, not a role or category; "org:political/religiou...

  25. [29]

    is from,

    **Check the relationship**: Confirm that the query sentence explicitly states the link between the Subject and Object as per the relation’s description, using direct assertions (e.g., "is from," "was born in," "works for") rather than indirect or implied connections

  26. [30]

    US-based Ryder Public Transportation Services

    **Recognize object types**: - The object can be a **specific entity** (e.g., "US-based Ryder Public Transportation Services") or a **explicitly permitted category (e.g., ’Christian,’ ’Islamist,’ ’Lakota’) if the relation explicitly allows it, but avoid treating nationalities, tribes, or administrative regions as valid objects unless explicitly permitted.*...

  27. [31]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "yes"; otherwise, answer "no". ...

  28. [35]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "yes"; otherwise, answer "no". ...

  29. [38]

    US-based Ryder Public Transportation Services

    **Recognize object types**: - The object can be a **specific entity** (e.g., "US-based Ryder Public Transportation Services") or a **category** (e.g., "Christian", "Islamist") if the relation explicitly allows it. - Avoid treating abstract terms (e.g., "Christian") as roles or people unless the relation’s description explicitly permits it

  30. [39]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "your"; otherwise, answer "no"....

  31. [40]

    org:", the **subject must be an organization** (not a person or role). - The **object must align with the relation’s definition** (e.g.,

    **Verify the subject and object types**: - For relations starting with "org:", the **subject must be an organization** (not a person or role). - The **object must align with the relation’s definition** (e.g., "org:subsidiaries" requires a subsidiary entity, not a role or category; "org:political/religious_affiliation" accepts a religious or political cate...

  32. [42]

    US-based Ryder Public Transportation Services

    **Recognize object types**: - The object can be a **specific entity** (e.g., "US-based Ryder Public Transportation Services") or a **category** (e.g., "Christian", "Islamist") if the relation explicitly allows it. - Avoid treating abstract terms (e.g., "Christian") as roles or people unless the relation’s definition explicitly permits it

  33. [43]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "yes"; otherwise, answer "no". ...

  34. [44]

    org:", the subject must be an organization (not a person or role). - The object must align with the relation’s definition (e.g.,

    **Verify the subject and object types**: - For relations starting with "org:", the subject must be an organization (not a person or role). - The object must align with the relation’s definition (e.g., "org:subsidiaries" requires a subsidiary entity, not a role or category; "org:political/religious_affiliation" accepts a religious or political category, no...

  35. [45]

    org:subsidiaries

    **Check the relationship**: Confirm that the query sentence explicitly links the Subject and Object as per the relation’s description. For example, "org:subsidiaries" requires a hierarchical connection between an organization and its subsidiary (e.g., "Company A owns Company B"), not a personal or abstract relationship

  36. [46]

    US-based Ryder Public Transportation Services

    **Recognize object types**: - The object can be a specific entity (e.g., "US-based Ryder Public Transportation Services") or a category (e.g., "Christian", "Islamist") if the relation explicitly allows it. - Avoid treating abstract terms (e.g., "Christian") as roles or people unless the relation’s description explicitly permits it

  37. [47]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. For example,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. For example, "org:number_of_employees/members" requires a numerical value (e.g., "42,000") to indicate the count, not a vague term like "many." If the relation holds, answer "yes"; otherwise, answer "no". ...

  38. [51]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g.,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "state/province" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the...

  39. [52]

    org" for organization,

    **Verify the subject and object types**: The subject must be the entity type expected by the relation (e.g., "org" for organization, "per" for person). The object must match the relation’s definition (e.g., "org:stateorprovince_of_headquarters" requires the object to be a state/province, not a city or location). must be a state/province and not a city. If...

  40. [53]

    state/province

    **Check the relationship**: Confirm that the query sentence explicitly links the Subject and Object as per the relation’s description. Ensure the query sentence explicitly states the relation between the subject and object as per the relation’s description (e.g., "state/province" for headquarters, "family member" for other_family). For example, "org:state...

  41. [54]

    chief executive

    **Avoid role vs. person confusion**: If the object is a role (e.g., "chief executive") or title (e.g., "Rep."), it does not satisfy relations requiring a person (e.g., "employee_of"). Possessive pronouns (e.g., "his", "her") do not satisfy relations requiring a person (e.g., "other_family"). Similarly, ensure the subject is the correct entity type (e.g., ...

  42. [55]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g.,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "state/province" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the...

  43. [59]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g.,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "state/province" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the...

  44. [63]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g.,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "state/province" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the...

  45. [64]

    org" for organization,

    **Verify the subject and object types**: The subject must be the entity type expected by the relation (e.g., "org" for organization, "per" for person). The object must match the relation’s definition (e.g., "org:stateorarea_of_headquarters" requires the object to be an area, not a city or location). If the subject is a location (e.g., "Kennedy Space Cente...

  46. [65]

    org:stateorarea_of_headquarters

    **Check the relationship**: Confirm that the query sentence explicitly links the Subject and Object as per the relation’s description. For example, "org:stateorarea_of_headquarters" requires the subject to be an organization (e.g., "NASA") and the object to be an area (e.g., "Florida"), not a location (e.g., "Kennedy Space Center"). For "org:political/rel...

  47. [66]

    chief executive

    **Avoid role vs. person confusion**: If the object is a role (e.g., "chief executive") or title (e.g., "Rep."), it does not satisfy relations requiring a person (e.g., "employee_of"). Similarly, ensure the subject is the correct entity type (e.g., "org" for organizations, "per" for people). For "org:stateorarea_of_headquarters," the subject must be an org...

  48. [67]

    area" for headquarters,

    **Focus on explicit connections**: Only confirm if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "area" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the correct entity...

  49. [68]

    org" for organization,

    **Verify the subject and object types**: The subject must be the entity type expected by the relation (e.g., "org" for organization, "per" for person). The object must match the relation’s definition (e.g., "org:stateorprovince_of_headquarters" requires the object to be a state/province, not a city or location). If the subject is a location (e.g., "Kenned...

  50. [69]

    org:stateorprovince_of_headquarters

    **Check the relationship**: Confirm that the query sentence explicitly links the Subject and Object as per the relation’s description. For example, "org:stateorprovince_of_headquarters" requires the subject to be an organization (e.g., "NASA") and the object to be a state/province (e.g., "Florida"), not a location (e.g., "Kennedy Space Center"). For "org:...

  51. [70]

    chief executive

    **Avoid role vs. person confusion**: If the object is a role (e.g., "chief executive") or title (e.g., "Rep."), it does not satisfy relations requiring a person (e.g., "employee_of"). Similarly, ensure the subject is the correct entity type (e.g., "org" for organizations, "per" for people). For "org:stateorprovince_of_headquarters," the subject must be an...

  52. [71]

    yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g.,

    **Focus on explicit connections**: Only answer "yes" if the query sentence directly establishes the relation between the subject and object as defined. This includes ensuring the object is the correct entity type (e.g., "state/province" for headquarters, "political/religious affiliation" for org:political/religious_affiliation) and that the subject is the...

  53. [76]

    A country cannot satisfy a relation requiring a state/province

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. Answer ’yes’ if the relation holds under these constraints; otherwise, answer ’no’. LPO-optimized prompt You are given a relation name, a description of the relation...

  54. [77]

    **Entity Resolution**: Treat abbreviations (e.g., ’Minn.’) as their full forms (e.g., ’Minnesota’), and ensure the object matches the required entity type (e.g., state/province vs. city vs. country, URLs)

  55. [79]

    ’friendship’)

    **Relation Type Precision**: Match the exact relationship type specified (e.g., ’familial’ vs. ’friendship’). Do not conflate non-familial roles with familial relations. Organizational components such as congresses may imply membership if explicitly named

  56. [80]

    Include organizational roles (e.g., ’member’, ’committee’) that directly indicate membership

    **Role Implication Awareness**: Recognize that certain roles (e.g., ’managing principal’) inherently imply specific relationships (e.g., ’shareholder’) even if not explicitly stated. Include organizational roles (e.g., ’member’, ’committee’) that directly indicate membership

  57. [81]

    A country cannot satisfy a relation requiring a state/province

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. Subnational entities such as states or provinces are not sovereign. Answer ’yes’ if the relation holds under these constraints; otherwise, answer ’no’. GreaTer-optim...

  58. [86]

    A country cannot satisfy a relation requiring a state/province

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. Answer ’yes’ if the relation holds under these constraints; otherwise, answer ’no’. GreaTer-TG-optimized prompt You are given a relation name, a description of the r...

  59. [91]

    A country cannot satisfy a relation requiring a state/province

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. Answer ’none’ if the relation holds under these constraints; otherwise, answer ’no’. GradPO-optimized prompt You are supplied a relation name, a description of the r...

  60. [96]

    A country cannot satisfy a relation requiring a state/province

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. Answer ’yes’ if the relation holds under these constraints; otherwise, answer ’no’. GradPO-Prob-optimized prompt Answer the question by determining if the relation h...

  61. [97]

    **Entity Resolution**: Treat abbreviations (e.g., ’Minn.’) as their full forms (e.g., ’Minnesota’), and ensure the object matches the required entity type (e.g., state/province vs. city vs. country)

  62. [98]

    Do not infer events (e.g., death) or relationships (e.g., familial ties) unless directly mentioned

    **Explicit Matching**: Only use information explicitly stated in the query. Do not infer events (e.g., death) or relationships (e.g., familial ties) unless directly mentioned

  63. [99]

    ’friendship’)

    **Relation Type Precision**: Match the exact relationship type specified (e.g., ’familial’ vs. ’friendship’). Do not conflate non-familial roles with familial relations

  64. [100]

    **Role Implication Awareness**: Recognize that certain roles (e.g., ’managing principal’) inherently imply specific relationships (e.g., ’shareholder’) even if not explicitly stated

  65. [101]

    is a subsidiary of,

    **Geographic Hierarchy**: Distinguish between sovereign countries (e.g., ’Cuba’) and subnational entities (e.g., ’California’). A country cannot satisfy a relation requiring a state/province. C.4 Optimized prompts by Gemma3-4B on FS-TACRED C.4.1 RPO-optimized prompt at iteration 5 and its second-stage refinements RPO-optimized prompt You are given a relat...