pith. machine review for the scientific record. sign in

arxiv: 2605.04643 · v1 · submitted 2026-05-06 · 💻 cs.CL

Graph-Augmented LLMs for Swiss MP Ideology Prediction

Pith reviewed 2026-05-08 16:14 UTC · model grok-4.3

classification 💻 cs.CL
keywords LLMknowledge graphideology predictionSwiss parliamentretrieval augmented generationpolitical scienceMP ideologygraph augmentation
0
0 comments X

The pith

Graph-augmented LLMs improve predictions of Swiss MPs' ideological positions by incorporating relational data from a political knowledge graph.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a framework called PG-RAG that uses retrieval-augmented generation to pull information from a political knowledge graph and feed it into large language models. This allows the models to use both the text of parliamentary data and the structured relationships between MPs, parties, and other entities. When tested on Swiss parliamentary records, these graph-augmented models outperform standard LLM approaches and other baselines in predicting MPs' ideologies. The improvement shows that domain-specific relational information adds value to modeling political behavior.

Core claim

The authors propose the PG-RAG framework, which implements a retrieval-augmented generation pipeline by first querying a political knowledge graph and then integrating the resulting graph-structured information into the LLM context. This captures both textual semantics and inter-MP relationships. Evaluation on Swiss parliamentary data shows that graph-augmented models improve prediction performance over state-of-the-art baselines.

What carries the argument

The PG-RAG retrieval-augmented generation pipeline that queries a political knowledge graph to enrich the context for ideology prediction.

Load-bearing premise

The political knowledge graph must accurately capture relevant relationships and entities in the parliamentary system without introducing noise or bias.

What would settle it

Running the same experiments on the Swiss dataset and finding that graph-augmented models perform no better or worse than the text-only baselines would falsify the main result.

Figures

Figures reproduced from arXiv: 2605.04643 by Laurence Brandenberger, Luis Salamanca, Sophia Schlosser, Yifei Yuan.

Figure 1
Figure 1. Figure 1: The overall framework of our proposed method view at source ↗
Figure 2
Figure 2. Figure 2: Vote-based ideology scores of Swiss members view at source ↗
Figure 3
Figure 3. Figure 3: Scatterplot of the best LLM prediction scores view at source ↗
Figure 5
Figure 5. Figure 5: Performance improvements from RAG over the non-RAG baseline for Qwen-8B and GPT-5. dictions reflect a more nuanced ideology placement that encompasses MPs’ legislative behavior outside voting view at source ↗
Figure 6
Figure 6. Figure 6: Graph demonstration of the MP-centric sub view at source ↗
Figure 7
Figure 7. Figure 7: Graph demonstration of the pursuit-centric view at source ↗
read the original abstract

Approximating the ideological position of Members of Parliament (MPs) is a fundamental task in political science, helping researchers understand legislative behavior, party alignment, and policy preferences. While Large Language Models (LLMs) have shown promising results in estimating MPs' ideological stances, there are more actors and elements in the parliamentary system, and relations between them, that could provide a wider and more informative picture. However, due to the complexity of integrating them in the prediction task, these additional elements are generally ignored. In this work, we propose an LLM framework, PG-RAG, that implements a retrieval-augmented generation pipeline: it first queries a political knowledge graph (KG) and then integrates the resulting graph-structured information into the context. This allows for capturing both textual semantics and inter-MP relationships, another relevant information source in any parliamentary system. We evaluate the approach on the task of ideology prediction, using data from a Swiss parliamentary dataset. When comparing graph-augmented models against several state-of-the-art baselines, the results demonstrate that incorporating this enriched information, which encodes information about different entities and relations, improves prediction performance. These results help to highlight the value of domain-specific relational information in modeling political behavior.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes PG-RAG, a retrieval-augmented generation framework that queries a political knowledge graph (KG) encoding entities and relations among Swiss MPs and other parliamentary actors, then injects the retrieved graph-structured information into the LLM context for ideology prediction. It evaluates this approach on a Swiss parliamentary dataset and reports performance gains relative to several LLM baselines, attributing the improvement to the incorporation of relational information beyond textual semantics.

Significance. If the empirical gains are robust and causally attributable to the relational structure rather than confounding factors such as context length or prompting differences, the work would provide concrete evidence for the utility of domain-specific KGs in political modeling tasks. The approach is timely given growing interest in augmenting LLMs with structured knowledge for social-science applications, and the Swiss parliamentary setting offers a well-defined testbed with public data.

major comments (3)
  1. [Abstract, §3] Abstract and §3 (method): The headline claim that 'incorporating this enriched information... improves prediction performance' and demonstrates 'the value of domain-specific relational information' is load-bearing, yet the manuscript provides no description of KG construction, entity/relation extraction method, data sources, coverage statistics, or validation (e.g., accuracy, expert review, or noise audit). Without these, it is impossible to rule out that observed gains arise from longer context, different prompting, or data leakage rather than the claimed relational enrichment.
  2. [§4] §4 (experiments): The evaluation reports performance gains but supplies no details on dataset size, train/test splits, statistical tests, error bars, or exact baseline implementations. This absence prevents verification that the cross-model comparison is fair and that the improvement is statistically reliable rather than an artifact of post-hoc choices or small-sample variance.
  3. [§3.2] §3.2 (PG-RAG pipeline): The integration step that 'integrates the resulting graph-structured information into the context' is described at a high level only; no ablation is presented that isolates the contribution of the graph structure (e.g., vs. simply retrieving and concatenating the same textual facts without graph topology). This leaves open whether the relational encoding itself, rather than additional text, drives the reported gains.
minor comments (2)
  1. [§3] Notation for the political KG (entities, relations, query mechanism) is introduced without a formal definition or diagram; a small schema figure would improve clarity.
  2. [Abstract, §4] The abstract states 'several state-of-the-art baselines' without naming them; the experimental section should explicitly list the baselines and their configurations in a table.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thoughtful and detailed report. The comments highlight important gaps in reproducibility and experimental validation that we will address through major revisions. Below we respond point-by-point to the major comments.

read point-by-point responses
  1. Referee: [Abstract, §3] Abstract and §3 (method): The headline claim that 'incorporating this enriched information... improves prediction performance' and demonstrates 'the value of domain-specific relational information' is load-bearing, yet the manuscript provides no description of KG construction, entity/relation extraction method, data sources, coverage statistics, or validation (e.g., accuracy, expert review, or noise audit). Without these, it is impossible to rule out that observed gains arise from longer context, different prompting, or data leakage rather than the claimed relational enrichment.

    Authors: We agree that the absence of KG construction details weakens the ability to attribute gains specifically to relational information. The current manuscript prioritizes the PG-RAG pipeline description but omits these elements. In the revised version we will insert a new subsection in §3 that fully documents: data sources (Swiss Federal Assembly open data plus linked parliamentary records), entity and relation extraction procedures (hybrid rule-based and LLM-assisted methods for MPs, parties, bills, and relations such as co-sponsorship or committee membership), coverage statistics (entity/relation counts and MP coverage), and validation steps (expert review by political scientists plus quantitative accuracy checks on a held-out sample). These additions will allow readers to assess potential confounds such as context length or leakage. revision: yes

  2. Referee: [§4] §4 (experiments): The evaluation reports performance gains but supplies no details on dataset size, train/test splits, statistical tests, error bars, or exact baseline implementations. This absence prevents verification that the cross-model comparison is fair and that the improvement is statistically reliable rather than an artifact of post-hoc choices or small-sample variance.

    Authors: We accept that the experimental section lacks necessary rigor for independent verification. We will expand §4 with: precise dataset statistics (number of MPs, labeled statements, and temporal range), train/test split protocol (including any temporal or stratified hold-out), statistical significance testing (e.g., paired tests with p-values), error bars derived from multiple random seeds or bootstrapping, and complete baseline specifications (exact prompts, model versions, and decoding parameters). We will also release code and processed data to support reproducibility. revision: yes

  3. Referee: [§3.2] §3.2 (PG-RAG pipeline): The integration step that 'integrates the resulting graph-structured information into the context' is described at a high level only; no ablation is presented that isolates the contribution of the graph structure (e.g., vs. simply retrieving and concatenating the same textual facts without graph topology). This leaves open whether the relational encoding itself, rather than additional text, drives the reported gains.

    Authors: This is a fair critique of causal attribution. We will add a targeted ablation study to the revised manuscript. The new experiment will compare (i) full PG-RAG using graph-structured retrieval and integration, (ii) a text-only retrieval baseline that linearizes the identical KG facts into sentences or lists without topology or edge information, and (iii) the original non-retrieval baselines. Results, including performance differences and qualitative analysis of cases where structure helps, will be reported in an updated §4. revision: yes

Circularity Check

0 steps flagged

No significant circularity in empirical evaluation

full rationale

The paper proposes an empirical LLM framework (PG-RAG) that augments prompts with retrieved information from a political knowledge graph and evaluates ideology prediction performance on Swiss parliamentary data against baselines. No equations, derivations, or first-principles results are present. The central claim rests on comparative experimental outcomes using external data rather than any reduction of predictions to fitted inputs, self-definitions, or self-citation chains. The framework is self-contained against the reported benchmarks with no load-bearing steps that collapse by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Only abstract available; no explicit free parameters, axioms, or invented entities are stated. The approach implicitly assumes the KG is a faithful representation of parliamentary relations.

axioms (1)
  • domain assumption The constructed political knowledge graph contains accurate and relevant entities and relations for ideology prediction.
    Invoked when claiming that graph integration improves performance; no validation of KG quality is described in the abstract.

pith-pipeline@v0.9.0 · 5518 in / 1190 out tokens · 33650 ms · 2026-05-08T16:14:30.697397+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 7 canonical work pages · 1 internal anchor

  1. [1]

    Annual Meeting of the Association for Computational Linguistics , year=

    Minimizing Manual Annotation Cost in Supervised Training from Corpora , author=. Annual Meeting of the Association for Computational Linguistics , year=

  2. [2]

    ArXiv , year=

    Top2Vec: Distributed Representations of Topics , author=. ArXiv , year=

  3. [3]

    Comparative Political Studies , volume=

    Turning a blind eye: Experimental evidence of partisan bias in attitudes toward corruption , author=. Comparative Political Studies , volume=. 2013 , publisher=

  4. [4]

    Philosophy & Public Affairs , volume=

    Beyond homo economicus: New developments in theories of social norms , author=. Philosophy & Public Affairs , volume=. 2000 , publisher=

  5. [5]

    Political Analysis , volume=

    Party system compactness: Measurement and consequences , author=. Political Analysis , volume=. 2004 , publisher=

  6. [6]

    American Journal of Political Science , pages=

    Information and influence: Lobbying for agendas and votes , author=. American Journal of Political Science , pages=. 1993 , publisher=

  7. [7]

    American Journal of Political Science , volume=

    The Jackie (and Jill) Robinson effect: Why do congresswomen outperform congressmen? , author=. American Journal of Political Science , volume=. 2011 , publisher=

  8. [8]

    , author=

    Categorization, belief similarity, and intergroup discrimination. , author=. Journal of personality and social psychology , volume=. 1975 , publisher=

  9. [9]

    European Journal of Social Psychology , volume=

    Political distinctiveness: An identity optimising approach , author=. European Journal of Social Psychology , volume=. 1994 , publisher=

  10. [10]

    ACM Transactions on Management Information Systems , volume=

    Large language models for conducting advanced text Analytics Information Systems Research , author=. ACM Transactions on Management Information Systems , volume=. 2025 , publisher=

  11. [11]

    The Journal of Legislative Studies , volume=

    Lawmaking and news making: Different worlds after all? A study on news coverage of legislative processes in the Netherlands , author=. The Journal of Legislative Studies , volume=. 2015 , publisher=

  12. [12]

    The Politics of Legislative Debate , year=

    France: Talkative MPs under Control , author=. The Politics of Legislative Debate , year=

  13. [13]

    Advanced Professional Communication , publisher =

    Ashman, Melissa and Cruthers, Arley , title =. Advanced Professional Communication , publisher =. 2021 , pages =

  14. [14]

    The Journal of Politics , volume=

    Self-fulfilling misperceptions of public polarization , author=. The Journal of Politics , volume=. 2014 , publisher=

  15. [15]

    Available at SSRN 4893164 , year=

    Distrust, not Dislike , author=. Available at SSRN 4893164 , year=

  16. [16]

    Third conference on applied natural language processing , pages=

    Automatic extraction of facts from press releases to generate news stories , author=. Third conference on applied natural language processing , pages=

  17. [17]

    American Economic Review , volume=

    Political losers as a barrier to economic development , author=. American Economic Review , volume=. 2000 , publisher=

  18. [18]

    American Political Science Review , volume=

    Explaining Senate Election Outcomes , author=. American Political Science Review , volume=. 1988 , publisher=

  19. [19]

    Political Research Quarterly , volume=

    Press coverage of mayoral candidates: The role of gender in news reporting and campaign issue speech , author=. Political Research Quarterly , volume=. 2008 , publisher=

  20. [20]

    BMC Public Health , volume=

    Innovation diffusion: how homogenous networks influence the uptake of community-based injectable contraceptives , author=. BMC Public Health , volume=. 2019 , publisher=

  21. [21]

    International Conference on Human-Computer Interaction , pages=

    AI Unreliable Answers: A Case Study on ChatGPT , author=. International Conference on Human-Computer Interaction , pages=. 2023 , organization=

  22. [22]

    Be less of a slave to the news

    “Be less of a slave to the news”: A texto-material perspective on news avoidance among young adults , author=. Journalism Studies , volume=. 2021 , publisher=

  23. [23]

    American Political Science Review , volume=

    Alienation and political behavior , author=. American Political Science Review , volume=. 1969 , publisher=

  24. [24]

    American political science review , volume=

    Political trust and racial ideology , author=. American political science review , volume=. 1970 , publisher=

  25. [25]

    Science Advances , year =

    Asikainen, Aili and Iñiguez, Gerardo and Ureña-Carrión, Javier and Kaski, Kimmo and Kivelä, Mikko , title =. Science Advances , year =. doi:10.1126/sciadv.aax7310 , url =

  26. [26]

    European Journal of Politics and Gender , year =

    Abou-Chadi, Tarik and Breyer, Magdalena and Gessler, Theresa , title =. European Journal of Politics and Gender , year =. doi:10.1332/251510821x16177312096679 , url =

  27. [27]

    Brookings papers on economic activity , volume=

    The European trust crisis and the rise of populism , author=. Brookings papers on economic activity , volume=. 2017 , publisher=

  28. [28]

    JCMS: Journal of Common Market Studies , volumne =

    Between Ideology and Nationality: Drivers of Legislative Oversight in the European Parliament's Economic Dialogues , author=. JCMS: Journal of Common Market Studies , volumne =. 2022 , publisher=

  29. [29]

    LeGes , volume =

    Albrecht, Martin , title =. LeGes , volume =

  30. [30]

    Congress & the Presidency: A Journal of Capital Studies , volume=

    What's Happened to the Watchful Eye? , author=. Congress & the Presidency: A Journal of Capital Studies , volume=. 2002 , organization=

  31. [31]

    Government and opposition , volume=

    Populism and liberal democracy: Populists in government in Austria, Italy, Poland and Switzerland , author=. Government and opposition , volume=. 2013 , publisher=

  32. [32]

    American Behavioral Scientist , volume=

    Changes in congressional oversight , author=. American Behavioral Scientist , volume=. 1979 , publisher=

  33. [33]

    The European Physical Journal B , pages =

    Arenas, A and Danon, L and Díaz-Guilera, A and Gleiser P M and Guimerá, R , title =. The European Physical Journal B , pages =. 2004 , doi =

  34. [34]

    Proceedings of the National Academy of Sciences , volume=

    Preventing extreme polarization of political attitudes , author=. Proceedings of the National Academy of Sciences , volume=

  35. [35]

    Digital Government: Research and Practice , volume=

    Citizen participation and machine learning for a better democracy , author=. Digital Government: Research and Practice , volume=. 2021 , publisher=

  36. [36]

    Renewable and Sustainable Energy Reviews , volume=

    Energy choices in Alaska: Mining people's perception and attitudes from geotagged tweets , author=. Renewable and Sustainable Energy Reviews , volume=. 2020 , publisher=

  37. [37]

    Journal of applied systems analysis , volume=

    From data to wisdom , author=. Journal of applied systems analysis , volume=

  38. [38]

    Paper presented at the Annual Meetings of the Midwest Political Science Association , year=

    Bill sponsorship activity and success in Congress: Why we should change the way we study legislative effectiveness , author=. Paper presented at the Annual Meetings of the Midwest Political Science Association , year=

  39. [39]

    The scope and urgency of legislation: Reconsidering bill success in the house of representatives , author=

  40. [40]

    New voices in the study of democracy in Latin America , pages=

    Analyzing legislative success in Latin America: The case of democratic Argentina , author=. New voices in the study of democracy in Latin America , pages=

  41. [41]

    Notizen eines Schweizer Parlamentariers

    Erlebtes Parlament. Notizen eines Schweizer Parlamentariers. , author=. 1984 , address =

  42. [42]

    Ethology , volume=

    Spatial proximity between newborns influences the development of social relationships in bats , author=. Ethology , volume=. 2012 , publisher=

  43. [43]

    Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 , pages=

    Attributed graph clustering: An attribute-aware graph embedding approach , author=. Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017 , pages=

  44. [44]

    and III, Charles Stewart , doi =

    Ansolabehere, Stephen and Snyder, James M. and III, Charles Stewart , doi =. Legislative Studies Quarterly , volume =

  45. [45]

    Proceedings of BioNLP 2011 workshop , pages=

    Medical entity recognition: a comparaison of semantic and statistical methods , author=. Proceedings of BioNLP 2011 workshop , pages=

  46. [46]

    Science , volume=

    Group organization among vertebrates , author=. Science , volume=. 1942 , publisher=

  47. [47]

    Forthcoming , journal =

    Angst, Mario and Brandenberger, Laurence , title =. Forthcoming , journal =

  48. [48]

    Democratization , volume=

    United we stand, divided we rule: how political polarization erodes democracy , author=. Democratization , volume=. 2021 , publisher=

  49. [49]

    2008 , publisher=

    Mostly harmless econometrics: An empiricist's companion , author=. 2008 , publisher=

  50. [50]

    Quality & Quantity , volume=

    Measuring populist discourse with semantic text analysis: an application on grassroots populist mobilization , author=. Quality & Quantity , volume=. 2018 , publisher=

  51. [51]

    PloS one , volume=

    The rise of partisanship and super-cooperators in the US House of Representatives , author=. PloS one , volume=. 2015 , publisher=

  52. [52]

    The Journal of Politics , volume=

    Is polarization a myth? , author=. The Journal of Politics , volume=. 2008 , publisher=

  53. [53]

    The Journal of politics , volume=

    Incumbency, redistricting, and the decline of competition in US House elections , author=. The Journal of politics , volume=. 2006 , publisher=

  54. [54]

    Journal of the American Statistical Association , volume=

    Improving and evaluating topic models and other models of text , author=. Journal of the American Statistical Association , volume=. 2016 , publisher=

  55. [55]

    American Political Science Review , volume=

    The learning of legislative norms , author=. American Political Science Review , volume=. 1973 , publisher=

  56. [56]

    2006 , publisher=

    The evolution of cooperation , author=. 2006 , publisher=

  57. [57]

    Procedia-Social and Behavioral Sciences , volume=

    Reciprocity, social curation and the emergence of blogging: A study in community formation , author=. Procedia-Social and Behavioral Sciences , volume=. 2011 , publisher=

  58. [58]

    Social Networks , volume=

    Introduction to the special issue on advances in two-mode social networks , author=. Social Networks , volume=

  59. [59]

    Policy studies journal , volume=

    Network Dynamics in Natural Resource Governance: A Case Study of S wiss Landscape Management , author=. Policy studies journal , volume=. 2017 , publisher=

  60. [60]

    Governance and Politics of the Netherlands , address =

    Andeweg, Rudy and Irwin, Galen , year =. Governance and Politics of the Netherlands , address =

  61. [61]

    Journal of Economic Behavior & Organization , volume=

    Institutions as cognitive media between strategic interactions and individual beliefs , author=. Journal of Economic Behavior & Organization , volume=. 2011 , publisher=

  62. [62]

    British journal of political science , volume=

    Strong states and weak states: Sectoral policy networks in advanced capitalist economies , author=. British journal of political science , volume=. 1989 , publisher=

  63. [63]

    Canadian Journal of Political Science/Revue canadienne de science politique , pages=

    Selecting policy instruments: Neo-institutional and rational choice interpretations of automobile insurance in Ontario , author=. Canadian Journal of Political Science/Revue canadienne de science politique , pages=. 1989 , publisher=

  64. [64]

    The Annals of Statistics , volume =

    Cox's regression model for counting processes: a large sample study , author=. The Annals of Statistics , volume =. 1982 , publisher=

  65. [65]

    Second Edition , author =

    Event History and Survival Analysis. Second Edition , author =. 2014 , publisher =

  66. [66]

    Sociological methodology , volume=

    Discrete-time methods for the analysis of event histories , author=. Sociological methodology , volume=. 1982 , publisher=

  67. [67]

    Political Science Quarterly , volume=

    The transition to Republican rule in the House: Implications for theories of congressional politics , author=. Political Science Quarterly , volume=. 1997 , publisher=

  68. [68]

    Academy of Management Review , volume=

    Boundary spanning roles and organization structure , author=. Academy of Management Review , volume=. 1977 , publisher=

  69. [69]

    Proceedings of the IEEE , volume=

    Keeping the energy debate clean: How do we supply the world's energy needs? , author=. Proceedings of the IEEE , volume=. 2010 , publisher=

  70. [70]

    1993 , publisher=

    Organizations working together , author=. 1993 , publisher=

  71. [71]

    Theories of the policy process , editor =

    The network approach , author=. Theories of the policy process , editor =. 2007 , publisher=

  72. [72]

    Journal of European public policy , volume=

    Comparative analysis of policy networks in Western Europe , author=. Journal of European public policy , volume=. 2006 , publisher=

  73. [73]

    Collaborative Governance in Theory and Practice

    Ansell, Chris and Gash, Alison , year =. Collaborative Governance in Theory and Practice. , pages =

  74. [74]

    Asserted rights: Rule activation Asserted Rights: Rule Activation Strategies in Water User Rivalries in Belgium and Switzerland , pages =

    Aubin, David , year =. Asserted rights: Rule activation Asserted Rights: Rule Activation Strategies in Water User Rivalries in Belgium and Switzerland , pages =

  75. [75]

    2016 , pages =

    Abisheva, Adiya and Garcia, David and Schweitzer, Frank , journal =. 2016 , pages =

  76. [76]

    Legislative Studies Quarterly , mendeley-groups =

    Alem. Legislative Studies Quarterly , mendeley-groups =

  77. [77]

    doi:10.1145/2556195.2566588 , eprint =

    Abisheva, Adiya and Garimella, Venkata Rama Kiran and Garcia, David and Weber, Ingmar , booktitle =. doi:10.1145/2556195.2566588 , eprint =

  78. [78]

    EPJ Data Science , volume=

    Sentiment cascades in the 15M movement , author=. EPJ Data Science , volume=. 2015 , publisher=

  79. [79]

    Small Group Behavior , volume=

    Group categorization and attribution of belief similarity , author=. Small Group Behavior , volume=. 1979 , publisher=

  80. [80]

    Annual Review of Statistics and Its Application , volume=

    Relational event modeling , author=. Annual Review of Statistics and Its Application , volume=. 2024 , publisher=

Showing first 80 references.