Interactive AI and Human Behavior: Challenges and Pathways for AI Governance

Cagatay Turkay; Daniel Bogiatzis-Gibbons; Yulu Pi

REVIEW 3 major objections 3 minor 6 references

Interactive AI and Human Behavior: Challenges and Pathways for AI Governance

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Interactive AI systems build long-term relationships with users, and this paper argues that AI governance must be rebuilt around evidence of how those relationships change people over time rather than around static risk rules.

desk verdict A clear, honest workshop-based position paper that names a real gap—behavioral evidence for relational AI—but its outcome-focused regulatory proposal is an agenda, not yet a framework. read the letter →

arxiv 2508.16608 v1 pith:DYWKQQBM submitted 2025-08-12 cs.CY cs.HC

Yulu Pi , Cagatay Turkay , Daniel Bogiatzis-Gibbons This is my paper · ORCID

classification cs.CYcs.HC

keywords interactiveAIgovernancebehavioralinsightshuman-AIrelationshipsoutcome-focusedregulationlongitudinalmethodsinteraction-centricknowledgesafety

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that interactive AI systems—chatbots and agents that remember users, form ongoing relationships, and act on their behalf—create harms that grow slowly through sustained engagement rather than through isolated incidents. Because current governance leans on rule-based and principle-based regulation plus pre-deployment risk assessment, it is poorly equipped to see those slow harms. The paper's central proposal is 'interaction-centric knowledge': longitudinal, mixed-method evidence about how human-AI interaction evolves and shapes behavior, feeding an outcome-focused regulatory model that monitors user autonomy, cognition, and well-being. It develops the proposal from a small interdisciplinary workshop and argues that behavioral research itself must change, combining long-term studies, participatory methods, and living evidence reviews.

What carries the argument

The carrying object is 'interaction-centric knowledge,' defined as evidence-based insight from real-world experience about how human-AI interactions evolve and shape behavior over time. It has three dimensions: users' engagement patterns, the system's adaptive responses to users, and the wider individual and societal outcomes of that co-evolution. This concept does the argument's work by turning interaction itself, rather than system capability or a one-time risk profile, into the object of governance. Methodologically, it is operationalized through longitudinal mixed-method studies and living evidence reviews; institutionally, it is paired with an outcome-focused regulatory model that monit

What would settle it

A multi-year longitudinal cohort study of people who use memory-enabled interactive AI daily, measuring autonomy, critical thinking, emotional dependence, and social functioning against matched controls, would settle the core premise: if heavy use shows no sustained divergence over two or more years, the claim that relational AI produces slow, governance-relevant harms is not supported. Conversely, if short-term six-week studies reproduce all long-term findings, the paper's case for discarding snapshot methods would weaken.

Watch

Extended reading notes

Core claim

On the paper's own terms, effective governance of interactive AI requires interaction-centric knowledge: evidence-based insights drawn from real-world experience about how human-AI interaction evolves and shapes behavior over time. The discovery is a three-part knowledge object: patterns of user engagement, the adaptive responses of AI systems to users, and the wider individual and societal implications of that co-evolution. Existing regulatory models and current behavioral methods both fail because they treat human-AI interaction as a static snapshot; the paper instead proposes outcome-focused regulation, modeled on a financial regulator's Consumer Duty approach, that defines outcome metric

Load-bearing premise

The argument's empirical footing is a single small, UK-centred workshop whose participants were mostly AI-safety and critical-theory researchers rather than AI builders; if a differently composed group would have produced substantially different themes, the generalized governance recommendations lose their evidence base.

Editorial extensions

If this is right

Regulators should define and track outcome metrics—user autonomy, cognitive development, psychological well-being—for interactive AI systems instead of relying only on capability tests or static risk categories.
Governments and funders should prioritize long-term, real-world studies of human-AI interaction, pairing API interaction logs with interviews, diaries, and ethnographic observation.
Open-access, regularly updated living evidence reviews should become a standard mechanism for translating behavioral findings into policy.
Behavioral AI research should standardize reporting of model specifications, system updates, and experimental protocols so findings can be replicated as systems change.
Policymakers and behavioral researchers should build proactive, sustained relationships, learning from the way the AI safety community aligned evidence with policymakers' preference for quantitative, actionable, modular interventions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the paper is right, independent research access to real interaction logs becomes a governance precondition; without vendor transparency, no outside body can produce the longitudinal behavioral evidence the model demands.
The outcome-focused model is only as strong as its measures: 'autonomy' and 'well-being' need operational definitions, or the approach could be captured by convenient, industry-friendly indicators.
A testable extension is that relational harms will concentrate unevenly—younger users and emotionally vulnerable users likely show faster autonomy and social-skill shifts—which would let regulators target oversight rather than apply it uniformly.
The same logic could be written into procurement: public institutions buying interactive AI could contractually require longitudinal outcome monitoring and data-sharing as a condition of deployment, not just pre-launch evaluation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a clear, honest workshop-based position paper that names a real gap: governance of interactive AI needs longitudinal, behavioral evidence. Its outcome-focused regulatory proposal is an agenda, not yet a worked framework.

What is new: it connects behavioral public policy to relational AI, coining 'interaction-centric knowledge' as a way to shift evidence gathering toward real-world, long-term interactions. The workshop methods are described transparently: 13 purposively sampled UK-based participants, reflexive thematic analysis with independent review, participant feedback, and an OSF appendix with materials. The authors disclose the small sample and UK-centricity in Section 6 and do not overclaim consensus. That is credible qualitative practice for a position paper.

The soft spots are real but proportionate. The empirical grounding for the headline harms mostly comes from cited literature and workshop impressions, not new data. That is acceptable for an agenda-setting paper, but not for an empirical contribution. The more load-bearing gap is the FCA analogy: the paper asserts that outcome-focused regulation, as in the FCA Consumer Duty, transfers to AI, but gives no evidence that the FCA model has improved measurable consumer outcomes, nor how 'user autonomy' or 'psychological well-being' would actually be defined, measured, monitored, and protected from gaming. The three bullet possibilities are headings, not instruments. So the central policy recommendation is a research call, not a ready-to-adopt framework.

The citation pattern is broad and appropriate; the one self-citation supports a side point and is not load-bearing.

Overall: the paper is serious, honest, and worth engaging. It deserves a serious referee because it proposes a plausible shift in how AI governance should gather evidence, and it says clearly what is missing. I would bring it to a reading group and would cite it as an agenda reference, while noting the FCA analogy needs testing.

Referee Report

3 major / 3 minor

Summary. The paper argues that 'interactive AI'—systems that adapt, remember, and form long-term relationships with users—creates risks that emerge through sustained relational engagement and are poorly captured by rule-based or principle-based regulation. To support this claim, it draws on a one-day interdisciplinary workshop held in the UK with 13 participants (policymakers, behavioral scientists, HCI researchers, civil-society practitioners). The workshop is used to identify governance risks (emotional manipulation, autonomy erosion, long-term cognitive and social harms), methodological gaps in studying dynamic human-AI interactions (artificial settings, temporality, context sensitivity, replication), and pathways for translating behavioral insights into policy. The paper recommends longitudinal mixed-method research, living evidence reviews, participatory co-production, proactive policy engagement, and a shift toward outcome-focused regulation. It introduces 'interaction-centric knowledge' as the proposed epistemic foundation for human-centric AI governance.

Significance. If the broad recommendation were operationalized, the paper would support a substantial reorientation of AI governance from static pre-deployment risk assessment to ongoing, outcome-focused evaluation of human-AI interaction. The paper has real strengths: the qualitative methodology is described transparently in Section 3, the limitations are acknowledged explicitly in Section 6, and the proposal to connect behavioral science with AI governance is timely and policy-relevant. The framework of 'interaction-centric knowledge' usefully names a gap. However, the significance is conditional on two unresolved issues: the evidence base is a single, small, UK-centric workshop, and the proposed outcome-focused regulatory model is not yet operationalized. As written, the paper is a plausible position statement rather than a fully supported governance framework.

major comments (3)

[Section 5, 'Focusing on Long-Term Effects and Outcomes'] The central policy recommendation rests on an analogy to the UK FCA Consumer Duty, but no evidence is provided that this regulatory model has improved measurable consumer outcomes. The cited documents (Financial Conduct Authority 2022, 2024) are primary regulatory materials, and the only secondary source is a non-academic explainer ('Initiatives 2024'). The three bullet 'possibilities'—defining outcome metrics, creating sandboxes, monitoring and evaluating outcomes—are headings, not an operational framework. No definitions are given for 'user autonomy', 'cognitive development', or 'psychological well-being', and there is no account of measurement timeframes, data sources, baselines, enforcement mechanisms, or resistance to gaming. Because this section is the paper's main answer to the governance gap, the recommendation is currently an assertion rather than a concrete, testable pathway.
[Section 3 and Section 6] The empirical grounding for the paper's generalized claims is a single purposive workshop of 13 participants. The paper itself discloses in Section 3 that 'most participants were focused on AI safety or critical theory around AI, rather than actively building AI tools' and in Section 6 that the findings are UK-centric and shaped by the authors' interpretive lens. Yet the abstract and Section 5 generalize to 'AI governance' and 'interactive AI systems' as a class without qualification. This mismatch is load-bearing because the workshop themes constitute the evidence base for the recommendations. The authors should either explicitly scope the contribution as an exploratory UK-based pilot or triangulate the themes with a wider empirical literature (e.g., systematic reviews of HCI studies, cross-cultural data, and developer perspectives).
[Section 5, 'Evidence on Human behavior is Essential'] The concept of 'interaction-centric knowledge' is introduced as the core epistemic foundation for governance, but its three dimensions are broad categories without specification of how they are measured, validated, or synthesized into policy decisions. The paper calls for longitudinal studies and living evidence reviews, but does not address who maintains these reviews, what inclusion/quality standards apply, or how the rapidly evolving and non-deterministic character of interactive AI—acknowledged in Section 4.2—can be reconciled with a stable, actionable knowledge base. As written, the term names a need rather than provides an operational pathway. This is a load-bearing gap because the paper's own argument is that governance should be grounded in exactly this kind of knowledge.

minor comments (3)

[Section 5] Typographical issues: 'Interacive' should be 'Interactive'; 'sandboxs' should be 'sandboxes'. Proofreading throughout would improve readability.
[Section 3] The total number of workshop participants appears only implicitly through Table 1 (5+4+4). The text says the workshop was 'intentionally kept small' but does not state the exact number in prose; stating it explicitly would strengthen transparency.
[Section 4.3] In the subsection 'Reflections on AI Safety Community's Successes', the phrase 'the pre-existing cultural sensation might helped attract' contains a verb-form error; also, the discussion of the AI safety community's influence would benefit from a clearer distinction between strategic success and normative desirability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's governance argument is grounded in workshop data and external literature, not derived from its own conclusions.

full rationale

I walked the paper's derivation chain. There are no equations, fitted parameters, or quantitative predictions, so the standard reduction-by-construction patterns do not apply. The central move is: workshop discussion -> thematic synthesis -> governance recommendation. This is an empirical-interpretive chain, and the paper explicitly disclaims that the themes are a direct representation of consensus (Section 3), noting the authors' interpretive role and the UK-centric, small, expert-skewed sample (Sections 3 and 6). Those are evidentiary limitations, not circularity. The proposed 'interaction-centric knowledge' is introduced by definition ('effective governance of interactive AI requires what we term interaction-centric knowledge—evidence-based insights drawn from real-world experiences...') and then argued for; the definition does not secretly contain the conclusion, and no empirical claim is being fitted to itself. The FCA Consumer Duty analogy in Section 5 is an external benchmark; whether the analogy holds is a correctness/evidentiary question, not a circular one. The only self-citation is Bogiatzis-Gibbons 2024, used in Section 5 to support public participation for democratic control of AI. This is a minor supporting citation; removing it leaves the workshop-based case for participatory governance intact, so it is not load-bearing. Hence no circularity is established by the quoted material.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

Qualitative policy paper; no numerical fits, so no free parameters. The central claim rests on four background assumptions that are stated or implicit: the generalizability of a 13-person UK workshop, the reality and scale of the harms motivating reform, the transferability of an outcomes-based regulatory model, and the feasibility of translating context-specific behavioral evidence into generalizable policy. All are domain assumptions rather than mathematical axioms; the paper explicitly flags the first and partially discusses the last.

assumptions (4)

domain assumption Insights from a 13-person, UK-centric expert workshop generalize enough to support global governance recommendations.
The evidence base is the workshop synthesis in Sections 3 and 4; the paper concedes in Section 6 that participants were predominantly UK-based, yet the recommendations are stated without geographic scope.
domain assumption The harms motivating reform (cognitive erosion, over-reliance, emotional dependence) occur at meaningful scale and are attributable to interactive AI.
Supported in Section 2 mainly by correlational or preliminary studies (e.g., Gerlich 2025; Kim and McGill 2025) and workshop opinion, not by the longitudinal causal evidence the paper itself calls for.
domain assumption An outcome-focused regulatory model (UK FCA Consumer Duty) transfers to interactive-AI governance despite undefined outcome metrics.
Section 5 proposes metrics for autonomy, cognition, and well-being but does not define them or show they are measurable; transfer rests on analogy.
domain assumption Behavioral evidence can be translated into policy despite the tension between context-specific HCI findings and the generalizable evidence policy requires.
Section 4 (Translating Behavioral Insights) itself describes this tension and cites Spaa et al. 2019; the central proposal depends on resolving it.

invented entities (1)

interaction-centric knowledge
purpose: Coined in Section 5 as the epistemic foundation for human-centered AI governance: evidence on how human-AI engagement patterns evolve, how systems adapt to users, and the societal implications; used to justify reorienting governance toward real-world behavioral evidence.
Defined by the paper itself with no falsifiable handle outside the text; it is a conceptual and rhetorical framing built from prior notions (situated interaction, relational autonomy) plus policy relevance, not an empirically detectable entity. Its value will be settled by institutional uptake and the quality of future studies it motivates.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interactive AI and Human Behavior: Challenges and Pathways for AI Governance." pith.science (2026). https://pith.science/paper/DYWKQQBM

@misc{pith2026250816608,
  author       = {Pith},
  title        = {Pith review of: Interactive AI and Human Behavior: Challenges and Pathways for AI Governance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DYWKQQBM}},
  note         = {Machine review of arXiv:2508.16608}
}

read the original abstract

As Generative AI systems increasingly engage in long-term, personal, and relational interactions, human-AI engagements are becoming significantly complex, making them more challenging to understand and govern. These Interactive AI systems adapt to users over time, build ongoing relationships, and even can take proactive actions on behalf of users. This new paradigm requires us to rethink how such human-AI interactions can be studied effectively to inform governance and policy development. In this paper, we draw on insights from a collaborative interdisciplinary workshop with policymakers, behavioral scientists, Human-Computer Interaction researchers, and civil society practitioners, to identify challenges and methodological opportunities arising within new forms of human-AI interactions. Based on these insights, we discuss an outcome-focused regulatory approach that integrates behavioral insights to address both the risks and benefits of emerging human-AI relationships. In particular, we emphasize the need for new methods to study the fluid, dynamic, and context-dependent nature of these interactions. We provide practical recommendations for developing human-centric AI governance, informed by behavioral insights, that can respond to the complexities of Interactive AI systems.

Figures

Figures reproduced from arXiv: 2508.16608 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

6 extracted references · 2 canonical work pages

[5]

Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support

Navigating The Future Of Education: The Impact Of Artiﬁcial Intelligence On Teacher-Student Dynamics.Edu- cational Administration: Theory and Practice, 30(4). Haghani, M. 2023. The notion of validity in experimental crowd dynamics.International Journal of Disaster Risk Re- duction, 93: 103750. Hallsworth, M. 2023. A Manifesto for Applying Be- havioural Sc...

work page Pith review arXiv 2023
[6]

Lichand, G.; Serdeira, A.; and Rizardi, B

Personal LLM Agents: Insights and Survey about the Capability, Efﬁciency and Security.ArXiv, abs/2401.05459. Lichand, G.; Serdeira, A.; and Rizardi, B. 2023. Motivating the Use of Behavioral Insights. InBehavioral Insights for Policy Design. Springer. Liu, X.; Y u, H.; Zhang, H.; Xu, Y .; Lei, X.; Lai, H.; Gu, Y .; Ding, H.; Men, K.; Y ang, K.; Zhang, S.;...

arXiv 2023
[2001]

Personal and Ubiquitous Computing, 5(1): 1–3

Situated Interaction and Context-Aware Computing. Personal and Ubiquitous Computing, 5(1): 1–3. Dezfouli, A.; Nock, R.; and Dayan, P . 2020. Adversarial vulnerabilities of human decision-making.Proceedings of the National Academy of Sciences, 117(46): 29221–29228. Durante, Z.; Huang, Q.; Wake, N.; Gong, R.; Park, J. S.; Sarkar, B.; Taori, R.; Noda, Y .; T...

arXiv 2020
[2019]

Technical report, Centre for International Governance Innovation

Models for Platform Governance. Technical report, Centre for International Governance Innovation. Financial Conduct Authority. 2022. Consumer Duty. Ac- cessed: 2025-05-06. Financial Conduct Authority. 2024. FCA AI Update. Ac- cessed: 2025-07-22. Gabriel, I.; Manzini, A.; Keeling, G.; Hendricks, L. A.; Rieser, V .; Iqbal, H.; Tomaˇsev, N.; Ktena, I.; Kento...

arXiv 2022
[2023]

InPro- ceedings of the F ourth Workshop on Insights from Negative Results in NLP, 1–10

Missing Information, Unresponsive Authors, Exper- imental Flaws: The Impossibility of Assessing the Repro- ducibility of Previous Human Evaluations in NLP. InPro- ceedings of the F ourth Workshop on Insights from Negative Results in NLP, 1–10. Dubrovnik, Croatia: Association for Computational Linguistics. Berlyne, D. E. 1975. Behaviourism? Cognitive theor...

arXiv 1975
[2024]

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24

How Culture Shapes What People Want From AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24. New Y ork, NY , USA: As- sociation for Computing Machinery. ISBN 9798400703300. Gerlich, M. 2025. AI Tools in Society: Impacts on Cognitive Ofﬂoading and the Future of Critical Thinking.Societies, 15(1). Gerlitz, C.; and H...

work page 2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [5]

Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support

Navigating The Future Of Education: The Impact Of Artiﬁcial Intelligence On Teacher-Student Dynamics.Edu- cational Administration: Theory and Practice, 30(4). Haghani, M. 2023. The notion of validity in experimental crowd dynamics.International Journal of Disaster Risk Re- duction, 93: 103750. Hallsworth, M. 2023. A Manifesto for Applying Be- havioural Sc...

work page Pith review arXiv 2023

[2] [6]

Lichand, G.; Serdeira, A.; and Rizardi, B

Personal LLM Agents: Insights and Survey about the Capability, Efﬁciency and Security.ArXiv, abs/2401.05459. Lichand, G.; Serdeira, A.; and Rizardi, B. 2023. Motivating the Use of Behavioral Insights. InBehavioral Insights for Policy Design. Springer. Liu, X.; Y u, H.; Zhang, H.; Xu, Y .; Lei, X.; Lai, H.; Gu, Y .; Ding, H.; Men, K.; Y ang, K.; Zhang, S.;...

arXiv 2023

[3] [2001]

Personal and Ubiquitous Computing, 5(1): 1–3

Situated Interaction and Context-Aware Computing. Personal and Ubiquitous Computing, 5(1): 1–3. Dezfouli, A.; Nock, R.; and Dayan, P . 2020. Adversarial vulnerabilities of human decision-making.Proceedings of the National Academy of Sciences, 117(46): 29221–29228. Durante, Z.; Huang, Q.; Wake, N.; Gong, R.; Park, J. S.; Sarkar, B.; Taori, R.; Noda, Y .; T...

arXiv 2020

[4] [2019]

Technical report, Centre for International Governance Innovation

Models for Platform Governance. Technical report, Centre for International Governance Innovation. Financial Conduct Authority. 2022. Consumer Duty. Ac- cessed: 2025-05-06. Financial Conduct Authority. 2024. FCA AI Update. Ac- cessed: 2025-07-22. Gabriel, I.; Manzini, A.; Keeling, G.; Hendricks, L. A.; Rieser, V .; Iqbal, H.; Tomaˇsev, N.; Ktena, I.; Kento...

arXiv 2022

[5] [2023]

InPro- ceedings of the F ourth Workshop on Insights from Negative Results in NLP, 1–10

Missing Information, Unresponsive Authors, Exper- imental Flaws: The Impossibility of Assessing the Repro- ducibility of Previous Human Evaluations in NLP. InPro- ceedings of the F ourth Workshop on Insights from Negative Results in NLP, 1–10. Dubrovnik, Croatia: Association for Computational Linguistics. Berlyne, D. E. 1975. Behaviourism? Cognitive theor...

arXiv 1975

[6] [2024]

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24

How Culture Shapes What People Want From AI. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24. New Y ork, NY , USA: As- sociation for Computing Machinery. ISBN 9798400703300. Gerlich, M. 2025. AI Tools in Society: Impacts on Cognitive Ofﬂoading and the Future of Critical Thinking.Societies, 15(1). Gerlitz, C.; and H...

work page 2024