Introduces Situated Interaction Auditing (SIA) to examine how user sociodemographic signals affect LLM response quality, content, and tone in personal interactions.
hub
Taxonomy of risks posed by language models
26 Pith papers cite this work, alongside 21 external citations. Polarity classification is still indexing.
26
Pith papers citing it
21
external citations · external index
hub tools
citation-role summary
background 4
citation-polarity summary
roles
background 4representative citing papers
PhantomBench is a new benchmark of 60K+ non-existent terms showing language models hallucinate at rates up to 86.7 percent even when inputs assume the concepts exist.
Controlled ablation finds Popperian code-generation skill adds no separable correctness benefit over labels-only scaffold; gains track structure not content.
VLMs preserve linearly separable visual magnitudes and can compare them, yet collapse at symbolic mapping because visual and textual number spaces remain fractured and disjoint.
FeDa4Fair is a new library and benchmark for creating federated datasets with heterogeneous client-level biases to standardize evaluation of fairness methods in federated learning.
Smoothie performs diffusion by smoothing token embeddings based on semantic similarity, outperforming prior diffusion models on sequence-to-sequence and unconditional text generation tasks.
LLMs default to responses more similar to opinions from the USA and some European and South American countries; prompting for a country shifts alignment but can introduce stereotypes, while translation does not reliably match language speakers.
The authors release BioStance, a dataset of 39,600 context-preserving Reddit post-comment pairs annotated for favor, against, or none stance on six bioethical controversy targets, with Krippendorff's alpha of 0.82.
LLM counter-arguments in ChangeMyView convey more trust and social-intent signals than human opinion-changing comments, and crowdworkers prefer them as more persuasive.
An adversarial methodology generates a multilingual cross-platform dataset of paired human-AI social messages, and models trained on it outperform prior detectors on real-world out-of-distribution data.
Exploratory interview study with 17 developers identifies four forms of emergent oversight work for software agents and documents situated challenges and heuristics.
A vector generalization of fusion-fission group dynamics from physics forecasts when AI behavior shifts to undesirable states, validated at 90 percent across seven models and prior to real-world data.
TreeTracer aggregates stochastic LLM generations into syntax-aligned Sankey trees with contrastive inference to visualize and quantify hidden representational biases across demographic ontologies.
Explicit demographic statements trigger higher refusal rates and lower semantic similarity in LLMs than implicit dialect cues, which reduce refusals but also reduce content sanitization.
Each tested LLM shows its own characteristic unreliability when engaging in repair during extended math-question dialogues.
BLOOM is a 176B-parameter open-access multilingual language model trained on the ROOTS corpus that achieves competitive performance on benchmarks, with improved results after multitask prompted finetuning.
LISE decomposes pretrained speaker embeddings into components that preserve ASV performance with negligible EER degradation and enable listeners to distinguish speakers at 83.9% accuracy.
SafeMoE isolates unsafe knowledge in domain-specific LoRA experts and routes them via a lightweight gate trained on safe responses to produce safer and more informative LLM outputs with zero-shot generalization.
Introduces the Mechanism Plausibility Scale, a four-level framework separating generative sufficiency from mechanistic plausibility in LLM-based agent-based models.
Generative AI boosts attackers' ability to create harmful content at scale while also enabling defenders to detect threats, support users, and improve moderation processes.
Proposes a modular architecture for LLM-based wellbeing recommenders using explicit constraints on guidance, explanations, directness, and user control to address trust calibration, intent alignment, and consequence awareness.
State-of-the-art LLMs respond inconsistently to queries from protected-group personas, with some responses omitting key information that should be provided.
Literature on system prompts for AI shows fragmented and contradictory claims that complicate policy efforts to use them as reliable governance mechanisms.
Authors introduce MLM and CLM specialization methods that avoid memorizing identifiers in sensitive training data while aiming for a privacy-utility tradeoff on medical datasets.
citing papers explorer
-
Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research
Introduces Situated Interaction Auditing (SIA) to examine how user sociodemographic signals affect LLM response quality, content, and tone in personal interactions.
-
PhantomBench: Benchmarking the Non-existential Threat of Language Models
PhantomBench is a new benchmark of 60K+ non-existent terms showing language models hallucinate at rates up to 86.7 percent even when inputs assume the concepts exist.
-
Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skill
Controlled ablation finds Popperian code-generation skill adds no separable correctness benefit over labels-only scaffold; gains track structure not content.
-
Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms
VLMs preserve linearly separable visual magnitudes and can compare them, yet collapse at symbolic mapping because visual and textual number spaces remain fractured and disjoint.
-
FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluation
FeDa4Fair is a new library and benchmark for creating federated datasets with heterogeneous client-level biases to standardize evaluation of fairness methods in federated learning.
-
Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation
Smoothie performs diffusion by smoothing token embeddings based on semantic similarity, outperforming prior diffusion models on sequence-to-sequence and unconditional text generation tasks.
-
Towards Measuring the Representation of Subjective Global Opinions in Language Models
LLMs default to responses more similar to opinions from the USA and some European and South American countries; prompting for a country shifts alignment but can introduce stereotypes, while translation does not reliably match language speakers.
-
A Context-Aware Dataset for Stance Detection in Bioethical Controversies on Reddit
The authors release BioStance, a dataset of 39,600 context-preserving Reddit post-comment pairs annotated for favor, against, or none stance on six bioethical controversy targets, with Krippendorff's alpha of 0.82.
-
"I understand your perspective": LLM Persuasion through the Lens of Communicative Action Theory
LLM counter-arguments in ChangeMyView convey more trust and social-intent signals than human opinion-changing comments, and crowdworkers prefer them as more persuasive.
-
Adversarial Creation and Detection of AI-Generated Social Bot Content
An adversarial methodology generates a multilingual cross-platform dataset of paired human-AI social messages, and models trained on it outperform prior detectors on real-world out-of-distribution data.
-
Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
Exploratory interview study with 17 developers identifies four forms of emergent oversight work for software agents and documents situated challenges and heuristics.
-
Fusion-fission forecasts when AI will shift to undesirable behavior
A vector generalization of fusion-fission group dynamics from physics forecasts when AI behavior shifts to undesirable states, validated at 90 percent across seven models and prior to real-world data.
-
Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation
TreeTracer aggregates stochastic LLM generations into syntax-aligned Sankey trees with contrastive inference to visualize and quantify hidden representational biases across demographic ontologies.
-
Dialect vs Demographics: Quantifying LLM Bias from Implicit Linguistic Signals vs. Explicit User Profiles
Explicit demographic statements trigger higher refusal rates and lower semantic similarity in LLMs than implicit dialect cues, which reduce refusals but also reduce content sanitization.
-
Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs
Each tested LLM shows its own characteristic unreliability when engaging in repair during extended math-question dialogues.
-
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
BLOOM is a 176B-parameter open-access multilingual language model trained on the ROOTS corpus that achieves competitive performance on benchmarks, with improved results after multitask prompted finetuning.
-
LISE : Listenable Interpretable Speaker Embeddings
LISE decomposes pretrained speaker embeddings into components that preserve ASV performance with negligible EER degradation and enable listeners to distinguish speakers at 83.9% accuracy.
-
Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
SafeMoE isolates unsafe knowledge in domain-specific LoRA experts and routes them via a lightweight gate trained on safe responses to produce safer and more informative LLM outputs with zero-shot generalization.
-
Mechanism Plausibility in Generative Agent-Based Modeling
Introduces the Mechanism Plausibility Scale, a four-level framework separating generative sufficiency from mechanistic plausibility in LLM-based agent-based models.
-
How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape
Generative AI boosts attackers' ability to create harmful content at scale while also enabling defenders to detect threats, support users, and improve moderation processes.
-
Designing Trustworthy LLM-based Wellbeing Recommendation through Controllable Interaction
Proposes a modular architecture for LLM-based wellbeing recommenders using explicit constraints on guidance, explanations, directness, and user control to address trust calibration, intent alignment, and consequence awareness.
-
Discriminatory Compliance: How LLMs Answer Queries from Protected Groups
State-of-the-art LLMs respond inconsistently to queries from protected-group personas, with some responses omitting key information that should be provided.
-
Prompt Governance? On Governing Technologies Governed by Natural Language
Literature on system prompts for AI shows fragmented and contradictory claims that complicate policy efforts to use them as reliable governance mechanisms.
-
Towards the Anonymization of the Language Modeling
Authors introduce MLM and CLM specialization methods that avoid memorizing identifiers in sensitive training data while aiming for a privacy-utility tradeoff on medical datasets.
-
LLM Harms: A Taxonomy and Discussion
Proposes a five-bucket taxonomy of LLM harms and calls for dynamic auditing, but the systematic review behind it is not reproducible and contains mismatched citations.
-
Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles
A scoping review of AIES and FAccT literature concludes that AI trustworthiness research prioritizes technical precision over social, ethical, and institutional factors, leaving the sociotechnical nature of AI systems underexplored.