Explainability-in-Action: Enabling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI
Reviewed by Pithpith:K2NXAW2Ropen to challenge →
read the original abstract
Explainable AI (XAI) in creative contexts can go beyond transparency to support artistic engagement, modifiability, and sustained practice. While curated datasets and training human-scale models can offer artists greater agency and control, large-scale generative models like text-to-image diffusion systems often obscure these possibilities. We suggest that even large models can be treated as creative materials if their internal structure is exposed and manipulable. We propose a craft-based approach to explainability rooted in long-term, hands-on engagement akin to Sch\"on's "reflection-in-action" and demonstrate its application through a model-bending and inspection plugin integrated into the node-based interface of ComfyUI. We demonstrate that by interactively manipulating different parts of a generative model, artists can develop an intuition about how each component influences the output.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
AttentionBender: Manipulating Cross-Attention in Video Diffusion Transformers as a Creative Probe
AttentionBender applies 2D transforms to cross-attention maps in video diffusion transformers, producing distributed distortions and glitch aesthetics that reveal entangled attention mechanisms while serving as both a...
-
Real-Time AttentionBender: Granular Interactive Network Bending of Video Diffusion Transformers
Real-Time AttentionBender enables live, granular manipulation of self-attention, cross-attention, and feed-forward layers in video diffusion transformers during generation.
-
How Creatives Approach GenAI Image Generation: Tensions Between Structured Guidance, Self-Experimentation, and Creative Autonomy
Creatives prefer self-experimentation over structured guidance for GenAI image tools to preserve creative freedom, even when guidance aids AI literacy.
-
How Creatives Approach GenAI Image Generation: Tensions Between Structured Guidance, Self-Experimentation, and Creative Autonomy
Qualitative studies show creatives prefer self-experimentation over structured guidance for GenAI image tools to preserve creative autonomy despite terminology barriers.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.