Schemex: Discovering Structural Abstractions from Examples
Pith reviewed 2026-05-22 20:58 UTC · model grok-4.3
The pith
Schemex decomposes schema induction into clustering examples, abstracting schemas, and contrastive refinement to yield more actionable structures.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Schemex is an interactive AI workflow that systematically supports schema induction by decomposing it into three tractable stages: clustering examples, abstracting candidate schemas, and contrastively refining them by generating new instances and comparing against originals. Studies show that Schemex produces more actionable schemas than a frontier baseline without sacrificing generalizability, with participants uncovering deep and nuanced structural patterns.
What carries the argument
The three-stage interactive workflow of clustering, abstracting, and contrastive refinement that turns raw examples into testable structural abstractions.
If this is right
- Users obtain structural patterns that transfer to new instances while remaining specific enough to guide action.
- The interactive comparison step reduces the chance that abstractions become either too vague or too narrow.
- Designers of creative tools gain a concrete process for supporting structure discovery rather than relying on direct prompting alone.
- The same staged approach can be applied to other domains where implicit rules underlie examples, such as music or software.
Where Pith is reading between the lines
- The contrastive generation step might be adapted to refine abstractions in non-creative domains like scientific hypothesis formation.
- Integrating user feedback loops at each stage could further reduce the surface-variation problem in larger example sets.
- The workflow suggests that pure generation models may benefit from explicit clustering and comparison modules when the goal is abstraction rather than imitation.
Load-bearing premise
Schema induction can be usefully split into the three stages of clustering examples, abstracting candidate schemas, and contrastively refining them by generating new instances and comparing against originals.
What would settle it
A controlled study in which participants using the three-stage workflow fail to produce schemas rated more actionable than those from the baseline system, or in which the schemas lose applicability to new examples.
Figures
read the original abstract
Creative and communicative work is often underpinned by implicit structures, such as the Hero's Journey in storytelling, design patterns in software, or chord progressions in music. People often learn these structures from examples - a process known as schema induction. However, because schemas are abstract and implicit, they are difficult to discover: shared structural patterns are obscured by surface-level variation, and balancing generality with specificity is challenging. We present Schemex, an interactive AI workflow that systematically supports schema induction by decomposing it into three tractable stages: clustering examples, abstracting candidate schemas, and contrastively refining them by generating new instances and comparing against originals. Studies show that Schemex produces more actionable schemas than a frontier baseline without sacrificing generalizability, with participants uncovering deep and nuanced structural patterns. We also discuss design implications for the cognitive role of interactive process in structure discovery.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces Schemex, an interactive AI workflow for schema induction that decomposes the process into three stages: clustering examples, abstracting candidate schemas, and contrastively refining them by generating new instances and comparing against originals. User studies are reported to show that Schemex produces more actionable schemas than a frontier baseline without sacrificing generalizability, with participants uncovering deep and nuanced structural patterns. The paper concludes with design implications for the cognitive role of interactive processes in structure discovery.
Significance. If the empirical results hold under scrutiny, this work offers a meaningful contribution to HCI and AI-assisted creative tools by providing a structured, interactive framework for discovering implicit structures across domains such as storytelling, design, and music. The three-stage decomposition is presented as a practical method rather than a universal axiom, and the absence of free parameters or invented entities (as noted in the supporting analysis) is a strength that keeps the approach grounded in examples. The emphasis on contrastive refinement and actionability could influence future systems that support human structure discovery.
minor comments (3)
- Abstract: The summary of study outcomes would be strengthened by a brief mention of key metrics (e.g., actionability ratings or generalizability scores) or participant numbers, even if full details appear in §4 or §5.
- Method section: Clarify how the frontier baseline was implemented and matched to Schemex in terms of interaction style and output format to ensure the comparison isolates the effect of the three-stage workflow.
- Discussion: The design implications for interactive processes could be tied more explicitly back to specific observations from the user studies rather than remaining at a high level.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of the Schemex manuscript, including the accurate summary of the three-stage workflow, the user study findings on actionability and generalizability, and the recommendation for minor revision. The significance statement correctly identifies the contribution to HCI and AI-assisted structure discovery. No major comments were provided in the report.
Circularity Check
No significant circularity
full rationale
The paper describes an empirical interactive AI system (Schemex) that decomposes schema induction into three stages presented as the proposed workflow rather than a derived result. No equations, fitted parameters, self-referential derivations, or load-bearing self-citations appear in the provided text. Claims rest on user studies comparing to a baseline, with the decomposition introduced as a practical method rather than proven from internal assumptions. The work is self-contained as a system description and evaluation.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Schema induction can be effectively supported by decomposing it into clustering, abstraction, and contrastive refinement stages.
Forward citations
Cited by 2 Pith papers
-
Discovery-Oriented Faceting: From Coverage to Blind-Spot Discovery
DOF ranks document categories by distinctiveness instead of size to promote blind-spot discovery, surfacing different content than coverage-based methods across four domains.
-
Narrix: Remixing Narrative Strategies from Examples for Story Writing
Narrix helps novices identify and reuse narrative strategies from examples through visualization and strategy-steered generation, improving retention, confidence, and adaptation over chat interfaces in a 12-person study.
Reference graph
Works this paper leans on
-
[1]
Maneesh Agrawala, Wilmot Li, and Floraine Berthouzoz. 2011. Design Principles for Visual Communication.Commun. ACM54, 4 (April 2011), 60–69. https: //doi.org/10.1145/1924421.1924439
-
[2]
Maneesh Agrawala and Chris Stolte. 2001. Rendering Effective Route Maps: Improving Usability Through Generalization. InProceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01). ACM, New York, NY, USA, 241–249. https://doi.org/10.1145/383259.383286
-
[3]
1977.A Pattern Language - Towns, Buildings, Construction
Christopher Alexander, Sara Ishikawa, Murray Silverstein, Max Jacobson, Ingrid Fiksdahl-King, and Shlomo Angel. 1977.A Pattern Language - Towns, Buildings, Construction. Oxford University Press
work page 1977
-
[4]
Khalid Alharbi and Tom Yeh. 2015. Collect, decompile, extract, stats, and diff: Mining design pattern changes in Android apps. InProceedings of the 17th in- ternational conference on human-computer interaction with mobile devices and services. 515–524
work page 2015
-
[5]
Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. InProceedings of the 2019 chi conference on human factors in computing systems. 1–13
work page 2019
-
[6]
Joseph Campbell. 2008. The hero with a thousand faces.New World Library (2008)
work page 2008
-
[7]
Minsuk Chang, Léonore V Guillain, Hyeungshik Jung, Vivian M Hare, Juho Kim, and Maneesh Agrawala. 2018. Recipescape: An interactive tool for analyzing cooking instructions at scale. InProceedings of the 2018 CHI conference on human factors in computing systems. 1–12
work page 2018
-
[8]
Lydia B Chilton, Greg Little, Darren Edge, Daniel S Weld, and James A Landay
-
[9]
InProceedings of the SIGCHI Conference on Human Factors in Computing Systems
Cascade: Crowdsourcing taxonomy creation. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1999–2008
work page 1999
-
[10]
Chilton, Ecenaz Jen Ozmen, Sam H
Lydia B. Chilton, Ecenaz Jen Ozmen, Sam H. Ross, and Vivian Liu. 2021. VisiFit: Structuring Iterative Improvement for Novice Designers. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/ 3411764.3445089
-
[11]
Chilton, Savvas Petridis, and Maneesh Agrawala
Lydia B. Chilton, Savvas Petridis, and Maneesh Agrawala. 2019. VisiBlends: A Flexible Workflow for Visual Blends. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, 172:1–172:14. https://doi.org/10.1145/3290605. 3300402
-
[12]
João Miguel Cunha, Pedro Martins, and Penousal Machado. 2018. How Shell and Horn Make a Unicorn: Experimenting with Visual Blending in Emoji. In Proceedings of the Ninth International Conference on Computational Creativity. 145–152. https://openreview.net/forum?id=u1mCZAjwX7
work page 2018
-
[13]
Leonidas AA Doumas, John E Hummel, and Catherine M Sandhofer. 2008. A theory of the discovery and predication of relational concepts.Psychological review115, 1 (2008), 1
work page 2008
-
[14]
Hakan Duman, Alex Healing, and Robert Ghanea-Hercock. 2009. Adaptive Visual Clustering for Mixed-Initiative Information Structuring. InHuman Interface and the Management of Information. Designing Information Environments: Symposium on Human Interface 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009, Procceedings, Part I. ...
work page 2009
-
[15]
Van Duyne, James Landay, and Jason I
Douglas K. Van Duyne, James Landay, and Jason I. Hong. 2002.The Design of Sites: Patterns, Principles, and Processes for Crafting a Customer-Centered Web Experience. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA
work page 2002
-
[16]
Alex Endert, Patrick Fiaux, and Chris North. 2012. Semantic interaction for visual text analytics. InProceedings of the SIGCHI conference on Human factors in computing systems. 473–482
work page 2012
-
[17]
2013.Constructivism: Theory, perspectives, and practice
Catherine Twomey Fosnot. 2013.Constructivism: Theory, perspectives, and practice. Teachers College Press
work page 2013
-
[18]
2013.Chord Progressions: Theory and Practice
Dan Fox and Dick Weissman. 2013.Chord Progressions: Theory and Practice. Alfred Music
work page 2013
-
[19]
1995.Design patterns: elements of reusable object-oriented software
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995.Design patterns: elements of reusable object-oriented software. Pearson Deutschland GmbH
work page 1995
-
[20]
Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia- Jun Li, and Simon Tangi Perrault. 2024. CollabCoder: a lower-barrier, rigorous workflow for inductive collaborative qualitative analysis with large language models. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–29
work page 2024
-
[21]
Simret Araya Gebreegziabher, Yukun Yang, Elena L Glassman, and Toby Jia-Jun Li. 2025. Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–18
work page 2025
-
[22]
Dedre Gentner. 1983. Structure-mapping: A theoretical framework for analogy. Cognitive science7, 2 (1983), 155–170
work page 1983
-
[23]
Dedre Gentner and Arthur B Markman. 1997. Structure mapping in analogy and similarity.American psychologist52, 1 (1997), 45. 13
work page 1997
-
[24]
Katy Ilonka Gero, Chelse Swoopes, Ziwei Gu, Jonathan K Kummerfeld, and Elena L Glassman. 2024. Supporting sensemaking of large language model outputs at scale. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–21
work page 2024
-
[25]
Mary L Gick and Keith J Holyoak. 1983. Schema induction and analogical transfer. Cognitive psychology15, 1 (1983), 1–38
work page 1983
-
[26]
Sumit Gulwani. 2011. Automating string processing in spreadsheets using input- output examples.ACM Sigplan Notices46, 1 (2011), 317–330
work page 2011
-
[27]
2010.Mixed-Initiative Clustering
Yifen Huang. 2010.Mixed-Initiative Clustering. Carnegie Mellon University
work page 2010
-
[28]
Jit, Jennifer Spinney, Priyank Chandra, Lydia B
Sophia S. Jit, Jennifer Spinney, Priyank Chandra, Lydia B. Chilton, and Robert Soden. 2024. Writing out the Storm: Designing and Evaluating Tools for Weather Risk Messaging. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, 502:1–502:16. https://doi.org/10.1145...
-
[29]
Peter H. Kahn, Nathan G. Freier, Takayuki Kanda, Hiroshi Ishiguro, Jolina H. Ruckert, Rachel L. Severson, and Shaun K. Kane. 2008. Design Patterns for Social- ity in Human-robot Interaction. InProceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction(Amsterdam, The Netherlands)(HRI ’08). ACM, New York, NY, USA, 97–104. https://do...
-
[30]
Charles Kemp and Joshua B Tenenbaum. 2008. The discovery of structural form. Proceedings of the National Academy of Sciences105, 31 (2008), 10687–10692
work page 2008
-
[31]
Bernstein, and Daniela Stein- sapir
Joy Kim, Mira Dontcheva, Wilmot Li, Michael S. Bernstein, and Daniela Stein- sapir. 2015. Motif: Supporting Novice Creativity Through Expert Patterns. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Comput- ing Systems(Seoul, Republic of Korea)(CHI ’15). ACM, New York, NY, USA, 1211–1220. https://doi.org/10.1145/2702123.2702507
-
[32]
Aniket Kittur, Andrew M Peters, Abdigani Diriye, and Michael Bove. 2014. Stand- ing on the schemas of giants: socially augmented information foraging. InPro- ceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 999–1010
work page 2014
-
[33]
Christian Kruschitz and Martin Hitz. 2010. Human-computer interaction design patterns: structure, methods, and tools.Int. J. Adv. Softw3, 1 (2010)
work page 2010
-
[34]
Ranjitha Kumar, Arvind Satyanarayan, Cesar Torres, Maxine Lim, Salman Ahmad, Scott R Klemmer, and Jerry O Talton. 2013. Webzeitgeist: design mining the web. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3083–3092
work page 2013
-
[35]
Michelle S Lam, Janice Teoh, James A Landay, Jeffrey Heer, and Michael S Bern- stein. 2024. Concept induction: Analyzing unstructured text with high-level concepts using lloom. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–28
work page 2024
-
[36]
Mackenzie Leake, Abe Davis, Anh Truong, and Maneesh Agrawala. 2017. Com- putational Video Editing for Dialogue-driven Scenes.ACM Trans. Graph.36, 4, Article 130 (July 2017), 14 pages. https://doi.org/10.1145/3072959.3073653
-
[37]
Michael Xieyang Liu, Tongshuang Wu, Tianying Chen, Franklin Mingzhe Li, Aniket Kittur, and Brad A Myers. 2024. Selenite: Scaffolding Online Sensemak- ing with Comprehensive Overviews Elicited from Large Language Models. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26
work page 2024
-
[38]
Arthur B Markman and Dedre Gentner. 1993. Structural alignment during similarity comparisons.Cognitive psychology25, 4 (1993), 431–467
work page 1993
- [39]
-
[40]
Paul Merrell, Eric Schkufza, Zeyang Li, Maneesh Agrawala, and Vladlen Koltun
-
[41]
InACM SIGGRAPH 2011 Papers(Vancouver, British Columbia, Canada)(SIGGRAPH ’11)
Interactive Furniture Layout Using Interior Design Guidelines. InACM SIGGRAPH 2011 Papers(Vancouver, British Columbia, Canada)(SIGGRAPH ’11). ACM, New York, NY, USA, Article 87, 10 pages. https://doi.org/10.1145/1964921. 1964982
-
[42]
Piotr Mirowski, Kory W Mathewson, Jaylen Pittman, and Richard Evans. 2023. Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals. InProceedings of the 2023 CHI conference on human factors in computing systems. 1–34
work page 2023
-
[43]
Nic Newman. 2022. How publishers are learning to create and distribute news on TikTok. (2022)
work page 2022
-
[44]
Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2–4
work page 2005
- [45]
-
[46]
Allison Sauppé and Bilge Mutlu. 2014. Design Patterns for Exploring and Pro- totyping Human-robot Interactions. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(Toronto, Ontario, Canada)(CHI ’14). ACM, New York, NY, USA, 1439–1448. https://doi.org/10.1145/2556288.2557057
-
[47]
Burr Settles. 2009. Active learning literature survey. (2009)
work page 2009
-
[48]
John Stasko, Carsten Gorg, Zhicheng Liu, and Kanupriya Singhal. 2007. Jigsaw: supporting investigative analysis through interactive visualization. In2007 IEEE Symposium on Visual Analytics Science and Technology. IEEE, 131–138
work page 2007
-
[49]
Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, and Haijun Xia. 2024. Lumi- nate: Structured generation and exploration of design space with large language models for human-ai co-creation. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26
work page 2024
-
[50]
John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science12, 2 (1988), 257–285
work page 1988
- [51]
-
[52]
Anh Truong, Peggy Chi, David Salesin, Irfan Essa, and Maneesh Agrawala
-
[53]
InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Automatic generation of two-level hierarchical tutorials from instruc- tional makeup videos. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16
work page 2021
-
[54]
Sitong Wang, Samia Menon, Tao Long, Keren Henderson, Dingzeyu Li, Kevin Crowston, Mark Hansen, Jeffrey V. Nickerson, and Lydia B. Chilton. 2024. ReelFramer: Human-AI Co-Creation for News-to-Video Translation. InPro- ceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, ...
-
[55]
Sitong Wang, Zheng Ning, Anh Truong, Mira Dontcheva, Dingzeyu Li, and Lydia B. Chilton. 2024. PodReels: Human-AI Co-Creation of Video Podcast Teasers. InProceedings of the 2024 ACM Designing Interactive Systems Conference (DIS ’24). Association for Computing Machinery, New York, NY, USA. https: //doi.org/10.1145/3643834.3661591
-
[56]
Sitong Wang, Savvas Petridis, Taeahn Kwon, Xiaojuan Ma, and Lydia B. Chilton
-
[57]
InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23)
PopBlends: Strategies for Conceptual Blending with Large Language Models. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). Association for Computing Machinery, New York, NY, USA, 435:1–435:19. https://doi.org/10.1145/3544548.3580948
-
[58]
Saelyne Yang, Anh Truong, Juho Kim, and Dingzeyu Li. 2025. VideoMix: Ag- gregating How-To Videos for Task-Oriented Learning. InProceedings of the 30th International Conference on Intelligent User Interfaces. 1564–1580
work page 2025
-
[59]
Lixiu Yu, Aniket Kittur, and Robert E Kraut. 2014. Distributed analogical idea generation: inventing with crowds. InProceedings of the SIGCHI conference on Human Factors in Computing Systems. 1245–1254
work page 2014
-
[60]
Yes" = Clearly demonstrates the feature
Lixiu Yu, Aniket Kittur, and Robert E Kraut. 2014. Searching for analogical ideas with crowds. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1225–1234. A PROMPTS USED IN SCHEMEX A.1 Get clusters I’m learning how to {content_type}. Analyze the following examples to identify clusters based on STRUCTURAL and RHETORICAL pattern...
work page 2014
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.