Pith. sign in

REVIEW 4 major objections 5 minor 71 references

Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Community Notes are a linter for misleading charts

desk verdict A lively, clearly-written provocation whose central analogy works better than its own conclusions admit; the fixable-rung overreach is real but fixable. read the letter →

arxiv 2502.07649 v1 pith:WFYPBEME submitted 2025-02-11 cs.HC

Anamaria Crisan , Andrew M. McNutt This is my paper · ORCID
classification cs.HC
keywords CommunityNoteshumancomputationlintingdatavisualizationsociotechnicallinterscrowdsourcedmoderationmisinformationalt.chi
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Community Notes—the crowd-sourced context boxes attached to posts on X—are effectively linters for data visualizations: they flag misleading or error-prone charts, point to the specific problem, and are approved or rejected by a community rating process. The authors extend the linter idea beyond code and beyond automated checks to human computation, then test the metaphor against the four-rung "linting ladder" of checkable, customizable, blamable, and fixable. They examine three real Community Notes that catch an inflation-unadjusted rent-versus-income chart, a cherry-picked leaderboard of language models, and a misleading choice of job-growth aggregation, showing that the same categories of issues automated visualization linters detect arise organically from the crowd. If the claim holds, crowd moderation becomes a legitimate form of visualization evaluation that adds social insight—cultural norms and community standards—that rule-based tools miss.

What carries the argument

The load-bearing mechanism is the "linting ladder" from the authors' prior work, a four-property model of the behavior expected from a linter: checkable, customizable, blamable, and fixable. The paper applies that model as a checklist to Community Notes and also situates notes on a two-axis alignment chart that classifies linters by input type (code, text, anything) and evaluation mode (static analysis, computer, human). The other piece of machinery is the Community Notes platform itself: notes are written only after enough people request them, rated for helpfulness by the community, and surfaced through a ranking algorithm, and this structured crowd process is what earns the label "human computation."

What would settle it

Collect a corpus of Community Notes attached to data visualizations on X and check whether any original chart is edited, deleted, or corrected after a note appears; finding zero fixable changes would weaken the ladder case. Separately, run a survey or field experiment comparing trust in un-noted misleading charts with noted ones; if viewers trust them equally, the implied-truth-effect implication drawn from treating notes as lints does not hold.

Watch

Extended reading notes

Core claim

In the paper's own framing, the central discovery is that Community Notes satisfy every rung of the linting ladder and therefore count as a genuine, human-powered linter of data visualizations. A note is checkable because it tests a chart against community norms about what counts as misleading; customizable because the community's ratings and the ranking algorithm determine whether a note appears at all; blamable because it names the specific failing, such as unadjusted scales, missing models, or a debatable aggregate; and fixable because the visible social pressure of a note can lead the original author to edit or delete the post. Because evaluation is done by people rather than static analysis, the paper classes Community Notes in the "evaluation rebel" cell of its alignment chart, with text as the input. The work is presented as a provocation rather than a user study: it reframes crowd moderation as linting and draws out consequences, including the implied truth effect, unresolvable conflicting lints, and the risks of non-expert or adversarial note-writers.

Load-bearing premise

The conclusion rests on accepting the four-rung linting ladder as the definitive model of what a linter is; if that prior-work checklist is not the right standard, the claim that Community Notes are linters loses its foundation.

Editorial extensions

If this is right

  • Crowd moderation of visualizations becomes a legitimate branch of visualization evaluation, complementing automated linters such as VizLinter.
  • Visualization tool builders can use the linting ladder as a checklist for whether their own feedback mechanisms are checkable, customizable, blamable, and fixable.
  • If the comparison holds, the absence of a note will be read as a signal of correctness, so platforms must design for note absence, not only note presence.
  • Human-powered linting catches issues that automated tools miss, including cherry-picked comparisons and aggregation choices that are socially misleading rather than formally wrong.
  • AI- or LLM-based linters should be judged by whether they reproduce the ladder and the social context that makes notes meaningful, not merely by whether they automate the checking step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A test the authors do not run: measure whether charts that receive Community Notes are subsequently edited or deleted, which would operationalize the fixable rung and separate social shaming from actual repair.
  • The same ladder-plus-alignment analysis could be applied to other community-moderated artifacts, such as Wikipedia edit reviews or mapping-platform change discussions, where crowds enforce norms on mutable objects.
  • If the implied truth effect applies to charts, a randomized field experiment comparing trust in un-noted misleading charts against noted ones would give a direct welfare test of the provocation.
  • The note-ranking algorithm gives the platform, not the individual viewer, control over the customizable rung; a user-level "hide note" option would be a concrete product experiment on how individual customization changes trust.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper argues that Community Notes on social media can be understood as a form of human computation that performs 'linting' of data visualizations. It extends a 'linting ladder' from the authors' prior work, consisting of the properties checkable, customizable, blamable, and fixable, and applies this ladder to Community Notes, using three annotated examples of misleading charts. The paper also situates linters on a puritanical/neutral/rebellious alignment chart and discusses implications for AI-powered linters, uncertainty, and the philosophy of linting. The authors explicitly frame the contribution as a provocation rather than an empirical study.

Significance. If the classification holds, the paper offers visualization researchers a generative lens: crowd moderation of charts as a sociotechnical complement to automated linting. The alignment-chart framing is thought-provoking, the three examples are real and instructive, and the discussion of implied truth effects and unresolvable lints opens concrete future work. The paper is also honest in its limitations section, acknowledging that it does not explore how general users understand the content. Its main weakness is that the central claim rests on a checklist application of the ladder despite admitted gaps in the 'fixable' rung, and the word 'demonstrate' in the abstract is stronger than the illustrative evidence provided.

major comments (4)
  1. [§4.2.2] The 'fixable' rung is not satisfied by the paper's own account. The paragraph 'Community Notes can Support Fixable Changes' concedes that the only mechanisms are social shaming/deletion and inconsistent editing, and even states that 'a standard linter would not suggest you should delete your entire code.' The subsequent conclusion nevertheless states that Community Notes 'can even guide fixable improvements.' If the ladder is a checklist, Community Notes do not qualify; if it is not a checklist, the paper needs to say which rungs are necessary and why. As written, the central claim that Community Notes 'conform to the expected behaviors of linters' is internally inconsistent.
  2. [§4.2.1, Figure 3] The evidence for the central claim consists of three hand-picked Community Notes with no selection criteria, no corpus, and no inter-rater reliability or alternative sampling. The abstract's 'We demonstrate' is accordingly stronger than the support provided; the paper itself labels the work a provocation and its limitations section says it does not explore how general users understand the content. Please either reframe the abstract and Section 4 as an illustrative analysis, or add a transparent and systematic selection procedure for the examples.
  3. [§3] The status of the linting ladder's rungs is underspecified. The ladder is introduced as 'a collection of four properties that broadly describe the behaviors expectable from a linter,' and then used as a checklist in §4.2.2. Yet earlier examples, such as StackOverflow downvotes and architectural review boards, lack the fixable property or apply it loosely. The paper should state whether the ladder is a set of necessary conditions, a set of common features, or a similarity metric; otherwise the classification of Community Notes as linters cannot be distinguished from a partial match.
  4. [Abstract, §4.3] The claim that human computation 'enhances traditional linting by offering social insights' is not supported by any comparative evaluation. The paper explains how Community Notes could surface contextual or cultural knowledge, but it does not show that they do so more effectively or differently than existing automated linters in a way that would justify 'enhances.' Please soften this to a proposal or provide evidence of the claimed enhancement.
minor comments (5)
  1. [§4.2.2] Typo: 'the issue switch a data visualization' should read 'the issue with a data visualization.'
  2. [§4.2.2] Typo: 'and and can even guide fixable improvements' contains a doubled 'and.'
  3. [§2.1] Typo: 'but that household incomehas' is missing a space between 'income' and 'has.'
  4. [References] References [36] and [37] are duplicate entries for the same Lazar et al. book chapter; please consolidate them.
  5. [§3, Figure 2] The word 'nonuplets' in the text preceding Figure 2 is unclear; if it is intended as a term of art, it should be defined.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper applies an independently defined conceptual model to a new case; the internal tension on the 'fixable' rung is a correctness issue, not a circular derivation.

full rationale

The paper's central claim that Community Notes can be seen as sociotechnical linters is an interpretive metaphor rather than a derivation or prediction. The linting ladder (checkable, customizable, blamable, fixable) was introduced in the authors' prior work [46], but it is restated in Section 3 as a checklist whose content does not depend on Community Notes. Applying one's own earlier framework to a new object is not circular, and the classification could in principle fail; indeed, the paper itself concedes in Section 4.2.2 that the fixable rung is only weakly satisfied ('Editing capabilities are inconsistent... a standard linter would not suggest you should delete your entire code'). That concession is a challenge to the strength of the argument, not evidence that the conclusion is definitionally forced. No fitted parameters are renamed as predictions, no equation is equivalent to an input, and no external uniqueness theorem is imported from the authors' prior work. The self-citation to McNutt et al. [46] is a normal citation to prior conceptual work and is not load-bearing in a reductive sense. The paper is an explicitly framed provocation, and its argument is transparently a conceptual mapping rather than a circular derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

This is a conceptual paper, so there are no fitted numeric parameters. The central claim rests on the linting ladder (from the authors' prior work), the classification of Community Notes as human computation, the representativeness of three chosen examples, and the newly proposed alignment-chart taxonomy. The only invented entity is the taxonomy itself.

assumptions (4)
  • domain assumption The linting ladder (checkable, customizable, blamable, fixable) is a valid model for evaluating linting behavior and transfers from code to social/human processes.
    The paper inherits the ladder from prior work (McNutt et al. 2024 [46]) and uses it to show Community Notes conform to expected linter behaviors in Section 4.2.2; if the ladder is not the right way to characterize linting, the central comparison weakens.
  • domain assumption Community Notes constitute human computation because they recruit a broad community and aggregate to a consensus result.
    Section 4.2 states this; this definitional premise underpins the framing of the paper. If Community Notes are not a form of human computation, the core claim collapses.
  • ad hoc to paper The three Community Notes in Figure 3 are representative examples of visualization linting rather than exceptional cases.
    The paper selects three examples of notes identifying visualization issues (scale mismatch, missing models, misleading aggregation) and treats them as evidence for the general claim that human-powered assessments identify misleading visualizations. No sampling or systematic search is described, so generalizability is assumed.
  • ad hoc to paper The puritanical/neutral/rebellious alignment-chart dimensions meaningfully organize the linting design space.
    Section 3 introduces this taxonomy as a framing device with no external validation; the placement of Community Notes in the 'rebel' quadrant is a judgment call.
invented entities (1)
  • Alignment-chart dimensions (puritanical, neutral, rebellious) for linting
    purpose: To organize prior and proposed linting systems and to situate Community Notes in a design space
    The axes of 'input' (purist to rebel) and 'evaluation' (purist to rebel) are newly proposed in Section 3 as a conceptual scaffold. No empirical validation or measurement procedure is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations." pith.science (2026). https://pith.science/paper/WFYPBEME

@misc{pith2026250207649,
  author       = {Pith},
  title        = {Pith review of: Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WFYPBEME}},
  note         = {Machine review of arXiv:2502.07649}
}
read the original abstract

Traditionally, linters are code analysis tools that help developers by flagging potential issues from syntax and logic errors to enforcing syntactical and stylistic conventions. Recently, linting has been taken as an interface metaphor, allowing it to be extended to more complex inputs, such as visualizations, which demand a broader perspective and alternative approach to evaluation. We explore a further extended consideration of linting inputs, and modes of evaluation, across the puritanical, neutral, and rebellious dimensions. We specifically investigate the potential for leveraging human computation in linting operations through Community Notes -- crowd-sourced contextual text snippets aimed at checking and critiquing potentially accurate or misleading content on social media. We demonstrate that human-powered assessments not only identify misleading or error-prone visualizations but that integrating human computation enhances traditional linting by offering social insights. As is required these days, we consider the implications of building linters powered by Artificial Intelligence.

Figures

Figures reproduced from arXiv: 2502.07649 by the authors.

Figure 1
Figure 1. A characterization of the input space for linting [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Different systems organized in an alignment chart format [ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example of three Community Notes as applied to data visualizations. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 38 canonical work pages

  1. [1]

    Ali Alkhatib and Michael Bernstein. 2019. Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions. In SIGCHI Conference on Human Factors in Computing Systems . ACM, Glasgow Scotland Uk, 1–13. https://doi.org/10. 1145/3290605.3300760

    arXiv 2019

  2. [2]

    Mohammad Allahbakhsh, Boualem Benatallah, Aleksandar Ignjatovic, Hamid Reza Motahari-Nezhad, Elisa Bertino, and Schahram Dustdar. 2013. Quality Control in Crowdsourcing Systems: Issues and Directions. IEEE Internet Computing 17, 2 (2013), 76–81. https://doi.org/10.1109/MIC.2013.20

    work page doi:10.1109/mic.2013.20 2013

  3. [3]

    AntV. 2023. Advisor.lint. ava.antv.antgroup.com/en/api/advice/advisor-lint

    work page 2023

  4. [4]

    Daniel W Barowy, Emery D Berger, and Benjamin Zorn. 2018. ExceLint: automati- cally finding spreadsheet formula errors. Proceedings of the ACM on Programming Languages 2, OOPSLA (2018), 1–26. https://doi.org/10.1145/3276518

    doi:10.1145/3276518 2018

  5. [5]

    Bernstein, Greg Little, Robert C

    Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, and Katrina Panovich. 2010. Soylent: a word processor with a crowd inside. In ACM Symposium on User Interface Software and Technology. Association for Computing Machinery, New York, NY, USA, 313–322. https://doi.org/10.1145/1866029.186607...

    arXiv 2010

  6. [6]

    Borgo, L

    R. Borgo, L. Micallef, B. Bach, F. McGee, and B. Lee. 2018. Information Visualiza- tion Evaluation Using Crowdsourcing. Computer Graphics Forum 37, 3 (2018), 573–595. https://doi.org/10.1111/cgf.13444

    doi:10.1111/cgf.13444 2018

  7. [7]

    David Borland and Russell M Taylor Ii. 2007. Rainbow color map (still) considered harmful. IEEE computer graphics and applications 27, 2 (2007), 14–17. https: //doi.org/10.1109/MCG.2007.46

    work page doi:10.1109/mcg.2007.46 2007

  8. [8]

    John Carpenter. 1988. They Live. Universal Pictures

    work page 1988

Show all 71 references
  1. [9]

    Joseph Chee Chang, Saleema Amershi, and Ece Kamar. 2017. Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets. In SIGCHI Conference on Human Factors in Computing Systems . Association for Computing Machinery, New York, NY, USA, 2334–2346. https://doi.org/...

    2017

  2. [11]

    Katherine Compton. 2019. Casual creators: Defining a genre of autotelic creativity support systems. University of California, Santa Cruz. https://www.escholarship. org/uc/item/4kg8g9gd

    2019

  3. [12]

    Michael Correll. 2021. The Clippy-ization of Human-Computer Design. Medium (2021). https://mcorrell.medium.com/the-clippy-ization-of-human-computer- design-c66e8042de88

    2021

  4. [13]

    Michael Correll and Michael Gleicher. 2014. Bad for Data, Good for the Brain : Knowledge-First Axioms For Visualization Design. In DECISIVe : Workshop on Dealing with Cognitive Biases in Visualisations . http://graphics.cs.wisc.edu/ Papers/2014/CG14a

    2014

  5. [14]

    Alan Cruickshank. 2023. Rolling out SQLFluff with a new team. docs.sqlfluff.com/en/stable/teamrollout.html

    2023

  6. [15]

    Keim, and Min Chen

    Alexandra Diehl, Alfie Abdul-Rahman, Mennatallah El-Assady, Benjamin Bach, Daniel A. Keim, and Min Chen. 2018. VisGuides: A Forum for Discussing Visu- alization Guidelines. In Eurographics Conference on Visualization . Eurographics Association, 61–65. https://doi.org/10.2312/E...

    2018 doi

  7. [16]

    Chiara Patricia Drolsbach, Kirill Solovev, and Nicolas Pröllochs. 2024. Community notes increase trust in fact-checking on social media.PNAS Nexus (2024), pgae217. https://doi.org/10.1093/pnasnexus/pgae217

    2024 doi

  8. [17]

    Eleventy. 2024. Inclusive Language Plugin. https://www.11ty.dev/docs/plugins/ inclusive-language/

    2024

  9. [19]

    Brian Ford. 2019. Write Good: Naive linter for English prose . github.com/btford/write-good

    2019

  10. [20]

    Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society 7, 1 (2020), 2053951719897945. https: //doi.org/10.1177/2053951719897945

    2020 doi

  11. [21]

    Madeleine Grunde-McLaughlin, Michelle S Lam, Ranjay Krishna, Daniel S Weld, and Jeffrey Heer. 2023. Designing LLM chains by adapting techniques from crowdsourcing workflows. arXiv preprint arXiv:2312.11681 (2023). https://doi. org/10.48550/arXiv.2312.11681

    2023 doi arXiv

  12. [22]

    Han L Han, Miguel A Renom, Wendy E Mackay, and Michel Beaudouin-Lafon

  13. [23]

    Kotaro Hara, Vicki Le, and Jon Froehlich. 2013. Combining crowdsourcing and google street view to identify street-level accessibility problems. In SIGCHI Con- ference on Human Factors in Computing Systems . Association for Computing Ma- chinery, New York, NY, USA, 631–640. htt...

    2013

  14. [24]

    Jeffrey Heer. 2019. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences 116, 6 (2019), 1844–1850. https://doi.org/10.1073/pnas.1807184115

    2019 doi

  15. [25]

    Jeffrey Heer and Michael Bostock. 2010. Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In SIGCHI Conference on Human Factors in Computing Systems . Association for Computing Machinery, New York, NY, USA, 203–212. https://doi.org/10.11...

    2010

  16. [26]

    Aspen K Hopkins, Michael Correll, and Arvind Satyanarayan. 2020. Visualint: Sketchy in situ annotations of chart construction errors. In Computer Graphics Forum. Wiley, 219–228. https://doi.org/10.1111/cgf.13975

    2020 doi

  17. [27]

    Wang, Seongmin Lee, Mahdi Roozbahani, and Duen Horng Chau

    Matthew Hull, Vivian Pednekar, Hannah Murray, Nimisha Roy, Emmanuel Tung, Susanta Routray, Connor Guerin, Justin Chen, Zijie J. Wang, Seongmin Lee, Mahdi Roozbahani, and Duen Horng Chau. 2023. VISGRADER: Automatic Grading of D3 Visualizations. Transactions on Visualization and...

    2023

  18. [28]

    Peter Hyams. 1994. TimeCop. Universal Pictures

    1994

  19. [29]

    Nick Hynes, D Sculley, and Michael Terry. 2017. Data linter: Lightweight, auto- mated sanity checking for ml data sets. In NIPS MLSys Workshop

    2017

  20. [30]

    Steven J Jackson and Sarah Barbrow. 2015. Standards and/as innovation: Protocols, creativity, and interactive systems development in ecology. InSIGCHI Conference on Human Factors in Computing Systems . 1769–1778. https://doi.org/10.1145/ 2702123.2702564

    2015

  21. [31]

    Alex Kale, Ziyang Guo, Xiao Li Qiao, Jeffrey Heer, and Jessica Hullman. 2023. EVM: Incorporating model checking into exploratory visual analysis.Transactions on Visualization and Computer Graphics (2023). https://doi.org/10.1109/TVCG. 2023.3326516

    2023

  22. [32]

    Uku Kangur, Roshni Chakraborty, and Rajesh Sharma. 2024. Who Checks the Checkers? Exploring Source Credibility in Twitter’s Community Notes. arXiv preprint arXiv:2406.12444 (2024). https://doi.org/10.48550/arXiv.2406.12444

    2024 doi

  23. [33]

    Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2023. A Systematic Mapping Study of Code Quality in Education. In Conference on Innovation and Technology in Computer Science Education V. 1. 5–11. https://doi.org/10.1145/3587102.3588777

    2023

  24. [34]

    Know Your Meme. 2022. Alignment Charts - Salad Alignment Chart. https: //knowyourmeme.com/photos/2382569-alignment-charts Online meme database entry

    2022

  25. [35]

    Robert Kosara. 2024. Business Data Visualization, Beyond the Boring . IEEE Computer Graphics and Applications 44, 05 (Sept. 2024), 153–158. https://doi.org/ 10.1109/MCG.2024.3407489

    2024

  26. [36]

    Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Chapter 14 - Online and ubiquitous HCI research. In Research Methods in Human Computer Interaction (Second Edition) (second edition ed.). Morgan Kaufmann, Boston, 411–453

    2017

  27. [37]

    Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Online and Ubiquitous HCI Research. In Research Methods in Human Computer Interaction . Morgan Kaufmann, Chapter 14, 411–453

    2017

  28. [38]

    Elsie Lee-Robbins and Andrew McNutt. 2023. Only YOU Can Make IEEE VIS Environmentally Sustainable. alt.vis (2023). https://doi.org/10.48550/arXiv.2308. 15429 https://arxiv.org/abs/2308.15429

    2023 doi arXiv

  29. [39]

    Fan Lei, Arlen Fan, Alan M MacEachren, and Ross Maciejewski. 2023. GeoLinter: A Linting Framework for Choropleth Maps. Transactions on Visualization and Computer Graphics (2023). https://doi.org/10.1109/TVCG.2023.3322372

    2023

  30. [40]

    Maxim Lisnic, Zach Cutler, Marina Kogan, and Alexander Lex. 2025. Visualization Guardrails: Designing Interventions Against Cherry-Picking in Interactive Data Explorers. (2025). To Appear

    2025

  31. [42]

    Andrew McNutt, Anamaria Crisan, and Michael Correll. 2020. Divining insights: Visual analytics through cartomancy. In SIGCHI Conference on Human Factors in Computing Systems (Extended Abstracts) . 1–16. https://doi.org/10.1145/3334480. 3381814

    2020 doi

  32. [43]

    Andrew McNutt and Gordon Kindlmann. 2018. Linting for visualization: To- wards a practical automated visualization guidance system. InVisGuides (IEEEVIS Workshop). 1–14

    2018

  33. [44]

    Andrew McNutt, Gordon Kindlmann, and Michael Correll. 2020. Surfacing visualization mirages. In SIGCHI Conference on Human Factors in Computing Systems. 1–16. https://doi.org/10.1145/3313831.3376420

    2020

  34. [45]

    Andrew McNutt, Anton Outkine, and Ravi Chugh. 2023. A Study of Editor Features in a Creative Coding Classroom. InSIGCHI Conference on Human Factors in Computing Systems. 1–15. https://doi.org/10.1145/3544548.3580683

    2023

  35. [46]

    Andrew McNutt, Maureen C Stone, and Jeffrey Heer. 2024. Mixing Linters with GUIs: A Color Palette Design Probe. IEEE Transactions on Visualization and Computer Graphics (2024). https://doi.org/10.1109/TVCG.2024.3456317

    2024

  36. [47]

    Andrew McNutt, Chenglong Wang, Robert A Deline, and Steven M Drucker. 2023. On the design of ai-powered code assistants for notebooks. In SIGCHI Conference on Human Factors in Computing Systems . 1–16. https://doi.org/10.1145/3544548. 3580940

    2023 doi

  37. [48]

    Joseph Bahman Moghadam, Rohan Roy Choudhury, HeZheng Yin, and Armando Fox. 2015. AutoStyle: Toward Coding Style Feedback at Scale. InACM Conference on Learning Scale . Association for Computing Machinery, New York, NY, USA, 261–266. https://doi.org/10.1145/2818052.2874315

    2015

  38. [49]

    Arpit Narechania, Adam Coscia, Emily Wall, and Alex Endert. 2021. Lumos: Increasing awareness of analytic behavior during visual data analysis. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 1009–1018. https://doi.org/10.1109/TVCG.2021.3114827

    2021

  39. [50]

    Dietmar Offenhuber. 2023. Autographic Design: The Matter of Data in a Self- Inscribing World. MIT Press. https://doi.org/10.7551/mitpress/14666.003.0010

    2023 doi

  40. [51]

    Gordon Pennycook, Adam Bear, Evan T Collins, and David G Rand. 2020. The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings.Management science 66, 11 (2020), 4944–4957. https://doi.org/10.1287/...

    2020

  41. [52]

    Polygon. 2019. Art Sqool took me back to real-life art school. https://www. polygon.com/2019/3/1/18246354/art-sqool-impressions-pc-mac. CHI EA ’25, April 26-May 1, 2025, Yokohama, Japan Anamaria Crisan and Andrew McNutt

    2019

  42. [53]

    Quinn and Benjamin B

    Alexander J. Quinn and Benjamin B. Bederson. 2011. Human computation: a survey and taxonomy of a growing field. In SIGCHI Conference on Human Factors in Computing Systems . Association for Computing Machinery, New York, NY, USA, 1403–1412. https://doi.org/10.1145/1978942.1979148

    2011

  43. [54]

    Katharina Reinecke and Krzysztof Z Gajos. 2015. LabintheWild: Conducting large-scale online experiments with uncompensated samples. In ACM conference on computer supported cooperative work & social computing . 1364–1378. https: //doi.org/10.1145/2675133.2675246

    2015

  44. [55]

    Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Ng. 2008. Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. In Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Honolulu, Hawaii, ...

    2008

  45. [56]

    Nicole Sultanum, Dennis Bromley, and Michael Correll. 2024. Data Guards: Chal- lenges and Solutions for Fostering Trust in Data. IEEE Visualization Conference (2024). https://doi.org/10.1109/VIS55277.2024.00019

    2024

  46. [57]

    Twitter Notes [n. d.]. Note ranking algorithm. https://communitynotes.x.com/ guide/en/under-the-hood/ranking-notes. https://doi.org/10.1007/s10288-003- 0021-7 Accessed 10/29/2024

    2024 doi

  47. [58]

    Huichen Will Wang, Larry Birnbaum, and Vidya Setlur. 2025. Jupybara: Opera- tionalizing a Design Space for Actionable Data Analysis and Storytelling with LLMs. In SIGCHI Conference on Human Factors in Computing Systems. ACM, 1–13

    2025

  48. [59]

    Zijie J Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering Responsible AI Awareness During AI Applica- tion Prototyping. In SIGCHI Conference on Human Factors in Computing Systems . https://doi.org/10.1145/3613904.364233

    2024

  49. [60]

    Colin Ware and Matthew D Plumlee. 2013. Designing a better weather dis- play. Information Visualization 12, 3-4 (2013), 221–239. https://doi.org/10.1177/ 1473871612465214

    2013

  50. [61]

    Colin Ware, Maureen Stone, and Danielle Albers Szafir. 2023. Rainbow colormaps are not all bad. IEEE Computer Graphics and Applications 43, 3 (2023), 88–93. https://doi.org/10.1109/MCG.2023.3246111

    2023

  51. [62]

    Brian Whitworth. 2005. Polite computing. Behaviour & Information Technology 24, 5 (2005), 353–363. https://doi.org/10.1080/01449290512331333700

    2005 doi

  52. [63]

    Wikipedia. 2024. Colorless green ideas sleep furiously — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=Colorless%20green% 20ideas%20sleep%20furiously&oldid=1258719573. [Online; accessed 24- November-2024]

    2024

  53. [64]

    Wesley Willett, Jeffrey Heer, and Maneesh Agrawala. 2012. Strategies for crowd- sourcing social data analysis. In SIGCHI Conference on Human Factors in Comput- ing Systems. Association for Computing Machinery, New York, NY, USA, 227–236. https://doi.org/10.1145/2207676.2207709

    2012

  54. [65]

    Hana Winchester, Ebtesam Al Haque, Alicia Boyd, and Brittany Johnson. 2023. HaTe Detector: A Tool for Detecting and Correcting Harmful Terminology in Computing Artifacts. In Symposium on Visual Languages and Human-Centric Computing. IEEE, 245–248. https://doi.org/10.1109/VL-HC...

    2023

  55. [66]

    Jo Wood, Alexander Kachkaev, and Jason Dykes. 2018. Design exposition with literate visualization. Transactions on Visualization and Computer Graphics 25, 1 (2018), 759–768. https://doi.org/10.1109/TVCG.2018.2864836

    2018

  56. [67]

    Aoyu Wu, Liwenhan Xie, Bongshin Lee, Yun Wang, Weiwei Cui, and Huamin Qu

  57. [68]

    Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. AI Chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In SIGCHI Conference on Human Factors in Computing Systems . 1–22. https: //doi.org/10.1145/3491102.3517582

    2022

  58. [69]

    Tongshuang Wu, Haiyi Zhu, Maya Albayrak, Alexis Axon, Amanda Bertsch, Wenxing Deng, Ziqi Ding, Bill Guo, Sireesh Gururaja, Tzu-Sheng Kuo, et al. 2023. Llms as workers in human-computational algorithms? replicating crowdsourcing pipelines with llms. arXiv preprint arXiv:2307.10...

    2023 doi arXiv

  59. [70]

    Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: generating struc- tured feedback on visual designs using a crowd of non-experts. In ACM Con- ference on Computer Supported Cooperative Work & Social Computing (CSCW ’14). Association for Computing Machinery, New York, ...

    2014

  60. [71]

    Zakas and Nicole Sullivan

    Nicholas C. Zakas and Nicole Sullivan. 2011. CSS Lint. csslint.net

    2011

  61. [72]

    Wei Zheng and Youssef Sherif. 2019. Visualization Linter. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    2019

  62. [2020]

    In SIGCHI Conference on Human Factors in Computing Systems

    Textlets: Supporting constraints and consistency in text documents. In SIGCHI Conference on Human Factors in Computing Systems . 1–13. https://doi. org/10.1145/3313831.3376804

  63. [2021]

    InSIGCHI Conference on Human Factors in Computing Systems

    Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison. InSIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 14, 13 pages. https://doi.org/10.1145/3411764.3445179

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.