Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

The State of the Art in Visualization Literacy

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey of 374 papers argues that visualization literacy research is best organized by five contribution types and four competency themes, and that this organization reveals systematic gaps—especially in critique, connection, accessibil

desk verdict A useful, transparent STAR with a plausible taxonomy, but the search string likely biases the gap claims and the corpus isn't released. read the letter →

arxiv 2509.01018 v1 pith:UDAWTF5C submitted 2025-08-31 cs.HC

Matthew Varona , Karen Bonilla , Maryam Hedayati , Alark Joshi , Lane Harrison , Matthew Kay , Carolina Nobre This is my paper · ORCID
classification cs.HC
keywords visualizationliteracytaxonomycompetencyframeworkstate-of-the-artsurveyassessmentinstrumentsdataeducationcriticalpopuliteracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a systematic survey of research on visualization literacy—the skills people need to read, make, question, and use data visualizations. It claims that the field can be organized by five kinds of research contributions (definitions, measurements, models of comprehension, population studies, and teaching interventions) crossed with four competency areas (consumption, construction, critique, connection). Applying this taxonomy to 374 papers shows the field is lopsided: most work measures how well people read charts, while critical evaluation and connecting charts to real-world knowledge are under-studied, and research outside Western contexts and on accessibility is nearly absent. The taxonomy gives researchers a shared vocabulary for saying exactly which skills they study and a map of where new work is needed.

What carries the argument

The central mechanism is the taxonomy itself. On one axis are five research categories derived from iterative open coding of the 374-paper corpus: ontology (definitions and scope), assessment (instruments for measuring literacy), mechanisms (models of user processes, barriers, and comprehension steps), populiteracy (descriptions of literacy within a specific population), and intervention (efforts to teach or improve literacy). On the other axis are four competency themes: consumption (reading and interpreting), construction (designing and implementing), critique (critical thinking about visualizations), and connection (relating visualizations to external knowledge and context). The taxonomy

What would settle it

A concrete check: rerun the same boolean keyword search against non-English and non-computer-science databases (e.g., ERIC, PsycINFO, CINAHL, SciELO, CNKI) and hand-code the results with the paper's inclusion criteria. If a substantial number of new papers in critique, connection, accessibility, or non-Western contexts emerges, the gap analysis would need revision. Alternatively, if the authors release the tagged corpus, an independent replication of the coding on a random subset could confirm the category counts and the claimed imbalances.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that visualization literacy research can be unified and usefully described by a two-dimensional taxonomy: five research categories (ontology, assessment, mechanisms, populiteracy, intervention) crossed with four competency themes (consumption, construction, critique, connection). The authors argue that visualization literacy should be operationalized as a context-dependent set of competencies rather than a single ability, and that doing so reveals systematic gaps. Consumption is heavily measured, interventions mostly target construction, critique and connection are rarely addressed across categories, and accessibility and non-Western populations

Load-bearing premise

The survey's findings rest on the 374-paper corpus being a representative sample of the field; if the keyword search, database choices, and manual tagging missed large bodies of relevant work—especially outside Western, English-language, computer-science-indexed venues—the reported gaps would be artifacts of the sampling frame rather than real properties of the literature.

Editorial extensions

If this is right

  • Assessment developers can use the framework to state explicitly which competencies their instrument measures and to justify creating new tests for construction and critique.
  • Intervention designers can see that most teaching targets construction and basic reading, making teaching of critique and connection comparatively open ground.
  • Researchers who adopt the competency themes will be forced to make their operationalization of visualization literacy explicit, making studies more comparable across the field.
  • Populiteracy studies receive a checklist of near-empty populations: non-Western groups, people with visual impairments or dyslexia, and specific domain specialists such as policymakers.
  • The taxonomy is structured to be extended or revised, so future surveys can treat it as a living classification rather than a fixed endpoint.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same taxonomy could be applied to adjacent literacies (data literacy, graph literacy, risk literacy) to test whether their research portfolios are similarly lopsided.
  • If the field shifts toward critique and connection, current default instruments like VLAT may lose status, and rubric-based or qualitative assessments may become more central.
  • The near-absence of non-Western studies suggests chart conventions themselves may be partly culture-specific; consumption measures developed in Western contexts would need cross-cultural validation before being trusted elsewhere.
  • A testable extension: rerun the same search protocol on non-English databases and compare category and theme distributions; if critique and connection papers are plentiful there, the taxonomy's gaps are a sampling artifact rather than a property of the field.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This state-of-the-art report surveys visualization literacy research, synthesizing 374 papers collected through a five-database keyword search, manual inclusion tagging, and snowballing. The authors propose a two-dimensional taxonomy: five research categories (ontology, assessment, mechanisms, populiteracy, intervention) and four competency themes (consumption, construction, critique, connection). They use this taxonomy to characterize the field, reporting that consumption is the most frequently measured competency while critique and connection receive less attention, and identifying gaps in non-Western and accessibility-focused research. The report also offers an operationalization framework for visualization literacy grounded in application contexts and competencies.

Significance. If the claims are accurate, this survey provides a valuable organizing structure for a fragmented field and a practical vocabulary for defining and comparing studies. Strengths include the transparent four-stage methodology (pilot coding with reported Cohen's kappa, explicit search string, manual inclusion tagging with dual annotation, snowballing) and the publication of corpus statistics. The four competency themes are intuitive and likely to be adopted by the community. However, the central gap claims rest entirely on the composition of the corpus, and the corpus search strategy is narrowly scoped. The survey's value depends on addressing the sampling threats identified below.

major comments (3)
  1. [Section 3.2 / Section 6] The Boolean search string requires visualization AND (literacy OR education OR teach* OR learn* OR student OR novice), plus a fixed set of graph-literacy terms. It does not include terms such as 'misleading', 'deceptive', 'critical', 'persuasion', 'rhetoric', 'decision-making', 'reasoning', or 'storytelling' unless accompanied by the enumerated literacy/education vocabulary. Because the pilot corpus (Section 3.1) was assembled from 'visualization literacy' searches, the extracted keywords likely inherit this narrow frame. Therefore, the conclusion in Section 6 that CRITIQUE and CONNECTION are under-studied, and the assessment-gap discussion in Section 5.2, may reflect the search frame rather than the field. This is load-bearing. I recommend supplementing the search with competency-specific terms and re-tagging the additional papers, or alternatively re-framing the claims as 'within the v
  2. [Section 3.2 / Section 6] The five databases (IEEE Xplore, ACM DL, DBLP, Scopus, Web of Science) are all English-centric, and the search string is English-only. The claim in Section 6 that non-Western contexts are nearly absent (echoed from [Sol22]) is directly threatened by the lack of non-English indexes and non-English keyword equivalents. Without a supplementary search in non-English venues, or at least a clear qualification that the non-Western gap holds only for the English-language indexed literature, this conclusion is unsupported.
  3. [Section 3.4] The only quantified inter-coder reliability is the inclusion kappa of 0.633 (Section 3.1). The final assignment of papers to categories and competency themes is described as consensus-based, but no reliability statistic or detailed rubric for competency tagging is provided. Since the gap analysis and the counts in Figure 3 depend on these tags, the authors should either report reliability for the final category/competency coding, or justify why dual annotation with consensus is sufficient. This is relevant to the central quantitative distribution claims.
minor comments (5)
  1. [Text encoding] Several encoding artifacts appear, e.g., 'â ˘AIJ', 'â ˘A¸ S', 'KÃ˝ urner'. These should be cleaned in the final PDF/source.
  2. [Appendix / reproducibility] The 374-paper corpus is not listed. Consider an appendix or online repository listing all included papers, enabling readers to audit the taxonomy and re-use the corpus.
  3. [Table 2] The 'Relevant Age Groups' row (Children, Adolescents) overlaps with the 'Students' grouping (e.g., primary school students). Clarify whether these are distinct population categories or a cross-cutting dimension.
  4. [Section 5.4.1] Reference [MHS15] is cited twice in close succession; coalesce or remove the duplicate citation.
  5. [Figure 3] Add axis labels and consider a companion table or matrix showing competency coverage per category; the current UpSet plot shows category intersections but not the competency themes that are central to the gap claims.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the taxonomy is an inductive summary of the surveyed corpus, not a fitted prediction.

full rationale

This is a descriptive systematic review rather than a predictive or fitted model. The taxonomic categories (ontology, assessment, mechanisms, populiteracy, intervention) and competency themes (consumption, construction, critique, connection) are explicitly induced from the survey corpus: 'We arrived at the framework via a thorough analysis of the competencies addressed by papers in our survey corpus' (Sec. 4). Applying an inductively derived scheme back to the same corpus is a standard grounded-theory loop; it means the reported category counts and gap summaries (Secs. 5-6) describe the corpus rather than test an external hypothesis, but no quantity is fitted and no prediction is made that could reduce to an input. The keyword string (Sec. 3.2) is bootstrapped from the pilot corpus, which could bias the sampling frame toward consumption/literacy vocabulary, but this is a coverage/validity limitation, not a circular equivalence. The paper's self-citations (e.g., [GHC*24], [HHK24], [NZM*24]) serve as ordinary examples and evidence; none is invoked as a uniqueness theorem or as the sole justification for the central taxonomy. Hence no load-bearing circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey's central synthesis rests on corpus completeness, the soundness of the proposed taxonomy, and the reliability of human coding. These are standard assumptions for grounded-theory reviews, but none is independently verified by the paper.

assumptions (3)
  • domain assumption The 4,519 unique papers found by the keyword search plus snowballing are representative of all visualization literacy research.
    Section 3.2-3.4. The search string and five databases define the field's boundary; relevant work using different terms or indexed outside these sources would be missed.
  • ad hoc to paper The four competency themes (consumption, construction, critique, connection) span the relevant space of visualization literacy abilities.
    Section 4. The authors propose this scheme; it is not proven exhaustive and they themselves call it a guide rather than an exhaustive list.
  • domain assumption Manual coding of papers into categories and competencies is accurate enough to support the reported distributions.
    Section 3.3-3.4. Only inclusion reliability is reported (kappa 0.633); inter-rater reliability for final category and competency tags is not given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The State of the Art in Visualization Literacy." pith.science (2026). https://pith.science/paper/UDAWTF5C

@misc{pith2026250901018,
  author       = {Pith},
  title        = {Pith review of: The State of the Art in Visualization Literacy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UDAWTF5C}},
  note         = {Machine review of arXiv:2509.01018}
}
read the original abstract

Research in visualization literacy explores the skills required to engage with visualizations. This state-of-the-art report surveys the current literature in visualization literacy to provide a comprehensive overview of the field. We propose a taxonomy of visualization literacy that organizes the field into competency themes and research categories. To address ambiguity surrounding the term ``visualization literacy'', we provide a framework for operationalizing visualization literacy based on application contexts (including domain, scenario, and audience) and relevant competencies, which are categorized under consumption, construction, critique, and connection. Research contributions are organized into five categories: ontology, assessment, mechanisms, populiteracy, and intervention. For each category, we identify key trends, discuss which competencies are addressed, highlight open challenges, and examine how advancements within these areas inform and reinforce each other, driving progress in the field.

Figures

Figures reproduced from arXiv: 2509.01018 by the authors.

Figure 1
Figure 1. A taxonomy of visualization literacy uniting research categories and competency themes. Research categories provide structure to the types of contributions found in our survey corpus of 374 papers. Competency themes broadly describe the skills necessary to engage with visualizations. Abstract Research in visualization literacy explores the skills required to engage with visualizations. This state-of-the-art report s… view at source ↗
Figure 2
Figure 2. The 4-stage methodology followed to acquire and tag relevant papers. In the pilot coding stage, an initial dataset of papers was coded to determine inclusion/exclusion criteria. Keyword search was conducted using phrases extracted from the pilot corpus. During the inclusion coding stage, the search results were each manually tagged with categories or marked for exclusion. Finally, each included paper was reviewed a … view at source ↗
Figure 3
Figure 3. The final survey corpus visualized in a modified UpSet plot. The bars on the left show the total number of papers in each category. Connected dots represent papers that fall under multiple categories, with the size of each intersection displayed as a number above the dots. This yielded 9,367 papers across all databases. After removing du￾plicate papers that were indexed by more than one database, 4,519 unique papers… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: A competency framework for visualization literacy. The four competency themes define the space of relevant visualization literacy competencies. Below each theme, example competencies illustrate some of the various levels of abstraction possible within each theme. Conte…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Read, Critique, or Sketch? Investigating Alternative Visualization Literacy Assessment Modalities

    cs.HC 2026-07 conditional novelty 6.0 of 10

    Sketching and think-aloud critique assessments distinguish visualization expertise levels better than multiple-choice tests and capture partially distinct literacy skills.

Reference graph

Works this paper leans on

22 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [3]

    In Innovative Methods in Media and Communication Research , Kubitschko S., Kaun A., (Eds.)

    11, 12 [GBMC16] G RAY J., B OUNEGRU L., M ILAN S., C IUCCARELLI P.: Ways of Seeing Data: Toward a Critical Literacy for Data Visualizations as Research Objects and Research Devices. In Innovative Methods in Media and Communication Research , Kubitschko S., Kaun A., (Eds.). Springer International Publish- ing, Cham, 2016, pp. 227–251. http://link.springer....

    arXiv 2016

  2. [7]

    doi:10.1145/3613905.3636289

    https://dl.acm.org/doi/10.1145/3613905.3636289. doi:10.1145/3613905.3636289. 2, 5 [Gil13] G ILBERT J. K.: Representations and Models. In Con- structing Representations to Learn in Science , Tytler R., Prain V ., Hubber P., Waldrip B., (Eds.). SensePublishers, Rotter- dam, 2013, pp. 193–198. http://link.springer.com/ 10.1007/978-94-6209-203-7_12 . doi:10.1...

    arXiv 2013

  3. [11]

    doi:10.1109/VAST.2011.6102435

    https://ieeexplore.ieee.org/document/6102435. doi:10.1109/VAST.2011.6102435. 12 [CMR23] C AMM J. D., M CCRAY G. E., R OEHM M. L.: More than just charts and graphs: What to teach in a data visualization course. Deci- sion Sciences Journal of Innovative Education 21 , 3 (July 2023), 112–

    work page arXiv 2011

  4. [14]

    There’s a Pan- demic Going On!

    https://ieeexplore.ieee.org/document/9646527/. doi:10.1109/MCG.2021.3117412. 2 [BHR23] B ERTLING J. G., H ODGE L., R YBA E.: “There’s a Pan- demic Going On!”: Data Visualization as Critical Place-Based Edu- cation in Challenging Times. Art Education 76 , 3 (May 2023), 24–

    work page arXiv 2021

  5. [15]

    doi:10.1109/VISUAL.2019.8933751

    https://ieeexplore.ieee.org/document/8933751/. doi:10.1109/VISUAL.2019.8933751. 5, 16, 17 [LWY∗16] L OPEZ K. D., W ILKIE D. J., Y AO Y., S OUSA V., FEBRETTI A., S TIFTER J., J OHNSON A., K EENAN G. M.: Nurses’ Numeracy and Graphical Literacy: Informing Studies of Clinical Decision Support Interfaces. Journal of Nursing Care Qual- ity 31 , 2 (Apr. 2016), 1...

    work page arXiv 2019

  6. [21]

    IEEE Computer Graphics and Applications 42 , 4 (July 2022), 20–26

    7 [Wan22] W ANG C.: VisVisual: A Toolkit for Teaching and Learning Data Visualization. IEEE Computer Graphics and Applications 42 , 4 (July 2022), 20–26. https://ieeexplore.ieee.org/document/ 9830795/. doi:10.1109/MCG.2022.3176199. 16 [WFL24] W OOD R., F ENG J. H., L AZAR J.: Health Data Visualization Literacy Skills of Young Adults with Down Syndrome and...

    work page arXiv 2022

  7. [27]

    EduVis: Workshop on Visualization Education, Literacy, and Activities

    16 [KHP∗23] K ECK M., H URON S., P ANAGIOTIDOU G., S TOIBER C., RAJABIYAZDI F., PERIN C., R OBERTS J. C., B ACH B.: EduVis: Work- shop on Visualization Education, Literacy, and Activities. https:// arxiv.org/abs/2303.10708, 2023. doi:10.48550/ARXIV. 2303.10708. 2 [KHTG14] K ÖRNER C., H ÖFLER M., T RÖBINGER B., G ILCHRIST I. D.: Eye Movements Indicate the ...

    work page Pith review arXiv doi:10.48550/arxiv.2303.10708 2023

  8. [31]

    doi:10.1080/00043125.2023

    https://www.tandfonline.com/doi/full/10.1080/ 00043125.2023.2167156. doi:10.1080/00043125.2023. 2167156. 5, 7 [BMBH16] B ÖRNER K., M ALTESE A., B ALLIET R. N., H EIMLICH J.: Investigating aspects of data visualization literacy using 20 infor- mation visualizations and 273 science museum visitors. Information Visualization 15 , 3 (July 2016), 198–213. http...

    work page arXiv 2023

Show all 22 references
  1. [54]

    doi:10.1109/EduVis60792.2023.00014

    https://ieeexplore.ieee.org/document/10344069. doi:10.1109/EduVis60792.2023.00014. 5, 17 [HK25] H EDAYATI M., K AY M.: What University Students Learn In Visualization Classes. IEEE Transactions on Visualization and Computer Graphics 31 , 1 (Jan. 2025), 1072–1082. https:// ieee...

    2023

  2. [78]

    Brickolage

    doi:10.1007/978-3-030-78635-9_78 . 17 [Str22] S TRANTZ A.: Data as “Brickolage”: Teaching Material Data Vi- sualization Design with LEGO. In The 40th ACM International Confer- ence on Design of Communication (Boston MA USA, Oct. 2022), ACM, pp. 130–135. https://dl.acm.org/doi/...

    2022

  3. [79]

    doi:10.1080/0013191830350108

    http://www.tandfonline.com/doi/abs/10.1080/ 0013191830350108. doi:10.1080/0013191830350108. 13 [RCP∗24] R UNGVIVATJARUS T., C HONG A. Z., P ATEL A., K HARE M., B IALOSTOZKY M., K UELBS C. L.: Training pediatric physicians and staff to obtain data from the electronic health rec...

    2024

  4. [102]

    doi:10.1002/bmb

    https://iubmb.onlinelibrary.wiley.com/doi/ 10.1002/bmb.2006.49403402094. doi:10.1002/bmb. 2006.49403402094. 7, 17 [SBN19] S TENLIDEN L., B ODÉN U., N ISSEN J.: Students as Produc- ers of Interactive Data Visualizations—Digitally Skilled to Make Their V oices Heard. Journal of ...

    2006

  5. [109]

    doi: 10.5220/0008987101000109

    http://www.scitepress.org/DigitalLibrary/ Link.aspx?doi=10.5220/0008987101000109. doi: 10.5220/0008987101000109. 7 [JGS22] J UNGJOHANN J., G EBHARDT M., S CHEER D.: Understand- ing and improving teachers’ graph literacy for data-based decision- making via video intervention. F...

    2022

  6. [122]

    doi:10.1111/dsji.12282

    https://onlinelibrary.wiley.com/doi/10.1111/ dsji.12282. doi:10.1111/dsji.12282. 16 [ ˇCP23] ˇCERNÝ J., P OTAN ˇCOK M.: Information literacy in in- ternational masters students: A competitive and business intelli- gence course perspective. Cogent Education 10 , 1 (Dec. 2023), ...

    2023

  7. [228]

    Choose-your-own

    https://onlinelibrary.wiley.com/doi/10.1111/ cgf.13975. doi:10.1111/cgf.13975. 17 [HHK24] H EDAYATI M., H UNT A., K AY M.: From pixels to practices: Reconceptualizing visualization literacy. https://osf.io/6mq42, May 2024. doi:10.31219/osf.io/6mq42. 2, 7 [HK23] H EDAYATI M., K...

    2024 doi

  8. [359]

    php?aid=6800

    http://www.hrpub.org/journals/article_info. php?aid=6800. doi:10.13189/ujer.2018.060219. 13 [OGCM18] O KAN Y., G ARCIA -RETAMERO R., C OKELY E. T., M AL- DONADO A.: Biasing and debiasing health decisions with bar graphs: Costs and benefits of graph literacy. Quarterly Journal ...

    2018

  9. [546]

    doi:10.1017/ S0003055401003100

    https://www.cambridge.org/core/journals/ american-political-science-review/article/ measurement-validity-a-shared-standard-for-qualitative-and-quantitative-research/ 91C7A9800DB26A76EBBABC5889A50C8B. doi:10.1017/ S0003055401003100. 2 [AGR21] A MABILI L., G UPTA K., R AIDOU R. ...

    2021

  10. [574]

    doi:10.1111/cura.12277

    https://onlinelibrary.wiley.com/doi/10.1111/ cura.12277. doi:10.1111/cura.12277. 13 [Wol15] W OLFE J.: Teaching Students to Focus on the Data in Data Visualization. Journal of Business and Technical Commu- nication 29 , 3 (July 2015), 344–359. https://journals. sagepub.com/doi...

    2015 doi

  11. [579]

    doi:10.1002/tea.3660230702

    https://onlinelibrary.wiley.com/doi/10.1002/ tea.3660230702. doi:10.1002/tea.3660230702. 15 [MPGZ22] M ACEDO M. P., P AIVA R. O. A., G ASPARINI I., Z AINA L. A. M.: Building information visualization of e-learning data with Vis2Learning guidelines. Journal on Interactive Syste...

    2022 doi

  12. [931]

    doi:10.1108/ ILS-03-2020-0054

    https://www.emerald.com/insight/content/doi/ 10.1108/ILS-03-2020-0054/full/html . doi:10.1108/ ILS-03-2020-0054 . 7 [SM24] S HREINER T. L., M ARTELL C. C.: Making race and racism invisible: A critical race analysis of data visualizations in online curric- ular materials for te...

    2020

  13. [1120]

    doi:10.1177/10525629241288266

    https://journals.sagepub.com/doi/10.1177/ 10525629241288266. doi:10.1177/10525629241288266. 7 [MGN∗24] M EDLEY -RATH S., G ILLESPIE M. D., N OVOSEL N., COMBS S., F EARNOW D.: Figures and Charts and Tables, Oh My!: A Content Analysis of Textbook Data Visualizations. Teaching So...

    2024 doi

  14. [1967]

    KN - Journal of Cartog- raphy and Geographic Information 70 , 1 (Apr

    16 [Nag20] N AGEL T.: Visually Analysing Urban Mobility: Results and Insights from Three Student Research Projects. KN - Journal of Cartog- raphy and Geographic Information 70 , 1 (Apr. 2020), 11–18. http: //link.springer.com/10.1007/s42489-020-00040-5 . doi:10.1007/s42489-020...

    2020 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.