{"id":"e57a04b9-13ec-42e4-92a5-4ccdc4b27646","arxiv_id":"2608.07093","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A text-to-chart system that visually preserves linguistic hedges helps trained readers match charts to uncertain text, though the cognitive-load evidence is not statistically significant.","lead":"This paper presents a chart-generation system that keeps hedge words like 'nearly' and 'approximately' visible in the graph, using wiggly bar tops, hatched fills, and hover explanations. It matters because today's automated charts from news and reports strip away uncertainty, and the authors show readers can match charts back to hedged text with 85 percent accuracy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Part 1 matching accuracy may stem from numeric alignment and taught mappings rather than the visual encodings, so the central claim that the encodings communicate uncertainty is not yet established.","rationale":"The paper makes a plausible and well-scoped contribution: a taxonomy of linguistic uncertainty and a set of visual encodings, with qualitative evidence that some encodings (squiggly borders, hatching) are intuitive and some (gradients, NA labels) fail. The authors honestly acknowledge the manual pre-selection of expressions and the non-significant cognitive-load trends. However, the headline quantitative evidence for the central claim—that the encodings communicate uncertainty—is weak because the matching tasks are not clean. The Match-Verify strategy reported in Section 7.3.1, combined with the orientation in Section 6.5 and the numeric variability in the Appendix B distractors, means participants could achieve high accuracy without relying on the visual encodings. This is precisely the reader's weakest assumption, and we agree it is load-bearing. A concrete audit of the stimuli can settle whether the confound is real. If it is, the central claim requires a redesigned evaluation; if not, the original interpretation may hold. Given that the paper has a promising core idea and the concern is testable, the CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":41569,"tokens_out":4716,"duration_ms":42919,"concrete_test":"Audit the 24 matching trials in Appendix B: for each trial, determine whether the correct text/chart is the only option whose numerical value(s) match the stimulus (ignoring all uncertainty markers and visual encodings). If numeric matching alone identifies the correct answer in more than half the trials, the reported 76%/85% accuracies are confounded; the authors should then re-run Part 1 with a no-orientation group and numeric values held constant across options to establish encoding effectiveness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the visual encodings successfully communicate uncertainty semantics rests on Part 1 matching accuracy (76% text-to-chart, 85% chart-to-text, Section 7.1.1). This evidence is confounded in two ways. First, Section 7.3.1 documents a Match-Verify strategy in which participants first check exact numeric alignment and only then evaluate uncertainty markers; the Appendix B materials often vary numbers across options, so numeric filtering can select the correct answer without interpreting the encoding. Second, Section 6.5 states that Part 1 began with an orientation explaining the taxonomy and the encodings, so participants were reproducing taught mappings rather than decoding the visual variables independently. If accuracy is driven by number matching and memorized conventions, the experiments do not establish that the encodings themselves carry uncertainty semantics, undermining the paper's central claim. The non-significant cognitive-load results (Section 7.2.2, p=0.054 and p=0.073) are a secondary issue; the primary load-bearing problem is the internal validity of the encoding-effectiveness evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces UncertaintyVis, an end-to-end pipeline that preserves linguistic uncertainty in automated text-to-chart generation. A formative corpus analysis of 211 uncertainty expressions across 12 documents and 8 domains yields a four-category taxonomy (Inferential Derivation, Precision Boundaries, Surface Form Normalization, Non-Inferable Gaps), which is mapped to chart-specific visual encodings and implemented using an LLM-based extraction pipeline. A two-part user study with 12 participants reports 85% chart-to-text and 76% text-to-chart matching accuracy, non-significant trends toward reduced cognitive workload, and 75% preference for uncertainty-aware visualizations. The paper's central claim is that these results demonstrate that visual encodings successfully communicate uncertainty semantics and that preserving linguistic uncertainty is practically beneficial.","tokens_in":41695,"tokens_out":4395,"duration_ms":39411,"significance":"If the central claim were established, the paper would address a real gap in text-to-chart systems: current tools flatten hedged expressions such as 'nearly 70,000 m²' into false precision. The strengths of the work are concrete: the corpus analysis is described in detail, the taxonomy is defined with examples, the system is implemented, and the study materials are reproduced in the appendices, which supports replication. The authors also candidly report the failures of line-chart encodings, gradient range markers, and N/A labels, and they do not hide non-significant results. However, I agree with the stress-test concern: the primary evidence for encoding effectiveness, the Part 1 matching accuracies, is confounded by the orientation procedure, the use of the same corpus for taxonomy development and evaluation, and the availability of number-based filtering in the matching tasks. As a result, the headline accuracy numbers do not yet establish that the visual encodings themselves carry the uncertainty semantics.","major_comments":[{"comment":"Section 6.5 states that Part 1 began with an orientation explaining the taxonomy categories and their visual encodings. Consequently, the reported 76% text-to-chart and 85% chart-to-text accuracies in Section 7.1.1 may reflect participants' ability to reproduce mappings they were just taught, rather than demonstrating that the visual encodings independently communicate uncertainty semantics. The central claim in Section 7.1.1 that 'our visual encoding strategies successfully communicate uncertainty semantics' therefore needs an evaluation condition without such orientation, or a transfer test to unseen expressions and encodings, to separate learning from decoding.","section":"Section 6.5"},{"comment":"Section 6.3 states that Part 2 documents were drawn directly from the 12-document corpus analyzed in the formative study, and Section 3.3 describes how the taxonomy was developed and refined on that same corpus. Evaluating the taxonomy on the same documents from which it was derived overestimates its generalizability. The authors should validate the taxonomy on held-out documents from different sources, or at minimum report which documents were used for taxonomy development versus evaluation and provide a per-document breakdown of the matching results.","section":"Section 6.3"},{"comment":"Section 7.3.1 reports that participants used a Match-Verify strategy, first checking exact numeric alignment before evaluating uncertainty markers. Inspection of Appendix B shows that the multiple-choice options frequently differ in numerical values (e.g., chart-to-text Q1, Q2, Q3, Q5, Q8, and many text-to-chart items), so a participant can select the correct option by number matching alone without decoding the uncertainty encoding. The reported 85% and 76% accuracies therefore do not establish that the visual encodings themselves carried the uncertainty semantics. The authors should either redesign distractors to hold numerical values constant or report accuracy separately for trials in which numeric filtering is insufficient to identify the correct answer.","section":"Section 7.3.1 and Appendix B"},{"comment":"Section 4.1 defines the proximity levels for directional comparisons as 3%, 5%, and 8% spatial differences, and Figure 5 uses the same values, but Appendix A.1 Prompt A instructs the LLM to use Close (3% offset), Moderate (5% offset), and Large (10% offset). This inconsistency means the implemented system may render directional comparisons with a 10% rather than 8% offset, contradicting the stated semantic-to-spatial mapping. The encoding specification and the LLM prompt must be reconciled, and any generated stimuli in the user study should be checked against the intended offsets.","section":"Section 4.1 and Appendix A.1"},{"comment":"The two workload results that most directly support the practical-benefit claim are not statistically significant (mental demand p=0.054, effort p=0.073). The sentence in Section 7.2.2 stating that 'when fully powered studies replicate this work, these trends will likely achieve statistical significance' is not an empirical finding but a speculative extrapolation. The authors should present these results as suggestive trends that require replication, and should temper the corresponding claims in the abstract and Section 9.","section":"Section 7.2.2"}],"minor_comments":[{"comment":"Section 5.1 contains an unresolved cross-reference: 'the large language model based uncertainty-aware pipeline (Section ??).' The section number should be inserted.","section":"Section 5.1"},{"comment":"In Section 7.1.3, 'diagonal hathcing ppatern or shaded bars' contains a typo; it should read 'diagonal hatching pattern'.","section":"Section 7.1.3"},{"comment":"The category name is spelled inconsistently: 'Non-Inferable Gaps' appears in the main text and figures, while Appendix A and the JSON schema use 'non_inferrable_gaps' and 'Non-Inferrable Gaps'. The spelling should be standardized.","section":"Throughout"},{"comment":"Section 4.2 refers to 'Figure 5i' when describing line chart adaptations, but the caption of Figure 5 labels the line chart adaptations as 'j-l'. The cross-reference should be corrected.","section":"Section 4.2"},{"comment":"In Table 1, question Q9 is a vertical bar chart with 67% accuracy, which is below the 60% threshold used to gray out rows in the table; please verify that the row-shading rule is applied consistently.","section":"Table 1"},{"comment":"The semantic suitability ratings are reported as averages such as '6 out of 7'; consider reporting standard deviations or full distributions so that the central tendency can be interpreted alongside the matching accuracy results.","section":"Section 7.1.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a solid kernel, but the evaluation design as reported does not currently support the headline matching-accuracy claims. The confounds identified in the major comments are addressable in principle through additional controls (no-orientation baseline, held-out corpus, number-matched distractors), which is why I recommend major revision rather than rejection. The authors' transparency about the failures of specific encodings and about non-significant workload results is a point in their favor, and the appendices make the study unusually easy to scrutinize."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2608.07093. The problem it targets is genuine: text-to-chart systems flatten \"nearly 70,000\" into 70,000, and that misleads readers. The paper's taxonomy—Surface Form Normalization, Precision Boundaries, Inferential Derivation, Non-Inferable Gaps—is a reasonable, literature-grounded framework for what gets lost, and the visual encodings (squiggly tops, hatching, darker borders) are sensible adaptations of known uncertainty visualization techniques. That is a real contribution.\n\nWhat is not established is the central empirical claim that the encodings communicate uncertainty to readers. The 85% chart-to-text and 76% text-to-chart matching numbers come from a design where participants were first taught the taxonomy and its visual encodings in an orientation, then asked to match. The stimuli in Appendix B often vary numbers across options (e.g., \"700,100\" vs. \"a little more than 700,000\"), so the Match-Verify strategy participants describe—check the number first, then the uncertainty marker—can pick the right answer on numeric alignment alone. The 85%/76% rates are therefore compatible with the simpler explanation that people reproduced taught mappings and matched numbers, not that the encodings carried the semantics.\n\nThe other soft spots are in proportion. Part 2 documents come from the same 12-document corpus used to derive the taxonomy, so generalization is untested. The cognitive-load results are not significant (p=0.054, 0.073), and leaning on effect sizes with n=12 is optimistic. There is an internal inconsistency in the proximity offsets: Section 4.1 says large proximity is 8%, the prompt in Appendix A says 10%. And the abstract calls the pipeline \"end-to-end\" while the workflow in Figure 9 has researchers manually selecting uncertainty expressions—the paper later admits this, but the self-description oversells. None of this kills the idea; it caps what the current evidence supports.\n\nThe paper does give real credit where due: the qualitative analysis is candid and specific. The interviews surface design failures (gradients confuse people, \"NA\" labels read as errors, line charts need different treatment) that will be useful to anyone building uncertainty-aware charts. The authors are also honest about the manual pre-selection and the non-significant p-values.\n\nThis is a paper for the visualization/HCI community working on uncertainty and text-to-chart generation. It deserves a serious referee: the taxonomy and the design space are worth engaging, and the evaluation is a useful case study in what confounds such studies. I would send it to review with instructions to the authors to either reframe the claims as a design exploration, or redo the matching part with an independent corpus, no training on the target encodings, and distractor options matched on numbers.","headline":"Real gap and a plausible taxonomy, but the matching study is too confounded to support the central claim.","tokens_in":42252,"tokens_out":3003,"would_cite":false,"duration_ms":28985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hedged numbers like 'nearly' and 'at least' can survive automated chart generation: readers decode them back to the source text with 85% accuracy, and the paper reports that these uncertainty-aware charts trend toward lower mental demand…","keywords":["linguistic uncertainty","uncertainty visualization","text-to-chart generation","data-rich documents","semantic preservation","visual encoding","large language models"],"falsifier":"A controlled study that strips the numeric anchors from the uncertainty encodings, such as showing a squiggly bar with no axis labels and asking participants to pick the matching verbal hedge, would show whether the visual cues alone communicate uncertainty; if accuracy collapses without the numbers, the 85% and 76% figures reflect number-matching plus taught mappings rather than the encodings themselves. A second decisive check would remove the orientation step or test with charts drawn from a held-out corpus.","tokens_in":41280,"feed_emoji":"📊","tokens_out":5011,"duration_ms":42630,"temperature":0.7,"pith_summary":"This paper argues that automated text-to-chart systems should preserve, not discard, the linguistic hedges authors attach to numbers such as 'nearly', 'approximately', and 'between'. It builds UncertaintyVis, a pipeline that classifies hedged statements into a four-part taxonomy and renders each type with a distinct visual cue, while keeping the underlying numbers spatially accurate. In a 12-participant study, readers matched charts back to source text 85% of the time and text to charts 76% of the time, with a non-significant trend toward lower mental demand and a 75% stated preference for the uncertainty-enhanced charts. If the claim holds, automated tools would stop turning deliberately imprecise prose into false-precision graphics.","feed_headline":"Charts that preserve hedged data match reader intent 85% of the time","feed_subtitle":"UncertaintyVis keeps 'nearly' and 'at least' visible in charts; readers decode them back to the source text at 85%.","key_machinery":"The load-bearing mechanism is the mapping from a four-category taxonomy of linguistic uncertainty to deterministic visual encodings: squiggly borders for approximation, diagonal hatching plus interactive question-mark badges for inferred values, darker thicker borders with hover-revealed originals for normalized formats, and N/A labels for missing data. A large language model classifies extracted text into these categories and outputs structured JSON metadata; a renderer applies the encodings on top of standard charts. The taxonomy itself comes from a corpus analysis of 211 uncertainty expressions across 12 documents and 8 domains, and the encoding choices draw on sketchiness and transparency conventions from uncertainty visualization.","core_discovery":"The paper's central claim is that linguistic uncertainty can be preserved in automated chart generation without sacrificing the spatial accuracy readers rely on. The authors derive a four-category taxonomy from a 12-document corpus (Inferential Derivation, Precision Boundaries, Surface Form Normalization, Non-Inferable Gaps), map each category to chart-specific visual encodings, and implement an end-to-end system that uses a large language model to extract data, classify uncertainty, and render encoded charts. In their evaluation, readers matched charts to source text at 85% accuracy and text to charts at 76%, and uncertainty-aware visualizations trended toward lower cognitive demand; 75% of participants preferred them to plain text. The paper also finds that bar and pie encodings work consistently while line chart encodings need redesign.","pith_inferences":["The same taxonomy could extend beyond numerical hedges to epistemic modal language such as 'may', 'likely', and 'suggests', which the paper names as future work but does not implement; visualizing those would cover a much larger share of real-world hedging.","If uncertainty-preserving charts become a convention, they could function as a lightweight verification standard for data journalism, analogous to source citations, a consequence the paper's 'trust infrastructure' framing gestures toward but leaves undeveloped.","A natural testable extension is automatic detection of uncertainty expressions, since the current system requires manual selection of excerpts; integrating detection would let the approach scale to full documents and would test whether the taxonomy is stable outside the corpus used to derive it.","The line-chart failure suggests a broader design principle: uncertainty encodings built for discrete marks such as bars and slices may not transfer to continuous marks such as lines and areas, so each mark type needs its own encoding palette worth testing across area charts and scatterplots."],"forward_implications":["Automated text-to-chart systems that adopt this approach will stop rendering 'nearly 70,000 m²' as an exact 70,000 m², reducing the false-precision effect in data journalism, policy briefs, and medical reports.","Readers of uncertainty-encoded charts can verify claims against source text, effectively turning the visualization into checkable evidence rather than an authoritative-looking graphic.","Bar and pie chart encodings are likely to work in deployed systems; line charts need new encodings such as confidence ribbons or interval markers before coverage is complete.","The trend toward lower mental demand at equal reading time suggests uncertainty annotations can be added without a speed or comprehension penalty, warranting a larger replication study.","The taxonomy gives NLP-based chart generation a target structure: instead of erasing hedges during extraction, the pipeline outputs structured uncertainty metadata that downstream renderers consume."],"supporting_citations":[{"why":"Represents the text-to-chart baseline the paper contrasts: generates charts from paragraphs but discards uncertainty markers during extraction.","marker":"[31]"},{"why":"Another LLM-based text-to-chart system that the paper identifies as flattening hedged values into precise chart marks.","marker":"[40]"},{"why":"Word-scale visualization system whose extraction pipeline drops 'nearly' and similar linguistic qualifiers, motivating the paper's preservation goal.","marker":"[45]"},{"why":"Supplies the argument that authors often avoid visualizing uncertainty and that charts can mislead when hedges are lost, grounding the paper's motivation.","marker":"[18]"},{"why":"Source taxonomy of textual uncertainty in the digital humanities that the four-category framework adapts and extends for visualization design.","marker":"[5]"},{"why":"Evaluation of sketchiness as a visual variable for qualitative uncertainty, grounding the squiggly-border encoding for approximation.","marker":"[6]"},{"why":"Sketchy rendering for information visualization, providing the hand-drawn aesthetic conventions used for uncertain boundaries.","marker":"[39]"},{"why":"Review of NLP measurement extraction that the paper reframes: extraction methods aim to eliminate uncertainty, motivating the preservation-focused alternative.","marker":"[15]"},{"why":"Survey of number representation in NLP, used to motivate the surface-form-normalization category and its visual treatment.","marker":"[36]"}],"fun_headline_variants":["Charts that keep 'nearly' visible match source text 85%","UncertaintyVis: hedged data charts hit 85% reader accuracy","Preserving hedging in charts: 85% accurate source matching","Automated charts that show uncertainty match source 85%","New system keeps 'nearly' and 'at least' in charts, 85% match"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the measured matching accuracy shows the visual encodings themselves carry the uncertainty semantics, but participants may have relied on simpler cues: they received an orientation on the taxonomy before Part 1, the Part 2 documents come from the same 12-document corpus used to build the taxonomy, and participants described a Match-Verify strategy that checks exact numeric alignment before evaluating uncertainty markers.","fun_headline_variants_meta":{"raw":{"variants":["Charts that keep 'nearly' visible match source text 85%","UncertaintyVis: hedged data charts hit 85% reader accuracy","Preserving hedging in charts: 85% accurate source matching","Automated charts that show uncertainty match source 85%","New system keeps 'nearly' and 'at least' in charts, 85% match"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000891,"raw_usage":{"total_tokens":3859,"prompt_tokens":977,"completion_tokens":2882,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":2784}},"tokens_in":593,"tokens_out":2882,"duration_ms":19466,"temperature":1.0,"reasoning_tokens":2784,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:02:21.170842+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled study that strips the numeric anchors from the uncertainty encodings, such as showing a squiggly bar with no axis labels and asking participants to pick the matching verbal hedge, would show whether the visual cues alone communicate uncertainty; if accuracy collapses without the numbers, the 85% and 76% figures reflect number-matching plus taught mappings rather than the encodings themselves. A second decisive check would remove the orientation step or test with charts drawn from a held-out corpus.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Represents the text-to-chart baseline the paper contrasts: generates charts from paragraphs but discards uncertainty markers during extraction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Another LLM-based text-to-chart system that the paper identifies as flattening hedged values into precise chart marks."},{"cited_title":"chart_type","cited_arxiv_id":null,"evidence_quote":"Word-scale visualization system whose extraction pipeline drops 'nearly' and similar linguistic qualifiers, motivating the paper's preservation goal."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the argument that authors often avoid visualizing uncertainty and that charts can mislead when hedges are lost, grounding the paper's motivation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source taxonomy of textual uncertainty in the digital humanities that the four-category framework adapts and extends for visualization design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Sketchy rendering for information visualization, providing the hand-drawn aesthetic conventions used for uncertain boundaries."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Review of NLP measurement extraction that the paper reframes: extraction methods aim to eliminate uncertainty, motivating the preservation-focused alternative."}],"review_version":1}