Pith. sign in

REVIEW 3 major objections 4 minor 26 references

Changing the Paradigm from Dynamic Queries to LLM-generated SQL Queries with Human Intervention

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This position paper argues that LLM-generated, human-editable SQL queries should replace the dynamic-query widgets of medical visualization systems, letting clinicians express analytical intent in natural language while preserving transpare

desk verdict A clearly argued position paper on LLM-generated SQL replacing dynamic query widgets, but the central premise is untested and the abstract overclaims. read the letter →

arxiv 2509.09461 v1 pith:M7PIBYSH submitted 2025-09-11 cs.HC

Ambre Assor , Hyeon Jeon , Sungbok Shin , Jean-Daniel Fekete This is my paper
classification cs.HC
keywords LLMnaturallanguagetoSQLdynamicquerieselectronichealthrecordsvisualizationinteractivefilteringhumanintervention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the dynamic-query sliders, checkboxes, and dropdowns used in traditional visualization systems do not scale to high-dimensional coded health records like the French national health database, which has thousands of attributes. The authors propose using an LLM as an interaction layer: a doctor states an analytical intent in natural language, the LLM produces one or more SQL queries, and the patient pathway visualization runs that SQL. To preserve trust and agency, the SQL is exposed for editing and paired with a plain-language explanation, so users can check for hallucination and learn the database's codes. They also propose bringing back dynamic widgets on demand for a selected subset of queries. A single-query demonstration on a flat table suggests feasibility, but the paradigm's value rests on LLM reliability over the full multi-table schema.

What carries the argument

The two-stage pipeline NL→SQL→Vis: (1) an LLM translates a doctor's natural-language request into one or more SQL queries, presented with an editable query box and a plain-language explanation; (2) the query is executed and drives the visualization. On-demand dynamic query widgets are generated by extracting SQL subtrees from prior or current queries, offering rapid, incremental, reversible filtering without sacrificing the scalability of the LLM approach.

What would settle it

Run the same or a similar LLM translator on the real multi-table schema with a battery of realistic clinician questions; if a substantial fraction of the generated SQL contains undetected errors—wrong joins, wrong code mappings, or hallucinated column names—that experts fail to catch in review, then the human-intervention model does not provide reliable exploration.

Watch

Extended reading notes

Core claim

The central claim is a paradigm shift: instead of building ever-larger panels of dynamic queries, visualization systems for medical data can treat a large language model as the query-generation layer. The user's natural-language intent is converted into editable, executable SQL, and the visualization runs that SQL; the generated code plus its natural-language explanation becomes a transparency mechanism that lets expert users detect mismatches, learn the database's coded vocabulary, and reuse or share queries. The paper also proposes a hybrid: extract SQL subtrees to dynamically generate traditional widgets when rapid incremental control is useful. This is demonstrated in ParcoursVis, an Eve

Load-bearing premise

The paradigm depends on LLMs translating natural-language medical queries into correct SQL over the full multi-table schema, but the paper's own test covers only a single query on a single flat table, and it acknowledges that hallucinations remain a problem.

Editorial extensions

If this is right

  • Clinicians can explore high-dimensional electronic health records without memorizing opaque field names or alphanumeric codes, reducing the need for large widget panels.
  • Exposing generated SQL as editable, scriptable code gives users ownership of the filtering logic and lets them save, share, and deterministically reuse queries.
  • Natural-language explanations paired with each query help users learn the database structure and spot LLM hallucinations before the query runs.
  • Reintroducing dynamic queries on demand, generated from SQL subtrees, preserves rapid incremental reversible control for the subset of filters that benefit from direct manipulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The single-query demonstration on a flattened table suggests that how an LLM maps natural-language terms to database codes is the bottleneck; a practical system might augment the LLM with an external dictionary lookup or retrieval step to ground those codes.
  • If the paradigm matures, a testable extension is to measure whether experts learn the database schema faster from explanations than from traditional widgets, and whether trust in LLM output changes after repeated editing.
  • A hybrid interaction model could automatically decide when to show a dynamic widget (for frequent, low-dimensional filters) and when to fall back to natural-language SQL (for rare, high-dimensional filters); the paper hints at this but leaves the decision rule unspecified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes replacing traditional dynamic-query widget panels in medical EHR visualization systems with an interaction layer in which users express analytical intents in natural language, an LLM translates these intents into editable, executable SQL queries, and each query is accompanied by a plain-language explanation. The authors argue this reduces visual clutter, removes the need to memorize database field names and system codes, and improves user agency and transparency. They illustrate the idea with ParcoursVis, an EventFlow-inspired scalable visualization system for French SNDS data, and present a single worked example of an LLM translating a natural-language request into SQL for the flat ER_PRSF table. They also discuss limitations, including LLM hallucinations and the complexity of multi-table schemas, and propose reintroducing dynamic queries on demand by extracting SQL subtrees. The paper is essentially a position/vision statement, but its abstract and conclusions claim that the paradigm is demonstrated in ParcoursVis.

Significance. If the central claim were validated, the contribution would be timely and potentially useful: replacing dense, space-limited dynamic-query widgets with editable LLM-generated SQL plus natural-language explanations could improve scalability and learnability for expert medical users exploring large coded EHR databases. The paper credits concrete, checkable design elements: it uses an open-source LLM with a specified prompt, exposes the generated SQL for editing, pairs it with a plain-language explanation, explicitly acknowledges hallucination as a risk, and proposes a concrete path for recovering dynamic-query behavior from SQL subtrees. These are real strengths for a vision paper. However, the significance for a research publication depends on evidence that the interaction model actually works: that LLM-to-SQL translation is sufficiently reliable over a realistic multi-table schema, and that medical users can detect and correct LLM errors. Currently the manuscript provides no user study, no accuracy measurement, and no comparison with dynamic-query interfaces, so the claims remain untested design hypotheses rather than demonstrated results.

major comments (3)
  1. [Abstract; §3] The abstract states 'We demonstrate this paradigm in ParcoursVis,' but §3 explicitly says the LLM-based step 'is not integrated into the final system yet.' The only evidence is a single natural-language-to-SQL example run against the flat ER_PRSF table, with the output shown as a prototype in Fig. 1(b). No integration with ParcoursVis's multi-table SNDS pipeline is shown. The load-bearing claim of a demonstrated paradigm is therefore not supported by the manuscript as written. The paper can be reframed as a position/vision paper, or it must provide an actual integration and demonstration against the real multi-table schema.
  2. [§3; §4] The central premise—that LLM-generated SQL can effectively replace dynamic-query widgets for real SNDS exploration—is not evaluated. §3 tests one query on a single flat table, while §4 itself enumerates the hard parts: multiple tables, opaque codes, duplicate records, join-path selection, and hallucinations. No accuracy metric, no workload of representative queries, and no comparison with dynamic-query baselines is reported. Because all downstream benefits (agency, transparency, scalability) are conditional on this premise, the paper either needs to be explicitly positioned as an untested research agenda or supplemented with a feasibility study measuring end-to-end correctness on realistic multi-table queries.
  3. [§2; §4] The human-in-the-loop validation mechanism is under-specified and potentially circular. §2 claims the natural-language explanation 'helps detect LLM mismatches or hallucinations,' but the explanation is generated by the same LLM that produced the SQL, so it is not an independent check. Moreover, the paper's core motivation is that medical users do not remember field names and system codes; without that schema knowledge, how are they to verify that the SQL joins the correct tables, disambiguates duplicate records, and uses the right codes? The manuscript should articulate an independent verification strategy (e.g., query-result statistics, code-level provenance, or a user study in which target users are tested on their ability to catch injected errors) before claiming that human intervention makes the paradigm safe.
minor comments (4)
  1. [§3, Fig. 1/2] The generated SQL itself is not shown in the text or figures; only the explanation is visible in Fig. 2. Without the SQL, the reader cannot assess the correctness of the example or the plausibility of the demo.
  2. [§2, Step 1] The phrase 'they are deterministic' is ambiguous: the saved SQL queries are deterministic, but the LLM translation process is explicitly non-deterministic. Clarify that the stored query artifacts, not the generation process, are repeatable.
  3. [Various] The paper alternates between 'dynamic queries' (the interaction principles from Shneiderman) and 'dynamic query widgets' (the concrete UI controls). Maintaining this distinction would help readers follow the argument that extracting SQL subtrees revives dynamic-query properties rather than re-adding widgets.
  4. [§4] The OMOP and flattening recommendations are plausible but introduced as remedies without analysis of their cost or their effect on the LLM-to-SQL task. At least a pointer to prior work on LLM performance over OMOP-flattened schemas would strengthen the argument.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a design proposal with no fitted equations, no derived predictions, and no load-bearing self-citations; acknowledged LLM limitations are honest caveats, not circular reductions.

full rationale

This paper is a position/vision paper rather than a derivation. It contains no equations, no fitted parameters, and no quantitative prediction that could reduce to its inputs by construction. The central proposal—LLM-generated editable SQL with natural-language explanations and human intervention—is presented as a design hypothesis. The only feasibility test (Section 3) is explicitly labeled preliminary: 'This step was part of an initial investigation to determine whether LLM-based query generation could be viable in our context (it is not integrated into the final system yet).' Section 4 explicitly acknowledges hallucinations, duplicate records, multi-table joins, and schema standardization issues, so the paper does not claim to have solved the central reliability problem. Self-citations to ParcoursVis [1], progressive analysis [5], and ComBiNet [16] provide context and prior mechanisms but are not load-bearing: removing them would not change the proposal's validity. The design's reliance on an LLM-generated explanation as a check on the LLM-generated SQL is a limitation of the proposed transparency mechanism, not a circular reduction of a derived result to its inputs. No step in the paper's argument is equivalent to its own premise by definition, and no fitted input is renamed as a prediction. The honest non-finding is therefore appropriate: score 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper's argument rests on three untested premises about user knowledge, LLM reliability, and the scalability limit of dynamic query widgets. None of these are derived or measured in the preprint.

assumptions (3)
  • domain assumption Medical users know the available filtering options but struggle to recall attribute names and code values.
    Stated in the Abstract and Section 2: 'we posit that medical users are trained to know the possible filtering options but challenged to remember the details of the attribute names and code values.' This motivates the entire approach and is not empirically tested.
  • domain assumption LLM-generated SQL for the SNDS schema is accurate enough for practical medical exploration.
    Section 3 tests one Mistral query against the flat ER PRSF table; Section 4 acknowledges hallucinations. The paper provides no accuracy measure, yet the viability of the paradigm depends on this premise.
  • domain assumption Dynamic query widgets cannot scale to the number of attributes in SNDS (over 3,000 variables).
    Section 1 argues 'the limited screen real estate cannot present all the dynamic query widgets'. This is a design constraint adopted without a formal or empirical demonstration that dynamic queries would fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Changing the Paradigm from Dynamic Queries to LLM-generated SQL Queries with Human Intervention." pith.science (2026). https://pith.science/paper/M7PIBYSH

@misc{pith2026250909461,
  author       = {Pith},
  title        = {Pith review of: Changing the Paradigm from Dynamic Queries to LLM-generated SQL Queries with Human Intervention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M7PIBYSH}},
  note         = {Machine review of arXiv:2509.09461}
}
read the original abstract

We propose leveraging Large Language Models (LLMs) as an interaction layer for medical visualization systems. In domains like healthcare, where users must navigate high-dimensional, coded, and heterogeneous datasets, LLM-generated queries enable expert medical users to express complex analytical intents in natural language. These intents are then translated into editable and executable queries, replacing the dynamic query interfaces used by traditional visualization systems built around sliders, check boxes, and drop-downs. This interaction model reduces visual clutter and eliminates the need for users to memorize field names or system codes, supporting fluid exploration, with the drawback of not exposing all the filtering criteria. We also reintroduce dynamic queries on demand to better support interactive exploration. We posit that medical users are trained to know the possible filtering options but challenged to remember the details of the attribute names and code values. We demonstrate this paradigm in ParcoursVis, our scalable EventFlow-inspired patient care pathway visualization system powered by the French National Health Data System, one of the largest health data repositories in the world.

Figures

Figures reproduced from arXiv: 2509.09461 by the authors.

Figure 1
Figure 1. (a): Dynamic query widgets from the current version of the ParcoursVis aggregated EHR sequence visualization system (see the Filter tab on the right panel at parcoursvis.lisn.upsaclay.fr). The panel has limited space and cannot show the large number of available attributes and value codes for filtering patients. (b): Prototype of our vision using an LLM to translate a natural language query into an SQL query featuri… view at source ↗
Figure 2
Figure 2. Example of explanations provided by our natural language [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 2 canonical work pages

  1. [1]

    Assor, M

    A. Assor, M. Sereno, and J.-D. Fekete. Visualization of electronic health record sequences at scale.arXiv preprint arXiv:2508.10700,

    arXiv

  2. [2]

    R. C. Basole and T. Major. Generative AI for Visualization: Oppor- tunities and Challenges.IEEE Computer Graphics and Applications, 44(2):55–64, 2024. doi: 10.1109/MCG.2024.3362168 2

    arXiv 2024

  3. [3]

    Ceneda, C

    D. Ceneda, C. Collins, M. El-Assady, S. Miksch, C. Tominski, and A. Arleo. A heuristic approach for dual expert/end-user evaluation of guidance in visual analytics.IEEE Trans. Vis. Comput. Graph., 30(1):997–1007, 2023. 2

    2023

  4. [4]

    Ceneda, T

    D. Ceneda, T. Gschwandtner, and S. Miksch. A review of guidance approaches in visual data analysis: A multifocal perspective.Com- puter Graphics Forum, 38(3):861–879, 2019. doi: 10.1111/cgf.13730 1

    doi:10.1111/cgf.13730 2019

  5. [5]

    Fekete, D

    J.-D. Fekete, D. Fisher, and M. Sedlmair.Progressive Data Analysis: Roadmap and Research Agenda. Eurographics, Nov. 2024. doi: 10. 2312/pda.20242707 2

    2024

  6. [6]

    Huang, W

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Trans. Inf. Syst., 43(2), article no. 42, 55 pages, Jan

  7. [7]

    Huang, L

    Y . Huang, L. Sun, H. Wang, S. Wu, Q. Zhang, Y . Li, C. Gao, Y . Huang, W. Lyu, et al. Position: TrustLLM: Trustworthiness in Large Lan- guage Models. InForty-first International Conference on Machine Learning, (ICML). OpenReview.net, 2024. 2

    2024

  8. [8]

    J. Kim, S. Lee, H. Jeon, K.-J. Lee, H.-J. Bae, B. Kim, and J. Seo. Phe- noFlow: A Human-LLM Driven Visual Analytics System for Explor- ing Large and Complex Stroke Datasets.IEEE Trans. Vis. Comput. Graph., 2024. 2

    2024

Show all 26 references
  1. [9]

    S. Lee, H. Song, J.-c. Lee, Y . J. Lee, B. Lee, H.-E. Lim, D. Kim, J. Seo, and B. Kim. MedOrchestra: A Hybrid Cloud-Local LLM Approach for Clinical Data Interpretation.arXiv preprint arXiv:2505.23806,

    arXiv

  2. [10]

    J. Li, B. Hui, G. Qu, J. Yang, B. Li, B. Li, B. Wang, B. Qin, R. Geng, N. Huo, X. Zhou, C. Ma, G. Li, K. C. Chang, F. Huang, R. Cheng, and Y . Li. Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs. InAdvances in Neural In...

    2023

  3. [11]

    Masson, S

    D. Masson, S. Malacria, G. Casiez, and D. V ogel. Directgpt: A di- rect manipulation interface to interact with large language models. In Proceedings of the ACM Conference on Human Factors in Computing Systems, article no. 975, 16 pages. Association for Computing Ma- chinery, ...

    2024

  4. [12]

    Ac- cessed: 2025-06-30

    Mistral-small-3.1-24b-instruct-2503.https://huggingface.co/ mistralai/Mistral-Small-3.1-24B-Instruct-2503. Ac- cessed: 2025-06-30. 2

    2025

  5. [13]

    Mohammadjafari, A

    A. Mohammadjafari, A. S. Maida, and R. Gottumukkala. From natural language to SQL: Review of LLM-based text-to-SQL systems.arXiv preprint arXiv:2410.01066, 2024. 2

    2024 arXiv

  6. [14]

    Monroe, R

    M. Monroe, R. Lan, H. Lee, C. Plaisant, and B. Shneiderman. Tempo- ral Event Sequence Simplification.IEEE Trans. Vis. Comput. Graph., 19(12):2227–2236, 2013. doi: 10.1109/TVCG.2013.200 1, 2

    2013 doi

  7. [15]

    Accessed: 2025-06-

    Standardized Data: The OMOP Common Data Model.https:// www.ohdsi.org/data-standardization/. Accessed: 2025-06-

    2025

  8. [16]

    Pister, C

    A. Pister, C. Prieur, and J.-D. Fekete. ComBiNet: Visual Query and Comparison of Bipartite Multivariate Dynamic Social Networks. Computer Graphics Forum, 42(1):290–304, 2023. doi: 10.1111/CGF .14731 2

    2023 doi

  9. [17]

    Shneiderman

    B. Shneiderman. Direct Manipulation: A Step Beyond Programming Languages.Computer, 16(8):57–69, 13 pages, Aug. 1983. doi: 10. 1109/MC.1983.1654471 1

    1983

  10. [18]

    Shneiderman

    B. Shneiderman. Dynamic queries for visual information seeking. IEEE Software, 11(6):70–77, 1994. doi: 10.1109/52.329404 1, 2

    1994 doi

  11. [19]

    Shneiderman

    B. Shneiderman. Human responsibility for autonomous agents.IEEE Intelligent Systems, 22(2):60–61, March 2007. doi: 10.1109/MIS. 2007.32 2

    2007 doi

  12. [20]

    Shneiderman.Human-centered AI

    B. Shneiderman.Human-centered AI. Oxford University Press, 2022. 2

    2022

  13. [21]

    Accessed: 2025-06-30

    Documentation collaborative du SNDS.https:// documentation-snds.health-data-hub.fr/. Accessed: 2025-06-30. 1

    2025

  14. [22]

    Tuppin, J

    P. Tuppin, J. Rudant, P. Constantinou, C. Gastaldi-M ´enager, A. Rachas, L. de Roquefeuil, G. Maura, H. Caillol, A. Tajahmady, J. Coste, C. Gissot, A. Weill, and A. Fagot-Campagna. Value of a national administrative database to guide public decisions: From the syst`eme nationa...

  15. [23]

    Wang and R

    Q. Wang and R. S. Laramee. EHR STAR: the state-of-the-art in inter- active EHR visualization.Computer Graphics Forum, 41(1):69–105,

  16. [2017]

    doi: 10.1016/j.respe.2017.05.004 1, 2

    R ´eseau REDSIAM. doi: 10.1016/j.respe.2017.05.004 1, 2

    2017 doi

  17. [2022]

    doi: 10.1111/CGF.14424 1

    doi

  18. [2025]

    doi: 10.1145/3703155 2

    doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.