Pith. sign in

REVIEW 4 major objections 5 minor 45 references

ChatVis: Large Language Model Agent for Generating Scientific Visualizations

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that retrieving ParaView documentation and feeding execution errors back to the LLM makes script generation succeed 95% of the time, against 50% for the best unassisted model.

desk verdict A useful RAG-plus-error-correction system for ParaView scripting with a solid benchmark, but the headline comparison is undercut by possible corpus contamination and uncontrolled baselines. read the letter →

arxiv 2507.23096 v1 pith:C7PRXVJK submitted 2025-07-30 cs.HC

Tom Peterka , Tanwi Mallick , Orcun Yildiz , David Lenz , Cory Quammen , Berk Geveci This is my paper
classification cs.HC
keywords largelanguagemodelsscientificvisualizationParaViewretrieval-augmentedgenerationcodevectordatabasebenchmarkingiterativeerrorcorrection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a large language model, without any retraining or fine-tuning, can become reliable at writing ParaView Python scripts for 3D and time-varying scientific visualization when wrapped in an assistant that retrieves API documentation and iteratively corrects execution errors. The evidence is a 20-task benchmark covering canonical visualizations, ParaView regression tests, and two real science workflows. ChatVis completes 95% of the tasks without syntax errors, while the best unassisted model completes 50%, and the rendered images are closer to hand-written references on SSIM, PSNR, and LPIPS. If the claim holds, it gives domain scientists a natural-language path to complex visualizations and shifts the problem of specialized-code generation from model training to external knowledge access.

What carries the argument

The load-bearing mechanism is a retrieval-augmented generation loop centered on a ParaView-specific vector database. A natural-language request is first simplified through chain-of-thought-style expansion into a list of concrete API operations; the operations are embedded and used to retrieve code examples and function documentation from a Faiss vector database built from the ParaView codebase and documentation; the retrieved text is appended to the prompt so the LLM generates pvpython code with correct function names and argument orders; and a traceback parser feeds execution errors back into the prompt for iterative repair until the script executes. The vector database supplies exactly the API knowledge that the base LLM hallucinates.

What would settle it

Embed the 20 reference scripts and query the vector database with the benchmark task prompts; if any reference script or near-duplicate appears among the retrieved chunks, ChatVis's 95% pass@1 and image scores are inflated because the answer is directly retrievable.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the gap between LLM-generated and expert-written visualization code is largely a knowledge-access gap rather than a reasoning gap. A user prompt is decomposed into individual ParaView operations, each operation is used to retrieve matching documentation and code snippets from a vector database built from the ParaView codebase and manual, and the retrieved context is fed to the LLM together with the original request. The generated pvpython script is executed, error messages are extracted from the traceback, and those messages are returned to the LLM for another round until the script runs. The paper reports that this pipeline reaches 95% pass@1 and average image scores of SSIM 0.80, PSNR 40.1 dB, and LPIPS 0.26 over the completed tasks, and that disabling retrieval drops pass@1 to 70%, showing the vector database is the main driver of the improvement.

Load-bearing premise

The comparison assumes the vector database does not already contain the hand-written reference scripts (or near copies) used to create the ground-truth images, because those scripts come from the same ParaView codebase and tests that the database is built from.

Editorial extensions

If this is right

  • Specialized visualization scripting becomes accessible without fine-tuning: any LLM can be upgraded by supplying an indexed API corpus and an error-feedback loop.
  • Retrieval, not model scale, accounts for most of the reliability gain: replacing RAG with few-shot prompting drops the completion rate from 95% to 70%.
  • Shorter, more conversational prompts still outperform unassisted full-prompt generation, so the assistant helps realistic users who omit details.
  • The same architecture should transfer to other Python-scriptable visualization tools such as VisIt, since it depends on documentation retrieval rather than ParaView-specific training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported image metrics are averaged only over tasks that ran without errors, so the unassisted models' quality scores are computed on an easier subset; a task-scaled comparison would widen the measured gap further.
  • Because the benchmark's regression tests are adapted from ParaView's own test suite and the RAG corpus comes from ParaView's codebase and documentation, the 95% figure is only trustworthy if those reference scripts are not retrievable from the vector database; a contamination check would settle this.
  • The current correction loop sees only syntax and runtime errors, not image content; extending it to compare rendered screenshots against the prompt could fix the residual color-range and camera-placement mismatches visible in the science cases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents ChatVis, an LLM-based assistant for generating ParaView Python scripts from natural-language prompts. The system combines chain-of-thought prompt decomposition, retrieval-augmented generation from a vector database of ParaView documentation and code examples, and an iterative error-correction loop that feeds execution error messages back to the LLM. The authors introduce a benchmark of 20 tasks (canonical visualizations, ParaView regression tests, and scientific use cases) with reference images, reference scripts, and two prompt variants, and they compare ChatVis against several unassisted LLMs using pass@1, SSIM, PSNR, and LPIPS. The central claims are that ChatVis achieves 95% pass@1 versus 50% for the best unassisted model, that image metrics are substantially better, that RAG outperforms few-shot prompting, and that longer full prompts outperform shorter quick prompts.

Significance. If the findings hold, this is a useful contribution to the emerging area of LLM-assisted scientific visualization. The paper provides a concrete benchmark suite with reference scripts and images, a reproducible pipeline based on open components, and a controlled comparison of RAG versus few-shot prompting that goes beyond the proof-of-concept workshop paper. The iterative error-correction loop and the explicit discussion of prompt ambiguity are valuable. However, the headline comparison against unassisted LLMs is not yet cleanly established: the evaluation has a plausible retrieval-contamination risk because the RAG corpus and ten benchmark tasks both derive from ParaView's own code base, and the image-quality metrics can penalize valid alternative visualizations. The benchmark itself and the RAG-versus-few-shot comparison are the most defensible parts of the paper, while the absolute comparison against unassisted models needs additional controls before the strongest claims can be accepted.

major comments (4)
  1. [Section 3.1 and Section 4.1 (Table 1)] The RAG corpus is built from the ParaView code base and documentation (Section 3.1), and ten of the twenty benchmark tasks are adapted from ParaView's own regression-test framework (Section 4.1, Table 1). The paper does not state whether the reference scripts for those regression-derived tasks, or near copies of them, were excluded from the vector database. If such scripts are retrievable, ChatVis's RAG component could return solutions very close to the ground truth for ten of twenty tasks, inflating both pass@1 and the image metrics in a way that does not reflect general capability. This is a load-bearing concern for the headline claim. The authors should report an overlap analysis between the benchmark reference scripts and the indexed corpus (e.g., exact-match and near-duplicate retrieval checks), exclude any leaked examples, and ideally re-evaluate on a held-out set of tasks not present in ParaView's code base.
  2. [Section 4.2, Table 2, and Section 4.4, Table 4] The main comparison is not controlled: ChatVis receives retrieval from a vector database and an iterative error-correction loop, while the unassisted LLMs generate a single script with no retrieval and no chance to fix errors. The 95% versus 50% pass@1 therefore conflates the assistant architecture with the underlying model capability. The RAG-versus-few-shot comparison in Table 4 is a step in the right direction, but the few-shot arm is explicitly capped at 5 iterations (Section 4.4) while the RAG arm is described as generally needing only one iteration. The authors should specify whether the RAG arm was also capped at 5 iterations, report sensitivity to the iteration cap, and present the unassisted-model comparison as a pipeline comparison rather than a model comparison unless the same correction budget is given to all arms.
  3. [Section 4.2, Eq. (1), and Section 5.1] The image metrics are averaged only over tasks that executed without syntax errors, so the number of contributing tasks varies widely across models (e.g., 3 of 20 for LLaMa-3.1-405B versus 19 of 20 for ChatVis in Table 2). This makes the average SSIM/PSNR/LPIPS values difficult to compare. More importantly, the metrics can penalize valid alternatives whose differences stem from prompt ambiguity: Section 5.1 explicitly notes that a color-bar range of [0,0.34] versus the ground truth's [0,0.46] arises from an unspecified order of operations and that both results are 'correct' with respect to the prompt. Because the benchmark deliberately leaves settings such as background color, camera position, and operation order unspecified, the reported image-quality differences are not a clean measure of correctness or quality. The authors should report the scaled metrics from Eq. (1), provide per-task metric tables, and either add a human evaluation or restrict the image-quality claim to tasks where prompt ambiguity cannot explain the difference.
  4. [Section 4.2 and Section 4.3] The paper defines pass@1 as whether the generated script runs without syntax errors, regardless of the content of the resulting image or animation (Section 4.2). Section 4.3 then states that ChatVis 'successfully completes 95% of the visualization tasks' and that o1pro is 'only able to complete half of the tasks.' This wording overstates what pass@1 measures: a script that executes but produces a wrong or meaningless visualization is counted as a success. The authors should either present a separate semantic-correctness metric or consistently describe pass@1 as 'syntax-error-free execution' rather than task completion, and adjust the abstract and conclusion accordingly.
minor comments (5)
  1. [Abstract and Section 4.3] The phrase 'all the metrics are significantly improved' suggests statistical significance, but no significance tests, confidence intervals, or repeated-run variance are reported anywhere in the paper. Please rephrase as 'improved in our experiments' or add appropriate statistical analysis.
  2. [Section 6.1] The conclusion refers to 'LIPS' in the sentence about image metrics; this should be 'LPIPS'.
  3. [Section 4.4, Table 4] The text says the few-shot prompting approach is limited to a maximum of 5 iterations but does not state whether the RAG approach was also limited. Please report the iteration counts for both arms explicitly.
  4. [Section 2.5.3] The paper justifies omitting code-comparison metrics by saying they 'did not agree with image metrics,' but no details or numbers are given. A brief illustrative example or reference to the workshop paper would make this methodological decision more transparent.
  5. [Figure 3 and Section 4.3] The reset-camera-direction example is convincing, but it is a single task. Consider showing a representative failure or success for at least one task from each benchmark category, especially since image metrics are averaged over an uneven number of tasks.

Circularity Check

1 steps flagged · score 4.0 of 10

RAG corpus is built from the same ParaView code base that supplies the reference scripts for ten of twenty benchmark tasks; without an exclusion statement, ChatVis's success on those tasks may be retrieval rather than generation.

  1. self definitional [Section 3.1 (Data Sources and Preprocessing) and Section 4.1 (Benchmark Example Tasks)]
    "The foundation of this methodology relies on two primary data sources: the ParaView code base and its corresponding documentation. ... Ten tasks in the benchmark are adapted from ParaView’s own regression testing framework. ... We then use the extracted list of operations to retrieve the corresponding code descriptions and code snippets from the vector DB."

    By the paper's own description, the RAG corpus is built from the ParaView code base (Sec. 3.1), and the reference scripts for ten benchmark tasks are 'adapted from ParaView's own regression testing framework', i.e., from that same code base (Sec. 4.1). Since retrieval returns 'code descriptions and code snippets from the vector DB' (Sec. 3.3), the correct solution for a regression-test task can be among the retrieved snippets; the paper does not state that reference or regression scripts were excluded from the corpus. Thus ChatVis's pass@1 and image metrics on half the benchmark can reduce to looking up the answer in its input source rather than predicting it, making the 95%-vs-50% comparison partially forced.

full rationale

One concrete overlap is identifiable from the paper's own text: the retrieval corpus and the benchmark's regression-test ground truth share the ParaView code base as their origin. This is a partial circularity in the evaluation for the ten regression-test tasks. However, the remaining ten tasks (canonical visualizations and scientific use cases) are independent of that particular source overlap, and the paper's ablations (RAG vs few-shot, full vs quick prompt) provide additional empirical content that does not reduce to a definition. The self-citation to the prior workshop paper [25] is contextual, not load-bearing. Overall the central claim has substantial independent content, but the headline comparison is partially contaminated by the unstated possible inclusion of reference-adjacent scripts in the vector DB.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The system's performance depends on several hand-chosen components and domain assumptions. The largest risk is that the test benchmark overlaps with the retrieval corpus, which would inflate the main results. The metrics also depend on assumptions about what counts as a correct visualization.

free parameters (3)
  • few-shot error-correction iteration cap = 5
    Section 4.4 limits the few-shot prompting baseline to a maximum of 5 correction iterations, while RAG generally required 1. This hand-chosen cap may disadvantage the baseline.
  • RAG retrieval top-k
    The number of retrieved snippets injected into the prompt is not reported; it is a hand-chosen design parameter that affects generation quality.
  • Embedding model = all-MiniLM-L6-v2
    Section 3.1 selects this Hugging Face model by hand; retrieval quality depends on this choice.
assumptions (5)
  • domain assumption The ParaView code base and documentation in the vector DB provide sufficient correct information for all 20 benchmark tasks.
    RAG performance depends on the corpus containing the needed API calls; if a task requires a function not in the corpus, retrieval cannot help.
  • domain assumption The reference scripts for the regression-test tasks are not included in or retrievable from the vector DB.
    The benchmark includes tasks adapted from ParaView regression tests while the RAG corpus is built from the ParaView code base; no exclusion is stated.
  • domain assumption pass@1 and image similarity metrics are valid proxies for visualization correctness.
    The paper notes prompts allow variability in background color and camera, so a valid alternative visualization could score poorly on SSIM, PSNR, or LPIPS.
  • domain assumption The ground truth images from hand-written scripts are the correct reference for all tasks.
    Prompts leave settings unspecified, so other outputs can be equally correct, as the authors acknowledge for the climate case in Section 5.1.
  • domain assumption Single-run generation is a meaningful estimate of model performance.
    Only one generation per model and task is reported, with no sampling variance or confidence intervals.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ChatVis: Large Language Model Agent for Generating Scientific Visualizations." pith.science (2026). https://pith.science/paper/C7PRXVJK

@misc{pith2026250723096,
  author       = {Pith},
  title        = {Pith review of: ChatVis: Large Language Model Agent for Generating Scientific Visualizations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C7PRXVJK}},
  note         = {Machine review of arXiv:2507.23096}
}
read the original abstract

Large language models (LLMs) are rapidly increasing in capability, but they still struggle with highly specialized programming tasks such as scientific visualization. We present an LLM assistant, ChatVis, that aids the LLM to generate Python code for ParaView scientific visualization tasks, without the need for retraining or fine-tuning the LLM. ChatVis employs chain-of-thought prompt simplification, retrieval-augmented prompt generation using a vector database of documentation and code examples, and error checking with iterative prompt feedback to correct errors until a visualization is produced. An integral part of our approach is a benchmark suite of canonical visualization tasks, ParaView regression tests, and scientific use cases that includes comprehensive evaluation metrics. We evaluate our visualization assistant by comparing results with a variety of top-performing unassisted LLMs. We find that all the metrics are significantly improved with ChatVis.

Figures

Figures reproduced from arXiv: 2507.23096 by the authors.

Figure 1
Figure 1. ChatVis-generated visualization of pathlines traced from a vector field simulated by an ocean climate model. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. System architecture for LLM-assisted ParaView scripting. The process involves embedding models, a vector database, and iterative code [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Generated images for reset-camera-direction task. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Climate science: pathlines traced from a vector field of the SOMA [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Materials science: comparison of coherent diffraction imaging of [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 27 canonical work pages

  1. [1]

    https://huggingface.co/ sentence-transformers/all-MiniLM-L6-v2 , 2025

    all-MiniLM-L6-v2. https://huggingface.co/ sentence-transformers/all-MiniLM-L6-v2 , 2025. [Online; accessed 27-March-2025]. 3

    work page 2025

  2. [2]

    U. Ayachit. The ParaView Guide: A Parallel Visualization Application. Kitware, Inc., 2015. 1

    work page 2015

  3. [3]

    Bhardwaj, I

    S. Bhardwaj, I. Fischer, J. Ballé, and T. Chinen. An unsupervised information-theoretic perceptual quality metric. Advances in Neural Infor- mation Processing Systems, 33:13–24, 2020. 3

    work page 2020

  4. [4]

    Chen, P.-H

    L. Chen, P.-H. Lin, T. Vanderbruggen, C. Liao, M. Emani, and B. de Supin- ski. LM4HPC: Towards Effective Language Model Application in High- Performance Computing. In International Workshop on OpenMP , pp. 18–33. Springer, 2023. 2

    work page 2023

  5. [5]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, et al. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374,

    arXiv

  6. [6]

    N. Chen, Y . Zhang, J. Xu, K. Ren, and Y . Yang. VisEval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics, 2024. 2

    work page 2024

  7. [7]

    Cheung and A

    S.-S. Cheung and A. Zakhor. Efficient video similarity measurement with video signaturem. IEEE Transactions on Circuits and Systems for video Technology, 13(1):59–74, 2003. 3

    work page 2003

  8. [8]

    Childs, E

    H. Childs, E. Brugger, B. Whitlock, J. Meredith, S. Ahern, D. Pugmire, K. Biagas, M. Miller, C. Harrison, G. H. Weber, et al. VisIt: An End-User Tool for Visualizing and Analyzing Very Large Data. 2012. 1

    work page 2012

Show all 45 references
  1. [9]

    X. Ding, L. Chen, M. Emani, C. Liao, P.-H. Lin, T. Vanderbruggen, Z. Xie, A. Cerpa, and W. Du. HPC-GPT: Integrating Large Language Model for High-Performance Computing. In Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Network...

    2023

  2. [10]

    Douze, A

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazaré, M. Lomeli, L. Hosseini, and H. Jégou. The Faiss library. 2024. 3

    2024

  3. [11]

    Energy Exascale Earth System Model (E3SM)

    E3SM Project. Energy Exascale Earth System Model (E3SM). [Computer Software] https://dx.doi.org/10.11578/E3SM/dc.20240301.3, mar 2024. doi: 10.11578/E3SM/dc.20240301.3 8

    2024 doi

  4. [12]

    Fuentes, R

    D. Fuentes, R. Bardeli, J. Ortega, and L. Gonzalez-Abril. A Similar- ity Measure Between Videos Using Alignment, Graphical and Speech Features. Expert Systems with Applications, 39(11):10278–10282, 2012. 3

    2012

  5. [13]

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, and H. Wang. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997, 2023. 2

    2023 arXiv

  6. [14]

    H. Guo, D. Lenz, J. Xu, X. Liang, W. He, I. R. Grindeanu, H.-W. Shen, T. Peterka, T. Munson, and I. Foster. FTK: A High-Dimensional Simplicial Meshing Framework for Robust and Scalable Feature Tracking. arXiv e-prints, pp. arXiv–2011, 2020. 8

    2011

  7. [15]

    E. Hoque. LLM4Vis: Large Language Models for Information Visu- alization. https://nlp4vis.github.io/, 2024. [Online; accessed 14-January-2025]. 2

    2024

  8. [16]

    M. S. Islam, M. T. R. Laskar, M. R. Parvez, E. Hoque, and S. Joty. Data- Narrative: Automated Data-Driven Storytelling with Visualizations and Texts. arXiv preprint arXiv:2408.05346, 2024. 2

    2024 arXiv

  9. [17]

    Jeong, J

    S. Jeong, J. Li, C. R. Johnson, S. Liu, and M. Berger. Text-Based Trans- fer Function Design for Semantic V olume Rendering. In 2024 IEEE Visualization and Visual Analytics (VIS), pp. 196–200. IEEE, 2024. 2

    2024

  10. [18]

    Jiang, F

    J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim. A Survey on Large Language Models for Code Generation. arXiv preprint arXiv:2406.00515,

    arXiv

  11. [19]

    Kantharaj, R

    S. Kantharaj, R. T. K. Leong, X. Lin, A. Masry, M. Thakkar, E. Hoque, and S. Joty. Chart-to-Text: A Large-Scale Benchmark for Chart Summa- rization. arXiv preprint arXiv:2203.06486, 2022. 2

    2022 arXiv

  12. [20]

    Kulal, P

    S. Kulal, P. Pasupat, K. Chandra, M. Lee, O. Padon, A. Aiken, and P. S. Liang. SPoC: Search-based Pseudocode to Code. Advances in Neural Information Processing Systems, 32, 2019. 3

    2019

  13. [21]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küt- tler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural infor- mation processing systems, 33:9459–9474, 2020. 2

    2020

  14. [22]

    S. Liu, Y . Chen, X. Xie, J. Siow, and Y . Liu. Retrieval-augmented generation for code summarization via hybrid gnn. arXiv preprint arXiv:2006.05405, 2020. 2

    2006 arXiv

  15. [23]

    S. Lu, N. Duan, H. Han, D. Guo, S.-w. Hwang, and A. Svyatkovskiy. Reacc: A retrieval-augmented code completion framework. arXiv preprint arXiv:2203.07722, 2022. 2

    2022 arXiv

  16. [24]

    S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664, 2021. 2

    2021 arXiv

  17. [25]

    Mallick, O

    T. Mallick, O. Yildiz, D. Lenz, and T. Peterka. ChatVis: Automating Scientific Visualization with a Large Language Model. In SC24-W: Work- shops of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 49–55. IEEE, 2024. 1, 2

    2024

  18. [26]

    Masry, D

    A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque. ChartQA: A Benchmark for Question Answering About Charts with Visual and Logical Reasoning. arXiv preprint arXiv:2203.10244, 2022. 2

    2022 arXiv

  19. [27]

    Nichols, A

    D. Nichols, A. Marathe, H. Menon, T. Gamblin, and A. Bhatele. HPC- Coder: Modeling Parallel Programs using Large Language Models. In ISC High Performance 2024 Research Paper Proceedings (39th International Conference), pp. 1–12. Prometeus GmbH, 2024. 2

    2024

  20. [28]

    Obeid and E

    J. Obeid and E. Hoque. Chart-to-Text: Generating Natural Language De- scriptions for Charts by Adapting the Transformer Model. arXiv preprint arXiv:2010.09142, 2020. 2

    2010 arXiv

  21. [29]

    M. R. Parvez, W. U. Ahmad, S. Chakraborty, B. Ray, and K.-W. Chang. Retrieval augmented code generation and summarization. arXiv preprint arXiv:2108.11601, 2021. 2

    2021 arXiv

  22. [30]

    Peveler, T

    M. Peveler, T. Gurjar, E. Maicus, A. Aikens, A. Christoforides, and B. Cut- ler. Lichen: Customizable, Open Source Plagiarism Detection in Submitty. In Proceedings of the 50th ACM Technical Symposium on Computer Sci- ence Education, pp. 1270–1270, 2019. 3

    2019

  23. [31]

    S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma. CodeBLEU: A Method for Automatic Evaluation of Code Synthesis. arXiv preprint arXiv:2009.10297, 2020. 3

    2009 arXiv

  24. [32]

    M. P. Sampat, Z. Wang, S. Gupta, A. C. Bovik, and M. K. Markey. Com- plex wavelet structural similarity: A new image similarity index. IEEE transactions on image processing, 18(11):2385–2401, 2009. 3

    2009

  25. [33]

    Seshadrinathan and A

    K. Seshadrinathan and A. C. Bovik. A Structural Similarity Metric for Video Based on Motion Models. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, vol. 1, pp. I–869. IEEE, 2007. 3

    2007

  26. [34]

    Severo, L

    D. Severo, L. Theis, and J. Ballé. The unreasonable effectiveness of linear prediction as a perceptual metric. arXiv preprint arXiv:2310.05986, 2023. 3

    2023 arXiv

  27. [35]

    H. Su, S. Jiang, Y . Lai, H. Wu, B. Shi, C. Liu, Q. Liu, and T. Yu. Arks: Active retrieval in knowledge soup for code generation. arXiv preprint arXiv:2402.12317, 2024. 2

    2024 arXiv

  28. [36]

    M. Tian, L. Gao, S. D. Zhang, X. Chen, C. Fan, X. Guo, R. Haas, P. Ji, K. Krongchon, Y . Li, et al. SciCode: A Research Coding Benchmark Curated by Scientists. arXiv preprint arXiv:2407.13168, 2024. 2

    2024 arXiv

  29. [37]

    Y . Tian, W. Cui, D. Deng, X. Yi, Y . Yang, H. Zhang, and Y . Wu. ChartGPT: Leveraging LLMs to Generate Charts from Abstract Natural Language. IEEE Transactions on Visualization and Computer Graphics, 2024. 2

    2024

  30. [38]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971,

    arXiv

  31. [39]

    Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402. Ieee, 2003. 2

    2003

  32. [40]

    P. J. Wolfram, T. D. Ringler, M. E. Maltrud, D. W. Jacobsen, and M. R. Pe- tersen. Diagnosing Isopycnal Diffusivity in an Eddying, Idealized Midlati- tude Ocean Basin via Lagrangian, In Situ, Global, High-Performance Par- ticle Tracking (LIGHT). Journal of Physical Oceanograph...

    2015

  33. [41]

    Y . Yao, H. Chan, S. Sankaranarayanan, P. Balaprakash, R. J. Harder, and M. J. Cherukara. AutoPhaseNN: Unsupervised Physics-Aware Deep Learning of 3D Nanoscale Coherent Imaging. arXiv e-prints, pp. arXiv– 2109, 2021. 8

    2021

  34. [42]

    Zhang, B

    F. Zhang, B. Chen, Y . Zhang, J. Keung, J. Liu, D. Zan, Y . Mao, J.-G. Lou, and W. Chen. Repocoder: Repository-level code completion through iterative retrieval and generation. arXiv preprint arXiv:2303.12570, 2023. 2

    2023 arXiv

  35. [43]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreason- able effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586–595, 2018. 3

    2018

  36. [44]

    Zheng, G

    T. Zheng, G. Zhang, T. Shen, X. Liu, B. Y . Lin, J. Fu, W. Chen, and X. Yue. Opencodeinterpreter: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402.14658, 2024. 2

    2024 arXiv

  37. [45]

    S. Zhou, U. Alon, F. F. Xu, Z. Wang, Z. Jiang, and G. Neubig. Docprompt- ing: Generating code by retrieving the docs. arXiv preprint arXiv: 2207.05987, 2022. 2

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.