Pith. sign in

REVIEW 5 cited by

Improving ChatGPT Prompt for Code Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.08360 v1 pith:YSOLGO5Y submitted 2023-05-15 cs.SE

Chao Liu , Xuanlin Bao , Hongyu Zhang , Neng Zhang , Haibo Hu , Xiaohong Zhang , Meng Yan This is my paper · ORCID
classification cs.SE
keywords generationcodechatgptpromptpromptsgeneratingguideincluding
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Automated code generation can be a powerful technique for software development, significantly reducing developers' efforts and time required to create new code by generating it automatically based on requirements. Recently, OpenAI's language model ChatGPT has emerged as a powerful tool for generating human-like responses to a wide range of textual inputs (i.e., prompts), including those related to code generation. However, the effectiveness of ChatGPT for code generation is not well understood, and the generation performance could be heavily influenced by the choice of prompt. To answer these questions, we conducted experiments using the CodeXGlue dataset to evaluate ChatGPT's capabilities for two code generation tasks, including text-to-code and code-to-code generation. We designed prompts by leveraging the chain-of-thought strategy with multi-step optimizations. Our results showed that by carefully designing prompts to guide ChatGPT, the generation performance can be improved substantially. We also analyzed the factors that influenced the prompt design and provided insights that could guide future research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation

    cs.SE 2026-08 conditional novelty 6.0 of 10

    Pressure-style prompt framings were associated with lower functional correctness and more security warnings than neutral prompts in LiveCodeBench, while most other influence tactics had minimal effects.

  2. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Prompt language affects LLM code generation, but English is not consistently best: Chinese prompts improve Python correctness on CoderEval, while quality and lexicon effects vary by model and programming language.

  3. SnipGen: A Mining Repository Framework for Evaluating LLMs for Code

    cs.SE 2025-02 conditional novelty 5.0 of 10

    A new framework mines recent GitHub commits into about 227K method-level code snippets with prompt templates for testing LLMs on code generation while reducing training-data contamination.

  4. Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineering

    cs.SE 2024-11 conditional novelty 5.0 of 10

    Interactive prompting (context enrichment, task decomposition, and question-asking) lifts LLM code snippet adaptation pass@1 from about 52% to 68-75% on ClassEval, with a multi-agent variant nearly matching human feedback.

  5. SmartLLMSentry: A Comprehensive LLM Based Smart Contract Vulnerability Detection Framework

    cs.CR 2024-11 conditional novelty 4.0 of 10

    A fine-tuned GPT-3.5 model generated static analysis detector conditions for five smart contract vulnerability classes with 92.1% exact match on a 38-sample test set.

Pith tools