Pith. sign in

REVIEW 3 cited by

Generative AI in Academic Writing: A Comparison of DeepSeek, Qwen, ChatGPT, Gemini, Llama, Mistral, and Gemma

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.04765 v2 pith:2SIRMS6P submitted 2025-02-11 cs.CY cs.HC

Omer Aydin , Enis Karaarslan , Fatih Safa Erenay , Nebojsa Bacanin This is my paper
classification cs.CYcs.HC
keywords contentacademicdetectionreadabilitywritinggeneratedmodelsplagiarism
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

DeepSeek v3, developed in China, was released in December 2024, followed by Alibaba's Qwen 2.5 Max in January 2025 and Qwen3 235B in April 2025. These free and open-source models offer significant potential for academic writing and content creation. This study evaluates their academic writing performance by comparing them with ChatGPT, Gemini, Llama, Mistral, and Gemma. There is a critical gap in the literature concerning how extensively these tools can be utilized and their potential to generate original content in terms of quality, readability, and effectiveness. Using 40 papers on Digital Twin and Healthcare, texts were generated through AI tools based on posed questions and paraphrased abstracts. The generated content was analyzed using plagiarism detection, AI detection, word count comparisons, semantic similarity, and readability assessments. Results indicate that paraphrased abstracts showed higher plagiarism rates, while question-based responses also exceeded acceptable levels. AI detection tools consistently identified all outputs as AI-generated. Word count analysis revealed that all chatbots produced a sufficient volume of content. Semantic similarity tests showed a strong overlap between generated and original texts. However, readability assessments indicated that the texts were insufficient in terms of clarity and accessibility. This study comparatively highlights the potential and limitations of popular and latest large language models for academic writing. While these models generate substantial and semantically accurate content, concerns regarding plagiarism, AI detection, and readability must be addressed for their effective use in scholarly work.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Let's Get You Hired: A Job Seeker's Perspective on Multi-Agent Recruitment Systems for Explaining Hiring Decisions

    cs.CY 2025-05 conditional novelty 6.0 of 10

    A multi-agent LLM chatbot for job seekers was perceived by 20 interviewed participants as more actionable, trustworthy, and fair than their recalled experiences with traditional hiring methods.

  2. Beyond Traditional Algorithms: Leveraging LLMs for Accurate Cross-Border Entity Identification

    cs.CL 2025-07 reject novelty 3.0 of 10

    A 65-case comparison claims commercial chatbot LLMs are the most accurate for Portuguese entity matching, but the reported false-positive rates contradict the claim.

  3. DeepSeek in Healthcare: A Survey of Capabilities, Risks, and Clinical Applications of Open-Source Large Language Models

    cs.CL 2025-06 conditional

    A narrative review of DeepSeek-R1's healthcare capabilities, risks, and applications, without new experiments.

Pith tools