Pith. sign in

REVIEW 10 cited by

C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.08322 v3 pith:F4423YOS submitted 2023-05-15 cs.CL

Yuzhen Huang , Yuzhuo Bai , Zhihao Zhu , Junlei Zhang , Jinghan Zhang , Tangjun Su , Junteng Liu , Chuancheng Lv
show 5 more authors
Yikai Zhang Jiayi Lei Yao Fu Maosong Sun Junxian He
This is my paper
classification cs.CL
keywords c-evalmodelschineseadvancedevaluationfoundationllmsabilities
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

New NLP benchmarks are urgently needed to align with the rapid development of large language models (LLMs). We present C-Eval, the first comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context. C-Eval comprises multiple-choice questions across four difficulty levels: middle school, high school, college, and professional. The questions span 52 diverse disciplines, ranging from humanities to science and engineering. C-Eval is accompanied by C-Eval Hard, a subset of very challenging subjects in C-Eval that requires advanced reasoning abilities to solve. We conduct a comprehensive evaluation of the most advanced LLMs on C-Eval, including both English- and Chinese-oriented models. Results indicate that only GPT-4 could achieve an average accuracy of over 60%, suggesting that there is still significant room for improvement for current LLMs. We anticipate C-Eval will help analyze important strengths and shortcomings of foundation models, and foster their development and growth for Chinese users.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Gated amplitude-only FFN interventions improve tool-structured LLM outputs on Qwen models by several points, while direction-changing repairs harm more than they fix.

  2. Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models

    cs.LG 2025-08 unverdicted novelty 6.0 of 10

    Amber Pruner proposes training-free N:M activation sparsity for LLM prefill; however, the supplied manuscript body is an unrelated paper.

  3. FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language

    cs.CL 2025-06 conditional novelty 6.0 of 10

    An adaptive, per-language data filtering and deduplication pipeline produces multilingual LLM pre-training corpora that beat prior public datasets on 11 of 14 evaluated languages, and a 20TB, 1,868 language-script dat...

  4. Serving Large Language Models on Huawei CloudMatrix384

    cs.DC 2025-06 conditional novelty 6.0 of 10

    CloudMatrix384 with CloudMatrix-Infer reports prefill 6,688 tokens/s per NPU and decode 1,943 tokens/s per NPU on DeepSeek-R1, exceeding published NVIDIA-based efficiencies.

  5. AutoEvoEval: An Automated Framework for Evolving Close-Ended LLM Evaluation Data

    cs.CL 2025-06 conditional novelty 5.0 of 10

    AutoEvoEval applies 22 atomic perturbations and multi-round chains to MCQ benchmarks, causing average accuracy drops of 7.283% and up to 52.932% for long chains.

  6. Enterprise Large Language Model Evaluation Benchmark

    cs.AI 2025-06 reject novelty 5.0 of 10

    A 14-task enterprise LLM benchmark built mostly from GPT-4o-generated labels and scored by GPT-4o-as-judge shows open-source models closing the reasoning gap, but the dataset is not public and the evaluation is partly...

  7. AI Flow: Perspectives, Scenarios, and Approaches

    cs.AI 2025-06 conditional novelty 5.0 of 10

    AI Flow proposes to combine device-edge-cloud deployment, feature-aligned model families, and multi-model collaboration to make large AI models cheaper, faster, and more widely accessible.

  8. A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A unified taxonomy and comparative meta-evaluation of automatic evaluation methods across text, vision, and speech generation, concluding that LLM-based evaluators dominate current practice.

  9. Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruning

    cs.LG 2025-09 conditional novelty 4.0 of 10

    FAPM prunes task vectors using a combined absolute and relative change magnitude criterion, reducing forgetting on general benchmarks to roughly a third of a percent while keeping downstream accuracy near the fine-tun...

  10. Technical Report of TeleChat2, TeleChat2.5 and T1

    cs.CL 2025-07 conditional novelty 4.0 of 10

    The released T1-115B open-weight model outperforms OpenAI's o1-mini and GPT-4o on MATH500, AlignBench, and IFEval, despite using a standard dense transformer architecture.

Pith tools