REVIEW 2 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
From KMMLU-Redux to KMMLU-Pro: A Professional Korean Benchmark Suite for LLM Evaluation
read the original abstract
The development of Large Language Models (LLMs) requires robust benchmarks that encompass not only academic domains but also industrial fields to effectively evaluate their applicability in real-world scenarios. In this paper, we introduce two Korean expert-level benchmarks. KMMLU-Redux, reconstructed from the existing KMMLU, consists of questions from the Korean National Technical Qualification exams, with critical errors removed to enhance reliability. KMMLU-Pro is based on Korean National Professional Licensure exams to reflect professional knowledge in Korea. Our experiments demonstrate that these benchmarks comprehensively represent industrial knowledge in Korea. We release our dataset publicly available.
Forward citations
Cited by 2 Pith papers
-
Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback
CAD generation agents are augmented with FEA feedback plus text blueprint and 21-view image signals, raising Box-IoU on S2O and Fusion360 while showing that base models produce no strict-passing FEA artifacts.
-
Self-Improving CAD Generation Agents with Finite Element Analysis as Feedback
CAD agents using finite element analysis feedback plus new text blueprint and multi-view image signals improve geometric accuracy on S2O and Fusion360 benchmarks while addressing physical validity gaps in prior genera...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.