Token pruning of non-Korean vocabulary in LLMs improves generation stability and often boosts machine translation on Korean tasks while cutting vocabulary size substantially.
InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7993–8007
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Optimizing Korean-Centric LLMs via Token Pruning
Token pruning of non-Korean vocabulary in LLMs improves generation stability and often boosts machine translation on Korean tasks while cutting vocabulary size substantially.