Byte-Level Distillation achieves competitive or superior results in cross-tokenizer LLM distillation by using byte-level probabilities as a shared interface instead of aligning mismatched vocabularies.
We then looked at training and val- idation losses over both bytes and tokens
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Cross-Tokenizer LLM Distillation through a Byte-Level Interface
Byte-Level Distillation achieves competitive or superior results in cross-tokenizer LLM distillation by using byte-level probabilities as a shared interface instead of aligning mismatched vocabularies.