HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training

Seungho Choi

REVIEW 4 major objections 6 minor 34 references

HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Appending all plausible Hanja spellings after ambiguous Korean words during continual pre-training lifts Korean understanding by about 21% relative to baseline, and the gain persists when Hanja is removed at inference.

desk verdict Sensible training-time trick with a plausible mechanism, but the 21% KoBALT claim rests on one post-hoc-selected run and needs replication before it is trusted. read the letter →

arxiv 2507.10920 v1 pith:5S43UXYZ submitted 2025-07-15 cs.CL cs.AI

Seungho Choi This is my paper · ORCID

classification cs.CLcs.AI

keywords KoreanLLMHanjaaugmentationhomophonedisambiguationcontinualpre-trainingtoken-levelknowledgedistillationSino-KoreanvocabularywordsenseKoBALTbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Korean LLMs underperform partly because Hangul collapses many distinct Sino-Korean words into a single spelling, so a model cannot tell '의사' the doctor from '의사' the intention. HanjaBridge counters this by appending every plausible Hanja (Chinese-character) reading after each ambiguous Hangul token during continual pre-training, using an attention mask that lets the Korean token consult each candidate while keeping candidates from blurring each other. The language-modeling loss is computed only on the original Hangul positions, so the Hanja act as latent cues that force contextual disambiguation. On the KoBALT-Hard benchmark, the $k{=}8$ configuration reaches an average score of 0.1437 versus 0.1189 for the baseline, about 21% relative improvement, and the benefit persists when Hanja is omitted at inference, so there is no extra run-time cost. Token-level knowledge distillation is added to keep the model from losing English competence, and the paper reports English scores well above the full continual-pretraining baseline.

What carries the argument

The central mechanism is the semantic slot: for each ambiguous Hangul token, the input is expanded with a group of all plausible Hanja candidates (for example, 가격 is followed by 價格 and 加擊), and the attention mask lets the Korean token attend to every candidate while blocking attention between candidates. The language-modeling loss is restricted to original Hangul positions, so Hanja tokens act only as latent disambiguation cues. A queue-based, token-level knowledge distillation with an instance queue and offset mapping aligns the student's hidden states with the frozen teacher's, preserving multilingual competence while the model learns the Hanja-augmented signal.

What would settle it

Train the same continual-pretraining pipeline with $k$ randomly chosen Hanja candidates instead of the dictionary's plausible readings and evaluate on KoBALT-Hard: if random candidates reproduce the roughly 21% relative gain, the claimed contextual-disambiguation mechanism is not what drives the improvement.

Watch

Extended reading notes

Core claim

The central claim is that presenting all plausible Hanja candidates after ambiguous Korean words during continual pre-training, rather than a single definitive character, substantially improves Korean language understanding. The paper states that on KoBALT-Hard, the $k{=}8$ model achieves an average score of 0.1437, outperforming the baseline of 0.1189 by about 21% relative improvement, and that on KoBEST-General it records the highest average among the compared models. A second component claim is that these gains persist even when Hanja augmentation is omitted at inference, so the learned disambiguation transfers to plain Hangul input. The paper also claims that reinforcing semantic alignment between Korean and Chinese through shared Hanja yields positive cross-lingual transfer without sacrificing English performance, and that attention-rollout analysis shows the model increasingly focuses on the context-appropriate Hanja candidate as training progresses.

Load-bearing premise

The load-bearing premise is that the privately constructed Hanja–Hangul mapping dictionary is accurate and comprehensive enough that the appended candidates are the correct interpretations of each homophone; if it omits legitimate readings or includes spurious ones, the training signal becomes noise and the attention analysis, which defines the 'correct' candidate by this same dictionary, becomes self-referential.

Editorial extensions

If this is right

On KoBALT-Hard, the $k{=}8$ HanjaBridge model scores 0.1437 on average, a roughly 21% relative improvement over the 0.1189 baseline.
On KoBEST-General, the $k{=}8$ model achieves the best average (0.6354) among the compared models, including full continual pre-training.
English performance is preserved: the $k{=}8$ model averages 0.6792 on the English benchmarks, well above the full continual-pretraining average of 0.6501 and close to the original model's 0.6976.
Attention-rollout accuracy on the correct Hanja candidate rises with training (for $k{=}8$, from 0.102 at 30k steps to 0.308 at 180k steps), supporting the claim that the model learns contextual disambiguation.
Omitting Hanja at inference retains most of the gain (0.568 versus 0.575 with Hanja present), so the method can be deployed with the original tokenizer and no added token cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The candidate-set supervision is answer-agnostic in a useful way: because the model is never told which Hanja is correct, the method may transfer to other homophone-heavy writing systems, such as Japanese kanji or Vietnamese Hán-Nôm, and to word-sense disambiguation beyond Korean.
A natural extension would be to ablate dictionary quality directly: training with a deliberately corrupted dictionary (random candidates, or candidates missing the true reading) would quantify how much of the 21% gain comes from candidate-set diversity versus from accurate dictionary coverage.
Because the 'correct' candidate in the attention analysis is defined by the same dictionary that generated the training data, an independent human-annotated disambiguation set would be needed to confirm that the attention shift reflects genuine semantic resolution rather than memorized pairings.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short take: this is a sensible training-time trick with a plausible mechanism, but the headline number is not yet trustworthy because it comes from one unreplicated run with the best hyperparameter chosen after the fact. The paper deserves a referee, not a desk reject, but the referee should push hard for seeds and the dictionary.

What is actually new: appending all candidate Hanja spellings in parallel after ambiguous Hangul tokens, blocking attention between candidates, and combining that with token-level distillation during continual pre-training. That specific combination is not in the prior Hanja work (Yoo et al. used deterministic selection; Kim et al. used Hanja for MT). The design decision to compute LM loss only on original tokens and treat Hanja as latent cues is clean, and the inference-time-free property is genuinely useful: Table 4 shows the training-only model gets 0.568 vs 0.503 pretrain on their probe, so most of the benefit survives without extra tokens. That is the paper's best piece of evidence because it is the least self-referential.

The soft spots are real and load-bearing. The 21% KoBALT-Hard gain is 0.1437 vs 0.1189 average over five sub-tasks, each a single accuracy number. k=8 was selected after seeing the benchmark, and k=4 ties the baseline exactly at 0.1189 while k=2 is worse. That pattern is exactly what selection noise looks like. No seeds, no error bars, no confidence intervals. The RQ1 attention analysis is partly circular: the 'correct' candidate is defined by the same private Hanja dictionary that generated the training augmentations, so rising attention to that candidate is expected and does not independently validate the dictionary or the mechanism. The dictionary itself is a black box—no construction details, size, or error rate—and the conclusion admits the method depends on it. Also, the abstract and Section 4 claim 'strong positive cross-lingual transfer' from Korean to Chinese, but there is no Chinese evaluation anywhere. That claim should be removed or tested.

The math and the formal statements (Eqs. 1–5) are straightforward and consistent. Citation pattern is fair; the prior Hanja augmentation work is cited. The paper is written clearly enough that a referee can tell exactly what was done.

Who is this for: people working on low-resource Korean LLMs, and more broadly anyone interested in script-level meaning injection for languages with homophones. It is a workshop-to-conference-level contribution with a practical angle. My recommendation: send it to review. A good referee will ask for multiple seeds, the dictionary (or a public subset), and deletion of the cross-lingual claim unless tested. If the effect holds under replication, it is a useful method paper.

Referee Report

4 major / 6 minor

Summary. The paper proposes HanjaBridge, a continual pre-training (CPT) method for Korean LLMs that appends all plausible Hanja (Chinese-character) candidates to ambiguous Hangul tokens, uses an attention mask that blocks inter-candidate attention, and applies token-level knowledge distillation to preserve the original model's multilingual abilities. Experiments on Qwen2.5-3B report a 21% relative improvement over the original model on the KoBALT-Hard benchmark, gains on KoBEST-General, preserved or improved English performance, an attention-rollout analysis (RQ1) showing increased attention to the 'correct' Hanja candidate over training, and a multiple-choice probe (RQ2) showing that Hanja can be omitted at inference time with little loss. The authors identify as a limitation that the method depends on the quality and coverage of the privately constructed Hanja-Hangul dictionary.

Significance. If the central finding is robust, HanjaBridge is a conceptually interesting and practically useful contribution: it addresses a real linguistic bottleneck for Korean LLMs (Hangul homophony), proposes a training-only augmentation with no inference-time overhead, and reports gains on a hard Korean benchmark while retaining English performance. The paper also includes explicit discussion of limitations, which is commendable. However, the current evidence is not yet sufficient for the strength of the claims: the headline result rests on single unreplicated runs with post hoc selection of the candidate count k, and the dictionary used to generate both the training signal and the RQ1 evaluation labels is not described or released. The idea merits further study, but the empirical foundation needs substantial strengthening before the claims can be accepted.

major comments (4)

[§4.2, Table 2] The central claim of a 21% relative improvement on KoBALT-Hard rests on a single run per condition. Table 2 reports one mean per task and model with no standard deviations, confidence intervals, or number of seeds, so the reader cannot distinguish a genuine effect from training or evaluation noise. The concern is amplified by the fact that k appears to have been selected post hoc: the paper compares k=2, 4, 8, 16 and presents k=8 as the proposed model, yet k=4 gives the same KoBALT-Hard average as the original model (0.1189) and k=2 is below it. To make the headline claim load-bearing, the authors should report multiple seeds for at least k=8 and the baseline (and ideally the full k sweep), include statistical significance testing (e.g., bootstrap or paired tests over tasks), or pre-register the choice of k on a held-out validation split. Without this, the 0.0248 absolute gap could easily be selection noise.
[§3.1, §5] The method's correctness depends entirely on the privately constructed Hanja-Hangul dictionary, but Section 3.1 gives no details about its construction, size, coverage, or error rate. The authors themselves acknowledge in Section 5 that 'HanjaBridge directly depends on the quality and coverage of the constructed Hanja-Hangul mapping dictionary,' and that neologisms or specialized terms may be missing. This is a load-bearing dependency: if the dictionary omits legitimate readings or includes spurious ones, the appended candidates inject noise rather than meaning. The manuscript should specify how the dictionary was built, report its size and accuracy (e.g., by human evaluation of a sample), and ideally release it. Without this information, both the training signal and the RQ1 attention analysis, which labels the 'correct' candidate using the same dictionary, are unverifiable.
[§4.3, Table 3] The RQ1 attention analysis is partly self-referential: the 'correct' Hanja candidate is defined by the same private dictionary that generated the training augmentations. Table 3 therefore measures whether the model learns to attend to the candidates that the authors' dictionary says are correct, not whether those candidates are semantically correct in the given context. To break the circularity, the authors should evaluate attention accuracy on an independent, human-annotated or publicly available Hanja-disambiguation test set. In addition, the claim that the model 'gradually focuses on the correct Hanja candidates' is only a correlation with training progress; without a control that measures attention accuracy for candidates that are wrong according to the dictionary, or a downstream task where the correct answer is not derived from the training dictionary, the analysis does not independently validate the central claim.
[§4.2, §4.4] Several comparison settings are under-specified, making the comparisons difficult to interpret. First, the 'Full CPT' baseline is described only as 'full-parameter CPT performed using Korean corpus only,' with no details on learning rate, batch size, training steps, or sequence length, while HanjaBridge is said to unfreeze 'selected layers' without specifying which layers. Second, the English performance claim that the k=8 model 'significantly higher than the benchmark of the Full CPT model' is not supported by any statistical test, and Table 2 shows only a 0.0291 average gap over four tasks with one run each. Third, the RQ2 comparison in Table 4 reports a single accuracy value per condition with no variance; the 0.007 gap between HanjaBridge with and without inference-time Hanja is described as 'not significant' without any test. These omissions should be addressed with experimental details and, where possible, variance estimates.

minor comments (6)

[§1, Figure 1 / Chinese example] The Chinese sentence '我今天始研究。' appears to be missing a character; it should likely read '我今天开始研究。' This is a minor typo, but in a paper about language ambiguity, accuracy of the linguistic examples matters.
[References] The reference 'Moosa, I. M.; Akhter, M. E.; and Habib, A. B. ???? Transliteration...' has no year and is incomplete. Please fix this citation.
[§3.1, Eq. (3)] In Eq. (3), the notation is overloaded: the denominator sums over d ∈ D+ but p(i,j) is indexed by j, while the text refers to 'z_i^T' and 'z_i^S'. Clarify the relationship between d_j, z_i^T, and the queue elements.
[§3.1, Figure 3] The attention-mask description says 'inter-candidate attention is blocked' but also says the Korean token can 'freely attend to its Hanja candidates.' It is unclear whether Hanja tokens can attend to other original tokens in the sentence, or only to the associated Korean token. A precise specification of which pairs (t,t') are allowed under the mask would improve reproducibility.
[§4.1] The hyper-parameter section lists sequence length, KD temperatures, and λ, but not the number of training steps, optimizer, learning rate, or batch size. Without these, the results cannot be reproduced. The caption of Table 3 mentions training steps from 30k to 180k, but the total number of steps is not stated in the setup.
[§4.3] Table 3 reports attention accuracy for k=2 through k=8, but the main experiments use k=8 as the proposed configuration. It would be helpful to also report the attention accuracy for k=16, since the paper discusses the performance decline at k=16 in Section 4.2; without this, the link between attention accuracy and benchmark performance is incomplete.

Circularity Check

2 steps flagged · score 4.0 of 10

The reported 21% KoBALT gain is the best-k selection on the same benchmark rather than an independent prediction, and the RQ1/RQ2 'correct Hanja' labels are drawn from the same private dictionary that generates the training augmentations.

fitted input called prediction [Section 4.2, Table 2; Section 4.2 'Effect of the number of Chinese character candidates (k)']
"We compare the optimal configuration (our model with k= 8 Hanja candidates) against the original pre-trained model and two baselines: a standard continual pre-training without our semantic augmentation (Full CPT), and a CPT with knowledge distillation but no Hanja augmentation (k=0). Table 2 summarizes the results. ... Through experiments, we also discovered a trend regarding the number of Chinese character candidates (k). The Korean benchmark performance improved overall as k increased from 2 to 8."

The paper reports the k=8 model as 'the optimal configuration' after observing that Korean benchmark performance improved as k increased from 2 to 8 on the same KoBALT and KoBEST benchmarks used for the final result. With a single run per k and no held-out validation split (Table 2 shows one number per task), the 21% relative improvement over the baseline is the selected maximum of the k-grid, not an unbiased prediction. The hyperparameter is therefore fitted to the evaluation set, and the resulting score is presented as evidence of the method's effect—a fitted input called a prediction.
self definitional [Section 3.1, Section 4.3 (Table 3), Section 4.4 (Table 4), Conclusion]
"HanjaBridge creates a single semantic slot by concatenating all candidate Hanja forms immediately after each ambiguous Hangul token. ... We constructed a joint Korean-Chinese Chinese character dictionary, then selected only Chinese characters corresponding to Korean prototypes and added them to a multilingual tokenizer. ... Table 3 shows the results ... demonstrating how much attention the model focuses on the correct character candidates (correct character tokens). ... HanjaBridge directly depends on the quality and coverage of the constructed Hanja-Hangul mapping dictionary."

The 'correct' Hanja labels used as targets in RQ1's attention analysis and as the correct option in RQ2's multiple-choice probes are defined by the same private Hanja-Hangul dictionary (Sec. 3.1) that supplies the Hanja candidates appended during training. Thus the internal evaluation checks whether the model reproduces the dictionary's mapping already embedded in the augmented corpus, rather than comparing against an independent semantic ground truth.

full rationale

The central endpoint (KoBALT and KoBEST averages) is an externally defined benchmark, so the paper's main empirical claim is not derived from the dictionary or from the method's definition; that part has independent content. However, two steps weaken the claimed derivation. First, the headline configuration k=8 is selected by scanning k on the same benchmarks that are then reported, with a single run per k, so the 21% relative gain is a selected maximum, not an unbiased predicted effect. The k=4 model ties the baseline average (0.1189), and k=2 is below baseline, further suggesting the apparent improvement may be selection noise. Second, the RQ1 attention analysis and RQ2 probe define 'correct' Hanja using the same private dictionary used to create the augmented training inputs; the internal validation is therefore partly self-referential. There are no load-bearing self-citations, no uniqueness theorem imported from the authors, and no renaming of a known result as a new method. For these reasons, the circularity score is 4 rather than higher: the central benchmark claim is not forced by construction, but the key supporting analyses and the choice of the reported configuration are not fully independent of the training resource and the evaluation set.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The method rests on two substantive domain assumptions: that Hanja spellings cleanly disambiguate Korean homophones when context is present, and that teacher hidden-vector distillation preserves multilingual competence. Both are standard in their communities but are not tested here. The free parameters are hyperparameters, one of which (k) is tuned on the evaluation benchmark.

free parameters (3)

k (number of Hanja candidates) = 8
Selected as the best-performing value across k=2,4,8,16 in Table 2; the headline 21% improvement is reported for this chosen value.
lambda (distillation loss weight) = 0.1
Set by hand in Section 4.1 without an ablation study.
distillation temperatures (tau_T, tau_S) = 0.01 and 0.2
Chosen in Section 4.1 without sensitivity analysis.

assumptions (3)

domain assumption The Hanja-Hangul dictionary accurately enumerates all plausible Hanja spellings for ambiguous Korean words.
The method's semantic injection depends entirely on this dictionary; the paper provides no statistics, examples of ambiguity resolution, or error analysis (Section 3.1, Conclusion).
domain assumption The linguistic context in the training corpus is sufficient to disambiguate Hanja candidates.
The training loss assumes the model can learn to select the contextually correct Hanja; if contexts are short or ambiguous, the appended candidates inject noise rather than signal (Section 3.1).
domain assumption Teacher token-level hidden representations are a reliable target for preserving multilingual competence.
The distillation objective (Eq. 3-5) assumes teacher features encode general knowledge worth preserving; this is a standard but unverified assumption for this base model and corpus (Section 3.2).

invented entities (1)

Parallel Hanja candidate slot
purpose: A training-time input structure that appends all candidate Hanja characters after each ambiguous Hangul token, with the candidate tokens acting as semantic hints during attention while excluded from the LM loss.
This is a model architecture and input construct rather than a physical entity; it has no falsifiable handle outside the paper's own benchmarks, and its benefit is only evidenced by the single-run Table 2 results.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training." pith.science (2026). https://pith.science/paper/5S43UXYZ

@misc{pith2026250710920,
  author       = {Pith},
  title        = {Pith review of: HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5S43UXYZ}},
  note         = {Machine review of arXiv:2507.10920}
}

read the original abstract

Large language models (LLMs) often show poor performance in low-resource languages like Korean, partly due to unique linguistic challenges such as homophonous Sino-Korean words that are indistinguishable in Hangul script. To address this semantic ambiguity, we propose HanjaBridge, a novel meaning-injection technique integrated into a continual pre-training (CPT) framework. Instead of deterministically mapping a word to a single Hanja (Chinese character), HanjaBridge presents the model with all possible Hanja candidates for a given homograph, encouraging the model to learn contextual disambiguation. This process is paired with token-level knowledge distillation to prevent catastrophic forgetting. Experimental results show that HanjaBridge significantly improves Korean language understanding, achieving a 21\% relative improvement on the KoBALT benchmark. Notably, by reinforcing semantic alignment between Korean and Chinese through shared Hanja, we observe a strong positive cross-lingual transfer. Furthermore, these gains persist even when Hanja augmentation is omitted at inference time, ensuring practical efficiency with no additional run-time cost.

Figures

Figures reproduced from arXiv: 2507.10920 by the authors.

**Figure 2.** End-to-end workflow of our method. teacher’s output logits during CPT.(Hinton, Vinyals, and Dean 2015) In this way, the student deepens its expertise in the new language while being explicitly regularized to preserve the teacher’s high-resource language knowledge. The benefit of KD has been corroborated in recent LLM studies on low-resource language adaptation. For example, (Raju et al. 2025) showed that applying KD… view at source ↗

**Figure 4.** The training strategy for the Teacher-Student mod [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Cumulative attention heat-maps (36 heads [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Examples of evaluation prompts (k = 3). difficulty level (k = number of candidates) interval. The experimental results show that accuracy steadily increases in all cases as learning progresses. For example, in the easy interval (k = 2), accuracy increased from 0.573 t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 27 canonical work pages

[1]

N.; Zhang, C.; Vechev, M.; and Toutanova, K

Alexandrov, A.; Raychev, V.; M \"u ller, M. N.; Zhang, C.; Vechev, M.; and Toutanova, K. 2024. Mitigating Catastrophic Forgetting in Language Transfer via Model Merging. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 17167--17186. Miami, Florida, USA: Association for Computation...

work page 2024
[2]

Chen, K.-M.; and Lee, H.-y. 2024. InstructionCP: A fast approach to transfer Large Language Models into target language. arXiv preprint arXiv:2405.20175

work page Pith review arXiv 2024
[3]

Clark, C.; Lee, K.; Chang, M.-W.; Kwiatkowski, T.; Collins, M.; and Toutanova, K. 2019. B ool Q : Exploring the Surprising Difficulty of Natural Yes/No Questions. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies...

work page 2019
[4]

Dou, L.; Liu, Q.; Zeng, G.; Guo, J.; Zhou, J.; Mao, X.; Jin, Z.; Lu, W.; and Lin, M. 2024. Sailor: Open Language Models for South- E ast A sia. In Hernandez Farias, D. I.; Hope, T.; and Li, M., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 424--435. Miami, Florida, USA: Association for...

work page 2024
[5]

Fang, Z.; Wang, J.; Wang, L.; Zhang, L.; Yang, Y.; and Liu, Z. 2021. \ SEED \ : Self-supervised Distillation For Visual Representation. In International Conference on Learning Representations

work page 2021
[6]

Fujii, K.; Nakamura, T.; Loem, M.; Iida, H.; Ohi, M.; Hattori, K.; Shota, H.; Mizuki, S.; Yokota, R.; and Okazaki, N. 2024. Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities. In First Conference on Language Modeling

work page 2024
[7]

Fujinuma, Y.; Boyd-Graber, J.; and Kann, K. 2022. Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1500--1512. Du...

work page 2022
[8]

Guo, P.; Ren, Y.; Hu, Y.; Li, Y.; Zhang, J.; Zhang, X.; and Huang, H. 2024. Teaching Large Language Models to Translate on Low-resource Languages with Textbook Prompting. In Calzolari, N.; Kan, M.-Y.; Hoste, V.; Lenci, A.; Sakti, S.; and Xue, N., eds., Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources ...

work page 2024

Show all 34 references

[9]

Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

2015 arXiv
[10]

S.; and Davis, E

Jang, M.; Kim, D.; Kwon, D. S.; and Davis, E. 2022. K o BEST : K orean Balanced Evaluation of Significant Tasks. In Calzolari, N.; Huang, C.-R.; Kim, H.; Pustejovsky, J.; Wanner, L.; Choi, K.-S.; Ryu, P.-M.; Chen, H.-H.; Donatelli, L.; Ji, H.; Kurohashi, S.; Paggio, P.; Xue, N...

2022
[11]

H.; Kim, S.; Park, T.; Kim, J.; Kang, S.; Ryu, N.-H.; Yoo, K

Kim, B.; Kim, H.; Lee, S.-W.; Lee, G.; Kwak, D.; Dong Hyeon, J.; Park, S.; Kim, S.; Kim, S.; Seo, D.; Lee, H.; Jeong, M.; Lee, S.; Kim, M.; Ko, S. H.; Kim, S.; Park, T.; Kim, J.; Kang, S.; Ryu, N.-H.; Yoo, K. M.; Chang, M.; Suh, S.; In, S.; Park, J.; Kim, K.; Kim, H.; Jeong, J...

2021
[12]

Kim, H.; Hirasawa, T.; and Komachi, M. 2020. K orean-to- J apanese Neural Machine Translation System using Hanja Information. In Nakazawa, T.; Nakayama, H.; Ding, C.; Dabre, R.; Kunchukuttan, A.; Pa, W. P.; Bojar, O.; Parida, S.; Goto, I.; Mino, H.; Manabe, H.; Sudoh, K.; Kuro...

2020
[13]

Kim, S.; Park, J.; Kim, Y.; and Lee, S. 2024. KOMBO : K orean Character Representations Based on the Combination Rules of Subcharacters. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 5102--5119. Bangkok,...

2024
[14]

Lee, J.; Moon, H.; Lee, S.; Park, C.; Eo, S.; Ko, H.; Seo, J.; Lee, S.; and Lim, H. 2024. Length-aware Byte Pair Encoding for Mitigating Over-segmentation in K orean Machine Translation. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computa...

2024
[15]

Li, H.; Sha, J.; and Shi, C. 2020. Revisiting back-translation for low-resource machine translation between Chinese and Vietnamese. IEEE Access, 8: 119931--119939

2020
[16]

I.; and Fung, P

Liu, Z.; Winata, G. I.; and Fung, P. 2021. Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2706--2718. ...

2021
[17]

Luo, Z.; Zhang, X.; Liu, X.; Li, H.; Gong, Y.; Qi, C.; and Cheng, P. 2024. Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training. arXiv preprint arXiv:2411.14318

2024 arXiv
[18]

M.; Akhter, M

Moosa, I. M.; Akhter, M. E.; and Habib, A. B. ???? Transliteration: A Simple Technique For Improving Multilingual Language Modeling
[19]

Nag, A.; Chakrabarti, S.; Mukherjee, A.; and Ganguly, N. 2025. Efficient Continual Pre-training of LLM s for Low-resource Languages. In Chen, W.; Yang, Y.; Kachuee, M.; and Fu, X.-Y., eds., Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associ...

2025
[20]

Navigli, R. 2009. Word sense disambiguation: A survey. ACM Comput. Surv., 41(2)

2009
[21]

Ogueji, K.; Zhu, Y.; and Lin, J. 2021. Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages. In Ataman, D.; Birch, A.; Conneau, A.; Firat, O.; Ruder, S.; and Sahin, G. G., eds., Proceedings of the 1st Workshop o...

2021
[22]

Park, J.; and Zhao, H. 2019. Korean-to-chinese machine translation using chinese character as pivot clue. arXiv preprint arXiv:1911.11008

2019 arXiv
[23]

A.; Von Werra, L.; Wolf, T.; et al

Penedo, G.; Kydl \' c ek, H.; Lozhkov, A.; Mitchell, M.; Raffel, C. A.; Von Werra, L.; Wolf, T.; et al. 2024. The fineweb datasets: Decanting the web for the finest text data at scale. Advances in Neural Information Processing Systems, 37: 30811--30849

2024
[24]

T.; and Camacho-Collados, J

Pilehvar, M. T.; and Camacho-Collados, J. 2019. W i C : the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North A merican Chapter of the Association...

2019
[25]

S.; Walia, J

Raju, J. S.; Walia, J. S.; Raghav, S.; Marivate, V.; et al. 2025. AfroXLMR-Comet: Multilingual Knowledge Distillation with Attention Matching for Low-Resource languages. arXiv preprint arXiv:2502.18020

2025 arXiv
[26]

A.; and Gordon, A

Roemmele, M.; Bejan, C. A.; and Gordon, A. S. 2011. Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. In AAAI spring symposium: logical formalizations of commonsense reasoning, 90--95

2011
[27]

Shin, H.; Lee, S.; Jang, D.; Song, W.; Kim, J.; Oh, C.; Jo, H.; Ahn, Y.; Oh, S.; Chang, H.; et al. 2025. KoBALT: Korean Benchmark For Advanced Linguistic Tasks. arXiv preprint arXiv:2505.16125

2025 arXiv
[28]

Tao, M.; Zhang, C.; Huang, Q.; Ma, T.; Huang, S.; Zhao, D.; and Feng, Y. 2024. Unlocking the Potential of Model Merging for Low-Resource Languages. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 8705...

2024
[29]

Vo, A.-D.; Jung, M.; Lee, W.; and Choi, D. 2024. Redwhale: An adapted korean llm through efficient continual pretraining. arXiv preprint arXiv:2408.11294

2024 arXiv
[30]

Xu, X.; Li, M.; Tao, C.; Shen, T.; Cheng, R.; Li, J.; Xu, C.; Tao, D.; and Zhou, T. 2024. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116

2024 arXiv
[31]

M.; Kim, T.; and Lee, S.-g

Yoo, K. M.; Kim, T.; and Lee, S.-g. 2019. Don ' t Just Scratch the Surface: Enhancing Word Representations for K orean with Hanja. In Inui, K.; Jiang, J.; Ng, V.; and Wan, X., eds., Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the ...

2019
[32]

Zellers, R.; Holtzman, A.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. H ella S wag: Can a Machine Really Finish Your Sentence? In Korhonen, A.; Traum, D.; and M \`a rquez, L., eds., Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4791--4...

2019
[33]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[34]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

N.; Zhang, C.; Vechev, M.; and Toutanova, K

Alexandrov, A.; Raychev, V.; M \"u ller, M. N.; Zhang, C.; Vechev, M.; and Toutanova, K. 2024. Mitigating Catastrophic Forgetting in Language Transfer via Model Merging. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 17167--17186. Miami, Florida, USA: Association for Computation...

work page 2024

[2] [2]

Chen, K.-M.; and Lee, H.-y. 2024. InstructionCP: A fast approach to transfer Large Language Models into target language. arXiv preprint arXiv:2405.20175

work page Pith review arXiv 2024

[3] [3]

Clark, C.; Lee, K.; Chang, M.-W.; Kwiatkowski, T.; Collins, M.; and Toutanova, K. 2019. B ool Q : Exploring the Surprising Difficulty of Natural Yes/No Questions. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies...

work page 2019

[4] [4]

Dou, L.; Liu, Q.; Zeng, G.; Guo, J.; Zhou, J.; Mao, X.; Jin, Z.; Lu, W.; and Lin, M. 2024. Sailor: Open Language Models for South- E ast A sia. In Hernandez Farias, D. I.; Hope, T.; and Li, M., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 424--435. Miami, Florida, USA: Association for...

work page 2024

[5] [5]

Fang, Z.; Wang, J.; Wang, L.; Zhang, L.; Yang, Y.; and Liu, Z. 2021. \ SEED \ : Self-supervised Distillation For Visual Representation. In International Conference on Learning Representations

work page 2021

[6] [6]

Fujii, K.; Nakamura, T.; Loem, M.; Iida, H.; Ohi, M.; Hattori, K.; Shota, H.; Mizuki, S.; Yokota, R.; and Okazaki, N. 2024. Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities. In First Conference on Language Modeling

work page 2024

[7] [7]

Fujinuma, Y.; Boyd-Graber, J.; and Kann, K. 2022. Match the Script, Adapt if Multilingual: Analyzing the Effect of Multilingual Pretraining on Cross-lingual Transferability. In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1500--1512. Du...

work page 2022

[8] [8]

Guo, P.; Ren, Y.; Hu, Y.; Li, Y.; Zhang, J.; Zhang, X.; and Huang, H. 2024. Teaching Large Language Models to Translate on Low-resource Languages with Textbook Prompting. In Calzolari, N.; Kan, M.-Y.; Hoste, V.; Lenci, A.; Sakti, S.; and Xue, N., eds., Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources ...

work page 2024