Paper Citation Record · LEDGER

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models

As of 18 August 2026, this Paper Citation Record lists 36 of 36 outbound references and 2 inbound Pith citation observations for arXiv:2412.06926.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.06926 v5

Coverage vector

measured 36 of 36 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-11T19:25:22.152544Z

measured 38 of 38 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-16T10:55:55.460756Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-04T04:39:34.656602Z

Reference resolution

36 of 36 outbound references displayed

verified exact1
verified fuzzy2
unresolved33
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 51720b38-9869-4158-ae8f-f9bb8beb6575 · outbound

This paper cites Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.056655Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 17cf01e3-9573-4395-a4c5-70f20dfccd92 · outbound

This paper cites Language Models are Few-Shot Learners.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Language Models are Few-Shot Learners

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.060360Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6043def0-88ca-44b4-bcd1-e126b0660993 · outbound

This paper cites Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki

Reference 3

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T19:25:22.480560Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 8545b9be-fb32-46ea-8015-d5721aaa00b7 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.066887Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 14a6a898-b4ef-4761-b192-cd8d6c3ac452 · outbound

This paper cites Getting the most out of your tokenizer for pre-training and domain adaptation.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Getting the most out of your tokenizer for pre-training and domain adaptation

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.070092Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3e7ce535-6458-4f8c-8b6a-af2d7a9614df · outbound

This paper cites BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.073299Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 887604cf-d2a5-43f5-a6ff-bdd32e6b1c6d · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.472945Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation d9a96d13-12fd-4b5b-a2d4-0017a7478305 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.079698Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c68e6069-a91a-4f68-bdc4-909124d1e8eb · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.082230Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation da4b618c-e179-4e97-9fc2-39c8889b919b · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.084667Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d7e50150-09ab-48db-a0b1-8b271f46c416 · outbound

This paper cites The Llama 3 Herd of Models.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models The Llama 3 Herd of Models

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.087056Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 795d1b2d-ea62-4a05-b114-c34f2442cbbc · outbound

This paper cites Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.090169Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 18ee15c1-4447-45a4-9d49-d709d12cb944 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 13

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.465279Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation ed137759-f8af-484b-84ca-ed6ab4f02134 · outbound

This paper cites Low-resource Languages: A Review of Past Work and Future Challenges.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Low-resource Languages: A Review of Past Work and Future Challenges

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.095097Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 28e134b2-db04-4550-8691-f0a7fa367d64 · outbound

This paper cites Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T19:25:22.456538Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation f7158363-8079-4502-b296-89e553b3ccb9 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 16

Resolution: verified exact; doi, observed 2026-08-11T19:25:22.191554Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation b8c417db-9ab5-422c-8fb2-e32e59599c64 · outbound

This paper cites A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.102716Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c37c32ba-e990-455c-960b-bb511e0f5a95 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 18

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.447015Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation fa61c109-b017-491e-af8c-6168aea6e493 · outbound

This paper cites GPT-4 Technical Report.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models GPT-4 Technical Report

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.107648Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f25817aa-4ee6-4b60-b2b6-49c50c437dc3 · outbound

This paper cites The LAMBADA dataset: Word prediction requiring a broad discourse context.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models The LAMBADA dataset: Word prediction requiring a broad discourse context

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.110282Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8f3d0465-aa73-4646-aace-d3067162310b · outbound

This paper cites Language Model Tokenizers Introduce Unfairness Between Languages.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Language Model Tokenizers Introduce Unfairness Between Languages

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.112975Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 679ce153-a688-4453-b7c4-f7c1063e7ae3 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 22

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.439237Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation c878b7df-5fb4-4505-b0fb-5617c76bb5f9 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.117661Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 33fdf193-adb9-4028-b21b-50f1956cecad · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.120049Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0c3fc772-0f05-4fce-86cc-faade0aab335 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.122282Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation afd4083e-b090-46fe-8f22-d2ab3ccb1d34 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 26

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.431371Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 032d1d6d-e205-49f8-a007-6722a45a082b · outbound

This paper cites Tokenization Is More Than Compression.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Tokenization Is More Than Compression

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.128101Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a58f0b5d-74f9-4135-9e9a-3fc4309e54bf · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 28

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.423545Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation d8506af5-1699-4158-907c-e3d8d2866223 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 29

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.415568Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 1ad9fc31-0882-4ba0-8f4c-311a49ae3cc3 · outbound

This paper cites Greed is All You Need: An Evaluation of Tokenizer Inference Methods.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Greed is All You Need: An Evaluation of Tokenizer Inference Methods

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.136962Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 82ac2e93-56b0-485e-a80c-a8017d75d31e · outbound

This paper cites Egalitarian Language Representation in Language Models: It All Begins with Tokenizers.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Egalitarian Language Representation in Language Models: It All Begins with Tokenizers

Reference 31

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.140105Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ccb193a0-747a-4e40-8aed-aef7dcf40493 · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.142947Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ff1acca1-06b5-43db-a08b-42d9dc54bc1e · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.145180Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e1ce7be9-2059-42e0-a329-203afb4b2c6d · outbound

This paper cites an unresolved cited work.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models Unresolved cited work

Reference 34

Resolution: unresolved; raw_fallback, observed 2026-08-11T19:25:22.397573Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation ea23ced9-9282-4f0e-8776-7c6ebd66f0ca · outbound

This paper cites online" 'onlinestring :=.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models online" 'onlinestring :=

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.149689Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation df4a8c44-be81-498c-a11f-39a5ff5ef582 · outbound

This paper cites write newline.

When Every Token Counts: Optimal Segmentation for Low-Resource Language Models write newline

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-11T19:25:22.152544Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 472eac41-6e95-4b58-9343-b1c85a10bd84 · inbound

Tokenization Matters: Improving Zero-Shot NER for Indic Languages cites this paper.

Tokenization Matters: Improving Zero-Shot NER for Indic Languages When Every Token Counts: Optimal Segmentation for Low-Resource Language Models

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-16T10:55:55.460756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7adebcde-5119-423a-a6da-c2bca8560b34 · inbound

Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet cites this paper.

Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet When Every Token Counts: Optimal Segmentation for Low-Resource Language Models

Reference 54

Resolution: verified exact; arxiv_id, observed 2026-07-04T04:39:34.657942Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.