LLM Vocabulary Compression for Low-Compute Environments

Anish Joishy; Ponnurangam Kumaraguru; Sreeram Vennam

LLM Vocabulary Compression for Low-Compute Environments

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.06371 v1 pith:35IJ4BND submitted 2024-11-10 cs.CL cs.LG

Sreeram Vennam , Anish Joishy , Ponnurangam Kumaraguru This is my paper · ORCID

classification cs.CLcs.LG

keywords environmentslow-computemethodbytecompresscompressiondatasetencoding

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

We present a method to compress the final linear layer of language models, reducing memory usage by up to 3.4x without significant performance loss. By grouping tokens based on Byte Pair Encoding (BPE) merges, we prevent materialization of the memory-intensive logits tensor. Evaluations on the TinyStories dataset show that our method performs on par with GPT-Neo and GPT2 while significantly improving throughput by up to 3x, making it suitable for low-compute environments.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

SpeLLM: Character-Level Multi-Head Decoding
cs.CL 2025-07 conditional novelty 6.0 of 10

SpeLLM converts a standard token-based LLM into a character-spelling model with multiple parallel output heads, achieving competitive downstream performance with a 5.1% average decoding speedup.

Pith tools