Pith. sign in

REVIEW 3 cited by

BitNet b1.58 2B4T Technical Report

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.12285 v2 pith:2U7CKZVZ submitted 2025-04-16 cs.CL cs.LG

Shuming Ma , Hongyu Wang , Shaohan Huang , Xingxing Zhang , Ying Hu , Ting Song , Yan Xia , Furu Wei This is my paper · ORCID
classification cs.CLcs.LG
keywords bitnetmodellanguageopen-sourceabilityachievesacrossadoption
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4 trillion tokens, the model has been rigorously evaluated across benchmarks covering language understanding, mathematical reasoning, coding proficiency, and conversational ability. Our results demonstrate that BitNet b1.58 2B4T achieves performance on par with leading open-weight, full-precision LLMs of similar size, while offering significant advantages in computational efficiency, including substantially reduced memory footprint, energy consumption, and decoding latency. To facilitate further research and adoption, the model weights are released via Hugging Face along with open-source inference implementations for both GPU and CPU architectures.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation

    cs.LG 2026-07 conditional novelty 7.0 of 10 partial

    For a fixed low-bit residual library, the distance to the closed relaxed reachable set is an exact structural floor that pure depth approaches at O(1/D), while write-back arithmetic can reverse the gain and accuracy m...

  2. BitNet Text Embeddings

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    BITEMBED trains 1.58-bit ternary-weight LLM embedders with contrastive pre-training, supervised distillation, and multi-precision output training, matching FP16 teachers within ~0.6 MMTEB points at ~2x CPU speed.

  3. Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodings

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A top-magnitude sign quantisation, grounded in a family of convex polytopes, encodes embeddings as ternary vectors and delivers fast, accurate approximate similarity search.

Pith tools