Pith. sign in

REVIEW 4 major objections 5 minor 57 references

DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DualComp is a single lightweight neural compressor that losslessly handles both images and text, matching billion-parameter language-model compressors while running near real time on a desktop CPU.

desk verdict Solid lightweight dual-modality compressor with honest ablations; the abstract overclaims parity with LLM compressors and the 9% Kodak gain rests on an unverified baseline. read the letter →

arxiv 2505.16256 v1 pith:QXJDB6TK submitted 2025-05-22 cs.CV cs.AIcs.MM

Yan Zhao , Zhengxue Cheng , Junxuan Zhang , Qunshan Gu , Qi Wang , Li Song This is my paper
classification cs.CVcs.AIcs.MM
keywords dual-modalitylosslesscompressionlearnedimagetextmixture-of-expertsmodality-switchingRWKVreparameterizationtrainingentropycoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single small neural network can losslessly compress both images and text, a job that previously required either separate single-modality models or billion-parameter language models. The proposed system, DualComp, builds on a lightweight recurrent backbone and adds three adjustments: a shared token vocabulary for both image sub-pixels and text, separate contextual-prediction layers that activate depending on the input modality, and a small mixture-of-experts layer that routes each token to specialized sub-networks. A reparameterization trick gives the model extra learning capacity during training without enlarging the model at deployment. The paper reports that DualComp matches far larger language-model compressors on both image and text benchmarks, and that its image-only variant beats the best previous image compressor on the Kodak dataset by about 9% while using about 1.2% of the parameters and decoding near real time on a desktop CPU. If true, learned lossless compression could be unified and practical, not just powerful.

What carries the argument

The load-bearing mechanism is the pair of modality-adaptive modules grafted onto the RWKV-7 backbone. Modality-switching contextual learning splits the R/K/V projection layers of each Time Mixing block into separate image and text sets, so each modality builds its own contextual predictions while sharing the rest of the block; this is what lets one model hold two different statistical models. The modality-routing mixture-of-experts replaces the final MLP with three smaller experts and a learned router that activates the top two per token, letting image and text tokens follow different nonlinear paths. Underneath both sits a unified tokenizer: text uses a 16K SentencePiece BPE vocabulary while image sub-pixels occupy a 256-token subspace, merged into one vocabulary with non-target logits masked before arithmetic coding. The reparameterization training strategy adds high-rank bypass branches to R/K/V projections during training and merges them away at inference, buying capacity without extra runtime cost.

What would settle it

Retrain the previous best image compressor, P2LLM, under DualComp-I's exact conditions, namely the same 5,500 ImageNet training images, the same 16×16×3 patch tokenization, and the same 16-epoch schedule, then measure its bits per byte on Kodak; if it reaches 2.571 or lower, the claimed 9% advantage disappears.

Watch

Extended reading notes

Core claim

The paper's central claim is that modality heterogeneity can be handled inside a single model rather than by separate modality-specific compressors or by flattening every input into text. DualComp tokenizes images as sequences of sub-pixel values and text as BPE tokens within one shared vocabulary, then applies modality-switching contextual learning: separate R/K/V (receptance/key/value) projection layers in each RWKV-7 Time Mixing block are activated depending on whether the input is an image or text, while the output and MLP layers stay shared. In the final block, a modality-routing mixture-of-experts with three experts replaces the MLP, and a learned router selects which experts process each token. A high-rank reparameterization branch on the R/K/V projections is trained and then merged away, so inference keeps a compact single-path structure. The reported outcome is that the 130M-parameter dual-modality model reaches 2.834 bits/byte on Kodak and 1.107 bits/byte on enwik9, comparable to large language model compressors, while the 96M-parameter image-only variant reaches 2.571 bits/byte on Kodak, about 9% better than the 8B-parameter P2LLM.

Load-bearing premise

The claimed 9% improvement over the previous best image compressor depends on treating that compressor's published compression numbers as directly comparable to DualComp-I's, even though the two models were trained on different data and with different preprocessing.

Editorial extensions

If this is right

  • A single DualComp model can replace separate image and text compressors in a deployment, cutting memory and system complexity because roughly two-thirds of its parameters are shared across modalities.
  • Billion-parameter LLM compressors are not needed for strong lossless ratios; the gains here come from structure, namely modality-specific context and routing, rather than model scale.
  • Near-real-time decoding on desktop CPUs and mobile NPUs, with small models reaching hundreds of kilobytes per second, makes learned lossless compression practical for everyday file and image archiving.
  • If the Kodak result transfers, the image-only variant sets a new state of the art for learned lossless image compression at 2.571 bits/byte with 96M parameters.
  • Adding a new modality becomes a modular operation: attach a tokenizer, modality-specific R/K/V branches, and MoE adaptation to the same shared backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An independent head-to-head rerun, retraining the 8B-parameter P2LLM baseline on the same 5,500 ImageNet images and patch preprocessing used for DualComp-I, would determine whether the 9% Kodak margin is an architecture effect or a training-protocol effect; the paper compares against the baseline's published numbers.
  • The same modality-switching plus routing recipe could be extended to audio and video by adding tokenizers and modality branches to the shared backbone, with the MoE router learning which experts serve which modality; the authors list this as future work rather than a demonstrated result.
  • The routing statistics themselves, with images showing higher expert variability than text, could be read as a per-modality complexity signal and used to allocate more parameters to harder modalities, a step the paper does not take.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes DualComp, a lightweight dual-modality (image and text) lossless compression model built on the RWKV-7 backbone. The method introduces modality-unified tokenization (a shared vocabulary of 256 image sub-pixel tokens and 16K SentencePiece BPE text tokens), modality-switching contextual learning (separate R/K/V projections per modality), and a modality-routing mixture-of-experts replacing the final MLP, plus a high-rank reparameterization training strategy. Experiments on Kodak, CLIC-P, CLIC-M, DIV2K, and enwik9 report bits/byte for several model scales, with CPU/NPU/GPU speed measurements. The abstract claims parity with SOTA LLM-based compressors for text and images, and a 9% improvement over the previous best image compressor on Kodak with 1.2% of the parameters.

Significance. The paper is practically motivated and the experimental effort is substantial: the authors reproduce classical baselines, measure inference speed on three platforms, and provide ablations for each proposed component. The single-modality variant, DualComp-I, demonstrates that reparameterization alone yields consistent gains on Kodak. However, the headline claims as stated are not supported by the paper's own tables: text compression at 1.107 bits/byte on enwik9 is far from the 0.722 bits/byte of Llama3-8B and 0.642 of Chinchilla-70B, and the 9% image gain is limited to Kodak and rests on cross-paper comparison with P2LLM. The architectural ideas are worth publishing after claim revisions and stronger baseline verification.

major comments (4)
  1. [Abstract; Section 4.2; Table 2] The claim that DualComp 'achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets' is internally contradicted for text. DualComp-130M compresses enwik9 to 1.107 bits/byte, while Table 2 lists Llama3-8B at 0.722 and Chinchilla-70B at 0.642; even classical learned compressors NNCP (0.853) and CMIX (0.879) are substantially better. Section 4.2 in fact narrows the claim to outperforming L3TC (1.280), which is a different statement. The abstract, Figure 1, and Figure 5 should be revised to reflect the actual scope of the text result, or the claim must be removed.
  2. [Section 4.2; Table 2; Table 6] The headline image result, a 9% gain over P2LLM on Kodak, is not supported as a general SOTA claim. DualComp-I-96M achieves 2.571 bits/byte vs. P2LLM's reported 2.830 on Kodak, but on CLIC-P it ties (2.350 vs 2.350), on CLIC-M it is 1% worse (2.110 vs 2.080), and on DIV2K it is 2% worse (2.547 vs 2.510). Moreover, the P2LLM values are taken from the original paper without retraining on the same data or preprocessing; the table's own footnote states that unmarked values are 'claimed by their original papers,' and the paper's backbone-selection experiments use a different grayscale-ASCII protocol. The 9% claim should be stated as a Kodak-specific improvement conditional on baseline comparability, or the baseline must be reproduced under identical conditions.
  3. [Section 4.4; Table 4; Section 3.5] Several hyperparameters are selected using the evaluation datasets. The auxiliary loss weight λ=0.01 (Section 3.5), the MoE expert count, top-k, hidden factor, and reparameterization rank are all chosen based on bits/byte on Kodak and enwik9 (e.g., Table 4 uses DualComp-0.3M evaluated on Kodak and enwik9). Tuning on the test sets makes the reported numbers optimistic and weakens the generality of the ablation conclusions. The authors should either use a held-out validation set for these choices or report sensitivity of the final models to these hyperparameters.
  4. [Section 4.1; Section 6.2] The training protocol for the main DualComp model is described only briefly (5,500 ImageNet images and 100MB of enwik8; three-stage training with specified epochs and learning rates), and no code or trained models are provided. Given that the central comparisons are sensitive to preprocessing and tokenization, the lack of a detailed protocol for image patch order, context length, and arithmetic coding implementation makes independent verification difficult. At minimum, the authors should provide an open-source implementation or exact protocol details.
minor comments (5)
  1. [Equation (1)] The target distribution q in the cross-entropy term L = -Σ q log p is never defined; please clarify whether q is the one-hot target distribution or something else.
  2. [Table 2 footnote] There is a typo: 'reprodeuced' should be 'reproduced', and 'presents' should be 'indicates' in the footnote about inference speeds.
  3. [Section 4.2] The sentence 'DualComp reaches comparable compression efficiency to SOTA approaches' is ambiguous. Given the text gap to LLM-based methods, specify that the comparison is against lightweight practical compressors such as L3TC, rather than all SOTA methods.
  4. [Section 3.1, Table 1] The backbone-selection experiment uses grayscale ASCII image encoding following [6], which differs from the final sub-pixel tokenization used in DualComp; a brief justification that the backbone ranking transfers to the final tokenization would be helpful.
  5. [Section 4.4] The statement that masking non-target logits 'brings up to 20% gains' is vague; please specify the baseline (which model, which dataset) and report the exact numbers in the ablation table.

Circularity Check

2 steps flagged · score 3.0 of 10

Mostly independent empirical system; mild circularity from test-set hyperparameter selection and enwik8/enwik9 training overlap.

  1. fitted input called prediction [Section 4.4 / Table 4]
    "We investigate the effect of different MoE configurations using the 0.3M DualComp model, as shown in Table. 4 and Fig. 7. It can be seen that replacing only the final MLP with an MoE achieves similar performance to replacing all MLPs ... Therefore, we adopt the most parameter-efficient configuration, as highlighted in Table. 4."

    Table 4 reports bits/Byte on Kodak and enwik9 for each MoE configuration. The number of MoE layers, top-k, expert count, and expert hidden factor are chosen because they minimize or match bits/Byte on these exact evaluation datasets. The final DualComp results in Table 2 are then presented as out-of-sample predictions, but the configuration was selected using the same test sets, so the reported numbers are partially fitted to the evaluation data rather than independent forecasts.

  2. other [Section 4.1, Dual-Modality Datasets]
    "The training set consists of 5,500 images from ImageNet [43] and 100MB of text from enwik8 [47]. ... For evaluation, we use Kodak [44], CLIC-Mobile, CLIC-Pro [48], and DIV2K [49] as image test sets, and enwik9 [50] (1GB of text) as the text test set."

    enwik8 is the standard 100MB prefix of enwik9, so the text model is trained on the first 10% of the reported test set. The reported enwik9 bits/Byte is therefore a mixture of memorized training prefix and unseen text, making the text-side 'prediction' partly a fit to the test set rather than an independent forecast. This does not affect the image results but undermines the text-side parity claim.

full rationale

The paper's core contributions are architectural (modality-unified tokenization, modality-switching contextual learning, modality-routing MoE, reparameterization) and are evaluated on external image benchmarks (Kodak, CLIC-P, CLIC-M, DIV2K) against external baselines. No result is derived from a self-citation chain or a uniqueness theorem, and the headline 9% image gain over P2LLM is an empirical comparison rather than a construction. However, two evaluation choices introduce mild circularity. First, Section 4.4 selects MoE hyperparameters using bits/Byte on Kodak and enwik9, the same datasets later reported in Table 2, so the reported numbers are selected rather than strictly out-of-sample. Second, the text model is trained on 100MB of enwik8, the prefix of enwik9 used as the text test set, so the 1.107 bits/Byte figure is partly memorization of the test set. The abstract's claim of text parity with SOTA LLM methods is also contradicted by the paper's own Table 2 (DualComp-130M: 1.107 vs Llama3-8B: 0.722), but that is a correctness/claim mismatch rather than a circularity. Overall the central image-side derivation is self-contained against external benchmarks, so the circularity score remains low.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a set of well-established techniques (entropy coding, RWKV-7) plus empirically chosen hyperparameters. No new physical entities are introduced. The most consequential choices are the MoE configuration and the assumption that external baseline results are comparable.

free parameters (6)
  • MoE expert count and top-k = 3 experts, top-2
    Selected by ablation on Kodak and enwik9 (Table 4), so tuned on the evaluation datasets.
  • MoE hidden factor = 2x embedding dim
    Chosen via ablation (Table 4 bottom) on the same test benchmarks.
  • Auxiliary loss weight lambda = 0.01
    Weight for expert balance loss in Eq. (1); chosen by hand without reported sensitivity analysis.
  • Reparameterization rank r = 4x embedding dim
    Taken from L3TC [10]; a hyperparameter not independently justified here.
  • Image patch size and scan order = 16x16x3, raster scan
    Ad hoc choice; no comparison of patch sizes or scan orders is reported, though performance on Kodak is attributed to small patches (Section 4.2).
  • Training schedule (epochs, learning rates) = 2/2/16 epochs; lr 2e-5/2e-5/cosine 1e-4 to 5e-6
    Three-stage training procedure (Section 4.1), chosen by the authors without systematic search.
assumptions (4)
  • standard math Arithmetic coding achieves near-entropy coding when predictive probabilities are accurate.
    Invoked in Introduction and Section 3 as the foundation for converting probabilities into bitstreams.
  • domain assumption RWKV-7's linear-time recurrent architecture is sufficient to model long-range dependencies in flattened image and text sequences.
    The paper selects RWKV-7 after a small comparison (Table 1) and relies on it as the backbone; no analysis of failure cases for long-range image dependencies.
  • domain assumption The evaluation datasets (Kodak, CLIC-P, CLIC-M, DIV2K, enwik9) are representative of natural images and text.
    Used to generalize the headline 9% claim to images/text at large (Section 4).
  • domain assumption Baseline numbers from cited papers are accurate and measured under comparable protocols.
    Table 2 uses unmarked values claimed by original papers, e.g., P2LLM's 2.830 bits/byte on Kodak, without re-running.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor." pith.science (2026). https://pith.science/paper/QXJDB6TK

@misc{pith2026250516256,
  author       = {Pith},
  title        = {Pith review of: DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QXJDB6TK}},
  note         = {Machine review of arXiv:2505.16256}
}
read the original abstract

Most learning-based lossless compressors are designed for a single modality, requiring separate models for multi-modal data and lacking flexibility. However, different modalities vary significantly in format and statistical properties, making it ineffective to use compressors that lack modality-specific adaptations. While multi-modal large language models (MLLMs) offer a potential solution for modality-unified compression, their excessive complexity hinders practical deployment. To address these challenges, we focus on the two most common modalities, image and text, and propose DualComp, the first unified and lightweight learning-based dual-modality lossless compressor. Built on a lightweight backbone, DualComp incorporates three key structural enhancements to handle modality heterogeneity: modality-unified tokenization, modality-switching contextual learning, and modality-routing mixture-of-experts. A reparameterization training strategy is also used to boost compression performance. DualComp integrates both modality-specific and shared parameters for efficient parameter utilization, enabling near real-time inference (200KB/s) on desktop CPUs. With much fewer parameters, DualComp achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets. Its simplified single-modality variant surpasses the previous best image compressor on the Kodak dataset by about 9% using just 1.2% of the model size.

Figures

Figures reproduced from arXiv: 2505.16256 by the authors.

Figure 1
Figure 1. Left: Most existing lossless compressors support only a single modality, whereas DualComp enables dual-modality compression in one model. Right: Lossless compression performance (bits/Byte) on image (Kodak) and text (enwik9) datasets. DualComp matches or surpasses SOTA methods with fewer parameters on both image and text. data as ASCII text, resulting in relatively poor performance on non-text modalities. In fact, d… view at source ↗
Figure 2
Figure 2. DualComp tokenizes text and image inputs with a unified vocabulary, then encodes them into a compressed bitstream via context probabilities and arithmetic coding. Built on a lightweight backbone, it further incorporates modality-switching context learning and a modality-routing mixture-of-experts for efficient dual-modality compression. Huffman coding [4], while bzip2 improves compression using the Burrows-Wheeler t… view at source ↗
Figure 3
Figure 3. Dual-modality tokenization: images are patched and scanned into 1D sequences, with each subpixel as a token. Text is tokenized using an SPM-BPE tokenizer. The two modalities share a unified vocabulary of 16K size. representation (Section 3.4). Besides, inspired by [10], we apply a high-rank reparameterization strategy to boost representation capacity without increasing inference complexity (Section 3.1). 3.1 Prelimi… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Percent of expert usage in each batch when compressing image (left) and text (right) using DualComp-16M. inherently two-dimensional and characterized by inter-pixel and inter-channel correlations. When flattened into 1D sequences via patching and raster scan, spatially…
Figure 5
Figure 5. Figure 5: Left: Learning-based methods’ image compression performance (bits/Byte vs. model size). The closer to the bottom-left corner, the smaller the model and the better the performance. Right: Dual-modality compression consistency. The x-axis and y-axis are bits/Byte on text…
Figure 6
Figure 6. Figure 6: Left: Parameter breakdown of DualComp at different scales (modality-specific vs. modality shared). Right: DualComp’s inference speed (KB/s) on a MacBook Air CPU at varying batch sizes (1, 16, 128, and 512). compresses enwik9 to 1.107 bits/Byte, yielding about 57% gains…
Figure 7
Figure 7. Figure 7: Discussion on modality-routing MoE configurations using the DualComp-0.3M model. The line plots (left y-axis) indicate compression performance (bits/Byte) on image (Kodak) and text (enwik9) datasets, while the bar charts (right y-axis) present the total and activated p…
Figure 8
Figure 8. Figure 8: Illustration of the high-rank repa￾rameterization training strategy. Specifically, instead of using parallel branches like 1 × 1 convolutions or shortcuts as in [54], we follow the strategy in [10] and reparameterize each branch as a product of two high-rank matrices: …
Figure 9
Figure 9. Figure 9: Lossless compression results (bits/Byte) on CLIC-P, CLIC-M, and DIV2K datasets. DualComp-I achieves SOTA performance on CLIC-P and ranks second on CLIC-M and DIV2K. DualComp ranks third on these datasets. (0.642 bits/Byte) with 70B parameters. Herein, we also include t…
Figure 10
Figure 10. Figure 10: DualComp’s inference speed (KB/s) on an iPhone 15 Pro’s NPU (left) and a NVIDIA A100 GPU (right) at varying batch sizes (1, 16, 128, and 512). Two model sizes (0.3M and 16M) are used for illustration. Pro. The DualComp-I models are even smaller and achieve faster infe…
Figure 11
Figure 11. Figure 11: Expert usage per batch when compressing CLIC-P, CLIC-M, and DIV2K using DualComp-16M. Expert 1 is most frequently used expert across image patches, likely because it captures more general image features. from the longer temporal dependencies created by flattening 2D i…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 39 canonical work pages

  1. [1]

    C. E. Shannon. A mathematical theory of communication.The Bell System Technical Journal, 27(3): 379–423, 1948. doi: 10.1002/j.1538-7305.1948.tb01338.x

    arXiv 1948

  2. [2]

    Cambridge university press, 2003

    David JC MacKay.Information theory, inference and learning algorithms. Cambridge university press, 2003

    2003

  3. [3]

    Howard and Jeffrey Scott Vitter

    Paul G. Howard and Jeffrey Scott Vitter. Analysis of arithmetic coding for data compression. InProceedings of Data Compression Conference (DCC), 1991

    work page 1991

  4. [4]

    David A. Huffman. A method for the construction of minimum-redundancy codes. InProceedings of the IRE, 1952

    work page 1952

  5. [5]

    Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding

    Jarek Duda. Asymmetric numeral systems: entropy coding combining speed of Huffman coding with compression rate of arithmetic coding. arXiv:1311.2540., 2013

    arXiv 2013

  6. [6]

    Language modeling is compression

    Gregoire Deletang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. Language modeling is compression. InThe Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview.net/forum?id=jznbgiynus

    work page 2024

  7. [7]

    Llmzip: Lossless text compression using large language models.arXiv preprint arXiv:2306.04050, 2023

    Chandra Shekhara Kaushik Valmeekam, Krishna Narayanan, Dileep Kalathil, Jean-Francois Chamberland, and Srinivas Shakkottai. Llmzip: Lossless text compression using large language models.arXiv preprint arXiv:2306.04050, 2023

    arXiv 2023

  8. [8]

    ts_zip: Text Compression using Large Language Models

    Fabrice Bellard. ts_zip: Text Compression using Large Language Models. https://bellard.org/ts_ zip/, 2023. Accessed: 2024-08-10

    work page 2023

Show all 57 references
  1. [9]

    Large language models for lossless image compression: Next-pixel prediction in language space is all you need.arXiv preprint arXiv:2411.12448, 2024

    Kecheng Chen, Pingping Zhang, Hui Liu, Jie Liu, Yibing Liu, Jiaxin Huang, Shiqi Wang, Hong Yan, and Haoliang Li. Large language models for lossless image compression: Next-pixel prediction in language space is all you need.arXiv preprint arXiv:2411.12448, 2024

    2024 arXiv

  2. [10]

    L3tc: Leveraging rwkv for learned lossless low-complexity text compression

    Junxuan Zhang, Zhengxue Cheng, Yan Zhao, Shihao Wang, Dajiang Zhou, Guo Lu, and Li Song. L3tc: Leveraging rwkv for learned lossless low-complexity text compression. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 13251–13259, 2025

    2025

  3. [11]

    Msdzip: Universal lossless compression for multi-source data via stepwise-parallel and learning-based prediction

    Huidong Ma, Sun Hui, Liping Yi, Ding Yanfeng, Gang Wang, et al. Msdzip: Universal lossless compression for multi-source data via stepwise-parallel and learning-based prediction. InTHE WEB CONFERENCE 2025

    2025

  4. [12]

    Practical full resolution learned lossless image compression

    Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Practical full resolution learned lossless image compression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10629–10638, 2019. doi: 10.1109...

    2019

  5. [13]

    Large language model for lossless image compression with visual prompts.arXiv preprint arXiv:2502.16163, 2025

    Junhao Du, Chuqin Zhou, Ning Cao, Gang Chen, Yunuo Chen, Zhengxue Cheng, Li Song, Guo Lu, and Wenjun Zhang. Large language model for lossless image compression with visual prompts.arXiv preprint arXiv:2502.16163, 2025

    2025 arXiv

  6. [14]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/ main/MODEL_CARD.md

    2024

  7. [15]

    NNCP v2: Lossless data compression with transformer.Technical report, Amarisoft, 2021

    Fabrice Bellard. NNCP v2: Lossless data compression with transformer.Technical report, Amarisoft, 2021

    2021

  8. [16]

    Cmix version 20, a lossless data compression program

    Byron Knoll. Cmix version 20, a lossless data compression program. http://www.byronknoll.com/ cmix.html, 2023. Accessed: 2024-08-10

    2023

  9. [17]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017. Accessed: 2024-08-10

    2017

  10. [18]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. neural computation.Neural Compu- tation, 1997

    1997

  11. [19]

    Rwkv-7" goose" with expressive dynamic state evolution

    Bo Peng, Ruichong Zhang, Daniel Goldstein, Eric Alcaide, Haowen Hou, Janna Lu, William Merrill, Guangyu Song, Kaifeng Tan, Saiteja Utpala, et al. Rwkv-7" goose" with expressive dynamic state evolution. arXiv preprint arXiv:2503.14456, 2025

    2025 arXiv

  12. [20]

    Multilayer perceptron: Architecture optimization and training

    Hassan Ramchoun, Youssef Ghanou, Mohamed Ettaouil, and Mohammed Amine Janati Idrissi. Multilayer perceptron: Architecture optimization and training. 2016. 10

    2016

  13. [21]

    A survey on mixture of experts.arXiv preprint arXiv:2407.06204, 2024

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts.arXiv preprint arXiv:2407.06204, 2024

    2024 arXiv

  14. [22]

    Richard C. Pasco. GZIP file format specification version 4.3.RFC 1952, 1996

    1952

  15. [23]

    W3C, 1997

    Thomas Boutell.PNG (Portable Network Graphics) Specification Version 1.0. W3C, 1997. URL https://www.w3.org/TR/PNG/

    1997

  16. [24]

    On the Performance of BWT Sorting Algorithms.Proceedings of the IEEE Data Compres- sion Conference 2000, 2000

    Julian Seward. On the Performance of BWT Sorting Algorithms.Proceedings of the IEEE Data Compres- sion Conference 2000, 2000

    2000

  17. [25]

    ZSTD, Zstandard - Fast real-time compression algorithm.https://github.com/facebook/zstd,

    Meta. ZSTD, Zstandard - Fast real-time compression algorithm.https://github.com/facebook/zstd,

  18. [26]

    A Universal Algorithm for Sequential Data Compression.IEEE Transactions on Information Theory, 1977

    Abraham Ziv, Jacob; Lempel. A Universal Algorithm for Sequential Data Compression.IEEE Transactions on Information Theory, 1977

    1977

  19. [27]

    An analysis of the burrows—wheeler transform.Journal of the ACM (JACM), 48(3): 407–430, 2001

    Giovanni Manzini. An analysis of the burrows—wheeler transform.Journal of the ACM (JACM), 48(3): 407–430, 2001

    2001

  20. [28]

    Move-to-front and inversion coding

    Ziya Arnavut. Move-to-front and inversion coding. InProceedings DCC 2000. Data Compression Conference, pages 193–202. IEEE, 2000

    2000

  21. [29]

    Finite state machine binary entropy coding

    Michael J Gormish and J Allen. Finite state machine binary entropy coding. InProc. Data Compression Conference, page 449. Citeseer, 1993

    1993

  22. [30]

    Webp image format

    Google. Webp image format. https://developers.google.com/speed/webp, 2010. Accessed: 2025-02-28

    2010

  23. [31]

    Flif - free lossless image format.https://flif.info/, 2015

    Jon Sneyers. Flif - free lossless image format.https://flif.info/, 2015. Accessed: 2023-10-01

    2015

  24. [32]

    Jpeg xl image coding system

    JPEG XL Team. Jpeg xl image coding system. https://jpeg.org/jpegxl/, 2021. Accessed: 2025- 02-28

    2021

  25. [33]

    Jpeg 2000 image coding system

    ISO/IEC. Jpeg 2000 image coding system. https://www.jpeg.org/jpeg2000/, 2000. Accessed: 2025-02-28

    2000

  26. [34]

    Bpg image format.https://bellard.org/bpg/, 2014

    Fabrice Bellard. Bpg image format.https://bellard.org/bpg/, 2014. Accessed: 2025-02-28

    2014

  27. [35]

    Context-adaptive binary arithmetic coding with fixed-length codewords.IEEE Transactions on Multimedia, 17(8):1385–1390, 2015

    Francesc Auli-Llinas. Context-adaptive binary arithmetic coding with fixed-length codewords.IEEE Transactions on Multimedia, 17(8):1385–1390, 2015

    2015

  28. [36]

    Deep lossy plus residual coding for lossless and near-lossless image compression.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3577–3594, 2024

    Yuanchao Bai, Xianming Liu, Kai Wang, Xiangyang Ji, Xiaolin Wu, and Wen Gao. Deep lossy plus residual coding for lossless and near-lossless image compression.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3577–3594, 2024

    2024

  29. [37]

    ivpf: Numerical invertible volume preserving flow for efficient lossless compression

    Shifeng Zhang, Chen Zhang, Ning Kang, and Zhenguo Li. ivpf: Numerical invertible volume preserving flow for efficient lossless compression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 620–629, 2021

    2021

  30. [38]

    iflow: Numerically invertible flows for efficient lossless compression via a uniform coder.Advances in Neural Information Processing Systems, 34: 5822–5833, 2021

    Shifeng Zhang, Ning Kang, Tom Ryder, and Zhenguo Li. iflow: Numerically invertible flows for efficient lossless compression via a uniform coder.Advances in Neural Information Processing Systems, 34: 5822–5833, 2021

    2021

  31. [39]

    Learning better lossless compression using lossy compression

    Fabian Mentzer, Luc Van Gool, and Michael Tschannen. Learning better lossless compression using lossy compression. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6638–6647, 2020

    2020

  32. [40]

    Finezip: Pushing the limits of large language models for practical lossless text compression.arXiv preprint arXiv:2409.17141, 2024

    Fazal Mittu, Yihuan Bu, Akshat Gupta, Ashok Devireddy, Alp Eren Ozdarendeli, Anant Singh, and Gopala Anumanchipalli. Finezip: Pushing the limits of large language models for practical lossless text compression.arXiv preprint arXiv:2409.17141, 2024

    2024 arXiv

  33. [41]

    Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  34. [42]

    Neural machine translation of rare words with subword units, 2016

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units, 2016. URLhttps://arxiv.org/abs/1508.07909. Accessed: 2024-08-10. 11

    2016 arXiv

  35. [43]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  36. [44]

    Kodak lossless true color image suite

    Eastman Kodak Company. Kodak lossless true color image suite. Internal Research Dataset, 1999. 24 uncompressed PNG images, 768x512 resolution

    1999

  37. [45]

    Apex (a pytorch extension)

    NVIDIA. Apex (a pytorch extension). https://nvidia.github.io/apex/optimizers.html, 2018. URLhttps://nvidia.github.io/apex/. API Documentation for NVidia’s Apex optimizers

    2018

  38. [46]

    Sgdr: Stochastic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts.arXiv preprint arXiv:1608.03983, 2016

    2016 arXiv

  39. [47]

    enwik8: First 100m bytes of the english wikipedia dump

    Wikipedia Community. enwik8: First 100m bytes of the english wikipedia dump. https://cs. fit.edu/~mmahoney/compression/enwik8.zip, 2006. URL https://cs.fit.edu/~mmahoney/ compression/textdata.html. Preprocessed version used for character-level language modeling bench- marks

    2006

  40. [48]

    CLIC: Workshop and challenge on learned image compression

    CLIC. CLIC: Workshop and challenge on learned image compression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021

    2021

  41. [49]

    Ntire 2017 challenge on single image super-resolution: Dataset and study

    Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. InProceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 126–135, 2017

    2017

  42. [50]

    enwik9: Complete english wikipedia dump (xml)

    Wikipedia Community. enwik9: Complete english wikipedia dump (xml). https://cs. fit.edu/~mmahoney/compression/enwik9.zip, 2007. URL https://cs.fit.edu/~mmahoney/ compression/textdata.html. Full Wikipedia XML dump preprocessed for large-scale language mod- eling

    2007

  43. [51]

    Learned lossless image compression through interpolation with low complexity.IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7832–7841, 2023

    Fatih Kamisli. Learned lossless image compression through interpolation with low complexity.IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7832–7841, 2023

    2023

  44. [52]

    Training compute-optimal large language models.arXiv preprint arXiv:2203.15556, 2022

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models.arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  45. [53]

    Large Text Compression Benchmark

    Matt Mahoney. Large Text Compression Benchmark. https://www.mattmahoney.net/dc/text. html, 2024. Accessed: 2024-08-10

    2024

  46. [54]

    Repvgg: Making vgg-style convnets great again

    Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making vgg-style convnets great again. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13733–13742, 2021

    2021

  47. [55]

    Rwkv: Reinventing rnns for the transformer era, 2023

    Bo Peng, Eric Alcaide, Quentin Anthony, et al. Rwkv: Reinventing rnns for the transformer era, 2023. URLhttps://arxiv.org/abs/2305.13048. Accessed: 2024-08-10

    2023 arXiv

  48. [56]

    Core ml, 2023

    Apple Inc. Core ml, 2023. URL https://developer.apple.com/documentation/coreml. Apple’s framework for integrating machine learning models into apps. 12 Supplementary Material The supplementary material provides additional implementation details and experimental results omitted...

    2023

  49. [2015]

    Accessed: 2024-08-10

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.