Pith. sign in

REVIEW 4 major objections 6 minor 19 references

LookME is the first framework to apply lookup-based external-embedding enhancement to multimodal (image and cross-modal) tokens in vision-language models, and it reports consistent gains over text-only PLE methods across 11 visual benchmark

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:28 UTC pith:JKN3AJOC

load-bearing objection A genuinely new extension of PLE to continuous visual and cross-modal tokens, but the empirical case is undermined by a missing supplement, no variance reporting, and a direct contradiction in the multi-image benchmark. the 4 major comments →

arxiv 2607.16305 v1 pith:JKN3AJOC submitted 2026-07-14 cs.CV cs.AI

LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Models

Zeyu Xu , Xingzhong Hou , Pengkai Guo , Siling Lin , Xiao Xu , Menghua Zhai , Haoyu Chen , Yunke Zhang
show 1 more author
Fei Huang
This is my paper
classification cs.CV cs.AI
keywords vision-language modelsper-layer embedding (PLE)multimodal embedding lookuphierarchical two-level retrievalsparse injectionROM storageedge deploymentvisual token enhancement
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LookME sets out to show that per-layer embedding (PLE) scaling—storing large external knowledge tables in ROM and doing lightweight lookups—can work for the visual token stream of a vision-language model, not just for text tokens. The paper introduces a hierarchical two-level lookup that first routes continuous image and cross-modal embeddings to coarse 'scenes', then softly combines the most relevant 'scene-primitives', plus a sparse injection scheme that gates which tokens get enhanced and propagates retrieved embeddings across layers. If the claim holds, VLM capacity could be scaled on edge devices by offloading knowledge to ROM and retrieving only relevant entries on demand, avoiding both the memory cost of dense models and the latency of loading experts. The empirical case is made on 11 visual benchmarks, where LookME consistently outperforms the base model and text-only PLE baselines, with the largest gains on fine-grained visual perception tasks.

Core claim

LookME's central claim is that continuous multimodal embeddings can be made lookup-addressable without discrete IDs, and that doing so is what PLE-style methods have been missing in VLMs. The paper argues that text-only lookup leaves the dominant visual pathway unenhanced, and proposes organizing a large external embedding table as a two-level structure of scenes and scene-primitives. Image embeddings are projected into the table's latent space, routed to the best scene by cosine similarity, and then combined with the top scene-primitives via soft weights—a design that avoids the routing imbalance and parameter under-utilization of a flat lookup. The same two-level lookup is applied to decod

What carries the argument

The central mechanism is the hierarchical two-level lookup table: an external embedding table reorganized as N_s scenes by N_p scene-primitives, replacing a flat |V|-way routing. Level-1 routes a projected continuous embedding to its nearest scene center via cosine similarity; Level-2 uses a shared MLP to score and softly combine the top-T primitives within that scene, so an arbitrary continuous embedding gets a relevant external representation without discrete IDs. Around this, LookME wraps three lookup pathways (text by ID, image by multi-scale two-level lookup, cross-modal by two-level lookup on hidden states), a Gumbel-Sigmoid per-token gate driven by the layer's attention contribution,

Load-bearing premise

The reported advantage rests on the assumption that all compared models were trained with the same data mixture, table-size budgets, and tuning hyperparameters; the paper defers those details to a supplementary document that is not present, so the gains could in principle come from the training recipe rather than from multimodal lookup.

What would settle it

Retrain the strongest text-only PLE baseline and LookME under strictly identical conditions—same 60M-sample corpus with disclosed sampling ratios, same total table parameter budget, same optimizer and compute—then compare on the 11 benchmarks; if the text-only baseline matches or beats LookME, the claim that multimodal lookup drives the improvement is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If LookME is correct, the PLE scaling paradigm extends from text to vision, letting VLMs grow knowledge capacity without proportional GPU memory growth.
  • Text-only PLE baselines underperform the base model on most benchmarks in this study, implying that multimodal lookup pathways—not just larger tables—are what a memory-augmented VLM needs.
  • Sparse injection via attention-conditioned gating and cross-layer propagation lets most layers benefit from one retrieval, so the efficiency penalty of lookup can be contained.
  • The largest gains appear on fine-grained perception and multi-image tasks, suggesting ROM-resident lookup memory is most valuable where evidence alignment and detail matter.
  • The ablation shows text, image, and cross-modal pathways each add independent performance, so future PLE designs should treat modality as a first-class axis of the lookup.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the scene-primitive factorization generalizes, the same two-level lookup could be applied to other continuous token streams—audio, video, or latent features—without redesign, since any continuous embedding can be routed to learned scene centers.
  • The attention-contribution gate (SKA) is a cheap, general heuristic: tokens that self-attention already updates most are the ones that benefit from external memory. This rule could be lifted out and tested in other memory-augmented architectures as a standalone selector.
  • Because all comparisons share one backbone and one self-curated 60M-sample corpus, the cleanest test of whether multimodal lookup itself causes the gains is to swap in a second base VLM and a fixed public data mixture; the paper does not report such a transfer experiment.
  • The single benchmark where a text-only PLE method edges out LookME (BLINK) is a multi-image reasoning set, which suggests cross-modal lookup alone may not fully cover compositional multi-image inference—an area where a dedicated inter-image pathway might be needed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces LookME, a framework that extends per-layer embedding (PLE) lookup from text-only operation to multimodal embeddings in vision-language models. It proposes a hierarchical two-level lookup: continuous image and cross-modal embeddings are first routed to a coarse 'scene' and then softly combined over 'scene-primitives', while text tokens retain ID-based lookup. A sparse injection mechanism—comprising an attention-contribution-based gate (SKA) and an inter-layer propagation path (NetProp)—selects which tokens receive external embeddings and allows non-LookME layers to reuse retrieved embeddings. Experiments on 11 benchmarks compare LookME with Qwen2.5-VL, a CPT baseline, and text-only PLE methods Engram and MeKi. The central claim is that LookME 'consistently outperforms baselines and text-only PLE methods across diverse visual benchmarks.'

Significance. The architectural idea is genuinely novel and timely: extending PLE-style lookup to continuous multimodal embeddings, with a scene-to-primitive factorization that avoids flat-table routing imbalance without increasing table size, is a plausible and useful direction for edge deployment of VLMs. The paper also provides useful ablations of lookup pathways, embedding dimension, injection layers, and NetProp range. If the empirical results survive matched-training scrutiny, the contribution would be publishable and practically relevant. However, the current manuscript does not yet establish that empirical claim: the fair-comparison setup is not verifiable from the text, the promised SKA ablation is absent, and a headline claim about multi-image gains is contradicted by the paper's own table.

major comments (4)
  1. [Experimental Setup] The fair-comparison assumption is unverifiable as stated. The paper says all models are pre-trained on the same 60M-sample unified dataset and that Engram/MeKi use 'the same hyperparameter settings as LookME', but the dataset composition, sampling ratios, training hyperparameters, and per-pathway table sizes are deferred to a supplementary that is not present in the submission. No error bars or repeated-seed results are reported. Therefore the reported +0.3–3.9 point gains over the strongest baseline could reflect data curation or tuning rather than the lookup mechanism. Please include the full training configuration for every method, data mixture weights, compute budgets, and at least three seeds per benchmark (or error bars), and specify table sizes and latency for each method.
  2. [Main Results / Table 1] The text repeatedly claims that LookME 'consistently achieves the best performance across all six task categories' and shows its 'largest gains' on multi-image evidence aggregation. Table 1 contradicts this: on BLINK, MeKi scores 49.0 while LookME scores 48.7, making LookME second-best in the Multi-Image category. Moreover, LookME's BLINK gain over the Qwen2.5-VL backbone is +0.1 (48.6→48.7), far smaller than the +2.8–3.9 gains on CV-Bench-3D and HRBench. Please correct these overstatements and rephrase the conclusions to match the data.
  3. [Effectiveness of Sparse Injection] The section states that 'we conduct separate ablations for each component' of sparse injection (SKA and NetProp), but only NetProp results are reported (Table 3). No ablation isolates the SKA intra-layer gate (e.g., gate vs. no gate; random gate vs. attention-contribution gate). Since SKA is a core contribution, the claim that sparse injection improves the efficiency/performance trade-off is not fully supported. Please add the missing SKA ablation and report results.
  4. [Eq. (10), Intra-Layer Sparse Activation] The SKA gate is described as a 'per-token, per-pathway binary gate' with 'distinct gate projections for multimodal pathways,' but Eq. (10) defines a single scalar gate g_i = σ(W_g Δ_i) with one W_g ∈ R^{d_h×1}. There is no pathway index or separate projection. Please clarify: if the gate is pathway-specific, update the equation; if it is shared, revise the text. This is important for reproducing the sparse injection behavior.
minor comments (6)
  1. [Eq. (7)] The notation 'argTop \tilde{T}' is not defined. It should be stated explicitly that this returns the indices of the top-\tilde{T} relevance scores, and the subsequent use of e_{S,p} should be clarified as the table embedding at those indices.
  2. [Eq. (9)] The symbol R_img is overloaded: it denotes both the per-scale retrieved embeddings (R_img,s) and the fused output on the left-hand side. Rename the per-scale outputs, e.g., to \tilde{R}_{img,s}, to avoid confusion.
  3. [Eq. (12)] The update R*_{(k)} = R*_{(k-1)} W_k^p + R*_{(k-1)} uses the same symbol on the right-hand side in two roles; please clarify whether the residual connection is intended and define W_k^p (dimensions, initialization, and whether it is a bottleneck).
  4. [Section 'Intra-Layer Sparse Activation'] Typo: 'controled' should be 'controlled'.
  5. [Table 1] The 'LLM Params (Backbone / External)' column does not specify whether 'External' counts table parameters, ROM storage, or active parameters. Please provide per-pathway table sizes and a breakdown of storage vs. active overhead for Engram, MeKi, and LookME.
  6. [Limitation] The limitation section is a single sentence about continual pre-training. Given the systems-oriented claims, the paper should report concrete latency, ROM, and memory measurements, and discuss failure cases where the lookup degrades performance (e.g., the BLINK result).

Circularity Check

0 steps flagged

No significant circularity: LookME's external tables are learned parameters; no benchmark result is forced by construction or by self-citation.

full rationale

The paper's central claim is that adding lookup-based multimodal embedding injection improves VLM benchmarks. The derivation chain is a standard trainable architecture: external tables E_text, E_img, E_cross are initialized by PCA from the pretrained backbone's embedding matrix (Methods, Multimodal Embedding Lookup), but are then updated during full fine-tuning and retrieved through learned routers, projections, gates, and fusion MLPs (Eqs. 4–12). No equation defines a reported benchmark number as a function of a fitted parameter that is then called a prediction. The ablation tables (Tables 2–5) are empirical comparisons of model variants, not analytic identities. The PCA initialization is an initialization choice, not a fitted estimate of the gains. The fair-comparison details (unified 60M corpus, hyperparameters) are deferred to a missing supplement, and BLINK shows MeKi 49.0 vs LookME 48.7, which undermines the 'consistently outperforms' phrasing; these are reproducibility and internal-consistency concerns, not circularity. There are no load-bearing self-citations, uniqueness theorems, or ansatz-smuggling via the authors' prior work. Therefore no circular step can be exhibited with the required quote-and-reduction standard.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 2 invented entities

The central claim is empirical, not a derivation, so circularity burden is low; however, the method rests on several untested modeling choices (PCA initialization, SKA proxy, NetProp) and unreported hyperparameters. The real knowledge added is architectural, and most capacity choices are tuned on the same reported benchmarks.

free parameters (7)
  • Embedding dimension d_e = 512
    Chosen by ablation on MMMU/MMBench/OCRB/HRBench (Table 4); best of 128/256/512.
  • Injection layers = [2/10/18]
    Chosen by ablation (Table 5) over three configurations using the same reported benchmarks.
  • Number of scenes N_s and primitives N_p = not reported
    Defines the external table size (0.82B params); values never stated, so capacity and retrieval budget are hidden choices.
  • Top-\tilde{T} primitives retrieved per token = not reported
    Controls retrieval cost and quality in Eq. 7; not specified.
  • NetProp range = Head-Tail
    Ablation Table 3 selects it as best; alternative ranges change scores by up to 2.6 points on HRBench.
  • Number of multi-scale image scales s = not reported
    Equations 4-9 assume s scales of visual encoder features; the paper never states s.
  • Scene router balancing bias b = learned/updated
    Per-scene bias adaptively updated from routing-frequency discrepancy (DeepSeek-V3 style), a hand-designed auxiliary balance mechanism.
axioms (5)
  • domain assumption Per-Layer Embedding with ROM-stored lookup tables is a working sparse-scaling paradigm for LLMs.
    Borrowed from Gemma 3n / MeKi / Engram; LookME's value depends on this premise being true.
  • ad hoc to paper PCA-initialized external tables from the pretrained backbone embedding matrix preserve a shared multimodal latent subspace and are a useful retrieval prior.
    Methods, Multimodal Embedding Lookup; no evidence given that PCA of the backbone is more informative than random initialization.
  • ad hoc to paper Attention contribution \Delta_i is a reliable per-token proxy for whether external embeddings will help.
    Eq. 10; the SKA gate keys solely on self-attention update magnitude, an unvalidated heuristic.
  • ad hoc to paper Adjacent decoder layers retrieve similar scene primitives, so propagating retrieved embeddings across non-LookME layers degrades little.
    Inter-Layer Sparse Activation; Table 3 shows Full propagation hurts on 2 of 4 benchmarks, so the assumption is only partially supported.
  • domain assumption Fine-tuning all models on the same 60M corpus yields a fair comparison.
    Experimental Setup; corpus composition and hyperparameters are not available, so fairness is unverified.
invented entities (2)
  • Scene (coarse lookup partition) no independent evidence
    purpose: Level-1 routing target for continuous image/cross-modal embeddings.
    Internal clustering structure of the external table; no external grounding, semantics are learned only during training.
  • Scene-primitive (fine lookup unit) no independent evidence
    purpose: Level-2 soft-combination target to synthesize retrieved embeddings.
    Internal units; their individual semantics are never evaluated or linked to external concepts.

pith-pipeline@v1.3.0-alltime-deepseek · 13711 in / 15330 out tokens · 148647 ms · 2026-08-02T06:28:49.363565+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) have achieved strong progress in multimodal understanding. However, scaling dense or sparse Mixture-of-Experts (MoE) models to improve performance limits deployment in resource-constrained environments due to the trade-off between high memory usage from full loading and increased latency from on-demand loading. Recently, the Per-Layer Embedding (PLE) architecture addresses this by scaling models with large external embedding tables stored in ROM and performing lightweight lookup to retrieve relevant embeddings to enhance token representations. Nevertheless, existing PLE-style methods are primarily designed for text embeddings due to the convenience of ID-based retrieval, limiting their effectiveness in VLMs where multimodal embeddings contain richer information for visual tasks. In this paper, we propose LookME, the first framework that enables lookup-based enhancement for multimodal embeddings in VLMs while supporting partitioned storage and on-demand loading. To efficiently lookup arbitrary continuous multimodal embeddings from large-scale embedding tables, we propose a hierarchical two-level lookup method employing a coarse-to-fine strategy that performs lookups from the scene-level to the intra-scene primitive-level. Furthermore, we integrate the lookup method with a sparse injection strategy, which adaptively prioritizes critical embeddings over voluminous multimodal embeddings within layers, and facilitates embedding table reuse across neighboring layers, improving the trade-off among efficiency, model size, and performance. Experiments on multiple visual benchmarks show that LookME outperforms text-only PLE-style methods, validating the effectiveness of lookup-based multimodal embedding enhancement.

Figures

Figures reproduced from arXiv: 2607.16305 by Fei Huang, Haoyu Chen, Menghua Zhai, Pengkai Guo, Siling Lin, Xiao Xu, Xingzhong Hou, Yunke Zhang, Zeyu Xu.

Figure 1
Figure 1. Figure 1: Comparison of architectures between different [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overall workflow of LookME. External embedding tables are separately stored in ROM from the main backbone. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 16 linked inside Pith

  1. [2]

    STEM: Scaling Transformers with Embedding Modules

    Sadhukhan, R.; Cao, S.; Dong, H.; Zhao, C.; Purpura- Pontoniere,A.;Tian,Y.;Liu,Z.;andChen,B.2026. STEM: Scaling Transformers with Embedding Modules. InPro- ceedings of the International Conference on Learning Rep- resentations (ICLR). Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q.; Hinton, G.; and Dean, J

    2026

  2. [4]

    DeepSeek-AI

    InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning.arXiv preprint arXiv:2305.06500. DeepSeek-AI

    Pith/arXiv arXiv

  3. [6]

    arXiv:2409.17146

    Molmo and PixMo: Open Weights and Open Data for State-of-the- Art Vision-Language Models. arXiv:2409.17146. Ding, N.; Liu, F.; Kim, K.; Hao, L.; Lee, K.-H.; Ko, H.; and Tang, Y

    Pith/arXiv arXiv

  4. [7]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N

    MeKi: Memory-based Expert Knowl- edge Injection for Efficient LLM Scaling.arXiv preprint arXiv:2602.03359. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N

    arXiv

  5. [9]

    arXiv:2410.18558

    Infinity-MM: Scaling Multimodal Perfor- mance with Large-Scale and High-Quality Instruction Data. arXiv:2410.18558. Guan, T.; Liu, F.; Wu, X.; Xian, R.; Li, Z.; Liu, X.; Wang, X.; Chen, L.; Huang, F.; Yacoob, Y.; Manocha, D.; and Zhou, T

    Pith/arXiv arXiv

  6. [11]

    Scaling Embeddings Outper- forms Scaling Experts in Language Models.arXiv preprint arXiv:2601.21204. Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan,Y.;Wang,J.;He,C.;Liu,Z.;etal.2024a.Mmbench:Is your multi-modal model an all-around player? InEuropean conference on computer vision, 216–233. Springer. Liu, Y.; Li, Z.; Huang, M.; Yang, B.; Yu...

    arXiv

  7. [12]

    Mathew,M.;Bagal,V.;Tito,R.P.;Karatzas,D.;Valveny,E.; andJawahar,C.V.2021

    DeepSeek-VL: Towards Real-WorldVision-LanguageUnderstanding.arXiv preprint arXiv:2403.05525. Mathew,M.;Bagal,V.;Tito,R.P.;Karatzas,D.;Valveny,E.; andJawahar,C.V.2021. InfographicVQA. VersionNumber:

    Pith/arXiv arXiv 2021

  8. [15]

    arXiv:2406.16860

    Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs. arXiv:2406.16860. Wang, K.; Pan, J.; Shi, W.; Lu, Z.; Ren, H.; Zhou, A.; Zhan, M.; and Li, H. 2024a. Measuring Multimodal Mathemati- cal Reasoning with MATH-Vision Dataset. InThe Thirty- eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Wang, W....

    Pith/arXiv arXiv

  9. [16]

    Wiedmann,L.;Zohar,O.;Mahla,A.;Wang,X.;Li,R.;Frere, T.;vonWerra,L.;Gosthipaty,A.R.;andMarafioti,A.2025

    CogVLM: Vi- sualExpert forPretrained LanguageModels.arXiv preprint arXiv:2311.03079. Wiedmann,L.;Zohar,O.;Mahla,A.;Wang,X.;Li,R.;Frere, T.;vonWerra,L.;Gosthipaty,A.R.;andMarafioti,A.2025. FineVision: Open Data Is All You Need. arXiv:2510.17269. Wu,Z.;Chen,X.;Pan,Z.;Liu,X.;Liu,W.;Dai,D.;Gao,H.; Ma, Y.; Wu, C.; Wang, B.; Xie, Z.; Wu, Y.; Hu, K.; Wang, J.; S...

    Pith/arXiv arXiv 2025

  10. [17]

    arXiv preprint arXiv:2412.10302

    DeepSeek-VL2: Mixture-of-Experts Vision- LanguageModelsforAdvancedMultimodalUnderstanding. arXiv preprint arXiv:2412.10302. Yuan, L.; Wang, J.; Sun, H.; Zhang, Y.; and Lin, Y

    Pith/arXiv arXiv

  11. [18]

    arXiv:2501.07888

    Tarsier2: Advancing Large Vision-Language Models from DetailedVideoDescriptiontoComprehensiveVideoUnder- standing. arXiv:2501.07888. Yue, X.; Ni, Y.; Zhang, K.; Zheng, T.; Liu, R.; Zhang, G.; Stevens, S.; Jiang, D.; Ren, W.; Sun, Y.; Wei, C.; Yu, B.; Yuan, R.; Sun, R.; Yin, M.; Zheng, B.; Yang, Z.; Liu, Y.; Huang, W.; Sun, H.; Su, Y.; and Chen, W

    Pith/arXiv arXiv

  12. [19]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    GLIPv2:UnifiedVision-LanguagePretrainingforLanguage Grounding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Zhang, X.; Li, D.; Liu, B.; Bao, Z.; Zhou, Y.; Yang, B.; Liu, Z.; Zhong, Y.; and Yuan, T. 2025a. Layer-wise Vision Injection with Disentangled Attention for Efficient LVLMs. InProceedings of the IEEE/CV...

    Pith/arXiv arXiv

  13. [2017]

    In Proceedings of the International Conference on Learning Representations (ICLR)

    Outrageously Large Neural Networks:TheSparsely-GatedMixture-of-ExpertsLayer. In Proceedings of the International Conference on Learning Representations (ICLR). Team, V.; Hong, W.; Yu, W.; Gu, X.; Wang, G.; Gan, G.; Tang, H.; Cheng, J.; Qi, J.; Ji, J.; Pan, L.; Duan, S.; Wang, W.; Wang, Y.; Cheng, Y.; He, Z.; Su, Z.; Yang, Z.; Pan, Z.; Zeng, A.; Wang, B.; ...

    Pith/arXiv arXiv 2025

  14. [2020]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S

    Scaling Laws for Neural Language Mod- els.arXiv preprint arXiv:2001.08361. Li, J.; Li, D.; Savarese, S.; and Hoi, S

    Pith/arXiv arXiv 2001

  15. [2022]

    Fu, X.; Hu, Y.; Li, B.; Feng, Y.; Wang, H.; Lin, X.; Roth, D.;Smith,N.A.;Ma,W.-C.;andKrishna,R.2024

    Switch Trans- formers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.Journal of Machine Learning Re- search (JMLR), 23(120): 1–39. Fu, X.; Hu, Y.; Li, B.; Feng, Y.; Wang, H.; Lin, X.; Roth, D.;Smith,N.A.;Ma,W.-C.;andKrishna,R.2024. BLINK: Multimodal Large Language Models Can See but Not Per- ceive.arXiv preprint arXiv:2404.12390....

    Pith/arXiv arXiv 2024

  16. [2023]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, TextReading,andBeyond.arXiv preprint arXiv:2308.12966. Bai,S.;Chen,K.;Liu,X.;Wang,J.;Ge,W.;Song,S.;Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.;...

    Pith/arXiv arXiv

  17. [2024]

    arXiv:2412.19437

    DeepSeek-V3 Technical Report. arXiv:2412.19437. Deitke, M.; Clark, C.; Lee, S.; Tripathi, R.; Yang, Y.; Park, J. S.; Salehi, M.; Muennighoff, N.; Lo, K.; Soldaini, L.; Lu, J.; Anderson, T.; Bransom, E.; Ehsani, K.; Ngo, H.; Chen, Y.; Patel, A.; Yatskar, M.; Callison-Burch, C.; Head, A.; Hendrix, R.; Bastani, F.; VanderBilt, E.; Lambert, N.; Chou, Y.; Chhe...

    Pith/arXiv arXiv

  18. [2025]

    Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923. Chen, G. H.; Chen, S.; Zhang, R.; Chen, J.; Wu, X.; Zhang, Z.;Chen,Z.;Li,J.;Wan,X.;andWang,B.2024a. ALLaVA: Harnessing GPT4V-synthesized Data for A Lite Vision- Language Model. arXiv:2402.11684. Chen, L.; Li, J.; Dong, X.; Zhang, P.; Zang, Y.; Chen, Z.; Duan, H.; Wang, J.; Qiao, Y.; Lin, D.; et ...

    Pith/arXiv arXiv 2023

  19. [2026]

    Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models.arXiv preprint arXiv:2601.07372. Dai,D.;Deng,C.;Zhao,C.;Xu,R.X.;Gao,H.;Chen,D.;Li, J.;Zeng,W.;Yu,X.;Wu,Y.;Xie,Z.;Li,Y.K.;Huang,J.;Luo, F.;Ruan,C.;Sui,Z.;andLiang,W.2024.DeepSeekMoE:To- wardsUltimateExpertSpecializationinMixture-of-Experts Language Models.arXiv preprin...

    Pith/arXiv arXiv 2024