{"id":"a3711228-0b4f-482c-a073-215979e76348","arxiv_id":"2505.05237","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Latte transfers LLM latent-state knowledge via a knowledge adapter and unsupervised meta-learning, claiming SOTA few-shot tabular performance, though its own evaluation contradicts that claim on Diabetes.","lead":"Latte is a training-time framework that distills hidden-state knowledge from a large language model into a small tabular model for few-shot prediction, using one LLM call per task. The paper reports strong gains across nine tabular benchmarks, but the claim of consistent state-of-the-art results is contradicted by its own table on the Diabetes dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 contradicts the headline: Latte loses to FeatLLM on Diabetes at every shot and at Blood 8-shot, so 'exceeds FeatLLM in all shot settings' is false.","rationale":"I agree with the rejection, but the decisive problem is not the reader's stated weakest assumption about whether the pooled LLM hidden state is informative. The stronger, more direct problem is that the paper's own Table 2 contradicts its central empirical claim: FeatLLM beats Latte on Diabetes at every shot setting, and at Blood shot 8. Six of 30 dataset-shot pairs are losses, so 'exceeds FeatLLM in all shot settings' is false as stated. The 4.22% average improvement also does not emerge from any obvious aggregation of Table 2; my recomputation over the 30 cells gives about 3.5 percentage points. Secondary verification problems reinforce this: the regression results are deferred to a missing Appendix B, dataset details to a missing Appendix A, Table 3 is garbled, and hyperparameters eta, tau, k, and L are under-specified. There is also a formal ambiguity in Eq. 6, where a KL divergence (a scalar) is assigned to qLLM and then used as a query vector in Eq. 7; this needs correction or code clarification. None of these concerns is an ad hominem; they are internal inconsistencies in the submitted evidence. Because the falsifying comparison is in the paper's own table, I would keep the reader's REJECT verdict unchanged. A rerun with the released code on Diabetes would settle whether the table contains a transcription error, but as submitted the manuscript does not support its headline.","tokens_in":14163,"tokens_out":9537,"duration_ms":90573,"concrete_test":"Run the released Latte code on Diabetes at shots 4/8/16/32/64 with three seeds and compare against the FeatLLM column in Table 2 (80.28/79.38/80.15/80.06/80.91). If Latte's reproduced AUCs remain in the 72-78 range, or otherwise fail to exceed FeatLLM on at least one shot, the Section 4.2 claim is false. Also recompute the average improvement over all 30 Table 2 cells; the 4.22% figure should be re-derived or corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 4.2 ('consistently outperforms all competing methods' and 'exceeds FeatLLM in all shot settings') is contradicted by the paper's own Table 2. Across the six classification datasets and five shot sizes there are 30 Latte-vs-FeatLLM comparisons, and Latte loses six of them. Diabetes is a clean counterexample: at shots 4/8/16/32/64, FeatLLM scores 80.28/79.38/80.15/80.06/80.91 while Latte scores 72.06/73.70/76.78/77.01/78.32, a gap of 2.6-8.2 AUC points in FeatLLM's favor. Blood at shot 8 is also a loss (69.97 vs 70.37). Therefore the 'all shot settings' claim is factually false, and the 'significant margin' conclusion does not follow. The reported 4.22% average improvement is also not reproducible by straightforward aggregation of Table 2: averaging Latte minus FeatLLM over the 30 cells gives roughly 3.5 percentage points, not 4.22. Because these numbers are the paper's only support for its state-of-the-art conclusion, the headline result fails on internal evidence, independent of any debate about the method's assumptions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Latte proposes a training-time framework for few-shot tabular learning. For a given tabular task, it feeds the dataset metadata (task and feature descriptions) into an LLM, average-pools the last-layer hidden states to obtain a 'latent knowledge' vector, and uses this vector to guide a downstream transformer-based tabular encoder through a knowledge adapter and KL-style losses. The method also includes an unsupervised meta-learning stage that clusters unlabeled data to generate pseudo-labels. Experiments are reported on nine datasets (six classification, three regression) against ten baselines, with the headline claim that Latte consistently outperforms all competing methods and exceeds FeatLLM at every shot setting.","tokens_in":14468,"tokens_out":8532,"duration_ms":75174,"significance":"If the results were as claimed, the paper would make a useful contribution: it avoids per-sample LLM calls at inference (one LLM call per task), combines LLM prior knowledge with unlabeled-data semi-supervision, and handles regression without architectural modification. The authors also provide code. However, the central empirical claim is contradicted by the paper's own Table 2, so the significance as stated is not established.","major_comments":[{"comment":"The sentence 'our method exceeds the performance of the state-of-the-art method FeatLLM in all shot settings' is contradicted by Table 2. On Diabetes, FeatLLM outperforms Latte at all five shot counts (80.28 vs 72.06 at 4 shots; 79.38 vs 73.70 at 8; 80.15 vs 76.78 at 16; 80.06 vs 77.01 at 32; 80.91 vs 78.32 at 64), and on Blood at 8 shots FeatLLM scores 70.37 vs Latte's 69.97. These are at least six of the thirty Latte-versus-FeatLLM comparisons in the classification table, so 'consistently outperforms all competing methods' is false as stated. The reported average improvement of 4.22% is not reproducible by straightforward aggregation of the classification results shown in Table 2. Because the abstract, Section 1, and Section 5 all rely on this claim, the paper's main conclusion is unsupported.","section":"§4.2, Table 2"},{"comment":"Equation (6) defines qLLM = KL(W0 hM/τ, q/τ), and Eq. (11) uses the same expression LKL = KL(W0 hM/τ, q/τ) as a scalar loss. This is not a well-defined use of KL divergence: KL divergence is defined between probability distributions, whereas W0 hM and q are raw real-valued vectors and no softmax or distributional normalization is specified. As written, Eq. (6) cannot produce a vector qLLM under the standard definition of KL, and Eq. (11) cannot be computed. The authors should specify the actual objective (for example, KL after converting both vectors to distributions, or an MSE/cosine alignment loss) and use consistent notation. Since this loss is the mechanism by which LLM knowledge is transferred, the current formulation is not reproducible.","section":"§3.4, Eqs. (6) and (11)"},{"comment":"The ablation study is conducted on only the Heart dataset, so it does not support the general claim that each component is crucial across datasets and task types. In addition, the table reports no significance tests, and several compared configurations are within one standard deviation of the full model (for example, at 4 shots the full model is 86.10±5.42 vs 85.16±4.44 for the configuration without the LLM-knowledge and meta components). The text's assertion that 'in all cases, modifying any of the ablated components leads to a decline in performance' is therefore stronger than the evidence provided.","section":"§4.3, Table 3"},{"comment":"The pre-training stage depends on clustering unlabeled data with k centroids (Eq. 10), and the text then says the procedure 'randomly select k samples from each cluster to create an N-way K-shot meta-training task.' The relationship between k, N, and K is never specified, and k appears both as the number of clusters and as the number of samples selected per cluster. Without this detail the unsupervised meta-learning procedure is not reproducible, and the dependence of the method on the choice of k is not analyzed.","section":"§3.5, Eq. (10)"}],"minor_comments":[{"comment":"The title contains a typo: 'Transfering' should be 'Transferring'.","section":"Title"},{"comment":"The dataset name is misspelled as 'Boold' in the header; it should be 'Blood'.","section":"Table 2"},{"comment":"The word 'Totle' in the table footer should be 'Total'.","section":"Table 4"},{"comment":"References [Han et al., 2024a] and [Han et al., 2024b] point to the same arXiv paper and should be consolidated.","section":"References"},{"comment":"Section 3.3 says the knowledge vector is obtained from 'the last transformer layer,' but the implementation details say 'The activation vector in the LLMs is obtained from the 30 layers.' If LLaMA2-7B has 32 layers, these statements conflict and should be reconciled.","section":"§3.3 vs §4.1"},{"comment":"The sentence 'we evaluate the proposed model against with baseline' is ungrammatical; it should read 'against baseline methods' or similar.","section":"§4.1"}],"recommendation":"reject","confidential_remarks":"The central issue is the direct contradiction between the text and Table 2. This is not a presentation issue: the abstract, introduction, and conclusion all assert a state-of-the-art result that the paper's own numbers disprove. If the authors have the raw results, rerunning the comparisons and rewriting the claims to match the evidence might be feasible, but in its current form the manuscript's main conclusion is not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"X,\n\nQuick take: the paper's actual contribution is a training-time LLM latent-knowledge transfer framework for few-shot tabular learning, and that part is coherent and interesting. The evaluation, however, contradicts the paper's own headline claim.\n\nWhat's genuinely new is the combination of one-shot-per-task LLM hidden-state extraction with a semantic tabular encoder and a knowledge adapter, plus meta-learning on pseudo-labeled unlabeled data. That design directly targets two real problems in prior work: test-time LLM inference cost and unreliable text-level features. Table 4 makes the efficiency case nicely: one LLM call per task, and inference is unaffected. That is a useful contribution even if the performance numbers were weaker.\n\nThe problems start with Section 4.2. The paper says Latte 'exceeds FeatLLM in all shot settings' and cites a 4.22% average improvement. Table 2 tells a different story: FeatLLM beats Latte on Diabetes at every shot (e.g., 80.28 vs 72.06 at 4 shots) and on Blood at 8 shots (70.37 vs 69.97). That is six outright losses out of 30 comparisons, and the claimed 4.22% does not reproduce from the table—I get roughly 3.5 points. This is not a minor quibble; it is the central empirical claim failing on the paper's own data.\n\nThe ablation table (Table 3) is garbled: the rows do not clearly mark which components are included, and the reader cannot tell what '✓ ✓ 13B' means. The regression results are deferred to a missing appendix. More concerning, Eq. (6) defines qLLM as the KL divergence between two vectors, but a KL divergence is a scalar, not a vector. That cannot be used as an attention query. The intended operation is probably something like a softmax or a projection, but as written it is formally undefined. The paper also leaves eta, tau, k, and the noise m under-specified.\n\nOn the citation pattern: the related work is reasonably covered, and self-citation is not an issue here.\n\nNet assessment: the framework is plausible and worth discussing, but as it stands the paper does not support its SOTA claims. I would not cite it as evidence of performance. It deserves a serious referee only if the authors first fix the evaluation and the formal description. Right now I would treat the results as unverified.\n\nRegards.","headline":"The core training-time LLM-knowledge-transfer idea is sound, but the paper's own Table 2 contradicts its SOTA claim, so the results should be treated as unverified.","tokens_in":14987,"tokens_out":3882,"would_cite":false,"duration_ms":37228,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Latte transfers an LLM's pooled hidden-state vector into a small tabular model through a KL-aligned attention query and reports consistent few-shot gains.","keywords":["few-shot tabular learning","latent-level knowledge transfer","LLM knowledge distillation","knowledge adapter","semantic tabular encoder","unsupervised meta-learning","tabular classification","tabular regression"],"falsifier":"Scramble or replace the metadata prompt with an irrelevant one and rerun Latte on the same datasets; if AUC and MSE do not clearly drop, the claimed transfer from the LLM is not happening. A sharper check is to train a linear probe on the pooled hidden-state vector to predict the target label: chance-level probe accuracy would show the vector carries no task signal.","tokens_in":13950,"feed_emoji":"📊","tokens_out":8778,"duration_ms":79704,"temperature":0.7,"pith_summary":"The paper introduces Latte, a training-time framework for few-shot tabular learning that treats a large language model as a teacher rather than as a test-time predictor. Its claim is that a single average-pooled vector of the LLM's last-layer hidden states, computed from a text prompt describing the task and its features, carries task-relevant prior knowledge that can be distilled into a small downstream model. That distillation works through a knowledge adapter that uses a KL divergence loss to reshape a query vector, which then attends over semantic feature embeddings to produce a task-aware row representation. The paper reports that this consistently beats prior text-rule feature engineering, with an average 4.22% AUC improvement over FeatLLM across shot settings, while calling the LLM only once per dataset. A reader should care because it points to a cheap way to use LLM priors: harvest knowledge once, offline, and run a small model at inference.","feed_headline":"Latte distills LLM latent knowledge into few-shot tabular models.","feed_subtitle":"One LLM call per dataset replaces per-sample inference, beating text-rule feature engineering by 4.22 percent.","key_machinery":"The load-bearing object is the latent knowledge vector $h_{\\mathcal{M}}$, obtained by averaging the last-layer hidden states of an LLM over a metadata prompt (Eq. 4). This vector is distilled into a global query $q$ via a KL divergence loss (Eq. 6) inside a knowledge adapter, and the resulting query $q_{\\text{LLM}}$ attends over feature embeddings produced by a semantic-aware tabular encoder. The encoder itself encodes each feature value using BERT: categorical values as pooled encodings of feature-name-plus-value text, numerical values as the feature-name embedding multiplied by the scalar value. The mechanism's job is to let the LLM's prior reweight which feature values matter, while a constant $\\eta$ blends the LLM-guided representation with a general [CLS] representation. All of this is trained first on pseudo-labeled clusters from unlabeled rows, then on the few labeled examples.","core_discovery":"Latte's central discovery is that the hidden states of an LLM, hooked at the last transformer layer and pooled over a metadata-only prompt, can act as a latent prior for a downstream tabular model. The paper claims that, unlike text-level rules generated autoregressively, this latent vector is more informative and less prone to hallucination, and it can be transferred by aligning the LLM's vector with a query vector produced by a GTransformer, then using attention over semantically encoded feature values. Combined with an unsupervised meta-learning stage that clusters unlabeled rows into pseudo-labeled N-way K-shot tasks, this enables both classification and regression from very few labeled samples. The paper reports consistent gains over ten baselines, including a 4.22% average improvement over the strongest text-engineering baseline, and shows the learned representations separate classes even with four labeled samples.","pith_inferences":["Editorial inference: the claim that a single pooled vector suffices could be tested by ablating the pooling step, replacing average-pooling with attention pooling or a [CLS]-style token to see whether the precise aggregation matters or any summarization works.","Editorial inference: because the prompt contains only metadata, the same recipe could be applied to wide or high-cardinality tables where serializing entire rows exceeds the LLM context window.","Editorial inference: the constant $\\eta$ that blends LLM-guided and general representations is fixed; a task-adaptive or learned $\\eta$ might be needed for domains where the LLM's priors are weak or outdated, and this is a natural extension the paper does not explore.","Editorial inference: probing the pooled hidden-state vector with a linear classifier before training would measure how much task signal it carries independently of downstream accuracy."],"forward_implications":["Because the LLM is invoked once at preprocessing time rather than once per test row, deployment latency and inference cost for Latte are independent of the number of test samples.","The same pipeline handles regression without modification, whereas several competing few-shot methods are classification-only.","Using unlabeled rows through clustering pseudo-labels extends the effective supervision beyond the labeled set, which matters when only a handful of labels exist.","The reported 4.22% average AUC gain over the best text-level method implies that latent-vector distillation can substitute for generated textual rules in feature engineering."],"supporting_citations":[{"why":"Defines FeatLLM, the text-level feature-engineering baseline that Latte targets and reports beating by 4.22% on average.","marker":"[Han et al., 2024a]"},{"why":"Supplies STUNT's self-generated tasks from unlabeled tables, which grounds Latte's meta-learning stage and the non-semantic encoder used in ablation.","marker":"[Nam et al., 2023]"},{"why":"Cited as evidence that latent-level hidden states are more informative and discriminative than text-level outputs.","marker":"[Zhou et al., 2024]"},{"why":"Cited for the observation that LLM hidden states can be used to detect hallucination, motivating the choice of latent over textual knowledge.","marker":"[Duan et al., 2024]"},{"why":"Describes the source model whose layer-30 hidden states are pooled to produce the latent knowledge vector.","marker":"[Touvron et al., 2023]"},{"why":"TabLLM is the parameter-efficient LLM fine-tuning baseline that Latte compares against.","marker":"[Hegselmann et al., 2023]"},{"why":"SCARF is the contrastive self-supervised baseline on unlabeled data that Latte also compares with.","marker":"[Bahri et al., 2021]"}],"fun_headline_variants":["Latte: one LLM call per dataset for few-shot tabular","LLM latent priors beat text rules in few-shot tabular learning","LLM hidden states lift few-shot tabular accuracy by 4.22%","Latte: LLM latent knowledge on tap for few-shot tables","Latte: training-time LLM extraction for few-shot tabular models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that one summary vector from the LLM's final layer, computed from a text description of the task and its features, contains useful task knowledge that a learning objective can press into a small model; if that vector is mostly prompt-formatting noise, the whole transfer step adds nothing.","fun_headline_variants_meta":{"raw":{"variants":["Latte: one LLM call per dataset for few-shot tabular","LLM latent priors beat text rules in few-shot tabular learning","LLM hidden states lift few-shot tabular accuracy by 4.22%","Latte: LLM latent knowledge on tap for few-shot tables","Latte: training-time LLM extraction for few-shot tabular models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001146,"raw_usage":{"total_tokens":4736,"prompt_tokens":911,"completion_tokens":3825,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":3728}},"tokens_in":527,"tokens_out":3825,"duration_ms":27643,"temperature":1.0,"reasoning_tokens":3728,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:09:00.268458+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Scramble or replace the metadata prompt with an irrelevant one and rerun Latte on the same datasets; if AUC and MSE do not clearly drop, the claimed transfer from the LLM is not happening. A sharper check is to train a linear probe on the pooled hidden-state vector to predict the target label: chance-level probe accuracy would show the vector carries no task signal.","supporting_citations":[{"cited_title":"Stunt: Few-shot tab- ular learning with self-generated tasks from unlabeled ta- bles","cited_arxiv_id":null,"evidence_quote":"Supplies STUNT's self-generated tasks from unlabeled tables, which grounds Latte's meta-learning stage and the non-semantic encoder used in ablation."},{"cited_title":"Tabllm: Few-shot classification of tab- ular data with large language models","cited_arxiv_id":null,"evidence_quote":"TabLLM is the parameter-efficient LLM fine-tuning baseline that Latte compares against."}],"review_version":1}