{"total":28,"items":[{"citing_arxiv_id":"2606.27941","ref_index":4,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"VASAE: Naming SAE Dictionary Directions with Vocabulary-Aligned Anchoring","primary_cat":"cs.CL","submitted_at":"2026-06-26T10:30:56+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"VASAE introduces vocabulary-aligned anchoring to train SAEs that yield features with intrinsic token names, reporting high alignment rates in early layers of GPT-2 and Llama-3.1 without reconstruction loss.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.27237","ref_index":60,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"LMs as Task-Specific Knowledge Bases: An Interpretability Analysis","primary_cat":"cs.CL","submitted_at":"2026-06-25T16:22:43+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LMs store facts in task-specific parameter subsets, shown by inconsistent emergence across tasks during training and distinct localized parameters for the same fact.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.21622","ref_index":49,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment","primary_cat":"cs.CL","submitted_at":"2026-06-19T17:23:23+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"Document-tuned transformers outperform base transformers by 13.4% Pearson r on person-level mental health prediction across two datasets and remain more accurate under text perturbations.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.19719","ref_index":33,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Closing the Calibration Gap in Semantic Caching","primary_cat":"cs.IR","submitted_at":"2026-06-18T02:34:33+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Introduces P-CHR AUC and CRR metrics to demonstrate that semantic caching model selection is limited by calibration quality rather than ranking performance.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.17468","ref_index":14,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"RSRank: Learning Relevance from Representational Shifts","primary_cat":"cs.IR","submitted_at":"2026-06-16T03:29:23+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"RSRank learns calibrated relevance scores from alignment between representational shifts induced by candidate documents and those from oracle document sets, enabling zero-threshold filtering.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.08810","ref_index":16,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Continuous Language Diffusion as a Decoder-Interface Problem","primary_cat":"cs.CL","submitted_at":"2026-06-07T20:00:33+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Continuous language diffusion works by entering high-margin decoder basins where frozen T5 embeddings recover 93-96% of native decisions and linear readouts reach 97.9% agreement, implying models should be evaluated as representation-decoder systems.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.08562","ref_index":6,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Inside the LLM Word Factory","primary_cat":"cs.CL","submitted_at":"2026-06-07T10:36:20+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Activation patching localizes English detokenization in Llama2-7B to a two-stage attention-then-MLP process at layer 1 that generalizes to 12 models from 8 families, with depth varying by positional encoding, plus an early-layer probe achieving 0.94-0.97 AUROC.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.08394","ref_index":3,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"When Correct Decisions Hide Internal Stress: Decision-State Probing in Multimodal Language Models","primary_cat":"cs.CL","submitted_at":"2026-06-07T01:11:36+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"S³E framework finds excess decision-state displacement under semantic stress in multimodal models despite consistent correct forced-choice behavior.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.07502","ref_index":10,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings","primary_cat":"cs.CL","submitted_at":"2026-06-05T17:54:32+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"EmbedFilter applies a linear filter derived from the LLM unembedding matrix to suppress high-frequency token influences in text embeddings, yielding improved zero-shot performance and inherent dimensionality reduction.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.03291","ref_index":11,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibility","primary_cat":"cs.CL","submitted_at":"2026-06-02T07:55:52+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Unlearning in multilingual LLMs suppresses rather than erases knowledge in later layers, with transfer varying by language similarity and reversible via inference-time steering.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.00909","ref_index":8,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"MLLM-Microscope: Unlocking Hidden Structure Within Multimodal Large Language Models","primary_cat":"cs.CL","submitted_at":"2026-05-30T22:20:50+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"MLLM-Microscope measures linearity, dimension and anisotropy of multimodal token streams in LLaVA-NeXT and OmniFusion, reporting high linearity overall and model-specific differences tied to modality fusion.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.30729","ref_index":6,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"SemStruct: Contextualizing Semantic Embeddings with Structural Information for Schema Matching","primary_cat":"cs.LG","submitted_at":"2026-05-29T01:45:45+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"SemStruct models tables as heterogeneous graphs with GNNs on frozen PLM embeddings to incorporate row co-occurrences for schema matching and reports SOTA results on Valentine and SOTAB-SM benchmarks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.29987","ref_index":1,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment","primary_cat":"cs.LG","submitted_at":"2026-05-28T14:22:25+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":3.0,"formal_verification":"none","one_line_summary":"MIC introduces Soft Collapse Regularization and Spectral Isotropy Regularization unified via self-distillation to maximize informational capacity in adaptive multi-granular embeddings.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.22223","ref_index":91,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"How Many Different Outputs Can a Transformer Generate?","primary_cat":"cs.LG","submitted_at":"2026-05-21T09:26:37+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Transformers are limited to a linearly growing number of accessible output sequences with prompt length, with exponential decay in accessible proportion beyond a critical point, even under unbounded context.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.08048","ref_index":35,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Accurate and Efficient Statistical Testing for Word Semantic Breadth","primary_cat":"cs.CL","submitted_at":"2026-05-08T17:38:36+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"A new permutation test uses Householder reflection to align word embedding clouds before testing dispersion differences, cutting Type-I error by 32.5% and speeding up 23x on GPU.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"tioning it as a corpus-driven proxy for meaning that can be compared across model sizes and architec- tures (Nagata and Tanaka-Ishii, 2025). At the same time, contextual embeddings exhibit non-trivial ge- ometry; for example, anisotropy and layer-wise dif- ferences complicate the interpretation of distances and norms, potentially confounding naive disper- sion comparisons (Ethayarajh, 2019). Related work also explores polysemy quantification from con- textual embedding geometry (Xypolopoulos et al., 2021), though these approaches often emphasize clustering or multi-sense structure rather than cali- brated hypothesis testing. Recent analyses further study relationships between the norm of mean con- textualized embeddings and variance (Yamagiwa"},{"citing_arxiv_id":"2605.05683","ref_index":37,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization","primary_cat":"stat.ML","submitted_at":"2026-05-07T05:19:56+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Spectral analysis of activations and gradients provides new diagnostics that link batch size to representation geometry, early covariance tails to token efficiency, and spectral shifts to learning dynamics in decoder-only LLMs, backed by a mechanistic model.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"Evolution of the spectral dimension of transformer activations. InOPT 2025: Optimization for Machine Learning, 2025. URL https://openreview.net/forum?id=Va5is76bTP. [36] Utkarsh Sharma and Jared Kaplan. Scaling laws from the data manifold dimension.Journal of Machine Learning Research, 23(9):1-34, 2022. URLhttps://www.jmlr.org/papers/v23/20-1 111.html. [37] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. InAdvances in Neural Information Processing Systems, 2007. URLhttps://papers.nips.cc/paper_fil es/paper/2007/hash/013a006f03dbc5392effeb8f18fda755-Abstract.html. [38] Zhichao Wang, Denny Wu, and Zhou Fan. Nonlinear spiked covariance matrices and signal propagation in deep neural networks."},{"citing_arxiv_id":"2605.01073","ref_index":17,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing","primary_cat":"cs.CL","submitted_at":"2026-05-01T20:12:06+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Nonlinear polynomial models fit local paraphrase embedding clouds more accurately than linear ones and support geometrically consistent synthetic point generation, yet this geometric fidelity does not improve classification performance.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"[16] demonstrated that representations obtained from pretrained language models may have an unfavorable anisotropic geometry, and proposed BERT-flow to transform their distribution into a smoother isotropic space. These works show that the geometry of representation space is not a secondary artifact, since it affects semantic similarity and downstream behavior. Closer to our setting is the work of Chu et al. [17], which proposes Refined SBERT, a method for redescribing Sentence-BERT representations in a manifold space while preserving local neighborhood structure. Even closer to the revised focus of the present paper is the work of Tehenan [18], where the problem is formulated directly as the semantic geometry of sentence embeddings. Our work continues this line while differing in two respects: we study controlled"},{"citing_arxiv_id":"2605.00618","ref_index":22,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Is Textual Similarity Invariant under Machine Translation? Evidence Based on the Political Manifesto Corpus","primary_cat":"cs.CL","submitted_at":"2026-05-01T12:41:00+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Machine translation preserves embedding similarity structure for ten languages but distorts it for four in the Manifesto Corpus, via a new non-inferiority testing framework.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"18653/v1/D19-1006. [21] F. Feng, Y . Yang, D. Cer, N. Arivazhagan, and W. Wang. Language- Agnostic BERT Sentence Embedding. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878-891, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.62. [22] L. Fu and L. Liu. What Are the Differences? A Comparative Study of Generative Artificial Intelligence Translation and Human Translation of Scientific Texts.Humanities and Social Sciences Communications, 11(1): 1-12, Sept. 2024. ISSN 2662-9992. doi: 10.1057/s41599-024-03726-7. [23] W. A. Gale and K. W. Church. A Program for Aligning Sentences in Bilingual Corpora."},{"citing_arxiv_id":"2604.20835","ref_index":29,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL","primary_cat":"cs.CL","submitted_at":"2026-04-22T17:58:36+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Parallel-SFT mixes parallel programs across languages during SFT to produce more transferable RL initializations, yielding better zero-shot generalization to unseen programming languages.","context_count":1,"top_context_role":"background","top_context_polarity":"support","context_text":"We compute the program representations via echo embedding (Springer et al., 2025) that repeats the program twice in a lightweight template which empirically produces strong sequence representations.16 Representation similarity is most straightforwardly computable by cosine similarity. It, however, can be misleading due to theanisotropyof the representation space (Ethayarajh, 2019). That is, it is possible that a model simply utilizes a small subset of its representation space for code, embeddingallprograms closer together, not just parallel programs. We account for this in two robust metrics: 16The intuition is that autoregressive models do not attend to the future, but future information is helpful for a good representation."},{"citing_arxiv_id":"2604.16576","ref_index":17,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"On the Robustness of LLM-Based Dense Retrievers: A Systematic Analysis of Generalizability and Stability","primary_cat":"cs.IR","submitted_at":"2026-04-17T13:02:29+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"LLM-based dense retrievers generalize better when instruction-tuned but pay a specialization tax when optimized for reasoning; they resist typos and corpus poisoning better than encoder-only baselines yet remain vulnerable to semantic perturbations, with larger models and certain embedding geometry,","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"studies centered specifically on modern LLM-based dense retrievers, including recent work on robustness scaling [44], remain limited in scope. This leaves a clear need for a systematic robustness analysis of contemporary LLM-based retrievers within a broader retrieval robustness framework. 2.3 Embedding Geometry and Isotropy The geometry of learned embedding spaces has long been studied as an indicator of representation quality. Ethayarajh [17] showed that contextual representations from pre-trained language models often occupy a narrow cone in high- dimensional space, a phenomenon known asanisotropy, which can reduce discriminability. Subsequent work proposed several geometric diagnostics. In particular, prior studies have characterized isotropy using average pairwise cosine similarity and principal-component-based analyses [ 7, 24], while Rudman and Eickhoff [54], Rudman et al ."},{"citing_arxiv_id":"2604.12469","ref_index":3,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Analyzing the Effect of Noise in LLM Fine-tuning","primary_cat":"cs.LG","submitted_at":"2026-04-14T08:54:49+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Label noise hurts fine-tuning performance most while grammatical and typographical noise sometimes act as mild regularizers, with changes concentrated in task-specific layers.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.10786","ref_index":15,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Do BERT Embeddings Encode Narrative Dimensions? A Token-Level Probing Analysis of Time, Space, Causality, and Character in Fiction","primary_cat":"cs.CL","submitted_at":"2026-04-12T19:23:48+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"BERT embeddings encode narrative dimensions of time, space, causality, and character at the token level, as a linear probe achieves 94% accuracy versus 47% on variance-matched random embeddings, though unsupervised clusters do not align with these categories.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.07562","ref_index":16,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Reasoning-Based Refinement of Unsupervised Text Clusters with LLMs","primary_cat":"cs.CL","submitted_at":"2026-04-08T20:02:48+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"LLM reasoning refines unsupervised text clusters via coherence checks, redundancy removal, and label grounding, yielding better coherence and human-aligned labels on social media data.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.16359","ref_index":32,"ref_count":4,"confidence":0.88,"is_internal_anchor":false,"paper_title":"LLM4Log: A Systematic Review of Large Language Model-based Log Analysis","primary_cat":"cs.SE","submitted_at":"2026-03-18T20:34:58+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":4.0,"formal_verification":"none","one_line_summary":"Systematic review of 145 papers on LLM-based log analysis, providing a unified taxonomy, common design patterns, evaluation practices, and challenges for deployment under drift and limited labels.","context_count":2,"top_context_role":"background","top_context_polarity":"support","context_text":"Instead of averaging context-free word vectors, they embed each entire log event with a pretrained encoder. The main advantage is robustness: contextual encoders model word order and compositional meaning within an event, and they can assign different representations to the same token across different contexts (which is common in fast-evolving software logs) [32, 128]. As a result, semantically equivalent events with heterogeneous phrasing tend to stay closer in the embedding space, while subtle boundary or wording changes are less likely to break downstream models, a property that is particularly useful under drift, long-tail patterns, and limited supervision. In practice, the workflow is lightweight. Each parsed event (often the template text, optionally concatenated"},{"citing_arxiv_id":"2506.02132","ref_index":12,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Models","primary_cat":"cs.CL","submitted_at":"2025-06-02T18:01:56+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Inflectional features stay linearly decodable across all layers while lexical identity weakens with depth in modern transformers.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2305.14299","ref_index":8,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings","primary_cat":"cs.CL","submitted_at":"2023-05-23T17:40:41+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"TaDSE learns dialogue sentence embeddings via template-guided self-supervised contrastive learning plus synthetic slot-filling augmentation and reports gains on five downstream benchmarks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2104.08821","ref_index":92,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"SimCSE: Simple Contrastive Learning of Sentence Embeddings","primary_cat":"cs.CL","submitted_at":"2021-04-18T11:27:08+00:00","verdict":"CONDITIONAL","verdict_confidence":"LOW","novelty_score":8.0,"formal_verification":"none","one_line_summary":"SimCSE achieves 76.3% unsupervised and 81.6% supervised Spearman's correlation on STS tasks with BERT-base, improving prior best results by 4.2% and 2.2% via simple contrastive learning.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"ing degenerated alignment via dropout noise, thus improving the expressiveness of the representa- tions. The same analysis shows that the NLI train- ing signal can further improve alignment between positive pairs and produce better sentence embed- dings. We also draw a connection to the recent find- ings that pre-trained word embeddings suffer from anisotropy (Ethayarajh, 2019; Li et al., 2020) and prove that-through a spectrum perspective-the contrastive learning objective \"flattens\" the singu- lar value distribution of the sentence embedding space, hence improving uniformity. We conduct a comprehensive evaluation of Sim- CSE on seven standard semantic textual similarity (STS) tasks (Agirre et al., 2012, 2013, 2014, 2015,"},{"citing_arxiv_id":"2010.03496","ref_index":12,"ref_count":1,"confidence":0.88,"is_internal_anchor":false,"paper_title":"Inductive Entity Representations from Text via Link Prediction","primary_cat":"cs.CL","submitted_at":"2020-10-07T16:04:06+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Entity representations learned from text via link prediction generalize to unseen entities and transfer to classification and retrieval with reported gains of 22% MRR, 16% accuracy, and 8.8% NDCG@10.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null}],"limit":50,"offset":0}