Pith. sign in

REVIEW 4 major objections 4 minor 11 references

Structure and Destructure: Dual Forces in the Making of Knowledge Engines

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This thesis claims that knowledge-graph models and large language models are driven by the same two forces—language modelling builds structure into model weights, and periodic embedding resets restore the plasticity needed to generalize to

desk verdict A well-written thesis whose synthesis is plausible but whose causal story on active forgetting needs an ablation that removes only the reset. read the letter →

arxiv 2509.00949 v1 pith:NNNL2M5G submitted 2025-08-31 cs.CL cs.AI

Yihong Chen This is my paper
classification cs.CLcs.AI
keywords knowledgeenginestructureformationactiveforgettingembeddingresetgraphcompletionlanguagemodelinterpretabilityplasticitycross-lingualtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis seeks to unify the structured knowledge-graph paradigm with the unstructured large-language-model paradigm by showing both are driven by the same two forces. Structure: language-modelling objectives cause models to encode recoverable relational patterns, whether the input is triples or raw text. Destructure: periodically resetting the embedding layer during training improves adaptation to unseen predicates and languages. If correct, the two paradigms share a common design recipe for knowledge engines that are simultaneously interpretable and adaptable.

What carries the argument

The unifying object is the embedding sandwich: any model with an input embedding layer, a central body, and an output unembedding layer. The load-bearing identity is the exact rewriting of gradient descent over entity embeddings as message-passing operations; this turns embeddings into caches of symbol-interaction messages. The other central mechanism is active forgetting—periodically resetting the embedding layer—which truncates that cache. For transformers, the corresponding object is the 2^L path decomposition of recursive residual networks, which turns the network into an ensemble of factorization models whose token-interaction databases can be read out as n-grams.

What would settle it

Take the same forgetting pretraining recipe but replace each periodic embedding reset with additive Gaussian noise of matched magnitude at the same intervals, holding everything else fixed. If the inductive graph and cross-lingual gains match the reset condition, the cache-truncation explanation is not necessary; if the gains disappear only when resets are applied to the embedding cache, the mechanism is supported.

Watch

Extended reading notes

Core claim

The central claim is that structure and destructure are complementary forces in making knowledge engines. Part I shows that a masked-language-modelling-style objective—predicting any symbol from its surrounding symbols—induces structure in both paradigms: adding relation prediction to standard entity-prediction training improves knowledge graph completion, especially on highly multi-relational graphs, and a transformer's residual computation can be decomposed into many input-to-output paths, each behaving like a factorization model, whose n-gram databases expose interpretable structure inside large language models. Part II argues that gradient descent on entity embeddings can be rewritten ex

Load-bearing premise

The claim that periodic embedding resets help generalization because they stop embeddings from caching excessive message-passing computations over symbols—rather than because the resets add noise, shrink model capacity, or interact with node-feature inputs—is the load-bearing premise; the thesis does not isolate it with an identification experiment.

Editorial extensions

If this is right

  • Adding relation prediction to standard KBC training—a masked-language-modelling-style objective over triples—improves link prediction, especially on datasets with many relation types.
  • A transformer's residual computation can be unrolled into many input–output paths, each resembling a factorization model; the resulting n-gram tables support data-free analysis of LLM internals, pretraining dynamics, and fine-tuning effects.
  • Reinterpreting gradient descent on entity embeddings as message passing shows factorization models are a special kind of GNN with infinite layers, a global normalizer, and layer-wise training; truncating this with finite layers and node features yields ReFactor GNNs, which keep transductive performance and generalize inductively.
  • Periodically resetting token embeddings during pretraining makes language models adapt faster and better to low-resource languages, with the largest gains for languages distant from the pretraining language.
  • Together the results define a recipe: model the seen through structure formation, and model the unseen through active destructuring of the learned cache.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the cache-truncation account is right, the same periodic-reset trick should generalize to other over-trained embedding layers in continual learning or model editing, with resets scheduled by how long a symbol class has been cached rather than by a fixed interval.
  • Beyond the paper: the n-gram extraction method could serve as a deployment-time audit tool—for example, checking whether fine-tuning or alignment edits actually remove undesirable knowledge, since the paper's own toxicity case study shows surface detoxification can leave embedded bigram mass almost unchanged.
  • Beyond the paper: the structure/destructure dichotomy suggests a testable scaling prediction—the farther a new language or predicate is from the training distribution, the larger the relative benefit of active forgetting; the paper's cross-lingual results are consistent with this but do not establish the scaling law.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This PhD thesis proposes a unified account of two paradigms for building knowledge engines: the structured paradigm (knowledge graphs, factorization models/GNNs) and the unstructured paradigm (transformer LMs). The claimed connection is two complementary forces: 'structure'—language-modelling objectives induce recoverable symbolic structure in both paradigms—and 'destructure'—periodically resetting embedding weights ('active forgetting') improves plasticity and generalization to unseen entities/predicates/languages. Part I (Chs 2–3) presents experiments: adding relation prediction to 1vsAll training improves KGC on several benchmarks; residual-stream decomposition of transformers yields n-gram 'databases' used for interpretability case studies. Part II (Chs 4–5) reinterprets GD on FM embeddings as message passing, introduces ReFactor GNNs for inductive KGC with periodic node-state cache resets, and applies periodic token-embedding resets during PLM pretraining to improve low-resource cross-lingual adaptation. The thesis concludes by proposing a 'structure/destructure recipe' for general knowledge engines.

Significance. If the central causal mechanism were established, this would be a valuable conceptual bridge: it identifies a shared architectural abstraction (the 'embedding sandwich'), gives a formal rewriting of FM training as message passing, and connects two usually separate literatures. Strengths include benchmark-backed experimental chapters (FB15k-237, WN18RR, CoDEx, XNLI, MLQA, XQuAD), statistical significance tests in Ch2, open-source code for Ch2, and generated predictions (forgetting improves low-resource adaptation) that are in principle falsifiable. However, the 'destructure' force is currently a plausible interpretation rather than an established cause: the resets are not isolated from confounds, and the integrated recipe is not validated on a fresh benchmark. The thesis's own limitations section (§6.2.1) acknowledges the theoretical scope, but the missing identification experiment is central, not peripheral.

major comments (4)
  1. [Ch2, Table 2.4] The CoDEx-S/M/L rows reproduce, exactly, the WN18RR/FB15k-237/Aristo-v4 rows from Table 2.3 (e.g., CoDEx-S '0.487/0.441/0.501/0.580' equals WN18RR 'EP=on, RP=off'). Since CoDEx has different statistics, this is not plausible unless the table is mislabeled. Please correct and re-run/verify; as printed, the RQ1 claim 'relation prediction consistently improves MRR and Hits@1 across CoDEx' is unsupported by the reported data.
  2. [§4.4–§4.5, Fig 4.4] The formal rewriting (Eq. 4.7–4.9) shows GD updates on embeddings can be expressed as message passing. The load-bearing causal step—that converged FMs cache infinite rounds and that the periodic external cache reset improves generalization by truncating this cache—is not identified. ReFactor GNNs change several factors at once: finite L, input node features, layer-wise training, global normalizer, and reset. The inductive gains in Fig 4.4 are measured against FMs/GNNs, not against a no-reset ReFactor with identical features/capacity. The thesis should add this ablation; otherwise the performance gain could be due to reduced capacity, added noise, or node features. §6.2.1 flags scope but does not supply the experiment.
  3. [§5.4–§5.5, Algorithm 5] The pretraining comparison in Ch5 is standard vs forgetting PLMs. Since resetting token embeddings every K updates is a strong stochastic/regularization intervention, the observed gains in XNLI/MLQA/XQuAD do not by themselves establish the cache-truncation mechanism. Required controls include a single-reset baseline, injecting equivalent noise without resets, varying embedding capacity, and comparing reset of embeddings vs reset of other layers, plus an ablation of K. Without these, 'destructure' remains a post-hoc interpretation, not a mechanism.
  4. [§1.4, §6.3] The abstract and conclusion present 'a new recipe for developing general knowledge engines,' but the recipe is assembled from the author's own published experiments and is not held out against any fresh, integrated validation. The framework is therefore partly post-hoc. Either add a prospective validation of the combined structure+destructure recipe or explicitly re-frame it as an interpretive synthesis rather than a predictive recipe.
minor comments (4)
  1. [Figure 2.2] The caption says 'Hits@1 versus embedding size' but the body text refers to 'MRR'. Please align the axis label and the description.
  2. [Eq. 4.5] The notation is inconsistent: Eq. 4.4 defines relation embeddings as ψ[r], but Eq. 4.5 uses g(r). Define g consistently with the encoder notation in Eq. 4.2.
  3. [Impact Statement] Typo: 'fundmentally' should be 'fundamentally'.
  4. [Tables 5.4–5.5] The repeated caption 'Accuracy comparison of forgetting and standard PLMs on XQuAD (table continued)' is redundant; one caption with a note that the table spans pages is sufficient.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: formal rewrites are exact, benchmarks are external, and the destructure mechanism is an underdetermined causal story rather than a self-referential derivation.

full rationale

I walked the paper's claimed derivation chain. In Chapter 2, relation prediction is an auxiliary objective term added to the standard 1vsAll objective; the evaluation is entity ranking on held-out triples against strong baselines. The entity-prediction improvement is an empirical result, not a quantity fitted by the auxiliary term by construction. In Chapter 3, the n-gram structures are extracted data-free from model weights via matrix products and softmax readouts; the paper's claim that LMs acquire these structures is supported by tracing bigram acquisition over pretraining and by comparison with corpus statistics, not merely by defining them into existence. Chapter 4's central identity (Eq. 4.7–4.9) is an exact rewriting of gradient descent over entity embeddings as message-passing; this is a mathematical equivalence, not a circular prediction. The inductive gains are evaluated on unseen graphs (e.g., FB15K237_v1_ind) against external methods (GraIL, NBFNet). ReFactor GNNs change multiple components simultaneously—finite layers, input node features, layer-wise training, a global normalizer, and the external cache reset—so the specific causal role of the reset is not isolated by an ablation that removes only the reset. That is an identification/confound concern, not circularity: no fitted parameter is renamed as a prediction, and the improvement does not reduce to the model's own training target. Chapter 5 compares forgetting pretraining with standard pretraining on XNLI, MLQA, and XQuAD; gains may come from regularization, altered capacity, or training/test distribution matching, but the outcomes are not encoded into the objective by definition. The overarching 'structure/destructure' synthesis is retrospective and partly semantic, but it does not substitute for evidence: each chapter provides its own external benchmarks and peer-reviewed provenance. Self-citations are declarations of provenance, not load-bearing arguments. The thesis itself flags theoretical scope and limitations (e.g., §6.2.1, §4.7, §3.6), which further supports treating the causal mechanism as an open interpretation rather than a circular derivation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central claims rest on a small set of background assumptions: the GD-as-message-passing rewriting (exact) plus its causal extension (assumed), the empirical sufficiency of LM objectives to induce recoverable structures, the approximate path decomposition of nonlinear transformers, and the rewiring framework as the plasticity testbed. The deposit is transparent about most of these; Ch3 and Ch6 flag the approximation and scope explicitly. Free parameters (lambda, K, L, sparsification thresholds, embedding size) are tuned to data and are not derived by the theory.

free parameters (5)
  • lambda (weight of relation prediction) = per-dataset grid-selected value; lambda=1 in ablations
    Ch2 Eq. 2.3; the benefit is lambda- and |R|-dependent (Fig 2.1), so the main Ch2 claim depends on this tuning.
  • K (forgetting interval) = tuned per experiment; exact value not visible in truncated text (Appendix C.5 explores frequency sensitivity)
    Algorithm 5 (Ch5); the plasticity gains depend on choosing K, so the 'destructure helps' claim is conditional on K.
  • L (number of ReFactor GNN message-passing layers) = empirically chosen (ablation on n[v] in Ch4 Table 4.2)
    Ch4; truncating message passing is the proposed mechanism, and the layer count is a design choice rather than a derived quantity.
  • trigram sparsification threshold = unspecified
    Algorithm 3: 'Sparsify based on threshold'; the extracted trigrams and the attribution claims (Table 3.2) depend on this choice.
  • embedding size = swept from 25 to 4000 in Ch2 ablations
    Ch2 Fig 2.2; relation prediction's benefit grows with embedding size, so the qualitative conclusion is size-dependent.
assumptions (5)
  • standard math Gradient descent on softmax scores equals message-passing; training to convergence therefore equals infinite message-passing rounds (Ch4, Section 4.4, Eqs. 4.7-4.9).
    The rewriting is exact for a single GD step; its use as a causal story about 'excessive caching' goes beyond the formal identity and needs the convergence and normalization assumptions stated in the chapter.
  • domain assumption Language modelling objectives induce recoverable global structures in embedding-encapsulated models (Ch2, Ch3, Part I summary).
    Shown empirically on benchmarks and case studies, not proven; Ch3 explicitly defers verification of the link between extracted n-grams and the training data distribution (Section 3.6).
  • domain assumption Transformers with nonlinearities can be decomposed into 2^L additive input-output paths for structure extraction (Ch3, Section 3.3.3).
    Exact only for linear gamma_l; for real transformers the author states the decomposition is approximate yet proceeds, validating mainly by case studies and interventions.
  • domain assumption The rewiring setup (frozen transformer body, relearned token embeddings, 125K adaptation steps) is a representative testbed for language plasticity (Ch5).
    The framework choice is discussed in Appendix C.1.2; the conclusion 'forgetting improves plasticity' is measured within this framework and may not transfer to full-model fine-tuning.
  • standard math Unigram scores are obtained from the steady state of a bigram Markov matrix (Algorithm 2).
    Requires the bigram probability matrix to be a well-behaved stochastic matrix with an eigenvalue 1; truncation effects could distort the steady state.
invented entities (2)
  • 'Destructure' as a complementary force
    purpose: Names the counterpart to structure formation; motivates periodic embedding resets as its operational form
    No new falsifiable handle beyond the experiments that motivated it; it is a post-hoc organizing label for active forgetting, which is prior work.
  • 'Embedding sandwich' abstraction
    purpose: Relabels the shared embedding-body-unembedding layout of KBC models and transformers to justify cross-paradigm transfer of insights
    A descriptive reorganization of existing architectures; no independent evidence is required or provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure and Destructure: Dual Forces in the Making of Knowledge Engines." pith.science (2026). https://pith.science/paper/NNNL2M5G

@misc{pith2026250900949,
  author       = {Pith},
  title        = {Pith review of: Structure and Destructure: Dual Forces in the Making of Knowledge Engines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NNNL2M5G}},
  note         = {Machine review of arXiv:2509.00949}
}
read the original abstract

The making of knowledge engines in natural language processing has been shaped by two seemingly distinct paradigms: one grounded in structure, the other driven by massively available unstructured data. The structured paradigm leverages predefined symbolic interactions, such as knowledge graphs, as priors and designs models to capture them. In contrast, the unstructured paradigm centers on scaling transformer architectures with increasingly vast data and model sizes, as seen in modern large language models. Despite their divergence, this thesis seeks to establish conceptual connections bridging these paradigms. Two complementary forces, structure and destructure, emerge across both paradigms: structure organizes seen symbolic interactions, while destructure, through periodic embedding resets, improves model plasticity and generalization to unseen scenarios. These connections form a new recipe for developing general knowledge engines that can support transparent, controllable, and adaptable intelligent systems.

Figures

Figures reproduced from arXiv: 2509.00949 by the authors.

Figure 1.1
Figure 1.1. Illustration of how “knowledge engines” in human minds facilitate diverse [PITH_FULL_IMAGE:figures/full_fig_p028_1_1.png] view at source ↗
Figure 1.2
Figure 1.2. A medical knowledge graph showing relationships between myocardial dis [PITH_FULL_IMAGE:figures/full_fig_p032_1_2.png] view at source ↗
Figure 1.3
Figure 1.3. Knowledge graphs use predicates to define entity relationships, enabling the [PITH_FULL_IMAGE:figures/full_fig_p042_1_3.png] view at source ↗
Figures from the paper (28 more)
Figure 2.1
Figure 2.1. Figure 2.1: Relative changes between ComplEx trained with and w/o Relation Prediction [PITH_FULL_IMAGE:figures/full_fig_p057_2_1.png]
Figure 2.2
Figure 2.2. Figure 2.2: Hits@1 versus embedding size for CP on FB15k-237, each point represents [PITH_FULL_IMAGE:figures/full_fig_p060_2_2.png]
Figure 2.3
Figure 2.3. Figure 2.3: t-SNE visualisations for ComplEx embeddings, trained with relation predic [PITH_FULL_IMAGE:figures/full_fig_p060_2_3.png]
Figure 3.1
Figure 3.1. Figure 3.1: The uncovered n-gram structures can be seen as a reformatting of the corre [PITH_FULL_IMAGE:figures/full_fig_p067_3_1.png]
Figure 3.2
Figure 3.2. Figure 3.2: Embedding “sandwiches” are typical architectures for dealing with discrete [PITH_FULL_IMAGE:figures/full_fig_p070_3_2.png]
Figure 3.3
Figure 3.3. Figure 3.3: Various expressions of residual stream, each emphasizing a different aspect. [PITH_FULL_IMAGE:figures/full_fig_p072_3_3.png]
Figure 3.4
Figure 3.4. Figure 3.4: Cartoon of the process of deriving bigram databases [PITH_FULL_IMAGE:figures/full_fig_p076_3_4.png]
Figure 3.5
Figure 3.5. Figure 3.5: Top 1K bigram hit ratios w.r.t. the final step. inspired by the one-to-one, one-to-many, and many-to-many relational analysis in the knowledge graph literature [Lacroix et al., 2018]. For example, “at least” is a few-to￾many bigram: there are many possible tokens tha…
Figure 3.6
Figure 3.6. Figure 3.6: Top [PITH_FULL_IMAGE:figures/full_fig_p089_3_6.png]
Figure 3.8
Figure 3.8. Figure 3.8: Visualization of OLMo-7B’s promotion and suppression dynamics of bi￾grams scores. ing is effective in acquiring relevant knowledge. Does RLHF finetuning remove toxicity? We compare the raw pretrained model, Llama￾2-7B, with its RLHF version, Llama-2-7B-Chat. RLHF ali…
Figure 3.9
Figure 3.9. Figure 3.9: Language modelling objective over embedding-encapsulated models drives [PITH_FULL_IMAGE:figures/full_fig_p096_3_9.png]
Figure 3.10
Figure 3.10. Figure 3.10: Not all structures encoded in the model are of positive roles. Some can be [PITH_FULL_IMAGE:figures/full_fig_p100_3_10.png]
Figure 3.11
Figure 3.11. Figure 3.11: Memory consists of four main operations: acquisition, consolidation, re [PITH_FULL_IMAGE:figures/full_fig_p102_3_11.png]
Figure 4.1
Figure 4.1. Figure 4.1: REFACTOR GNN bridges factorization-based models and graph neural net [PITH_FULL_IMAGE:figures/full_fig_p107_4_1.png]
Figure 4.2
Figure 4.2. Figure 4.2: ReFactor GNN architecture. The left figure describes messages from the local neighbourhood {(v2, r1, v1),(v3, r2, v1),(v1, r3, v4)} (the orange and blue edges, which depend on the type of relationship of the edges) and a global normaliser term induced by the partitio…
Figure 4.3
Figure 4.3. Figure 4.3: Illustration of the external node state cache used during training. External cache, its push, pull, and clear. Implementation-wise, such a training dy￾namic equals to maintaining an external memory for storing and retriving historical node states h t−1 to compute h t…
Figure 4.4
Figure 4.4. Figure 4.4: Inductive KGC performance, trained on the KG [PITH_FULL_IMAGE:figures/full_fig_p128_4_4.png]
Figure 4.5
Figure 4.5. Figure 4.5: Performance vs parameter efficiency on FB15K237_v1. Left axis is Test MRR while right axis is #parameters. The solid lines and dashed lines indicate the changes of Test MRR and the changes of #parameters [PITH_FULL_IMAGE:figures/full_fig_p129_4_5.png]
Figure 5.1
Figure 5.1. Figure 5.1: Rewiring via relearning token embeddings: where the transformer body (the purple part) is “frozen” and reused for a new language, but the token embeddings are relearned to suit the new language. see this as a promising testing ground for forgetting techniques. Our fo…
Figure 5.2
Figure 5.2. Figure 5.2: Unsupervised zero-shot cross-lingual transfer. [PITH_FULL_IMAGE:figures/full_fig_p139_5_2.png]
Figure 5.3
Figure 5.3. Figure 5.3: The rewiring performance for standard PLMs (blue dashed line) drops dras [PITH_FULL_IMAGE:figures/full_fig_p140_5_3.png]
Figure 5.4
Figure 5.4. Figure 5.4: Pretraining losses of forgetting and standard language models. The forgetting [PITH_FULL_IMAGE:figures/full_fig_p143_5_4.png]
Figure 5.5
Figure 5.5. Figure 5.5: Adaptation curves on XNLI, MLQA, and XQuAD. Numbers aggregated [PITH_FULL_IMAGE:figures/full_fig_p148_5_5.png]
Figure 5.6
Figure 5.6. Figure 5.6: Relative gains of forgetting PLMs over standard PLMs across languages for [PITH_FULL_IMAGE:figures/full_fig_p149_5_6.png]
Figure 5.7
Figure 5.7. Figure 5.7: Relative gains of forgetting over standard across languages for MLQA. For [PITH_FULL_IMAGE:figures/full_fig_p150_5_7.png]
Figure 5.8
Figure 5.8. Figure 5.8: Adaptation curves on XNLI within 5K updates for individual languages: Bulgaria, Greek, Spanish, French, Russian, Swahili, Vietnamese and Chinese. For all languages except Urdu, the forgetting PLMs converge faster than the standard PLMs during the language adaptation …
Figure 5.9
Figure 5.9. Figure 5.9: The active forgetting mechanism periodically resets the embedding layer [PITH_FULL_IMAGE:figures/full_fig_p155_5_9.png]
Figure 5.10
Figure 5.10. Figure 5.10: The reality is always changing. We use different shapes to indicate the [PITH_FULL_IMAGE:figures/full_fig_p156_5_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages

  1. [1]

    Componentscorrespondingtothetripletswhere v =v^ w6=v. Thesumofthese components is given by: X (v,r, w)∈T ∂logP ( wjv, r) ∂ϕ[v] = X (v,r, w)∈T " ∂Γ(v, r, w) ∂ϕ[v] X u P (ujv, r)∂Γ(v, r,u ) ∂ϕ[v] # = X (r, w)∈N 1 +[v] ϕ[ w] g(r) X (v,r, w)∈T X u P (ujv, r)g(r) ϕ[u]

    work page

  2. [2]

    Componentscorrespondingtothetripletswhere v6=v^ w=v. Thesumofthese components is given by: X ( v,r,v)∈T ∂ logP (vj v,r) ∂ϕ[v] = X ( v,r,v)∈T " ∂Γ( v,r,v ) ∂ϕ[v] X u P (uj v,r)∂Γ( v,r,u ) ∂ϕ[v] # = X ( v,r)∈N 1 −[v] g(r) ϕ[ v] (1P (vj v,r))

    work page

  3. [3]

    v’s neighbourhood! v

    Componentscorrespondingtothetripletswhere v6=v^ w6=v. Thesumofthese 193 components is given by: X ( v,r, w)∈T ∂ logP ( wj v,r) ∂ϕ[v] = X ( v,r, w)∈T " 0 X u P (uj v,r)∂Γ( v,r,u ) ∂ϕ[v] # = X ( v,r, w)∈T P (vj v,r)∂Γ( v,r,v ) ∂ϕ[v] . = X ( v,r, w)∈T P (vj v,r)g(r) ϕ[ v]. Collecting these three categories, the GD operator overϕ[v], or rather the node repre-...

    work page 2018

  4. [4]

    Eberhard Fuchs and Gabriele Flügge

    URLhttps://openreview.net/forum?id=rJl-b3RcF7. Eberhard Fuchs and Gabriele Flügge. Adult neuroplasticity: more than 40 years of re- search. Neural plasticity, 2014(1):541870, 2014. 155 A Garcez, M Gori, LC Lamb, L Serafini, M Spranger, and SN Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reaso...

    arXiv 2014

  5. [8]

    % TuckER (200, 100) 0.10 10 0.10 NA 0.920 %

    URLhttps://arxiv.org/abs/2309.05660. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang,MyraCheng,MiaGlaese,Bo...

    arXiv 2021

  6. [27]

    How Does Knowledge Graph Embedding Extrapolate to Unseen Data: A Semantic Evidence View

    Open Review, 62(1), 2022. 162 Seungpil Lee, Woochang Sim, Donghyeon Shin, Wongyu Seo, Jiwon Park, Seokki Lee, Sanha Hwang, Sejin Kim, and Sundong Kim. Reasoning abilities of large language models: In-depthanalysisontheabstractionandreasoningcorpus. ACMTrans.Intell. Syst.Technol., January 2025. ISSN 2157-6904. doi: 10.1145/3712701. URLhttps: //doi.org/10.1...

    work page Pith review arXiv 2022

  7. [2003]

    DEPT: Decoupled Embeddings for Pre-training Language Models

    URLhttps://doi.org/10.1016/j.websem.2003.07.001. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. InInter- national Conference on Learning Representations, 2021. 159 Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. Gpt-gnn: Generative ...

    work page Pith review arXiv 2003

  8. [2019]

    Divyanshu Aggarwal, Ashutosh Sathe, and Sunayana Sitaram

    URLhttps://api.semanticscholar.org/CorpusID:198147334. Divyanshu Aggarwal, Ashutosh Sathe, and Sunayana Sitaram. Exploring pretraining viaactiveforgettingforimprovingcrosslingualtransferfordecoderlanguagemodels. arXiv preprint arXiv:2410.16168, 2024. Jethro Akroyd, Sebastian Mosbach, Amit Bhave, and Markus Kraft. Universal digital twin - a dynamic knowled...

    arXiv 2024

Show all 11 references
  1. [2020]

    Jerome Bruner.The Process of Education

    URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. Jerome Bruner.The Process of Education. Harvard University Press, 1960. Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. Imbalanced learning: A com- prehensive evalua...

    2020 arXiv

  2. [2023]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov

    URL https://proceedings.neurips.cc/paper_files/paper/2023/ file/34e1dbe95d34d7ebaf99b9bcaeb5b2be-Paper-Conference.pdf. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. XNLI: Evaluating cross-lingual sentence re...

    2023 arXiv

  3. [2024]

    Free eBooks from the public domain

    URLhttps://www.gutenberg.org. Free eBooks from the public domain. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Smaranda Muresa...

    2022 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.