Multi-Way Representation Alignment

Akshit Achara; Donato Crisostomi; Emanuele Rodol\`a; Mateo Mahaut; Melih Barsbey; Pritish Chakraborty; Tatiana Gaintseva; Viktor Stenby Johansson

REVIEW 4 major objections 5 minor 2 cited by

Multi-Way Representation Alignment

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes that aligning three or more representation spaces through a shared 'universe' with a post-hoc consensus correction — its GCPA method — simultaneously preserves internal geometry and improves any-to-any retrieval beyond p

desk verdict GCPA is a practical, honestly evaluated method for aligning M≥3 representation spaces, but its retrieval gains partly just confirm that optimizing cosine agreement improves cosine-agreement metrics, and the consensus assumption is only tested in strong-correspondence settings. read the letter →

arxiv 2602.06205 v2 pith:DIJ4S5Q7 submitted 2026-02-05 cs.LG cs.AI

Akshit Achara , Tatiana Gaintseva , Mateo Mahaut , Pritish Chakraborty , Viktor Stenby Johansson , Melih Barsbey , Emanuele Rodolà , Donato Crisostomi This is my paper · ORCID

classification cs.LGcs.AI

keywords multi-wayrepresentationalignmentgeneralizedprocrustesanalysiscanonicalcorrelationshareduniversezero-shotretrievalmodelstitchingconsensuscorrectionPlatonichypothesis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tackles a scaling problem: aligning M independently trained neural networks with pairwise maps costs O(M^2) maps and gives translations that can disagree depending on path. It argues for a shared-universe factorization (one map per model, O(M) total) and shows that a pure isometric universe (GPA) preserves geometry but underperforms on retrieval, while agreement-maximizing methods (GCCA) distort geometry. Its proposed GCPA keeps the GPA universe as a scaffold and applies a small shared correction that pulls each model's directions toward a per-sample consensus direction, controlled by a trust penalty. Experiments across multilingual, cross-camera, and multimodal benchmarks claim GCPA consistently beats pairwise, GPA, and GCCA on retrieval while keeping the reusable shared reference. If right, this gives a practical recipe for making many pretrained models interoperable without retraining.

What carries the argument

The consensus direction c_i = norm((1/M) sum_m û_{m,i}) — the normalized average of the M unit directions a matched sample takes in the GPA universe — together with the shared residual map T_theta (a small MLP) trained to minimize distance to c_i under a trust-region penalty on angular drift. The formal workhorse is Proposition 3.2, which equates average consensus agreement with the norm of the mean direction and shows it monotonically controls total pairwise agreement.

What would settle it

Take a set of, say, three models with deliberately corrupted training correspondences for one model (as in the paper's edge-map experiment) and measure GCPA versus GPA retrieval as anchors are removed one by one. If GCPA's accuracy drops below GPA's whenever the consensus is built from fewer than two clean models, that would demonstrate the correction depends on consensus quality rather than on the geometry scaffold itself.

Watch

Extended reading notes

Core claim

GCPA is the claim: first construct a Generalized Procrustes Analysis universe — orthogonal maps into a common reference that preserve each model's internal distances and angles — then train one small shared MLP in universe coordinates to nudge each sample direction toward the multi-model consensus direction c_i, with a trust penalty that caps how far it may deviate from the trusted GPA geometry. The paper proves an identity (Proposition 3.2): for a fixed sample, raising each view's cosine similarity to the consensus direction monotonically raises the sum of pairwise cosine similarities between all views, so the correction directly targets retrieval agreement. The result is a universe that re

Load-bearing premise

The load-bearing premise is that the per-sample consensus direction — the normalized average of all models' directions in the universe — points at the semantically correct alignment target; the paper's own Figure 8 shows that when few or corrupted models support that consensus, GCPA's correction can reinforce the mismatch and fall behind plain GPA.

Editorial extensions

If this is right

Translating between any two models via the universe is path-independent: composing through the shared reference always gives the same map as a direct one, which pairwise pipelines cannot guarantee.
Adding a new model costs one fitted map into the fixed universe rather than M−1 new pairwise maps, with little loss in cross-model accuracy versus full refitting.
Retrieval in the corrected universe beats both strict isometries and agreement-maximizing GCCA across multilingual, cross-camera, and cross-modal benchmarks.
A shared universe also stabilizes weak pairwise links: a poorly-corresponding pair becomes stitched more accurately when routed through a universe of healthy anchor models.
The trust-region correction creates a tunable geometry/agreement trade-off for practitioners.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The consensus-correction module is a plug-in: the same T_theta could be applied on top of any base alignment, not just GPA, so the idea likely transfers to non-orthogonal or functional universe constructions.
The identity behind GCPA suggests a broader principle: for unit vectors, maximizing average agreement with the mean direction is equivalent to maximizing pairwise sum-of-cosines, so any method that targets consensus implicitly optimizes pairwise agreement — a testable design rule for future multi-view objectives.
The failure mode shown in Figure 8 — GCPA underperforming when correspondences are weak — implies the method's benefit is conditional on consensus quality; a practical recipe would be to gate the correction by an estimate of consensus confidence per sample.
One extension worth testing: applying the correction per-sample with different trust thresholds based on agreement level could improve robustness to noisy correspondences beyond the fixed (τ, λ) used here.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

GCPA is a solid, useful method. The combination of GPA as a shared orthogonal universe plus a learned, shared correction toward the per-sample consensus is a natural idea, and the paper carries it through with serious experiments across multilingual, cross-camera, and multimodal retrieval, plus clustering and stitching. It also avoids the usual oversell: the paper explicitly shows in Figure 8 that GCPA can hurt when the consensus is poorly supported, which is more than many methods papers do. The code is a nice bonus.

The main caveat is that the retrieval task is evaluated by cosine agreement, and GCPA is trained to maximize cosine agreement to the consensus. So the gains over GPA and GCCA are less of a surprise and more of a confirmation that optimizing the evaluation objective works. The paper reports no error bars or significance tests on the retrieval tables, so I would not take the precise numbers too seriously; the trends are consistent enough, though.

The "retaining geometry" selling point is also softer than the abstract implies. Appendix C.2 shows GCPA introduces controlled drift, which is fine, but it means the shared reference is not isometric anymore. The paper is upfront about this, and the trust penalty limits the drift. Still, the headline should probably say "approximately retaining geometry" or "with a tunable geometry trade-off."

The theoretical side is thin. The GCCA derivation is standard and Proposition 3.1 mostly restates the theorem. The consensus identity, Proposition 3.2, is a two-line calculation, not a deep insight. That is okay if presented as intuition, but the paper treats it as the engine of the method.

The stress-test about the consensus direction being reliable is on target. The paper's own evidence shows GCPA can reinforce a noisy anchor, and the retrieval experiments use models with strong mutual correspondences. A critical reader will ask: what happens if one model is systematically biased? The paper does not answer that, and the "consistent improvement" claim should be scoped accordingly.

I would send this to peer review. It deserves a careful referee, mostly to demand proper statistical reporting and a more measured discussion of the consensus assumption. The central claim—that GCPA is competitive with or better than existing multi-way alignment on retrieval—is well supported in the tested regime.

For my own work, I would cite it for the GPA universe framing and the drift analysis. I am not sure I would adopt GCPA as my default, but it is a credible alternative.

Best, [Name]

Referee Report

4 major / 5 minor

Summary. The paper addresses the alignment of M≥3 neural representation spaces by constructing a shared universe. It adapts Generalized Procrustes Analysis (GPA) to produce an orthogonal universe, argues that strict isometry is suboptimal for retrieval relative to agreement-maximizing methods such as GCCA, and proposes Geometry-Corrected Procrustes Alignment (GCPA), which starts from a GPA scaffold and applies a shared nonlinear correction toward per-sample consensus directions. Experiments cover multilingual retrieval, cross-camera person re-identification, multimodal retrieval, clustering, and probing/stitching, reporting that GCPA improves retrieval over pairwise alignment, GPA, and GCCA while retaining a reusable reference space.

Significance. If the reported results hold, GCPA is a practically useful multi-way alignment method: it reduces the number of learned maps from O(M^2) to O(M), supports incremental model addition, and improves retrieval in several benchmarks. The paper ships a public implementation, and its formal statements (Proposition 3.2, Theorem B.1) are concrete and checkable. However, the central empirical claim is weakened by the fact that the GCPA objective directly optimizes the cosine-agreement metric used for evaluation, by the absence of error bars/significance tests on the headline retrieval tables, and by the paper's own Figure 8 showing that GCPA can underperform GPA under weak correspondences. These issues make the 'consistently improves' claim currently conditional rather than established.

major comments (4)

[§4.4, Table 2 and Figure 4] Retrieval results are reported as point estimates without error bars, confidence intervals, or significance tests. Several gains are small (e.g., Table 2: GCPA vs GCCA at M=3 is 0.637 vs 0.628; at M=10 it is 0.503 vs 0.487). Without multiple seeds or paired statistical tests, 'consistently improves' is not supported. Please add variability estimates and, where possible, paired tests across language subsets or cross-validation folds.
[§3.4, Eq. (5), Eq. (8), Proposition 3.2] The GCPA objective maximizes cosine similarity to the consensus direction c_i, and Proposition 3.2 proves this monotonically increases pairwise cosine similarity, which is exactly the retrieval evaluation metric. Moreover, c_i in Eq. (5) averages over all M models including the model being corrected, so the objective contains a self-contribution: each model is pulled toward a target that partly consists of its own vector. This self-referential bias should be quantified or removed by, e.g., computing a leave-one-out consensus c_i^{(m)} excluding model m during training, and reporting whether retrieval gains persist. As stated, the improvement of GCPA over GPA on cosine-based retrieval is close to a consequence of the training objective rather than an independent validation of the method.
[Appendix A.1, Figure 8; Abstract and §5] The abstract and conclusions claim GCPA 'consistently improves any-to-any retrieval,' but Figure 8 shows that on edge-heavy CIFAR-100, GCPA can underperform GPA when the universe is small or the anchor models are weak. The paper frames this as expected behavior, but it directly contradicts the unqualified 'consistent' claim. Please revise the claim to state the conditions under which GCPA helps, and ideally add a retrieval experiment with corrupted or biased anchors to show how retrieval performance degrades in the regime identified by Figure 8.
[Appendix E, Eq. (14)] GCPA applies a shared nonlinear corrector Tθ before mapping back to a target space, and the paper acknowledges that 'strict orthogonal cycle consistency need not hold after correction.' This undermines the 'shared universe' framing: the corrected coordinates are no longer a common coordinate system in the usual sense, because the composition of maps through the universe is path-dependent. The contribution claims a 'practical shared reference space'; please clarify what invariant property is retained after the nonlinear correction, and report whether any-to-any translation consistency is measurably affected.

minor comments (5)

[Notation throughout] The use of 'Ω' and 'U' is sometimes inconsistent (e.g., Eq. (1) uses X n Ω_{m←n} while the text refers to maps in different orders). Please standardize notation for maps and spaces.
[§3.3, Proposition 3.1] Proposition 3.1 is stated without a complete, self-contained formulation; it refers to Theorem B.1 and the 'retrieval gap' is described only informally. Consider stating the precise optimization and what 'suboptimal' means formally.
[§4.5, Figure 5] The bar chart reports mean changes but no variance or number of trials per condition. Adding per-triad values or error bars would help judge robustness.
[Appendix A.3] For Market-1501, there are no cross-camera image-level correspondences, yet the alignment uses identity-level correspondences. Please explain how the matched rows for GPA/GCPA are constructed from different images of the same identity, since this is nonstandard.
[References] The reference to 'Cannistraci et al.' is incomplete (no year, title, or venue). Please complete it or remove.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GCPA's gains are empirical and held-out; the consensus identity is a true algebraic lemma, not a self-smuggled prediction.

full rationale

The claimed derivation chain is not circular. GPA (Eq. 3) and GCPA (Eq. 8) are constructive algorithms; the consensus direction (Eq. 5) is defined from the aligned unit vectors, and Proposition 3.2 is a correct algebraic identity (proved in Appendix B) showing that maximizing agreement to the consensus increases pairwise cosine agreement. This identity is tautological in the sense that the consensus is the normalized sum of the vectors, but it is used only to motivate the objective, not as evidence for the empirical retrieval results. The retrieval improvements are measured on held-out test splits after fitting alignment parameters on the training split (Section 4.1), and GCPA is compared against GCCA, a strong independent baseline that also optimizes agreement; outperforming GCCA is therefore not forced by the GCPA training objective. The paper's own Fig. 8 and Appendix A.1 disclose a genuine limitation — under weak/corrupted correspondences the consensus correction can reinforce mismatch — which qualifies the 'consistently improves' claim but is a correctness/robustness caveat, not circularity. Self-citations to earlier representation-alignment work are background references and are not used as load-bearing uniqueness theorems or as substitutes for the experiments. Hence no circular step; score 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

GCPA introduces no new physical or conceptual entities. Its load-bearing assumptions are the availability of correspondences, the choice of cosine agreement as the retrieval surrogate, and the reliability of the per-sample consensus direction, the last of which the paper's own robustness experiment shows is fragile under noisy correspondences. The main fitted quantities are the trust hyperparameters and the shared MLP weights, whose exact settings are not reported.

free parameters (4)

trust tolerance τ = not specified; described as a fixed conservative value, swept in Appendix C
Threshold below which GCPA's trust penalty is inactive; exact value is not reported in the main text, only sensitivity plots.
trust penalty weight λ = not specified; fixed across benchmarks, swept in Appendix C
Controls how strongly GCPA penalizes drift from the GPA universe; exact value is not stated.
shared MLP Tθ parameters = trained on the training split with the GCPA objective (Eq. 13)
The correction network is fitted to training correspondences; architecture, learning rate, and training epochs are not specified.
PCA common dimension d = not specified
Applied to retrieval features when model dimensions differ; the chosen dimension affects all retrieval results.

assumptions (6)

domain assumption Matched sample correspondence is available across all M spaces during alignment fitting.
All methods (PW, GPA, GCCA, GCPA) fit maps using matched samples on the training split, and evaluation assumes identity-level or sample-level correspondences.
domain assumption All representations are standardized within each space before alignment.
Stated at the start of Section 3; standardization is part of the problem setup.
domain assumption Cosine similarity in the aligned space is the appropriate retrieval metric.
Retrieval experiments use cosine similarity, and GCPA's objective is also cosine-based; the choice binds the method to the evaluation.
ad hoc to paper The per-sample average of unit-normalized matched directions is a semantically meaningful consensus target.
Introduced in Eq. 5 and used as the GCPA training target. The paper itself shows in Figure 8 that this consensus can be distorted under weak correspondences, so this is a load-bearing assumption specific to GCPA.
domain assumption Orthogonal maps preserve internal geometry and this is desirable for stitching/probing.
The motivation for GPA relies on the claim that intra-model distances and angles should be preserved; the paper does not independently validate this for all downstream tasks.
standard math Rayleigh–Ritz/Ky Fan variational characterization of extremal eigenvalues.
Used in Appendix B to solve the spectral problem for GCCA; this is a standard linear algebra tool.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Way Representation Alignment." pith.science (2026). https://pith.science/paper/DIJ4S5Q7

@misc{pith2026260206205,
  author       = {Pith},
  title        = {Pith review of: Multi-Way Representation Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DIJ4S5Q7}},
  note         = {Machine review of arXiv:2602.06205}
}

abstract

The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of $M \ge 3$ models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.

Figures

Figures reproduced from arXiv: 2602.06205 by the authors.

**Figure 2.** Multi-way alignment stabilizes fragile connections. On edge-heavy CIFAR-100, we isolate a “weak” model pair with poor alignment. By progressively expanding the universe with robust models and refitting the universe, we observe a monotonic increase in stitching accuracy between the original fragile pair. We first establish a baseline by fitting a direct pairwise orthogonal map between the fragile pair. Due to the d… view at source ↗

**Figure 3.** Cross-model probing on CIFAR-100. Adding a new model by fitting only ΩM+1 into a fixed universe (GPAADD) approaches refitting the universe (GPA-REFIT) and outperforms PW alignment. To cover diverse scenarios, we use four different base model sets where the first two (from the left) sets consist of three models and the next two consist of five models each. Once a universe is learned, it can be reused rather than reb… view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Cross-lingual retrieval on TED-Multi (rank-1). GCPA outperforms GCCA, GPA, and pairwise orthogonal alignment. M = 3 M = 5 M = 10 Method Avg ↑ Worst ↑ Avg ↑ Worst ↑ Avg ↑ Worst ↑ PW 0.571 0.468 0.471 0.303 0.430 0.230 GPA 0.572 0.469 0.474 0.308 0.433 0.236 GCCA 0.628 0…

**Figure 5.** Figure 5: Robustness to correspondence noise on TEDMULTI. Rank-1 retrieval accuracy (%) on the clean test split relative to the unshuffled baseline. Solid bars average over the six directed pairs within the triad; hatched bars average over all directed pairs that involve at lea…

**Figure 6.** Figure 6: Cross-camera retrieval on MARKET1501 (mAP, %). Geometry-Corrected Procrustes Alignment (GCPA) improves over GCCA, GPA, and PW [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Rank-1 cross-modal retrieval (%) on FLICKR8K. GCPA improves audio↔ image ↔text retrieval while retaining the GPA universe. To understand the geometric mechanism behind these gains, we analyze how alignment changes (i) the typical distance between matched crossmodal pa…

**Figure 8.** Figure 8: Weak-link probing on edge-heavy CIFAR-100 under an expanding alignment set. We plot the change in probing accuracy (in %) for the same fragile pair as additional models are added. GPA improves steadily with more anchors, while GCPA can be unstable for very small sets b…

**Figure 9.** Figure 9: Sensitivity of GCPA to the trust penalty on cross-camera retrieval. We sweep the trust-region parameters (τ, λ) and report crosscamera mAP (%, higher is better). The trust penalty in GCPA acts as a soft trustregion as it allows the corrector to nudge a point toward t…

**Figure 10.** Figure 10: Sensitivity of GCPA to the trust penalty on drift. We sweep the trust-region parameters (τ, λ) and report median drift (%, lower is better). The top-right corner consists of higher λ and lower τ and therefore has lower drift values. GCPA improves retrieval by applyin…

**Figure 11.** Figure 11: UMAP representation of images from 20 representative classes from the Market-1501 dataset. The first column computes distances between images directly in the original model space, while the others are the universal spaces made using GPA, GCPA, and GCCA. Each line of s…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

One Adapter Pair per Model: A Universal Activation Interface for Language Models
cs.AI 2026-08 conditional novelty 6.0 of 10

A frozen shared activation bus with one encoder-decoder adapter per model lets probes, a single SAE, and a carrier NLA be reused across compatible language models.
CertBind from Multimodal Connectivity to Certifiable Retrieval Decisions
cs.LG 2026-08 conditional novelty 6.0 of 10

CertBind provides four interlocking certificates that let deployed multimodal connector routes preserve native retrieval, recover flagged routes through failure-diverse paths, and abstain when a top-k answer cannot be...

Reference graph

Works this paper leans on

10 extracted references · 4 linked inside Pith · cited by 2 Pith papers

[2]

3 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

URL http://arxiv.org/abs/ 2509.24734. 3 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and shor...

arXiv 2019
[5]

3 Fabian Gröger, Shuo Wen, Huyen Le, and Maria Brbi´c

URL https://proceedings.mlr.press/v89/grave19a.html. 3 Fabian Gröger, Shuo Wen, Huyen Le, and Maria Brbi´c. With limited data for multimodal alignment, let the structure guide you.arXiv preprint arXiv:2506.16895,

arXiv
[6]

Deep multimodal semantic embeddings for speech and images

3 David Harwath and James Glass. Deep multimodal semantic embeddings for speech and images. In 2015 IEEE workshop on automatic speech recognition and understanding (ASRU), pp. 237–244. IEEE,

2015
[7]

3 Jon R Kettenring

URLhttps://arxiv.org/abs/2505.12540. 3 Jon R Kettenring. Canonical analysis of several sets of variables.Biometrika, 58(3):433–451,

arXiv
[9]

Towards universal backward-compatible representation learning.arXiv preprint arXiv:2203.01583,

3 Binjie Zhang, Yixiao Ge, Yantao Shen, Shupeng Su, Fanzi Wu, Chun Yuan, Xuyuan Xu, Yexin Wang, and Ying Shan. Towards universal backward-compatible representation learning.arXiv preprint arXiv:2203.01583,

arXiv
[2019]

An image is worth 16x16 words: Transformers for image recognition at scale

9 Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

arXiv 2010
[2020]

We introduce a distribution shift by replacing 85% of the training images for these two mod- els with binary Canny edge maps (Canny, 2009), retaining only 15% as original RGB

trained on a corrupted version of CIFAR-100 (Krizhevsky et al., 2009). We introduce a distribution shift by replacing 85% of the training images for these two mod- els with binary Canny edge maps (Canny, 2009), retaining only 15% as original RGB. This de- grades the correlation between their learned fea- tures compared to models trained on standard data. ...

2009
[2021]

Escaping platos cave: Jam for aligning independently trained vision and language models.arXiv preprint arXiv:2507.01201,

3 Yisong Yue, Been Kim, et al. Escaping platos cave: Jam for aligning independently trained vision and language models.arXiv preprint arXiv:2507.01201,

arXiv

Show all 10 references

[2025]

2 Irene Cannistraci, Luca Moschella, Marco Fumero, Valentino Maiorca, and Emanuele Rodolà

URLhttps://arxiv.org/abs/2505.17101. 2 Irene Cannistraci, Luca Moschella, Marco Fumero, Valentino Maiorca, and Emanuele Rodolà. From bricks to bridges: Product of invariances to enhance latent space communication. InThe Twelfth International Conference on Learning Representati...

arXiv
[6542]

URL https://doi.org/10.1093/pnasnexus/ pgac039

doi: 10.1093/pnasnexus/pgac039. URL https://doi.org/10.1093/pnasnexus/ pgac039. 2 John C Gower. Generalized procrustes analysis.Psychometrika, 40(1):33–51,

doi

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [2]

3 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

URL http://arxiv.org/abs/ 2509.24734. 3 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and shor...

arXiv 2019

[2] [5]

3 Fabian Gröger, Shuo Wen, Huyen Le, and Maria Brbi´c

URL https://proceedings.mlr.press/v89/grave19a.html. 3 Fabian Gröger, Shuo Wen, Huyen Le, and Maria Brbi´c. With limited data for multimodal alignment, let the structure guide you.arXiv preprint arXiv:2506.16895,

arXiv

[3] [6]

Deep multimodal semantic embeddings for speech and images

3 David Harwath and James Glass. Deep multimodal semantic embeddings for speech and images. In 2015 IEEE workshop on automatic speech recognition and understanding (ASRU), pp. 237–244. IEEE,

2015

[4] [7]

3 Jon R Kettenring

URLhttps://arxiv.org/abs/2505.12540. 3 Jon R Kettenring. Canonical analysis of several sets of variables.Biometrika, 58(3):433–451,

arXiv

[5] [9]

Towards universal backward-compatible representation learning.arXiv preprint arXiv:2203.01583,

3 Binjie Zhang, Yixiao Ge, Yantao Shen, Shupeng Su, Fanzi Wu, Chun Yuan, Xuyuan Xu, Yexin Wang, and Ying Shan. Towards universal backward-compatible representation learning.arXiv preprint arXiv:2203.01583,

arXiv

[6] [2019]

An image is worth 16x16 words: Transformers for image recognition at scale

9 Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929,

arXiv 2010

[7] [2020]

We introduce a distribution shift by replacing 85% of the training images for these two mod- els with binary Canny edge maps (Canny, 2009), retaining only 15% as original RGB

trained on a corrupted version of CIFAR-100 (Krizhevsky et al., 2009). We introduce a distribution shift by replacing 85% of the training images for these two mod- els with binary Canny edge maps (Canny, 2009), retaining only 15% as original RGB. This de- grades the correlation between their learned fea- tures compared to models trained on standard data. ...

2009

[8] [2021]

Escaping platos cave: Jam for aligning independently trained vision and language models.arXiv preprint arXiv:2507.01201,

3 Yisong Yue, Been Kim, et al. Escaping platos cave: Jam for aligning independently trained vision and language models.arXiv preprint arXiv:2507.01201,

arXiv