Meta-Learning Transformers to Improve In-Context Generalization

Anna Vettoruzzo; Joaquin Vanschoren; Lorenzo Braccaioli; Mohamed-Rafik Bouguelia; Nicola Conci; Prabhant Singh

REVIEW 3 major objections 6 minor 4 references

Meta-Learning Transformers to Improve In-Context Generalization

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Meta-training a transformer on a curated collection of 30 small, domain-specific datasets yields cross-domain generalization on par with — and on low-overlap domains better than — training the same model on ImageNet-1k, with added…

desk verdict A careful empirical study of training in-context learners on curated multi-domain collections; the headline claim is real but more conditional than the abstract admits. read the letter →

arxiv 2507.05019 v2 pith:NT4N4TN7 submitted 2025-07-07 cs.LG cs.AI

Lorenzo Braccaioli , Anna Vettoruzzo , Prabhant Singh , Joaquin Vanschoren , Mohamed-Rafik Bouguelia , Nicola Conci This is my paper

classification cs.LGcs.AI

keywords in-contextlearningmeta-learningfew-shotimageclassificationmulti-domaindatasetscontinualcurriculumunsupervisedtransformergeneralization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that in-context learning does not need a giant, unstructured training corpus: a deliberately curated collection of small, domain-specific datasets can generalize as well or better. The authors train GEOM, a transformer, by meta-learning on the 30 datasets of the Meta-Album collection (ten visual domains), then test it on domains withheld from training, in a sequential streaming scenario, and without labels. GEOM matches or exceeds a same-architecture model trained on ImageNet-1k on most domains, and its accuracy rises steadily as more datasets and classes are added to the collection. If the finding stands, in-context learners could be built from replaceable, privacy-friendly dataset building blocks instead of monolithic web-scale corpora.

What carries the argument

The load-bearing object is the non-causal task sequence: each training episode concatenates $N \times K$ embedded context images with their encoded class labels and one query embedding, forming a permutation-invariant sequence processed by a transformer encoder $M_\theta$ with a single classification layer on the query position. This turns meta-learning into a sequence prediction objective — minimize expected cross-entropy over the query label given the context — without any causal masking or weight-update adaptation step. The architecture pairs a frozen ImageNet-pretrained ResNet-50 feature extractor $f_\psi$ with a learnable linear class encoder $g_\phi$; the same mechanism carries all three variants (offline GEOM, sequential GEOM-S, unsupervised GEOM-U), so differences in results are attributable to data organization rather than architecture.

What would settle it

Build a few-shot test set from domains that share no classes, sensors, or image sources with either Meta-Album or ImageNet-1k. If GEOM's accuracy there falls toward chance while the ImageNet-trained model stays strong, the claimed diversity-driven generalization is confined to domains near the curated collection; if a same-size randomly assembled collection reproduces the gains instead, curation is not the active ingredient.

Watch

Extended reading notes

Core claim

GEOM casts few-shot classification as a non-causal sequence modeling problem: a task is a sequence of labeled context images followed by an unlabeled query image, and a transformer encoder must predict the query's label by attending to the context. Meta-trained on the small, domain-separated datasets of Meta-Album, GEOM generalizes to completely held-out domains, and its accuracy rises monotonically as more datasets (hence more classes) are added to the training collection. The paper's central empirical discovery is that diversity of classes and domains, not raw image count, drives out-of-domain generalization: moving from 20-class Micro to full-class Mini datasets yields the largest gains, while multiplying images per class adds little. In the sequential setting, GEOM-S accumulates knowledge across 30 datasets with no rehearsal and ends with positive backward transfer; curricula that order datasets by difficulty outperform both random and domain-grouped orders; and in the unsupervised setting GEOM-U, trained only on pseudo-labeled augmentations, beats its ImageNet-trained counterpart on nearly every domain.

Load-bearing premise

The load-bearing premise is that a curated collection built from ten chosen domains (Meta-Album) is a fair stand-in for unseen tasks in general, and that the ImageNet-1k baseline is a fair comparison even though many of its classes overlap with the Meta-Album test domains — an overlap the paper itself documents and partially sets aside when claiming advantages.

Editorial extensions

If this is right

A curated multi-domain collection matches the large-scale baseline: GEOM ties or beats GEOM-IN on domains with little class overlap with ImageNet-1k, and only lags where overlap enables memorization.
Class and domain diversity, not image count, drives cross-domain gains: moving from Micro to Mini (more classes) gives the largest accuracy jump, while the fivefold image increase of Extended adds little.
Streaming training without rehearsal avoids catastrophic forgetting: with a static 20-epoch-per-dataset schedule, backward transfer turns positive across the 30-dataset stream, and difficulty-ordered curricula (hard-to-easy by transfer-learning score, easy-to-easy by optimal-transport distance) beat the domain-grouped order.
The curation advantage survives without labels: GEOM-U, trained on pseudo-labeled augmentations from Meta-Album, outperforms the same learner trained unsupervised on ImageNet-1k on all but the overlap-heavy Large Animals domain.
Meta-trained in-context learners tolerate corrupted demonstrations: accuracy stays high until more than a quarter of the context labels are wrong, in line with the view that they use demonstration structure rather than memorized input-label pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test the paper leaves to future work is to hold total images fixed while varying the number of classes; if the Micro-to-Mini gain reproduces fully, collection designers should spend their budget on more classes rather than more examples per class.
If part of GEOM-IN's advantage is overlap-driven memorization, as the paper's own analysis suggests, then leave-one-domain-out protocols like this one are a more honest yardstick for genuine generalization than accuracy on benchmarks whose classes already appeared in pretraining.
The modular structure implies an untested practical recipe: each dataset owner could contribute a small labeled or unlabeled corpus, with domains swapped in and out without retraining. The paper motivates this for quality control and unlearning but does not evaluate a distributed setting.
The label-noise results raise the testable question of whether these learners can train directly on noisy annotation streams; the paper corrupts up to half the context labels but does not test fully poisoned or wrong-class label sets.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, this is a well-executed empirical study, not a new method: GEOM is CAMeLU's architecture with Meta-Album as training data, and the main comparison is an ablation of training data using the same pipeline. Second, the abstract's claim that curated small datasets achieve 'comparable performance' with a single large dataset is conditional in ways the paper mostly acknowledges but does not fully resolve.

What's actually new: a systematic LOO evaluation across all ten Meta-Album domains, a clean class-count vs image-count experiment showing that more classes (not more images per class) drives OOD generalization, a sequential setting with positive backward transfer, curriculum ordering comparisons (TL- and OT-based), and an unsupervised variant. They report means and standard deviations over three runs, and the code is public. That is real, reproducible evidence.

Where it wobbles: the GEOM-vs-GEOM-IN comparison, which carries the 'single large-scale dataset' claim, is weakened on both sides. The overlap analysis in Appendix B.2 matches class names and CLIP-similar label strings; it never checks whether the actual images overlap. So when GEOM-IN wins big on Large Animals and Human Actions, the paper calls it 'memorization rather than true generalization,' but it could be genuine transfer from many images of the same classes. Conversely, since both variants share the ImageNet-pretrained ResNet-50 backbone, GEOM's 'test domain completely excluded from training knowledge' is only true for the meta-training episodes, not for the backbone's pretraining. This doesn't sink the paper—GEOM vs GEOM-M is unaffected by ImageNet overlap and still favors per-domain training—but it should temper the abstract's phrasing.

The bigger gap is the absence of external few-shot baselines. Everything is GEOM trained on different data; there is no MAML, ProtoNet, or Meta-Dataset comparison. That makes it hard to say whether the curated collection is good in absolute terms.

Bottom line: useful for anyone designing data mixtures for ICL or studying data curation for few-shot learners. It deserves a serious referee, and a revision that adds an image-level overlap check and one external baseline would make the claim much sturdier. I'd send it out.

Referee Report

3 major / 6 minor

Summary. The paper proposes training a transformer-based in-context learner (GEOM) on a collection of multiple small-scale, domain-specific datasets from Meta-Album, rather than on a single large-scale dataset such as ImageNet-1k. It evaluates this paradigm in three settings: a supervised leave-one-out (LOO) scenario, a sequential/continual scenario, and an unsupervised scenario. The authors claim that the curated collection improves out-of-domain generalization and achieves performance comparable to models trained on a single large-scale dataset, while offering modularity and privacy benefits.

Significance. The paper contributes a substantial body of experiments: three training scenarios, multiple baselines, means and standard deviations over three runs, and publicly available code. If the headline claim were supported, the result would be practically relevant. However, the central GEOM-versus-GEOM-IN comparison is confounded by the shared ImageNet-1k pre-trained backbone and by a label-based rather than image-based overlap analysis, so the significance is conditional on a thorough revision of the experimental interpretation.

major comments (3)

[§3.3, §5, Abstract] The LOO evaluation is described as a 'controlled environment, where the test domain is completely excluded from the training knowledge' (Abstract; §5). This is not the case: the feature extractor fψ is a ResNet-50 pre-trained on ImageNet-1k (§3.3), and Appendix B.2 shows that ImageNet-1k contains classes from the left-out domains (e.g., 120/120 overlap for dataset 44298). Hence GEOM's feature representation has already been trained on images from the test domain, and the LOO claim is false. The comparison with GEOM-IN is therefore confounded; a test of the claim requires training the feature extractor from scratch or measuring the effect of ImageNet pre-training on the LOO evaluation.
[§5.1, Appendix B.2] The overlap analysis in Appendix B.2 computes exact class-name matches and CLIP cosine similarity of label strings (Fig. 11). This measures concept/name overlap, not image-level duplication. The statement in §5.1 that GEOM-IN's strong results in Large Animals and Human Actions are 'relying on memorization rather than true generalization' is therefore unsupported: GEOM-IN may simply be transferring from different images of the same classes. To justify discounting those results, the authors need an image-level overlap check (e.g., near-duplicate detection) or a comparison restricted to non-overlapping classes.
[§5.1, Fig. 2, Tab. 7] The abstract's claim of 'comparable performance with models trained on a single large-scale dataset' is not supported by the aggregate data in Tab. 7. On datasets 44298, 44305, 44284, 44291, 44301, 44288, 44294, and 44304, GEOM-IN outperforms GEOM by large margins (e.g., 98.49 vs 63.03 on 44298; 88.91 vs 72.66 on 44284). These are precisely the datasets that the paper's own overlap analysis (Fig. 2, right) flags as having high class overlap with ImageNet-1k. If the authors wish to claim comparability, they must report results separately for overlap and non-overlap classes, or exclude overlapping classes from the evaluation; the current reporting hides the fact that the overall comparability is driven by non-overlap domains.

minor comments (6)

[§7, Tab. 17] The statement that 'The only cases where GEOM-U underperforms CAMeLU are in the Large Animals domain' is inaccurate; Tab. 17 shows GEOM-U also underperforms on 44284 (89.20 vs 90.52), 44291 (77.27 vs 79.82), and 44294 (74.97 vs 76.62).
[Eq. (4)] The term 'p-Warssertein' is a typo for 'p-Wasserstein'; the notation in the definition of dZ uses p and W_p inconsistently and should be clarified.
[§6.3.2] The curriculum name 'Hard-to-Hasy' should be 'Hard-to-Hard'.
[Appendix B.1] The appendix refers to the external evaluation datasets 'as described in Section 3.3'; the actual description appears in Section 5.3.
[Fig. 2 (right)] The overlap counts are listed in an order that does not match the left-panel dataset ordering, making the figure hard to parse.
[§5.3] The conclusion that class diversity is more important than image count is based on comparing Micro, Mini, and Extended, which differ in both class count and image count simultaneously, and Extended excludes OCR; a controlled ablation (e.g., subsampling classes or images) would strengthen this claim.

Circularity Check

2 steps flagged · score 4.0 of 10

Headline 'comparable performance' claim is conditioned on a fitted class-label overlap classification that discounts ImageNet-trained wins as 'memorization,' with no image-level check; method recipe is inherited from the authors' CAMeLU.

fitted input called prediction [Sect. 5.1 and App. B.2 (overlap analysis); headline claim in Abstract]
"When comparing GEOM to GEOM-IN, GEOM achieves superior or comparable performance in datasets with minimal class overlap between Meta-Album and ImageNet-1k. In domains with significant class overlap, such as Large Animals and Human Actions, GEOM-IN benefits from the knowledge acquired during training, relying on memorization rather than true generalization. [Abstract: 'achieving comparable performance with models trained on a single large-scale dataset.']"

The abstract's 'comparable performance with models trained on a single large-scale dataset' is established in Sect. 5.1 by discounting the domains where GEOM-IN wins (e.g., 44298: 63.03 vs 98.49; 44284: 72.66 vs 88.91) as 'memorization rather than true generalization.' That discounting is licensed by an overlap analysis (App. B.2) that matches only class-name strings and CLIP label embeddings, with a threshold 'set at 0.83' that is itself fitted from the per-dataset 90th percentiles of those similarity distributions; no image-level overlap is checked. The verdict 'GEOM achieves superior or comparable performance in datasets with minimal class overlap' is parameterized by the same fitted measure used to discard counter-evidence (and Sect. 7 repeats the move for GEOM-U vs CAMeLU).
other [Abstract and Sect. 5 intro ('test domain is completely excluded'); Sect. 3.3 (ImageNet-1k pre-trained backbone)]
"Firstly, we show the performance in a controlled environment, where the test domain is completely excluded from the training knowledge. ... we project input images to the feature space with a ResNet-50 (He et al., 2016) feature extractor fψ pre-trained on ImageNet-1k (Deng et al., 2009)"

This premise is contradicted by the paper's own apparatus. The abstract's 'controlled environment, where the test domain is completely excluded from the training knowledge' is violated for every variant, including GEOM, because the frozen ResNet-50 feature extractor is pre-trained on ImageNet-1k, and the paper's own App. B.2/Fig. 11 quantifies substantial class overlap between ImageNet-1k and the test domains later called leaky (Large Animals, Human Actions). Thus the promised exclusion fails through the same overlap channel invoked against GEOM-IN, and no image-level check is offered to separate true generalization from shared-backbone transfer on either side.

full rationale

The raw accuracies are genuine measurements, and I found no equation-level reduction in which a predicted quantity is computed from its own inputs; the paper is therefore not circular in the strongest sense. The most consequential structural issue is that the central 'comparable performance' verdict is conditioned on a fitted, label-level overlap classification (App. B.2) that decides which GEOM-IN wins count as 'memorization' and are discarded, with no image-level leakage check; and the same classification shows the shared ImageNet-1k pre-trained backbone already puts test-domain classes into GEOM's own feature extractor, undermining the 'completely excluded' LOO framing. These two flagged steps (fitted_input_called_prediction; other) make the abstract's unconditional claim partially reducible to the paper's own framing, although they are validity and fairness gaps as much as circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its claims rest on several practical design choices: the exact task configuration (5-way 5-shot), the sampling scheme, the fixed 20-epoch budget per dataset, the mixup hyperparameters, and the CLIP overlap threshold. These choices are reasonable but not systematically varied, so the generality of the central claim is not fully tested.

free parameters (5)

Task sampling probability p(Da) proportional to |Da| = p(Da) = |Da| / sum |Da|
Introduced heuristically for task sampling in Sect. B.3; no analysis of its effect on the multi-dataset generalization claim.
N and K (5-way, 5-shot) = N=5, K=5
Fixed task configuration, not fitted, but central to the cross-domain generalization conclusions; no sweep over N or K.
Training epochs/iterations = 300k iterations, 600 epochs, 20 epochs per dataset in static sequential setting
Chosen by practice; directly influences sequential/curriculum results, especially the 20-epoch static allocation.
Beta distribution parameters for mixup = alpha=1, beta=1, lambda in (0,0.5)
Adopted from CAMeLU without independent justification in this setting.
CLIP similarity threshold for overlap analysis = 0.83 (90th percentile median)
Used to quantify ImageNet-1k overlap in Appendix B.2; defined from the data, not a standard threshold.

assumptions (3)

domain assumption Meta-Album LOO evaluation is a valid proxy for cross-domain generalization in ICL.
Sect. 5 assumes that leaving out a whole Meta-Album domain yields a fair OOD test; this depends on the Meta-Album benchmark faithfully representing real domain shifts.
domain assumption A ResNet-50 pretrained on ImageNet-1k is a neutral feature extractor for comparing GEOM and GEOM-IN.
Sect. 3.3 uses a frozen ImageNet pretrained backbone for both models, which may bias against models trained on smaller collections and is acknowledged only in the overlap analysis.
domain assumption Backward transfer evaluation with 20 epochs per dataset is a meaningful measure of forgetting.
Sect. 6.2 computes BWT after a fixed 20 epochs, not after convergence; this is a design choice that may understate forgetting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Meta-Learning Transformers to Improve In-Context Generalization." pith.science (2026). https://pith.science/paper/NT4N4TN7

@misc{pith2026250705019,
  author       = {Pith},
  title        = {Pith review of: Meta-Learning Transformers to Improve In-Context Generalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NT4N4TN7}},
  note         = {Machine review of arXiv:2507.05019}
}

read the original abstract

In-context learning enables transformer models to generalize to new tasks based solely on input prompts, without any need for weight updates. However, existing training paradigms typically rely on large, unstructured datasets that are costly to store, difficult to evaluate for quality and balance, and pose privacy and ethical concerns due to the inclusion of sensitive information. Motivated by these limitations and risks, we propose an alternative training strategy where we leverage a collection of multiple, small-scale, and domain-specific datasets. We empirically demonstrate that the increased quality and diversity of such data improve the generalization abilities of in-context learners beyond their training domain, while achieving comparable performance with models trained on a single large-scale dataset. We investigate this paradigm by leveraging meta-learning to train an in-context learner on the Meta-Album collection under several settings. Firstly, we show the performance in a controlled environment, where the test domain is completely excluded from the training knowledge. Secondly, we explore the robustness of these models to forgetting in a continual scenario where the information is accessible for a limited time. Finally, we explore the more challenging unsupervised scenario. Our findings demonstrate that transformers still generalize for in-context prediction when trained on a curated dataset collection while offering advantages in modularity and replaceability.

Figures

Figures reproduced from arXiv: 2507.05019 by the authors.

**Figure 2.** (Left) Accuracy comparison between GEOM, GEOM-M, and GEOM-IN for all the Meta-Album [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Comparison of GEOM training only on datasets from the first release ( [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (13 more)

**Figure 4.** Figure 4: Validation performance of GEOM while trained on the Mini and Extended size of Meta-Album. The Mini size achieves peak performance early but declines due to overfitting, while the Extended size shows steady improvement over longer training periods, indicating the impact…

**Figure 5.** Figure 5: Relative performance of GEOM-S using a static and proportional approach for assigning training epochs to each dataset compared to the offline baseline, where all the datasets are available simultaneously. The relative accuracy is calculated as the difference between th…

**Figure 6.** Figure 6: Heatmap showing the performance difference, used to compute the BWT, on datasets from the first release of Meta-Album Mini (one per domain), training GEOM-S with the static approach and the domain-based streaming scenario described in Sect. 6.1. Each entry er,c repres…

**Figure 7.** Figure 7: Relative accuracy of the E2H and H2E curricula compared to the domain-based order baseline. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Comparison of learning trends for E2H and [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Relative accuracy of the E2E, H2H, and Switch curricula compared to the domain-based order [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

**Figure 10.** Figure 10: Relative accuracy of GEOM-U compared to CAMeLU in the unsupervised scenario computed [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

**Figure 12.** Figure 12: Cosine similarity distribution between CLIP (Radford et al., 2021) embeddings of ImageNet-1k labels and the Meta-Album labels that have no exact match. Horizontal bars represent the 90th percentile of similarity values for each dataset. Datasets from the Small Anima…

**Figure 13.** Figure 13: Comparison of GEOM training only on datasets from the first release ( [PITH_FULL_IMAGE:figures/full_fig_p033_13.png]

**Figure 14.** Figure 14: Relative validation accuracy of different TL curricula when the feature extractor is trained from [PITH_FULL_IMAGE:figures/full_fig_p036_14.png]

**Figure 15.** Figure 15: Heatmap representing the dataset similarity for all datasets in Meta-Album Mini computed with [PITH_FULL_IMAGE:figures/full_fig_p037_15.png]

**Figure 16.** Figure 16: Dataset similarity for all datasets in Meta-Album Mini computed with OTDD (Alvarez-Melis [PITH_FULL_IMAGE:figures/full_fig_p038_16.png]

**Figure 17.** Figure 17: Model robustness to input-label mapping perturbations by varying the proportion of correctly [PITH_FULL_IMAGE:figures/full_fig_p039_17.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

4 extracted references · 3 canonical work pages

[1]

A pre-processing step is applied to remove special characters and convert all names to lowercase, ensuring consistency in the comparison

Label matching: Class names in Meta-Album and ImageNet-1k are compared by identifying matching words. A pre-processing step is applied to remove special characters and convert all names to lowercase, ensuring consistency in the comparison

work page
[2]

Scores above a certain threshold are considered indicative of overlap

Concept similarity: Using CLIP (Radford et al., 2021) embeddings, we calculate cosine similarity scores between Meta-Album and ImageNet-1k labels to identify overlapping concepts. Scores above a certain threshold are considered indicative of overlap. The threshold is computed considering the distribution of cosine similarity values for each dataset, ident...

work page 2021
[2009]

doi: 10.1109/CVPR.2009.5206848. 19 Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. Zhengxiao Du, Aohan Zeng,...

arXiv 2009
[2025]

Benchmarking General-Purpose In-Context Learning

URL https://openreview.net/forum?id=Jprs1v2wPA. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016. 23 Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds- 200-2011 dataset. Technic...

work page Pith review arXiv 2016

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

A pre-processing step is applied to remove special characters and convert all names to lowercase, ensuring consistency in the comparison

Label matching: Class names in Meta-Album and ImageNet-1k are compared by identifying matching words. A pre-processing step is applied to remove special characters and convert all names to lowercase, ensuring consistency in the comparison

work page

[2] [2]

Scores above a certain threshold are considered indicative of overlap

Concept similarity: Using CLIP (Radford et al., 2021) embeddings, we calculate cosine similarity scores between Meta-Album and ImageNet-1k labels to identify overlapping concepts. Scores above a certain threshold are considered indicative of overlap. The threshold is computed considering the distribution of cosine similarity values for each dataset, ident...

work page 2021

[3] [2009]

doi: 10.1109/CVPR.2009.5206848. 19 Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. Zhengxiao Du, Aohan Zeng,...

arXiv 2009

[4] [2025]

Benchmarking General-Purpose In-Context Learning

URL https://openreview.net/forum?id=Jprs1v2wPA. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016. 23 Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds- 200-2011 dataset. Technic...

work page Pith review arXiv 2016