In Context Learning with Vision Transformers: Case Study

Alex Proshkin; Antony Zhao; Fergal Hennessy; Francesco Crivelli

REVIEW 3 major objections 5 minor 12 references

In Context Learning with Vision Transformers: Case Study

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a decoder-only transformer can infer image mappings in-context from a handful of downscaled grayscale examples, matching or beating models trained from scratch.

desk verdict A plausible extension of Garg et al. to image inputs, but the central 'matches or exceeds' claim rests on an unverified baseline budget and no quantitative reporting. read the letter →

arxiv 2505.20872 v1 pith:P3AAYGJL submitted 2025-05-27 cs.CV cs.AIcs.LG

Antony Zhao , Alex Proshkin , Fergal Hennessy , Francesco Crivelli This is my paper

classification cs.CVcs.AIcs.LG

keywords in-contextlearningvisiontransformersfew-shotimageregressiondecoder-onlytransformerconvolutionalneuralnetworksCIFAR-10curriculumtraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that in-context learning, previously demonstrated on vector-valued functions such as linear maps and small neural networks, transfers to the image domain. The authors train a decoder-only transformer jointly with an image encoder (a CNN or a ViT) on prompt sequences of downscaled CIFAR-10 images paired with outputs of random image-to-vector functions. They report that with only a handful of context pairs the model can infer the mapping—linear, small convolutional, or small ViT—on 8×8 grayscale images, approaching the least-squares solution for linear targets and matching or exceeding from-scratch-trained MLP, CNN, and ViT baselines on nonlinear targets. If the paper is right, a transformer can adapt to a new image function at inference time without gradient updates, a capability relevant to few-shot and small-data vision settings.

What carries the argument

The central object is a decoder-only GPT-2-style transformer fused with a learned image encoder (a CNN in E1, a ViT in E2–E4), trained end-to-end from random initialization. The prompt is the ordered sequence $(x_1, f(x_1), x_2, f(x_2), \ldots, x_k, f(x_k))$, with causal masking so each predicted $f(x_i)$ can only use preceding context pairs; the MSE loss is weighted to emphasize the latter half of the sequence. A curriculum grows the visible image dimension $d$ from 2 to 8 and the number of prompt samples $n = kd + 1$ as training proceeds.

What would settle it

Train the MLP, CNN, and ViT baselines from Section 5.2 to convergence (or over a sweep of step budgets and learning rates) on the same random two-layer CNN and ViT targets and compare MSE with the in-context learner; if any converged baseline clearly wins, the 'matches or exceeds' claim fails. A second check is to run the same protocol on semantically meaningful image mappings such as edge detection or grayscale-to-color functions, and see whether the in-context learner still beats the mean baseline.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a decoder-only transformer can perform in-context regression over image inputs: given a sequence of image–output pairs where the outputs come from a randomly initialized convolutional network or ViT, the model learns to predict the output for a query image without fine-tuning. Across the four experiments the authors find that the in-context learner approaches the pseudo-inverse solution for linear image functions, beats nearest-neighbour and mean baselines, and matches or exceeds fresh models trained from scratch on the same support examples, most clearly when context is scarce. The claimed scope is deliberately narrow: simple convolutional mappings on small grayscale images, with sample complexity that grows as image dimensionality increases.

Load-bearing premise

The comparative claim that the in-context learner matches or exceeds from-scratch models presumes that a fixed 5000-step Adam budget represents full training for those baselines, and that random-weight, semantically meaningless target functions are representative of the image mappings of interest.

Editorial extensions

If this is right

If the central claim holds, a transformer can adapt to a new image-to-scalar function from context alone, without gradient updates, making in-context learning a viable few-shot mechanism for image regression.
On linear image functions the in-context learner approaches the pseudo-inverse least-squares solution and beats 3-nearest-neighbours and mean baselines, so in-context learning on images can track an optimal linear estimator.
On nonlinear targets (a two-layer CNN and a ViT), a handful of context examples suffices for the in-context learner to match or beat from-scratch-trained MLP, CNN, and ViT baselines trained on the same examples.
Larger input dimensionality requires more context and slows convergence, implying that in-context learning on images carries an intrinsic sample-complexity cost as images grow.
The image encoder (CNN or ViT) and the sequence model are trained jointly from random initialization without pretraining, so the reported in-context behavior is not inherited from a large pretrained backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, a natural testable extension is to use fully trained models as targets instead of random-weight ones; if the transformer can in-context learn a class of trained image models, that would connect in-context learning directly to practical few-shot adaptation.
Beyond the paper, the curriculum over visible image dimension suggests a scaling recipe: the same kind of progressive curriculum could be applied over patch embeddings or feature channels for larger or multi-channel images, potentially reducing the number of examples needed.
Beyond the paper, because all target functions are random-weight and semantically meaningless, the demonstration concerns low-level pixel-to-output mappings; whether in-context learning extends to semantically meaningful image functions such as object classification or segmentation is left open by this study.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper does something genuinely new: it takes the Garg et al. in-context regression setup and moves it from random vectors to 8x8 CIFAR images, with a CNN or ViT encoder feeding a GPT-2-style decoder that predicts the outputs of random linear, CNN, and ViT functions. That is a real extension, and the experimental design is sensible. The linear experiments are benchmarked against the pseudo-inverse solution, which is the right external reference, and the nonlinear experiments are compared to separately trained from-scratch models. The authors are also honest that compute limited them to simple models and small images, and they do not oversell the scope. On the merits, the direction is worth exploring.

The soft spots are real and mostly in the reporting. The prose contains no actual MSE numbers; everything lives in figures that are not readable from the text. No seeds, no error bars, and the early stopping differs across experiments (500k, ~300k, 80k steps). The code link is a bare mention with no URL. The Figure 4 "out-of-distribution" label overstates what is a within-class shift. These are all fixable, but they make it impossible to verify the claims from the text alone.

The load-bearing problem is baseline fairness in Section 5.2. The from-scratch MLP/CNN/ViT baselines get a fixed 5000 Adam steps with no convergence check, while the in-context learner is trained for 80k or 300k steps on a curriculum of many tasks. If the baselines are undertrained, the "matches or exceeds" claim is an artifact. The stress-test note is right to flag this; it is the one assumption that, if false, would change the main conclusion. The authors do not report baseline training losses or convergence curves, so the concern cannot be dismissed from the text.

The linear results are on firmer ground because the comparison is to least-squares, not to a trained baseline, and those results are consistent with Garg et al. But the nonlinear claims are the interesting part, and those are exactly where the baseline budget matters. This is an externally testable issue, not a deep philosophical flaw.

Who is this for? Researchers working on few-shot vision or ICL for non-language modalities. It is a case study, not a definitive result, and it would benefit from a serious referee who insists on numbers, error bars, and a convergence check for the baselines. The core idea deserves referee time, but the paper as submitted is not ready to be taken at face value.

Referee Report

3 major / 5 minor

Summary. The paper studies in-context learning (ICL) for image inputs with a decoder-only GPT-style transformer combined with CNN or ViT encoders. Random linear, convolutional, and ViT target functions are applied to downscaled 8×8 CIFAR-10 images, and the model is trained on prompt sequences of (x_i, f(x_i)) pairs to predict f(x_i) via masked MSE loss. Four experiments (E1–E4) compare the ICL model against least-squares, 3-NN, mean predictors, and from-scratch baselines. Section 7 concludes that a decoder-only transformer can in-context learn simple convolutional mappings on small images with few examples, and Section 5.2 claims parity or superiority over from-scratch models on the support set.

Significance. The paper addresses a timely question—whether ICL, established for vectors in Garg et al., transfers to image inputs with structured encoders. The experimental design is largely sound: linear experiments are benchmarked against the least-squares solution, and nonlinear experiments against independently trained MLP/CNN/ViT baselines, providing an external reference point. The use of curriculum training and loss weighting is plausible. If the central claims are verified, the results would be a useful case study for the ICL community. However, the current reporting (no numerical values, no seeds, no baseline convergence checks) prevents verification, and the baseline-fairness issue is material to the main comparative claim.

major comments (3)

[Section 5.2, Figures 7–8, Appendix Table 2] The central comparative claim—that the in-context learner 'matches or exceeds fresh models trained from scratch (MLP, CNN, ViT) on the same support examples'—is not supported as reported because the from-scratch baselines use a fixed 5000-step Adam budget (lr=1e-3, per Appendix Table 2) with no convergence check, no validation-based early stopping, and no budget sweep. The ICL model is trained for ~300k steps (E3) or 80k steps (E4) on a curriculum of many tasks, giving it a far larger effective compute budget. If the baselines are undertrained on their 41- or 65-example support sets, the comparison is biased in favor of the ICL learner. The paper should report baseline training and validation curves, or run a budget sweep (e.g., 5k, 50k, 500k steps) and show that performance plateaus before making the comparative claim.
[Sections 5.1–5.2, Figures 3–8] The manuscript contains no numerical MSE values, no error bars, and no standard deviations across seeds. The prose statements that the transformer 'approaches the pseudo-inverse solution' (E1/E2) and 'matches or exceeds' baselines (E3/E4) rest entirely on the figures, whose axes and captions do not report sample sizes, seed counts, or error statistics. Please provide tables of mean±std MSE over at least three seeds and state how many target functions were sampled per experiment. Without this, the magnitude and reliability of the reported ICL effects cannot be evaluated from the text.
[Section 4, Eq. (1)] The method description is insufficient for reproduction. For each experiment, the paper does not specify the value of k in n = kd + 1 (only 'such as 5 or 8'), the increment schedule for d, or the exact form of the loss weighting mask. Furthermore, Eq. (1) overloads k: it is used both as the prompt-length multiplier in Section 4 and as the index in the weighting formula '(2k/n)^2 for k=1,2,...,n'. Please give a precise specification per experiment (E1–E4) and use distinct notation for the index.

minor comments (5)

[Section 6] The statement 'The code used to generate this document is available on Github' provides no URL or repository name; please make the code link explicit.
[References] Several entries in the bibliography are not cited in the main text (e.g., [5], [9], [10], [11], [12]), and Steiner et al. (2021) is discussed in Related Work but does not appear in the reference list; please reconcile the citations.
[Section 7] The conclusion refers to 'small grayscale images,' but the method described in Section 4 operates on 8×8 downscaled CIFAR-10 images, which are color; please clarify whether images were converted to grayscale or correct the wording.
[Figures 3–8] Figure captions are minimal; they do not state the metric (MSE), the number of seeds or target functions, or the context sizes. Adding this information would make the figures self-contained.
[Appendix Figures 9–12] The loss curves in the appendix are not referenced in the main text and do not include baseline training curves; if they are meant to document convergence, they should be integrated into the relevant experiment section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the in-context learner is evaluated against external optima (pseudo-inverse) and independent from-scratch baselines on held-out functions.

full rationale

The paper's derivation chain is self-contained and externally anchored. In E1/E2 (Section 5.1), the linear in-context learner is compared to the pseudo-inverse/least-squares solution and to nonparametric baselines; the pseudo-inverse is an external optimum computed from the context, not a quantity fitted into the transformer, so approaching it is a meaningful empirical result. In E3/E4 (Section 5.2 and Figures 7-8), the nonlinear in-context learner is compared to MLP/CNN/ViT models trained from scratch on the same support examples; those baselines are independent of the transformer's training and their MSE is not used to define the learner's predictions. Although the fixed 5000-step Adam budget without a convergence check could unfairly flatter the ICL model, that is an experimental-fairness/correctness concern, not a circular reduction: baseline underperformance is not an input to the ICL learner. The training protocol samples fresh random functions from the target class and evaluates on held-out functions (standard ICL practice), so the target is not defined in terms of the model's outputs. The loss weighting mask in Section 4 is a training hyperparameter, not a fitted prediction. No load-bearing self-citation appears: the references (Garg et al., Min et al., etc.) are external prior work, and no uniqueness theorem is imported. Section 7 explicitly limits claims to simple models and small images, a scope restriction rather than a circular device. Accordingly, no step reduces to its own input, and the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the ICL training recipe imported from Garg et al. (2023): a decoder-only transformer, causal masking, curriculum over input dimension d, and prompts of length n = kd + 1. On top of that the paper adds a hand-chosen quadratic loss weighting mask, hand-set learning rates, per-experiment early stopping, and a fixed 5000-step Adam budget for from-scratch baselines; none of these are derived or ablated. The empirical claims assume 8x8 grayscale CIFAR-10 retains the structure needed to test image functions, that random-weight CNN/ViT targets form learnable ICL function classes, and that the baseline budget fairly represents from-scratch performance. No new entities (particles, forces, dimensions) are postulated; the only object is the trained ICL model itself.

free parameters (5)

Curriculum input-dimension schedule = d = 2 through 8, incremented every 5000 updates
Section 4: the schedule is chosen by hand to start from low-dimensional masked blocks; the increment rate and endpoint are not derived.
Loss weighting mask = (2k/n)^2 over prompt positions k = 1..n
Section 4: introduced because it 'improved performance on the final query prompt'; the quadratic form is a hand choice, not derived, and the equation is garbled in the preprint.
Prompt length multiplier k = 5 or 8
Section 4: prompts use n = kd + 1 samples with k set to 'some value such as 5 or 8'; the value is a free choice.
Baseline from-scratch training budget = 5000 Adam steps at learning rate 1e-3
Section 5.2: all gradient-descent baselines get a fixed 5000-step budget with no convergence check; the 'matches or exceeds' comparison is sensitive to this arbitrary cap.
Per-experiment early stopping = 500k steps (E1), ~300k (E2-E3), 80k (E4)
Section 5.1: training is halted at a loss plateau with no common criterion, so cross-experiment comparisons mix model capability with training budget.

assumptions (5)

standard math Least-squares (pseudo-inverse) optimality is the correct external benchmark for linear in-context regression
Imported from Garg et al. (2023) and used for E1/E2; the claim that the ICL model 'approaches the pseudo-inverse solution' presumes this optimality reference.
domain assumption Downsampled 8x8 grayscale CIFAR-10 images retain enough structure for the studied function classes
Section 4 downscales 32x32 to 8x8 and z-scores; the paper asserts there are 'inherent patterns in the image that could be interesting to study' without demonstrating what structure survives downscaling.
domain assumption Random-weight CNN and ViT target functions form learnable in-context function classes
Section 5.2 freezes random 2-layer CNN and random 4-layer ViT targets; the paper assumes their outputs are predictable from context, an empirical assumption the experiments are designed to test.
ad hoc to paper The curriculum schedule and quadratic loss weighting are sufficient for in-context learning to emerge
Section 4 introduces both as training choices ('improved performance on the final query prompt'); neither is derived nor ablated, so the results are conditional on these choices.
standard math GPT-2 decoder-only and ViT/CNN encoder behavior is as described in the cited prior literature
Section 4 assumes the decoder-only GPT-2 recipe and the CNN/ViT encoder recipes behave as in the cited literature (Olsson et al., Dosovitskiy et al.) without re-derivation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of In Context Learning with Vision Transformers: Case Study." pith.science (2026). https://pith.science/paper/P3AAYGJL

@misc{pith2026250520872,
  author       = {Pith},
  title        = {Pith review of: In Context Learning with Vision Transformers: Case Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P3AAYGJL}},
  note         = {Machine review of arXiv:2505.20872}
}

read the original abstract

Large transformer models have been shown to be capable of performing in-context learning. By using examples in a prompt as well as a query, they are capable of performing tasks such as few-shot, one-shot, or zero-shot learning to output the corresponding answer to this query. One area of interest to us is that these transformer models have been shown to be capable of learning the general class of certain functions, such as linear functions and small 2-layer neural networks, on random data (Garg et al, 2023). We aim to extend this to the image space to analyze their capability to in-context learn more complex functions on the image space, such as convolutional neural networks and other methods.

Figures

Figures reproduced from arXiv: 2505.20872 by the authors.

**Figure 2.** An example of the masking process, where we take one of the images and then zero out (which is [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Linear regression results (E1 and E2). Lower is better. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (9 more)

**Figure 4.** Figure 4: Out-of-distribution linear convolution results. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Prediction error on convolutional targets (E3). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Prediction error on ViT targets (E4). 6 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

**Figure 7.** Figure 7: E3 vs. gradient-descent-trained baselines. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

**Figure 8.** Figure 8: E4 vs. gradient-descent-trained baselines. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

**Figure 9.** Figure 9: Loss curves from training experiment 1 (CNN + Transformer on linear functions) over 500k epochs. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]

**Figure 10.** Figure 10: Loss curves from training experiment 2 (ViT + Transformer on linear functions) over 300k epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

**Figure 11.** Figure 11: Loss curves from training experiment 3 (ViT + Transformer on convolutional networks) over 300k [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]

**Figure 12.** Figure 12: Loss curves from training experiment 4 (ViT + Transformer on ViTs) over 80k epochs. [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 5 canonical work pages

[1]

(2023).What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2023).What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. arXiv preprint arXiv:2208.01066.https://arxiv.org/abs/ 2208.01066

arXiv 2023
[2]

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022). Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?arXiv preprint arXiv:2202.12837.https://arxiv.org/abs/2202.12837

arXiv 2022
[3]

Self-Promoted Supervision for Few-Shot Transformer

Dong, B., Zhou, P., Yan, S., & Zuo, W. (2022).Self-Promoted Supervision for Few-Shot Transformer. arXiv preprint arXiv:2203.07057.https://arxiv.org/abs/2203.07057

work page Pith review arXiv 2022
[4]

Are Transformers More Robust Than CNNs?

Bai, Y., Mei, J., Yuille, A., & Xie, C. (2021).Are Transformers More Robust Than CNNs?arXiv preprint arXiv:2111.05464.https://arxiv.org/abs/2111.05464

work page Pith review arXiv 2021
[5]

(2022).In-context learning and induction heads

Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., et al. (2022).In-context learning and induction heads. Anthropic Transformer Circuits.https: //transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html

work page 2022
[6]

(2016).Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016).Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778

work page 2016
[7]

(2021).An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., et al. (2021).An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv preprint arXiv:2010.11929

arXiv 2021
[8]

P., & Ba, J

Kingma, D. P., & Ba, J. (2015).Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (ICLR). arXiv preprint arXiv:1412.6980

arXiv 2015

Show all 12 references

[9]

(2019).Decoupled Weight Decay Regularization

Loshchilov, I., & Hutter, F. (2019).Decoupled Weight Decay Regularization. International Conference on Learning Representations (ICLR). arXiv preprint arXiv:1711.05101

2019 arXiv
[10]

(2021).Training data-efficient image transformers & distillation through attention

Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2021).Training data-efficient image transformers & distillation through attention. International Conference on Machine Learning (ICML). arXiv preprint arXiv:2012.12877

2021 arXiv
[11]

(2021).Swin Transformer: Hier- archical Vision Transformer using Shifted Windows

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021).Swin Transformer: Hier- archical Vision Transformer using Shifted Windows. IEEE/CVF International Conference on Computer Vision (ICCV). arXiv preprint arXiv:2103.14030

2021 arXiv
[12]

M., Wang, X., Shu, R., & Urtasun, R

Ziko, I. M., Wang, X., Shu, R., & Urtasun, R. (2022).Vision Transformers for Few-Shot Learning: A Survey. arXiv preprint arXiv:2209.07864. 8 8 Appendix 8.1 Hyperparameters Embedding Size #Layers #Heads 256 12 8 Table 1: GPT-2 Model Experiment #Convolutional/ViT Layers #Channel...

2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

(2023).What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2023).What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. arXiv preprint arXiv:2208.01066.https://arxiv.org/abs/ 2208.01066

arXiv 2023

[2] [2]

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022). Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?arXiv preprint arXiv:2202.12837.https://arxiv.org/abs/2202.12837

arXiv 2022

[3] [3]

Self-Promoted Supervision for Few-Shot Transformer

Dong, B., Zhou, P., Yan, S., & Zuo, W. (2022).Self-Promoted Supervision for Few-Shot Transformer. arXiv preprint arXiv:2203.07057.https://arxiv.org/abs/2203.07057

work page Pith review arXiv 2022

[4] [4]

Are Transformers More Robust Than CNNs?

Bai, Y., Mei, J., Yuille, A., & Xie, C. (2021).Are Transformers More Robust Than CNNs?arXiv preprint arXiv:2111.05464.https://arxiv.org/abs/2111.05464

work page Pith review arXiv 2021

[5] [5]

(2022).In-context learning and induction heads

Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., et al. (2022).In-context learning and induction heads. Anthropic Transformer Circuits.https: //transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html

work page 2022

[6] [6]

(2016).Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016).Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778

work page 2016

[7] [7]

(2021).An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., et al. (2021).An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv preprint arXiv:2010.11929

arXiv 2021

[8] [8]

P., & Ba, J

Kingma, D. P., & Ba, J. (2015).Adam: A Method for Stochastic Optimization. International Conference on Learning Representations (ICLR). arXiv preprint arXiv:1412.6980

arXiv 2015