Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning

Fabio Ferreira

REVIEW 3 major objections 4 minor 18 references

Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Meta-learned selectors can pick and finetune pretrained models better than default pipelines.

desk verdict A solid cumulative dissertation whose vision chapters are empirically serious, but the abstract's 'outperform finetuning foundation models' claim leans on an unreleased, synthetic-only LLM experiment that needs real-QA confirmation. read the letter →

arxiv 2506.12161 v1 pith:MQXJJX75 submitted 2025-06-11 cs.LG stat.ML

Fabio Ferreira This is my paper

classification cs.LGstat.ML

keywords meta-learningautomatedmachinelearningpipelineselectionfinetuningsyntheticdataaugmentationself-supervisedreinforcement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The dissertation's central claim is that meta-learning — learning from prior tasks — can automate the two most tedious parts of deep learning practice: choosing which pretrained model to finetune for a new dataset, and deciding how to finetune it. Against the backdrop of hundreds of thousands of available pretrained models, it proposes zero-shot and few-shot surrogate models (ZAP and Quick-Tune) that rank pipelines from cheap dataset descriptions and partial learning curves, and reports that these outperform the winners of the AutoDL challenge and standard hyperparameter optimization baselines. In the second half, it argues that the same meta-learning tool can be turned on the data itself: meta-learned augmentation strategies improve self-supervised pretraining, and meta-learned synthetic environments and reward networks train reinforcement-learning agents more efficiently and robustly than the real environments. If these results hold, practitioners could replace manual model selection, hyperparameter tuning, and expensive real-world data collection with learned surrogates and synthetic generators. The strongest version of the claim, stated in the abstract, is that this approach can outperform finetuning foundation models.

What carries the argument

The load-bearing machinery is the meta-dataset paired with a learned surrogate. A meta-dataset records which pipeline (pretrained model plus finetuning hyperparameters) achieved what performance on which dataset; ZAP's cost matrix holds 525 by 525 evaluations, and Quick-Tune's holds learning curves for 24 models over 87 datasets. The surrogate is the object that carries transfer: ZAP's neural network trained with a pairwise ranking loss over pipelines seen as points in a geometric space, and Quick-Tune's deep-kernel Gaussian process that consumes partial learning curves together with a multilayer-perceptron cost estimator inside a cost-sensitive Expected Improvement acquisition function. For the synthetic-data half, the carrying mechanism is bi-level (inner and outer loop) optimization: an agent trains on the synthetic proxy in the inner loop, its performance on the real environment is measured in the outer loop, and the proxy parameters are updated to maximize that performance; the one-shot world model replaces the outer loop with a supervised in-context-learning objective on synthetically sampled trajectories.

What would settle it

Build a held-out set of image classification datasets that deliberately differs from the meta-training set, for example only medical or only very high-resolution satellite images, and run ZAP or Quick-Tune on it; if the meta-learned selector does not beat random pipeline selection and the single best pipeline on average, the transfer claim fails. For the reinforcement learning half, give the one-shot world model 1,000 transitions from a control environment with dynamics not covered by its synthetic prior; if the trained agent cannot reach competitive performance on that environment, the one-shot world model claim fails.

Watch

Extended reading notes

Core claim

The dissertation establishes a unified thesis: the combined algorithm-selection and hyperparameter-optimization problem (CASH), long studied for classical machine learning, can be extended to deep learning by meta-learning surrogates on large meta-datasets of pipeline evaluations. ZAP learns a zero-shot surrogate that maps dataset meta-features (resolution, number of classes, number of images) to a ranking over 525 deep learning pipelines, trained with a pairwise ranking objective; Quick-Tune instead meta-learns a deep-kernel Gaussian process performance predictor and a cost estimator, then runs cost-sensitive Bayesian optimization on partial learning curves to jointly select a pretrained model and its finetuning hyperparameters. On the data side, the dissertation claims that data augmentation is an underestimated lever in self-supervised learning, proposes adversarially selected hard views (Hard View Pretraining) as a single-task meta-learning augmentation strategy, and shows that bi-level meta-learning can produce synthetic environments and reward networks that act as proxies for reinforcement learning environments, as well as a transformer-based one-shot world model trained purely on synthetic priors. The paper's own empirical claims are that ZAP outperforms the ChaLearn AutoDL challenge winners, Quick-Tune outperforms default finetuning and state-of-the-art hyperparameter optimization across the Meta-Album benchmark, and the synthetic-proxy methods train agents more efficiently and robustly than the real environments.

Load-bearing premise

The methods assume the meta-datasets they train on are representative of the new datasets and tasks the system will actually face; if future tasks are far from the training-task distribution, the learned selectors and synthetic generators have no guarantee of transferring.

Editorial extensions

If this is right

Practitioners facing a new dataset could get a strong finetuning pipeline immediately from cheap dataset meta-features, with no exploratory model runs (ZAP), or within a user-specified time budget (Quick-Tune).
The same meta-learned selection machinery transfers across modalities: the dissertation reports it moves from image classification to finetuning large language models on synthetically generated question-answer datasets, beating random search and default finetuning.
A pool of diverse pretrained models, searched jointly with hyperparameters, can outperform finetuning a single large foundation model, including a 1-billion-parameter model finetuned with LoRA or linear probing.
Data augmentation, not training hyperparameters, is the dominant lever in self-supervised pretraining; adversarially selected hard views improve downstream performance and make models more robust to hyperparameter variation.
Meta-learned synthetic environments and reward networks can train reinforcement learning agents more efficiently, make them more robust to agent hyperparameters, and transfer to unseen agent algorithms; the one-shot world model extends this to multiple environments from a context of 1,000 transitions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the meta-dataset approach scales, the economics of applied deep learning could shift: instead of each practitioner re-tuning an existing model hub, shared surrogate models trained on broad meta-datasets could be reused across organizations, with the main remaining cost being the one-time collection of pipeline-evaluation data.
A natural stress test the dissertation does not run is out-of-distribution detection on meta-features: adding a fallback that recognizes when a new dataset lies outside the meta-training manifold, and switches to ordinary hyperparameter optimization, would make the automated pipeline robust to the weakest assumption.
The synthetic-environment result suggests a possible union with the world-model approach: use the cheap synthetic prior to initialize the search, then run the bi-level meta-learning loop on the real environment, a combination that could preserve both generality and hyperparameter robustness.
The language-domain result is currently limited to synthetically generated question-answer datasets; the same teacher-student framework could in principle be applied to code, reasoning, or instruction-tuning data, where synthetic generation already plays a large role.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, the real substance of this dissertation is the vision chapters — ZAP and Quick-Tune — and they are empirically solid. The meta-datasets are large (2,871 GPU days for ZAP, 32 GPU months for Quick-Tune), the evaluation protocols (leave-one-out, error bars, ablations) are careful, and the comparisons against AutoDL winners and HPO baselines are credible. Second, the abstract's headline claim that 'our approach can outperform finetuning foundation models' is not supported by what is actually inspectable. It leans on Chapter 8, the LLM finetuning work, which is the only chapter without code release and which is evaluated on synthetic QA datasets generated by Llama-3.1-70B, with a Llama-3.1 teacher as judge. That same-model-family training and scoring setup is exactly where an artifact would hide.

What is new here is not the individual methods — they were peer-reviewed at ICML, ICLR, and NeurIPS workshops — but the integrative framing: casting DL pipeline selection as CASH, the geometric pipeline space for the zero-shot surrogate, and the meta-learned cost/performance predictors in Quick-Tune. The background chapters on meta-learning and synthetic data are competent and useful. The dissertation also gets credit for releasing code and data for almost all chapters; Chapter 8 is the admitted exception.

Soft spots, in proportion. The Chapter 8 problem is the load-bearing one, because it is what makes the language-domain generality claim. If the claim were scoped to vision, the dissertation would be on much firmer ground. The representative-task-distribution assumption you flagged is real, but it is the standard meta-learning caveat, and the vision chapters actually test it under distribution shift (AutoDL challenge). The sharpest testable instance is Chapter 8's synthetic QA prior, where shift is most likely and least controlled. I'd also note the ZAP cost matrix is expensive to reproduce, but the authors acknowledge that in the limitations.

Who is this for? Practitioners who want a practical answer to model selection in a model hub, and researchers working on AutoML/meta-learning. The vision chapters deserve a serious referee. The dissertation as a whole is honest about what it does not cover, except that the abstract oversells the language part.

My recommendation: send it to peer review, but condition acceptance on either adding real-QA evaluation for Chapter 8 or scoping the abstract claim to the evaluated benchmarks. The underlying work is solid enough that the revision cost is modest.

Referee Report

3 major / 4 minor

Summary. This cumulative dissertation proposes to extend Automated Machine Learning to deep learning pipelines by meta-learning over prior tasks and by using synthetic data and data augmentation. Part II introduces ZAP, a zero-shot surrogate that selects a pretrained model and its finetuning hyperparameters from a 525-by-525 cost matrix of evaluated DL pipelines; Quick-Tune, a gray-box, cost-aware Bayesian optimization method that meta-learns performance and cost predictors from a 20,000-configuration meta-dataset; Quick-Tune-Tool, a software interface; and a language-domain extension that uses synthetic QA datasets to meta-learn finetuning configurations for LLMs. Part III contributes an empirical study of hyperparameters and data augmentation for self-supervised learning, Hard View Pretraining (HVP), Synthetic Environments and Reward Networks for RL, and a One-Shot World Model trained on a synthetic prior. The central claim is that meta-learned surrogates and synthetic data generators can automate model selection and finetuning and can outperform finetuning of large foundation models, with the vision chapters providing the strongest evidence.

Significance. If the results hold, the empirical contribution is substantial and unusually well resourced: the ZAP cost matrix required 2,871 GPU days, Quick-Tune's meta-dataset required 32 GPU months, and both flagship chapters use careful leave-one-core-dataset-out protocols, repeated seeded runs, and comparisons against strong baselines such as the AutoDL challenge winners, ASHA, BOHB, DEHB, and DyHPO. Code is released for all chapters except the LLM chapter, which is explicitly noted in Section 2.2 as the only work without a code release. The vision-side claims are credible and largely reproducible. The conceptual novelty is moderate, since the methods build on established algorithm selection and transfer-HPO ideas, but the scale and benchmarking are valuable. The language-domain extension, however, is not supported by evidence of comparable quality and currently underlies the abstract's cross-domain claim that "our approach can outperform finetuning foundation models."

major comments (3)

[§2.2 / Chapter 8] The abstract's headline claim that "our approach can outperform finetuning foundation models" is not yet supported for the language domain. The only LLM evidence, summarized in §2.2 and Appendix D, is a meta-dataset built by prompting Llama-3.1-70B to extract atomic facts from 30 scientific papers into synthetic QA datasets, with evaluation performed on further synthetic QA datasets and scored by a Llama-3.1 teacher model. Because the same model family generates the training tasks and judges the responses, the reported improvements over random search and default finetuning may be an artifact of the synthetic QA distribution and LLM-as-judge bias rather than evidence of general transfer. This is a load-bearing gap for the cross-domain research question (RQ3). I ask the authors to either add validation on real, externally labeled language benchmarks or revise the abstract and Chapter 2 so that the cross-domain claim is explicitly delimited as a preliminary, synthetic-only result.
[§3.1] The central meta-learning assumption, stated in §3.1 as "the closer the training tasks sampled from p(T) are to a new task t_new, the better the learner f_theta can transfer," is load-bearing for every chapter but is never empirically characterized. ZAP and Quick-Tune are evaluated on held-out datasets from the same broad benchmark families used to construct their meta-datasets (TFDS and Meta-Album), and the AutoDL benchmark comprises only five final datasets. No diagnostic quantifies how performance degrades as p(T) shifts, for example to medical imaging, non-natural images, or label distributions outside the training range. This is a limitation of the central claim rather than a correctness error, but it should be addressed explicitly, ideally with a distribution-shift experiment or a clear statement in Chapter 13 of which extrapolation claims are and are not made.
[§6.2 / Table 3] The claim that Quick-Tune "outperforms finetuning foundation models" is worded more broadly than the evidence in Table 3, where the large-model baselines are Dinov2 with LoRA and Dinov2 with linear probing rather than full finetuning of the foundation model. While Table 2 does compare against default finetuning of BEiT, XCiT, and DLA, the abstract and Chapter 2 do not distinguish these two comparisons. I recommend a more precise statement that separates "outperforms specific default finetuning configurations" from "outperforms efficient finetuning of a 1B-parameter foundation model."

minor comments (4)

[Table of Contents / §7.5 and §9.5] The table of contents lists Section 7.5 as "Expriments and Results" and Chapter 9 as "Conclusion and Limitatons"; these typos should be corrected.
[§5 / abstract] The statement that the ZAP meta-dataset is "over 1000 times larger than previous meta-datasets" is not backed by explicit size comparisons with Meta-Dataset or VTAB; please add the comparison or soften the claim.
[§6.2 / Table 2] The default-configuration baselines in Table 2 are three specific models selected from the Pareto front (BEiT-large, XCiT-small, DLA46x); the conclusion should state that Quick-Tune outperforms these particular default configurations, not all possible default finetuning recipes.
[§8.4 / §2.2] The description of the LLM meta-dataset says the method is "tested on unseen synthetically generated QA datasets," but the text does not specify how the synthetic test datasets are generated, whether the underlying 30 scientific papers are disjoint from the meta-training split, or how leakage is prevented; these details should be provided in the main text or Appendix D.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central claims rest on held-out external benchmarks; self-citations are provenance only.

full rationale

The dissertation's derivation chain is self-contained against external benchmarks. ZAP trains zero-shot surrogates on a 525x525 cost matrix and evaluates via leave-one-core-dataset-out plus the ChaLearn AutoDL benchmark; Quick-Tune meta-learns performance/cost predictors on held-out Meta-Album folds and benchmarks against ASHA/BOHB/DEHB/DyHPO; the SSL chapters evaluate on ImageNet/CIFAR; the RL chapters evaluate on control environments. None of the central claims are obtained by fitting a parameter to the test quantity and renaming it a prediction. The only self-citations are provenance for the consolidated papers, and the LLM chapter (Ch. 8) uses a synthetic QA distribution with Llama-3.1 teacher evaluation, which is a distribution-shift and metric-validity limitation rather than a by-construction equivalence: the Quick-Tune surrogate does not incorporate the teacher model at inference. I find no step where an equation reduces to its own input by definition or where a fitted value is presented as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 4 invented entities

The dissertation introduces no new physical axioms; its assumptions are domain-level. The central method relies on the representativeness of meta-datasets and on the transferability of learned surrogates and generators. The 'invented entities' are algorithmic constructs (Synthetic Environments, Reward Networks, One-Shot World Model, Hard View Pretraining), all provided with empirical evaluations, so they carry independent evidence rather than being unfalsifiable postulates.

free parameters (4)

ZAP cost matrix training budget = 10 minutes per pipeline
Hand-chosen proxy for the 20-minute AutoDL budget; ZAP assumes rankings at 10 minutes transfer to 20 (ZAP Section 4.3).
Quick-Tune epoch step Delta_t = 1 epoch
Chosen to allow quick discarding of bad configurations during finetuning (Quick-Tune Section 6.1).
ZAP meta-feature set = number of images, channels, resolution, classes
Hand-selected cheap meta-features; the paper notes the choice impacts zero-shot selection performance (ZAP Section 4.1).
Quick-Tune model hub composition = 24 Pareto-optimal timm models
Selected by multi-objective optimization over ImageNet accuracy and parameter count; defines the search space for model selection (Quick-Tune Section 5.1).

assumptions (4)

domain assumption Meta-learning transfers across tasks: closer training tasks yield better transfer to a new task.
Stated in Chapter 3.1 as the principle underlying all methods in the dissertation.
domain assumption The meta-training task distribution is representative of future target tasks.
ZAP uses 35 TFDS datasets, Quick-Tune uses Meta-Album; the dissertation does not prove coverage of all possible downstream distributions.
domain assumption Partial learning curves are predictive of final pipeline performance.
Quick-Tune's gray-box surrogate assumes early epochs inform later performance (Quick-Tune Section 4.2).
domain assumption Synthetic data and augmented views preserve task-relevant information.
Part III assumes synthetic data or adversarial augmentations improve downstream learning rather than introducing harmful noise.

invented entities (4)

Synthetic Environments (SEs) independent evidence
purpose: Meta-learned neural proxies for RL environments that mimic state dynamics and rewards.
Evaluated by training RL agents in classic control environments and comparing to real environments (Chapter 11).
Reward Networks (RNs) independent evidence
purpose: Neural networks modeling only rewards of target RL environments.
Evaluated in the same setting as SEs (Chapter 11).
One-Shot World Model (OSWM) independent evidence
purpose: Transformer-based general synthetic simulator trained on random synthetic priors, adaptable via 1,000 context transitions.
Evaluated on GridWorld, CartPole, custom control, Reacher (Chapter 12).
Hard View Pretraining (HVP) independent evidence
purpose: Adversarial selection of hardest augmented views during SSL pretraining.
Evaluated on ImageNet with four SSL methods and ViT-B/16 (Chapter 10).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning." pith.science (2026). https://pith.science/paper/MQXJJX75

@misc{pith2026250612161,
  author       = {Pith},
  title        = {Pith review of: Meta-Learning and Synthetic Data for Automated Pretraining and Finetuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQXJJX75}},
  note         = {Machine review of arXiv:2506.12161}
}

read the original abstract

The growing number of pretrained models in Machine Learning (ML) presents significant challenges for practitioners. Given a new dataset, they need to determine the most suitable deep learning (DL) pipeline, consisting of the pretrained model and the hyperparameters for finetuning to it. Moreover, as models grow in scale, the increasing reliance on real-world data poses a bottleneck for training and requires leveraging data more effectively. Addressing the first challenge often involves manual model selection and hyperparameter tuning. At the same time, as models grow larger and more and more of the available human-generated data is being used for training, data augmentation and synthetic data become critical elements. Automated machine learning offers a path to address these challenges but is traditionally designed for tabular data and classical ML methods. This dissertation adopts meta-learning to extend automated machine learning to the deep learning domain. We propose empirical approaches to automate DL pipeline selection for Computer Vision tasks using prior task knowledge to learn surrogate models for pipeline ranking. Extending these methods to the language domain, we learn to finetune large language models. As a result, we show that our approach can outperform finetuning foundation models. Additionally, we meta-learn data augmentation and synthetic data to enhance performance in up-stream and down-stream tasks. We empirically show the underestimated importance of data augmentation when using Self-Supervised Learning and meta-learn advanced data augmentation strategies. Leveraging synthetic data, we also propose to meta-learn neural synthetic data generators as proxies for Reinforcement Learning (RL) environments. Additionally, we learn a multiple-environment world model in an in-context learning fashion by purely using synthetic, randomly sampled data.

Figures

Figures reproduced from arXiv: 2506.12161 by the authors.

**Figure 1.1.** (a) The Combined Algorithm and Hyperparameter Optimization (CASH) [PITH_FULL_IMAGE:figures/full_fig_p016_1_1.png] view at source ↗

**Figure 1.2.** Illustrative example of the large set of over 700 pretrained image classification models available from the timm library, with still 24 models on the Pareto front (illustration from the Quick-Tune paper (Arango et al., 2024)). However, with the recent developments in model scaling, we argue that selecting the best pipeline alone is insufficient without leveraging data effectively. For instance, Hoffmann et al. (20… view at source ↗

**Figure 4.1.** Predicting image rotations instead of classes in SelfSupervised Learning (visualization from Giradis et al. (2018)). Data augmentation applies transformations to existing data. Assume F is a set of sample level transformations. Each transformation f ∈ F maps Dt to an augmented dataset D˜ t . On a sample level, data augmentation transforms each sample x ∈ Dt independently. In computer vision, these transformatio… view at source ↗

Figures from the paper (82 more)

**Figure 4.2.** Figure 4.2: Left: Example views after applying geometric and appearance transformations (visualization from Ferreira et al. (2025)). Right: SimSiam’s architecture asymmetry to facilitate contrastive learning without negative image pairs (visualization from X. Chen and He (2021))…

**Figure 1.** Figure 1: ZAP consists of two stages. In the meta-train stage, the cost matrix on the source tasks is leveraged to learn a joint response surface conditioned on the meta-features and pipelines. During the meta-test stage, ZAP assigns scores to the pipelines of the unseen dataset…

**Figure 2.** Figure 2: Overview of our pairwise ranking objective [PITH_FULL_IMAGE:figures/full_fig_p047_2.png]

**Figure 3.** Figure 3: Critical difference diagram comparing loss functions using the Wilcoxon-Holm signed-rank (5% significance level). 4. ZAP Meta-Dataset Design In this section, we introduce a novel meta-dataset (PinedaArango et al., 2021), that will ultimately allow us to perform zero…

**Figure 4.** Figure 4: Cost matrix C as a heatmap Color indicates the ALC score (higher is better). We observe that some datasets (dark rows) are more complex and some pipelines (dark columns) generalize worse across datasets systematically than others. 5. Experiments Our experiments are des…

**Figure 5.** Figure 5: ALC scores of our approach vs. winner baselines over 525 datasets and 10 repetitions. Our ZAP methods clearly improve over the challenge winners (higher is better), by almost 1 point. Our geometry-aware zero-shot HPO version of ZAP with its binary pairwise ranking obje…

**Figure 1.** Figure 1: Ranks of model performances across datasets. Before introducing our method, we want to remind the reader about the importance of searching for the optimal pretrained neural network from a pool of models. Our main premise is that there is no silver bullet model that fi…

**Figure 2.** Figure 2: The subset of Pareto optimal pretrained models with respect to the predictive accuracy and model size. While our proposed method is agnostic to the application domain, the set of pretrained models and hyperparameter space to choose from, we need to instantiate these …

**Figure 3.** Figure 3: Comparison against state-of-the-art HPO methods. [PITH_FULL_IMAGE:figures/full_fig_p064_3.png]

**Figure 4.** Figure 4: Comparing Quick-Tune with (+) and without (-) (M)eta-learning and (C)ost-Awareness, [PITH_FULL_IMAGE:figures/full_fig_p065_4.png]

**Figure 5.** Figure 5: Varying the model hub size. behavior is consistent among datasets of different sizes and present in all three meta-dataset versions. We attribute the search efficiency to our careful search space design, which includes both large and small models, as well as regulariza…

**Figure 6.** Figure 6: Comparison with a two-stage search for models and hyperparameters. [PITH_FULL_IMAGE:figures/full_fig_p066_6.png]

**Figure 1.** Figure 1: Quick-Tune-Tool Architecture. In summary, using Quick-Tune-Tool starts by defining the search space and having the ConfigManager generate initial configurations. QuickTuner then employs the Optimizer to suggest configurations based on previous evaluations, e.g. Histor…

**Figure 2.** Figure 2: Objective Function. ConfigManager. The ConfigManager, as the name suggests, manages the configurations and processes them to be input to the optimizer. It takes a configuration space as input, i.e., the pipeline search space over which the optimization is performed. …

**Figure 3.** Figure 3: Result Dict. Objective Function. The Objective Function is invoked by the tuner during optimization. It can be any function that accepts configuration, budget, and optional task-related information. It returns results as a dictionary or a list of dictionaries. For effi…

**Figure 4.** Figure 4: A simple example of using the Quick-Tune-Tool. [PITH_FULL_IMAGE:figures/full_fig_p077_4.png]

**Figure 5.** Figure 5: Evaluations on common vision datasets using Quick-Tune-Tool (QT) and Random Search as optimizers. We present results for Δ𝑡 ∈ {1, 2} 6 Conclusion and Outlook In this paper, we introduced Quick-Tune-Tool, a tool that simplifies the automated selection and finetuning of …

**Figure 1.** Figure 1: Method Overview. We generate new NLP datasets from scientific papers and then create a meta-dataset, which we use for transfer learning to finetune by pre-training Quick-Tune (left). For a new dataset, we compute meta-features and then apply the pre-trained Quick-Tune …

**Figure 2.** Figure 2: Our Meta-Dataset. For each run stored in our meta-dataset, represented by a blue circle, we present the accuracy and finetuning time in seconds. For each paper, we randomly sample finetuning pipelines from a search space based on hyperparameters for LoRA (Hu et al., 2…

**Figure 3.** Figure 3: Optimizer Performance Over Time. We visualize the average validation (left) and test (right) performance across the eight datasets over time. At each time point, we evaluated the best pipeline found so far. We observe that DEHB and Quick-Tune (default) stagnant after 1…

**Figure 4.** Figure 4: Final Performance. We show the validation (left) and test (right) learning curve of the best pipeline returned by the optimizers after 5 hours, averaged across eight datasets. The finetuning pipeline returned by Quick-Tune (ours) performs best. 4 Results Experimental S…

**Figure 1.** Figure 1: Density estimates for the probability of applying a grayscale augmentation as sampled in our optimization for SimSiam’s augmentation strategy on CIFAR-100. We show density estimates for the best and worst-performing configurations (top and bad), all configurations (al…

**Figure 1.** Figure 1: (a) HVP first samples N views, pairs them, and adversarially selects the hardest pair, i.e., the one with the worst loss according to the current model state. (b) Examples (left) and sampled views (right) after transformations. Hard pairs selected by HVP are shown with…

**Figure 2.** Figure 2: Left: In over 40% of the cases, the adversarially selected view pair has also the lowest Intersection over Union throughout SimSiam+HVP pretraining. We attribute the early spike to the random initialization of the embedding. Right: HVP (blue) shows a shift to smaller I…

**Figure 3.** Figure 3: Left: The average IoU of view pairs selected by SimSiam+HVP (blue) compared against the default SimSiam training (green). Right: Using static color augmentation for all pairs before the selection increases the dependency on the IoU. 6.1.2 Importance of Augmentation Met…

**Figure 4.** Figure 4: With HVP, SimSiam appears more robust to augmentation hyperparameter vari [PITH_FULL_IMAGE:figures/full_fig_p110_4.png]

**Figure 1.** Figure 1: We use an agent-agnostic meta-learning approach to learn neural proxy RL environments [PITH_FULL_IMAGE:figures/full_fig_p117_1.png]

**Figure 2.** Figure 2: Multiple NES runs of Alg. 1 for CartPole [PITH_FULL_IMAGE:figures/full_fig_p120_2.png]

**Figure 3.** Figure 3: Top row: Densities based on each 4000 cumulative test rewards collected by DDQN (left), Dueling DDQN (center), and discrete TD3 (right) agents on CartPole. We show three settings: agents trained on a real environment without any involvement of SEs (blue, baseline), on …

**Figure 4.** Figure 4: Top row: Histograms of next state s ′ and reward r produced by 10 DDQN agents when trained on a CartPole SE (blue) and afterwards tested for 10 episodes on a real environment (orange). We also show the SE responses when fed with real environment data seen during testin…

**Figure 5.** Figure 5: The average cumulative test rewards of agents when trained on different RN variants for one episode and evaluated on the real environments for one episode alternatingly. Top row: Training and evaluation with the same agent using default agent hyperparameters (HPs). Cen…

**Figure 1.** Figure 1: OSWM is trained on synthetic data sampled from a prior distribution of randomly initial [PITH_FULL_IMAGE:figures/full_fig_p131_1.png]

**Figure 2.** Figure 2: Evaluation scores for RL agent training on the OSWM for GridWorld, CartPole-v0, and [PITH_FULL_IMAGE:figures/full_fig_p136_2.png]

**Figure 3.** Figure 3: Typical distribution patterns generated by the NN prior: (a) highly peaked, (b) wide or [PITH_FULL_IMAGE:figures/full_fig_p136_3.png]

**Figure 4.** Figure 4: Reward distributions for the real and OSWM GridWorld and CartPole environments. [PITH_FULL_IMAGE:figures/full_fig_p137_4.png]

**Figure 7.** Figure 7: Dataset augmentation flow the Normalized Area Under ROC Curve (AUC): si = 2 ∗ AUC(⃗pi) − 1 (7) We then convert this score to a time-sensitive step function s(t) = step fn(⃗s) (8) and we also transform the time non-linearly between [0, 1] such that the performance on th…

**Figure 8.** Figure 8: Distribution of the meta-features where each point corresponds to a dataset. Blue points come from our meta-dataset, whilst red ones are the datasets provided by AutoDL challenge. Star and point markers are public AutoDL datasets and private AutoDL datasets (from feedb…

**Figure 9.** Figure 9: Learning curve of a task is the step function of normalized AUC (NAUC) scores received during the 20 minutes and the light blue area underneath is the area under the learning curve(ALC). Every dark blue point (steps) corresponds to a set of predictions made and y-axis …

**Figure 7.** Figure 7: Evaluation of Quick-Tune on Datasets outside Meta-Album. [PITH_FULL_IMAGE:figures/full_fig_p168_7.png]

**Figure 8.** Figure 8: Results for finetuning the last layer of DINOv2. We relax the efficiency conditions by [PITH_FULL_IMAGE:figures/full_fig_p168_8.png]

**Figure 6.** Figure 6: Evaluation results. Search Space. A subset of the hyperparameters (see [PITH_FULL_IMAGE:figures/full_fig_p177_6.png]

**Figure 7.** Figure 7: ImageFolder format 14 [PITH_FULL_IMAGE:figures/full_fig_p179_7.png]

**Figure 5.** Figure 5: We depict row-wise ten example images from the ImageNet train set along with [PITH_FULL_IMAGE:figures/full_fig_p198_5.png]

**Figure 6.** Figure 6: The training loss over 100 epochs. Comparing the DINO vanilla method with [PITH_FULL_IMAGE:figures/full_fig_p199_6.png]

**Figure 7.** Figure 7: Setting the number of views too high can result in performance deterioration. [PITH_FULL_IMAGE:figures/full_fig_p199_7.png]

**Figure 8.** Figure 8: Application of fANOVA Hutter et al. (2014) on logged training data to determine [PITH_FULL_IMAGE:figures/full_fig_p200_8.png]

**Figure 9.** Figure 9: Illustration of adversarial learning with a Spatial Transformer Network (STN) [PITH_FULL_IMAGE:figures/full_fig_p202_9.png]

**Figure 10.** Figure 10: Example for static output behavior of the STN. [PITH_FULL_IMAGE:figures/full_fig_p202_10.png]

**Figure 2.** Figure 2: CIFAR-10 with SimSiam data augmentation search space [PITH_FULL_IMAGE:figures/full_fig_p217_2.png]

**Figure 3.** Figure 3: CIFAR-100 with SimSiam data augmentation search space [PITH_FULL_IMAGE:figures/full_fig_p218_3.png]

**Figure 4.** Figure 4: DermaMNIST with SimSiam data augmentation search space [PITH_FULL_IMAGE:figures/full_fig_p219_4.png]

**Figure 5.** Figure 5: CIFAR-10 with SimSiam training hyperparameters search space [PITH_FULL_IMAGE:figures/full_fig_p220_5.png]

**Figure 6.** Figure 6: CIFAR-100 with SimSiam training hyperparameters search space. top all bad 0.01 0.1 pt_learning_rate top all bad collapsing 0 5 10 15 20 25 30 #configs with optimizer sgd adamw lars top all bad 0.0001 0.001 0.01 pt_weight_decay_end top all bad 0.0001 0.001 0.01 pt_weigh…

**Figure 7.** Figure 7: DermaMNIST with SimSiam training hyperparameters search space [PITH_FULL_IMAGE:figures/full_fig_p221_7.png]

**Figure 6.** Figure 6: Results from 40 different NES runs with 16 workers each (using random seeds) show that [PITH_FULL_IMAGE:figures/full_fig_p229_6.png]

**Figure 7.** Figure 7: Evaluation of performance and transferability of SEs on the Acrobot-v1 task. [PITH_FULL_IMAGE:figures/full_fig_p229_7.png]

**Figure 8.** Figure 8: Histograms of approximate next state s ′ and reward r distributions produced by 10 DDQN or Dueling DDQN agents when trained on an SE (blue) and when afterwards tested for 10 episodes on a real environment (orange) for each task. 1st row: CartPole and DDQN, 2nd row: Car…

**Figure 9.** Figure 9: Performance of different RN variants for the HalfCheetah-v3 environment. Left: perfor [PITH_FULL_IMAGE:figures/full_fig_p233_9.png]

**Figure 10.** Figure 10: This is the zoomed in version of Figure 9. [PITH_FULL_IMAGE:figures/full_fig_p233_10.png]

**Figure 11.** Figure 11: Learned Cliff Walking reward for different reward network types. Shown is the start state [PITH_FULL_IMAGE:figures/full_fig_p235_11.png]

**Figure 12.** Figure 12: The Cliff environment RNs optimized with the AUC objective. [PITH_FULL_IMAGE:figures/full_fig_p240_12.png]

**Figure 13.** Figure 13: The Cliff environment RNs optimized with the max reward objective. [PITH_FULL_IMAGE:figures/full_fig_p240_13.png]

**Figure 14.** Figure 14: The Cliff environment RNs optimized with the reward threshold objective (similar to the [PITH_FULL_IMAGE:figures/full_fig_p240_14.png]

**Figure 15.** Figure 15: The CartPole environment RNs optimized with the AUC objective. [PITH_FULL_IMAGE:figures/full_fig_p241_15.png]

**Figure 16.** Figure 16: The CartPole environment RNs optimized with the max reward objective. [PITH_FULL_IMAGE:figures/full_fig_p241_16.png]

**Figure 17.** Figure 17: The CartPole environment RNs optimized with the reward threshold objective (similar to [PITH_FULL_IMAGE:figures/full_fig_p241_17.png]

**Figure 18.** Figure 18: The MountainCarContinuous environment RNs optimized with the AUC objective. [PITH_FULL_IMAGE:figures/full_fig_p242_18.png]

**Figure 19.** Figure 19: The MountainCarContinuous environment RNs optimized with the reward threshold [PITH_FULL_IMAGE:figures/full_fig_p242_19.png]

**Figure 20.** Figure 20: The HalfCheetah environment RNs optimized with the AUC objective. [PITH_FULL_IMAGE:figures/full_fig_p242_20.png]

**Figure 21.** Figure 21: The HalfCheetah environment RNs optimized with the reward threshold objective (simi [PITH_FULL_IMAGE:figures/full_fig_p242_21.png]

**Figure 22.** Figure 22: Top: Comparing the cumulative test reward densities of agents trained on SEs (green and orange), supervised baseline (purple), and baseline on real environment (blue). Agents trained on the supervised model underperform the SE models and the real baseline. Bottom: Com…

**Figure 23.** Figure 23: Top: Comparing the cumulative test reward densities of agents trained on SEs (green and orange), supervised baseline (purple), and baseline on real environment (blue). Agents trained on the supervised model underperform the SE models and the real baseline and using on…

**Figure 24.** Figure 24: Evaluation of different score transformation schemes for synthetic environments: Shown [PITH_FULL_IMAGE:figures/full_fig_p248_24.png]

**Figure 25.** Figure 25: Model-wise evaluation: 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p249_25.png]

**Figure 26.** Figure 26: Model-wise evaluation based on the cumulative rewards of 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p249_26.png]

**Figure 27.** Figure 27: Model-wise evaluation based on the cumulative rewards of 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p250_27.png]

**Figure 28.** Figure 28: Model-wise evaluation based on the cumulative rewards of 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p250_28.png]

**Figure 29.** Figure 29: Model-wise evaluation based on the cumulative rewards of 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p250_29.png]

**Figure 30.** Figure 30: Model-wise evaluation based on the cumulative rewards of 1000 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p251_30.png]

**Figure 5.** Figure 5: Typical distribution patterns generated by the Momentum prior: (a) broad, (b) multi-modal, [PITH_FULL_IMAGE:figures/full_fig_p258_5.png]

**Figure 6.** Figure 6: Visualization of the custom GridWorld environment. Terminal states are in red, goal states [PITH_FULL_IMAGE:figures/full_fig_p260_6.png]

**Figure 7.** Figure 7: Reward distributions for the real and OSWM Pendulum environments. [PITH_FULL_IMAGE:figures/full_fig_p261_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 10 canonical work pages

[1]

Accurate structure prediction of biomolecular interactions with AlphaFold 3

257 Abramson, J., J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. (2024). “Accurate structure prediction of biomolecular interactions with AlphaFold 3”. In: Nature, pp. 1–3 (cit. on p. 3). Achiam, J., S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Al- tenschmidt,...

arXiv 2024
[10]

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

URL: https://deepmind.google/discover/blog/genie-2-a-large-scale-founda tion-world-model/ (cit. on p. 136). Parker-Holder, J., R. Rajan, X. Song, A. Biedenkapp, Y. Miao, T. Eimer, B. Zhang, V. Nguyen, R. Calandra, A. Faust, F. Hutter, and M. Lindauer (2022). “Automated Reinforcement Learning (AutoRL): A Survey and Open Problems”. In: Journal of Artificial...

work page 2022
[13]

Learning from synthetic data: Addressing domain shift for semantic segmentation

MIT Press. Chap. 8, pp. 318–362 (cit. on p. 3). Sankaranarayanan, S., Y. Balaji, A. Jain, S. N. Lim, and R. Chellappa (2018). “Learning from synthetic data: Addressing domain shift for semantic segmentation”. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’18). Computer Vision Foundation and IEEE Computer S...

work page 2018
[15]

A survey on image data augmentation for deep learning

Proceedings of Machine Learning Research. PMLR, pp. 20026–20040 (cit. on p. 27). Shorten, C. and T. M. Khoshgoftaar (2019). “A survey on image data augmentation for deep learning”. In: Journal of big data 6.1, pp. 1–48 (cit. on pp. 6, 25). Simard, P., D. Steinkraus, and J. Platt (2003). “Best practices for convolutional neural net- works applied to visual...

work page 2019
[16]

Viewmaker Networks: Learning Views for Unsupervised Representation Learning

Proceedings of Machine Learning Research (cit. on pp. 4, 6, 27). Tamkin, A., M. Wu, and N. D. Goodman (2021). “Viewmaker Networks: Learning Views for Unsupervised Representation Learning”. In: Proceedings of the International Conference on Learning Representations (ICLR’21). Published online: iclr.cc. ICLR (cit. on p. 27). Thornton, C., F. Hutter, H. Hoos...

arXiv 2021
[17]

Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions

Proceedings of Machine Learning Research. PMLR. URL: https://icml.cc/virtual/2 022/20697 (cit. on p. 81). Wang, R., J. Lehman, J. Clune, and K. O. Stanley (2019). “Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions”. In: arXiv preprint arXiv:1901.01753 (cit. on p. 136). Wan...

arXiv 2019
[37]

Chapter Title (e.g., Trends in AI Development)

Omni- press, pp. 2113–2122 (cit. on p. 28). Maslej, N., L. Fattorini, R. Perrault, et al. (2024a). “Chapter Title (e.g., Trends in AI Development)”. In: The AI Index 2024 Annual Report. Stanford, CA: Institute for Human- Centered AI, Stanford University. Chap. Chapter 1: Research and Development, pp. 52– 55 (cit. on pp. 4, 10). Maslej, N., L. Fattorini, R...

work page 2024
[48]

Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook

Proceedings of Machine Learning Research (cit. on p. 23). Schmidhuber, J. (May 1987). “Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook”. Diploma Thesis. Technische Universitat Munchen, Germany. URL: http://www.idsia.ch/~juergen/diploma.htm l (cit. on pp. 5, 21). Schmidhuber, J. (1992). Learning To...

work page 1992

Show all 18 references

[70]

Unsupervised Representation Learning by Predicting Image Rotations

Proceedings of Machine Learning Research, pp. 1126–1135 (cit. on pp. 5, 23). 260 Bibliography Giradis, S., P. Singh, and N. Komodakis (2018). “Unsupervised Representation Learning by Predicting Image Rotations”. In: Proceedings of the International Conference on Learning Repre...

2018 arXiv
[97]

Denoising diffusion probabilistic models

Proceedings of Machine Learning Research, pp. 2731–2741 (cit. on p. 25). Ho, J., A. Jain, and P. Abbeel (2020). “Denoising diffusion probabilistic models”. In: Proceedings of the 34th International Conference on Advances in Neural Information Processing Systems (NeurIPS’20). E...

2020 arXiv
[98]

Exploring Simple Siamese Representation Learning

Proceedings of Machine Learning Research, pp. 1597–1607 (cit. on pp. 10, 26). Chen, X. and K. He (2021). “Exploring Simple Siamese Representation Learning”. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’21). Computer Vision Fo...

2021 arXiv
[139]

Robust Speech Recognition via Large-Scale Weak Supervision

Proceedings of Machine Learning Research. PMLR, pp. 8748–8763 (cit. on pp. 3, 135). Radford, A., J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever (2023). “Robust Speech Recognition via Large-Scale Weak Supervision”. In: Proceedings of the 40th International Confere...

2023
[162]

A Survey on Transfer Learning

Proceedings of Machine Learning Research. PMLR, pp. 1128–1135. URL: https://icml.cc/virtual/2022/spotlight/18008 (cit. on pp. 19, 31). 263 Pan, S. J. and Q. Yang (2010). “A Survey on Transfer Learning”. In:IEEE Trans. Knowl. Data Eng. 22.10, pp. 1345–1359 (cit. on p. 22). Park...

2010
[202]

Language models are unsupervised multitask learners

Proceedings of Machine Learning Research. PMLR (cit. on p. 3). Radford, A., J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever (2019). “Language models are unsupervised multitask learners”. In: OpenAI blog 1.8, p. 9 (cit. on p. 3). Rapant, I., L. Purucker, F. Ferreira, S. P...

2019 arXiv
[251]

Learning Synthetic Environments and Reward Networks for Reinforcement Learning

Proceedings of Machine Learning Research. PMLR (cit. on p. 3). Ferreira, F., T. Nierhoff, A. Sälinger, and F. Hutter (2022). “Learning Synthetic Environments and Reward Networks for Reinforcement Learning”. In: Proceedings of the International Conference on Learning Representa...

2022
[2021]

A Simple Framework for Contrastive Learning of Visual Representations

Ed. by D. A. Shell, M. Toussaint, and M. Ani Hsieh (cit. on p. 10). Chen, T., S. Kornblith, M. Norouzi, and G. E. Hinton (2020). “A Simple Framework for Contrastive Learning of Visual Representations”. In:Proceedings of the 37th International Conference on Machine Learning (IC...

2020
[2024]

Transformers Can Do Bayesian Inference

URL: https://gith ub.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md (cit. on pp. 4, 134). Müller, S., N. Hollmann, S. Arango, J. Grabocka, and F. Hutter (2022). “Transformers Can Do Bayesian Inference”. In: Proceedings of the International Conference on Le...

2022
[7317]

Parameter-Efficient Fine- Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

Lecture Notes in Computer Science. Springer, pp. 228–241 (cit. on p. 5). Xu, L., H. Xie, S.-Z.. J. Qin, X. Tao, and F. L. Wang (2023). “Parameter-Efficient Fine- Tuning Methods for Pretrained Language Models: A Critical Review and Assessment”. In: arXiv:2312.12148 [cs.CL] (cit...

2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Accurate structure prediction of biomolecular interactions with AlphaFold 3

257 Abramson, J., J. Adler, J. Dunger, R. Evans, T. Green, A. Pritzel, O. Ronneberger, L. Willmore, A. J. Ballard, J. Bambrick, et al. (2024). “Accurate structure prediction of biomolecular interactions with AlphaFold 3”. In: Nature, pp. 1–3 (cit. on p. 3). Achiam, J., S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Al- tenschmidt,...

arXiv 2024

[2] [10]

Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

URL: https://deepmind.google/discover/blog/genie-2-a-large-scale-founda tion-world-model/ (cit. on p. 136). Parker-Holder, J., R. Rajan, X. Song, A. Biedenkapp, Y. Miao, T. Eimer, B. Zhang, V. Nguyen, R. Calandra, A. Faust, F. Hutter, and M. Lindauer (2022). “Automated Reinforcement Learning (AutoRL): A Survey and Open Problems”. In: Journal of Artificial...

work page 2022

[3] [13]

Learning from synthetic data: Addressing domain shift for semantic segmentation

MIT Press. Chap. 8, pp. 318–362 (cit. on p. 3). Sankaranarayanan, S., Y. Balaji, A. Jain, S. N. Lim, and R. Chellappa (2018). “Learning from synthetic data: Addressing domain shift for semantic segmentation”. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’18). Computer Vision Foundation and IEEE Computer S...

work page 2018

[4] [15]

A survey on image data augmentation for deep learning

Proceedings of Machine Learning Research. PMLR, pp. 20026–20040 (cit. on p. 27). Shorten, C. and T. M. Khoshgoftaar (2019). “A survey on image data augmentation for deep learning”. In: Journal of big data 6.1, pp. 1–48 (cit. on pp. 6, 25). Simard, P., D. Steinkraus, and J. Platt (2003). “Best practices for convolutional neural net- works applied to visual...

work page 2019

[5] [16]

Viewmaker Networks: Learning Views for Unsupervised Representation Learning

Proceedings of Machine Learning Research (cit. on pp. 4, 6, 27). Tamkin, A., M. Wu, and N. D. Goodman (2021). “Viewmaker Networks: Learning Views for Unsupervised Representation Learning”. In: Proceedings of the International Conference on Learning Representations (ICLR’21). Published online: iclr.cc. ICLR (cit. on p. 27). Thornton, C., F. Hutter, H. Hoos...

arXiv 2021

[6] [17]

Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions

Proceedings of Machine Learning Research. PMLR. URL: https://icml.cc/virtual/2 022/20697 (cit. on p. 81). Wang, R., J. Lehman, J. Clune, and K. O. Stanley (2019). “Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions”. In: arXiv preprint arXiv:1901.01753 (cit. on p. 136). Wan...

arXiv 2019

[7] [37]

Chapter Title (e.g., Trends in AI Development)

Omni- press, pp. 2113–2122 (cit. on p. 28). Maslej, N., L. Fattorini, R. Perrault, et al. (2024a). “Chapter Title (e.g., Trends in AI Development)”. In: The AI Index 2024 Annual Report. Stanford, CA: Institute for Human- Centered AI, Stanford University. Chap. Chapter 1: Research and Development, pp. 52– 55 (cit. on pp. 4, 10). Maslej, N., L. Fattorini, R...

work page 2024

[8] [48]

Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook

Proceedings of Machine Learning Research (cit. on p. 23). Schmidhuber, J. (May 1987). “Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta...-Hook”. Diploma Thesis. Technische Universitat Munchen, Germany. URL: http://www.idsia.ch/~juergen/diploma.htm l (cit. on pp. 5, 21). Schmidhuber, J. (1992). Learning To...

work page 1992