Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Efficient Task Grouping Through Samplewise Optimisation Landscape Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes SCA, an inter-task affinity score computed from one-step gradient updates on a randomly initialised shared model, and claims it identifies effective task groupings for multi-task learning and gradient-based…

desk verdict Cheap and often effective task-affinity heuristic, but the proof that supposedly backs it does not establish the claimed link between SCA distances and joint trainability. read the letter →

arxiv 2412.04413 v2 pith:BH6UPLHL submitted 2024-12-05 cs.LG

Anshul Thakur , Yichen Huang , Soheila Molaei , Yujiang Wang , David A. Clifton This is my paper · ORCID
classification cs.LG MSC 68T0768T05
keywords taskgroupingmulti-tasklearningnegativetransfersample-wiseoptimisationlandscapeaffinitygraphattentionnetworksgradient-basedmeta-learningSCAscores
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Task grouping—deciding which tasks to train together—is normally expensive because it requires training candidate multi-task models or measuring gradient effects inside a shared model. This paper claims the decision can be made before any shared training: from a random initialisation, take one gradient step per task on each of a few hundred samples, and measure the average $\ell^1$ distance between the points reached. Those distances, called sample-wise convergence-based affinity (SCA) scores, act as task affinities, and a graph-attention plus clustering module turns them into task groups. If the claim is right, task grouping becomes a cheap preprocessing step, and the reported experiments on eight datasets show accuracy comparable to or better than state-of-the-art methods with roughly five times less GPU time.

What carries the argument

The load-bearing construction is the sample-wise optimisation landscape: each training sample is split into $T$ task-specific learning problems, and one gradient step from a random shared initialisation $\theta_0$ yields task-specific 'sample-wise optima' $\theta^{t*}_i$. Theorem 4.1 shows the shared optimum is their average, and Theorem 4.2 bounds the joint loss by the density $\psi_{\theta_i}$ of their pairwise $\ell^1$ distances. The pairwise distance, averaged over samples, is the SCA score that weights the edges of a task graph; a two-layer graph attention network learns node embeddings that encode higher-order task interactions, and Gaussian mixture clustering on those embeddings produces the final task groups.

What would settle it

Train pairs of tasks fully from multiple random initialisations and measure the actual joint-training loss, then compare that ordering with SCA affinity; if SCA ranks pairs opposite to the measured losses in a setting where the gradient norm after one step is far from zero, the theorem's premise fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the average multi-task training loss is controlled by the density of one-step task-specific sample-wise optima. Writing $\theta^{t*}_i = \theta_0 - \eta \nabla_\theta \ell_t$ for the point reached by one gradient step on task $t$ and sample $i$, the shared optimum is $\theta^*_i = \frac{1}{T}\sum_t \theta^{t*}_i$, and Theorem 4.2 gives $J_i \le T^3 \psi_{\theta_i}^2$ for the per-sample average loss, where $\psi_{\theta_i}$ is the pairwise $\ell^1$ scatter of the task-specific optima. Over the whole dataset the bound becomes $J \le \frac{T^3}{2n}\sum_{i=1}^n \psi_{\theta_i}^2 + \frac{n^3}{2}\Psi_{\theta^*}^2$ (Eq. 15). The paper therefore proposes pairwise $\ell^1$ distances between one-step optima, averaged over samples (Eq. 16), as SCA affinities that identify good task groupings without training a shared model. Experiments report that these affinities are stable during training and across sample sizes, and that groupings found this way match or beat HOA and TAG on eight datasets while cutting GPU time roughly five-fold.

Load-bearing premise

The theory assumes that one gradient step from the random initialisation already puts the model at a true sample-wise optimum, where the loss and its gradient are both zero; with a finite learning rate, one update generally does not reach such a point.

Editorial extensions

If this is right

  • Task affinities can be computed from a randomly initialised network with one forward and backward pass per sampled example, before any shared model is trained.
  • The same SCA computation applies to gradient-based meta-learning: for one inner Reptile step the shared optimum is again an average of task-specific optima, so grouping can be done inside meta-learning pipelines.
  • Grouping becomes a graph-clustering problem rather than a search over $2^{|T|}$ model combinations, sidestepping the NP-hard affinity-maximisation formulation.
  • The reported five-fold GPU-time reduction is accompanied by accuracy that matches or exceeds HOA and TAG across QM9, CelebA, MIMIC-III, ECG waveform, and image-classification tasks.
  • Only about 100 samples per task are needed to stabilise the SCA scores, so the preprocessing cost does not grow with dataset size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the one-step-is-an-optimum assumption is relaxed, SCA can still serve as a first-order proxy for gradient conflict; a cheap probe would be computing SCA after several gradient steps and checking whether task rankings change.
  • Because SCA depends on the architecture and initialisation, it could double as a tool for choosing which shared architecture to use for a fixed task set, not just which tasks to group.
  • The memory bottleneck of comparing full parameter vectors, acknowledged by the authors, could be removed with layer-wise or projected distances, making SCA practical for very large models.
  • The same affinity signal could be used outside static grouping, for example to order tasks in a curriculum or to select collaborators in federated learning where full joint training is too costly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SCA (sample-wise convergence-based affinity), a task grouping method for multi-task learning and gradient-based meta-learning that avoids training shared models to measure task affinities. It computes, for a small set of samples, one-step gradient updates from a shared initialization and defines pairwise task affinity as the average L1 distance between the resulting task-specific parameter vectors. A graph attention network and Gaussian mixture model clustering then produce task groups. The paper claims a theoretical bound (Theorem 4.2 and Eq. 15) connecting the density of task-specific sample-wise optima to the average multi-task training loss, thereby justifying the use of SCA distances for grouping. Experiments on several datasets (CelebA, QM9, MIMIC-III, ECG waveforms, and image classification benchmarks) show that SCA groupings perform comparably to TAG and HOA while requiring substantially less computation.

Significance. If the theoretical connection were established, this would be a valuable contribution: task grouping without training any shared model would reduce the computational bottleneck of existing methods. The empirical portion is reasonably broad and the reported speedups are concrete; the results on image classification and ECG tasks show SCA matching or slightly exceeding TAG/HOA, which is credible evidence that the heuristic has practical merit. However, the central theoretical claim is not currently supported: the proof of Theorem 4.2 relies on an unproven and generally false premise, and Eq. (11) contains an algebraic error. The paper therefore stands as an empirically motivated heuristic with an unsupported theoretical narrative. That is still a publishable contribution if reframed, but the manuscript as written overclaims the theoretical grounding.

major comments (3)
  1. [Section 4.1, proof of Theorem 4.2, Eqs. (8)–(9)] The step from Eq. (8) to Eq. (9) requires that each task-specific sample-wise optimum θ^{t*}_i satisfies ℓ_t(f_{θ^{t*}_i}(x_i), y_i^t) = 0 and ∇θℓ_t(f_{θ^{t*}_i}(x_i), y_i^t) = 0. The manuscript itself only labels this as "arguably assumed" in Section 3.2, and it is not proven. For a generic smooth loss and finite learning rate η, a single gradient step does not reach a stationary point, let alone a zero-loss stationary point. When the residual loss and gradient are nonzero, Eq. (9) contains extra first-order and zeroth-order terms, so the density ψ_{θ_i} no longer upper-bounds J_i. Because Eq. (15) is the theoretical bridge from SCA distances to joint trainability, this assumption is load-bearing. I recommend stating explicit sufficient conditions under which the bound holds (e.g., the per-sample loss is a convex quadratic with appropriate scaling and η chosen to reach the minimum), or reformulating Theorem 4.2 as an approximation with explicit residual terms, or deleting the theorem and presenting SCA as a heuristic.
  2. [Eq. (11) in the proof of Theorem 4.2] The proof writes θ*_i = Σ_{j=1}^{T} θ^{j*}_i, but Theorem 4.1, specifically Eq. (6), establishes θ*_i = (1/T) Σ_{j=1}^{T} θ^{j*}_i. The missing factor of 1/T changes the subsequent constants: the T^3 factor in the stated bound is not justified even if the zero-loss/gradient premise were granted. Please re-derive the constant with the correct convex combination.
  3. [Section 4.1, "Extension to meta-learning"] The claim that "Theorem 4.1 holds for both" multi-task learning and gradient-based meta-learning is asserted without proof. For Reptile with a single inner iteration on a single sample, the update reduces to a form similar to Eq. (4), but the actual Reptile update in Algorithm 1 uses inner-loop batches and multiple inner steps, after which the shared optimum of a training unit is not the average of one-step sample-wise optima from a common initial point. Since the meta-learning experiments in Section 6.2 are a stated contribution, this extension needs a proper derivation for the specific algorithm used, or the claim must be restricted to the one-step, one-sample case.
minor comments (5)
  1. [Abstract and Introduction] The abstract says the empirical assessment covers 8 datasets, while the Introduction says "5 distinct datasets" and Table 1 lists nine datasets (CelebA, QM9, MIMIC-III, MIMIC-III Waveform, CIFAR-10, SVHN, STL-10, Colorectal, Malaria). Please make the counts consistent.
  2. [Figure 3 caption] The caption contains the typo "SVNH" where the intended abbreviation is "SVHN".
  3. [Equation (16)] The index i is used both for the outer sample summation and for the two task indices in the inner expression, which is confusing. Please use distinct indices, e.g., a_{τ_p,τ_q} = (1/n) Σ_{k=1}^{n} ||θ^{τ_p*}_k - θ^{τ_q*}_k||_1.
  4. [Section 1, reference citation] The citation for gradient-based meta-learning in the Introduction appears as "[ ?], [11]"; a placeholder reference is unresolved.
  5. [Section 6.4, Figures 11 and 12 captions] The captions for Figures 11 and 12 both say "(a) CIFAR" and "(b) SVHN", but the text says these figures concern patient care tasks; the labels appear to be copy-paste errors.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SCA affinities are initial-state geometric quantities that are never fit to the joint-training outcome; the Theorem 4.2 gap is a soundness concern, not a circular reduction.

full rationale

The derivation chain is not circular. SCA affinities (Eq. 16) are computed directly from one-step task-specific updates θt*_i = θ0 − η∇θℓ_t (Eq. 5) and the density ψ (Eq. 7); no parameter of the affinity measure is fitted to the joint-training loss J or to the final grouping quality, and the graph-attention clustering (Section 4.2.2) is trained only to reconstruct its own input affinity rows. Theorem 4.1 is an algebraic identity (average of gradients), and Theorem 4.2 / Eq. 15 is a conditional smoothness bound, not a tautology: even if it were unsound, the weakness is the explicitly 'arguably assumed' property that one gradient step reaches a zero-loss, zero-gradient sample-wise optimum (Section 3.2, proof of Theorem 4.2), plus an algebraic slip (Eq. 11 writes θ⋆ as an unnormalised sum). Those are correctness/soundness objections, not reductions of the prediction to its inputs. The only self-citation, Ref. [12], supplies the LSTM architecture for the patient-care experiments (Section 5.2) and is not load-bearing for the central theoretical claim. The conclusion (Section 7) honestly limits SCA scores to 'ease of joint optimisation' and notes architecture dependence, which further undercuts any reading of the affinities as a fitted proxy. Evaluation is anchored to external baselines (TAG, HOA, random splits, MTL methods), so results do not depend on the authors' own prior claims. The manuscript also contains a placeholder citation '[?]' in Section 1; that is a completeness issue, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The ledger shows the method itself is parameter-light: the SCA affinity is a distance between one-step updates, with no fitted constants to match the final task grouping performance. The load is instead carried by axioms: the single-step zero-loss assumption that makes the density bound true, the asserted extension to meta-learning, and the heuristic that GAT embeddings solve the NP-hard grouping objective. The main unstated free parameter is the learning rate used to compute the one-step optima, which directly scales the SCA distances.

free parameters (3)
  • Learning rate η for SCA sample-wise updates = not reported
    Eqs. 4-5 define task-specific optima as a single gradient step with learning rate η; the SCA distances in Eq. 16 scale with eta, and the paper never states the value used to compute affinities, so this is an unstated hand-chosen quantity.
  • Number of samples N for SCA computation = 100
    Section 5.3 samples N=100 examples per dataset to compute SCA scores; Section 6.4 shows the scores are stable as the sample count grows, so this is a deliberate cost-accuracy choice, not a fitted value.
  • GAT and GMM hyperparameters = 2 GAT layers, 100 epochs, Adam lr 0.001, GMM with b mixtures
    Chosen by hand in the supplementary; the final grouping depends on the learned embeddings, but these settings are not tuned to the evaluation metric and follow standard defaults.
assumptions (4)
  • standard math Loss functions are H-smooth with finite Hessian bound H; standard Taylor inequality.
    Used in Theorem 4.2 proof, Eq. 8-9.
  • domain assumption A single gradient update from random initialization reaches a sample-wise global optimum with zero loss and zero gradient.
    Stated in Section 3.2 (citing Zhang and Jia [18]) and used in Theorem 4.2 proof; for finite learning rates this is generally false.
  • ad hoc to paper The same average-of-task-specific-optima characterization applies to gradient-based meta-learning (Reptile).
    Section 4.1 'EXTENSION TO META LEARNING' asserts Theorem 4.1 holds for meta-learning without proof.
  • domain assumption GAT embeddings trained to reconstruct the affinity matrix capture higher-order interactions sufficient for near-optimal groupings of the NP-hard average-affinity maximization.
    Section 4.2.2; no correspondence between GAT embeddings and the NP-hard objective is shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Task Grouping Through Samplewise Optimisation Landscape Analysis." pith.science (2026). https://pith.science/paper/BH6UPLHL

@misc{pith2026241204413,
  author       = {Pith},
  title        = {Pith review of: Efficient Task Grouping Through Samplewise Optimisation Landscape Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BH6UPLHL}},
  note         = {Machine review of arXiv:2412.04413}
}
read the original abstract

Shared training approaches, such as multi-task learning (MTL) and gradient-based meta-learning, are widely used in various machine learning applications, but they often suffer from negative transfer, leading to performance degradation in specific tasks. While several optimisation techniques have been developed to mitigate this issue for pre-selected task cohorts, identifying optimal task combinations for joint learning - known as task grouping - remains underexplored and computationally challenging due to the exponential growth in task combinations and the need for extensive training and evaluation cycles. This paper introduces an efficient task grouping framework designed to reduce these overwhelming computational demands of the existing methods. The proposed framework infers pairwise task similarities through a sample-wise optimisation landscape analysis, eliminating the need for the shared model training required to infer task similarities in existing methods. With task similarities acquired, a graph-based clustering algorithm is employed to pinpoint near-optimal task groups, providing an approximate yet efficient and effective solution to the originally NP-hard problem. Empirical assessments conducted on 8 different datasets highlight the effectiveness of the proposed framework, revealing a five-fold speed enhancement compared to previous state-of-the-art methods. Moreover, the framework consistently demonstrates comparable performance, confirming its remarkable efficiency and effectiveness in task grouping.

Figures

Figures reproduced from arXiv: 2412.04413 by the authors.

Figure 1
Figure 1. Illustration of a standard task grouping framework addressing three patient-care prediction tasks. A single multi [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the nature of shared global optima [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. An illustration of higher-order interactions captured [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Illustration of the process of obtaining task groupings from pair-wise SCA scores (affinity matrix) using graph [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Performance and computational efficiency of the proposed [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Performance and computational efficiency of the pro [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Running time of different task grouping methods on [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Dynamics of the SCA scores as the joint training of [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Dynamics of the SCA scores as the joint training of [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Impact of sample size on SCA scores computed for [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Impact of sample size on SCA scores computed for [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 1
Figure 1. Figure 1: (a) t-SNE representations of sample-wise converged Resnet-50 models for CIFAR-10, street view house number (SVHN) and colorectal histology datasets. (b) The proposed SCA and (c) task-affinity groupings (TAG)-based task similarities computed among these datasets. Both S…
Figure 2
Figure 2. Figure 2: Performance of different QM9 tasks in a multi-task (two task learning) setup when paired with the closest (best) [PITH_FULL_IMAGE:figures/full_fig_p016_2.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 46 canonical work pages

  1. [1]

    Multi-task learning for dense prediction tasks: A survey,

    S. Vandenhende, S. Georgoulis, W. Van Gansbeke, M. Proesmans, D. Dai, and L. Van Gool, “Multi-task learning for dense prediction tasks: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 7, pp. 3614–3633, 2021. 1

    work page 2021

  2. [2]

    A survey on multi-task learning,

    Y. Zhang and Q. Yang, “A survey on multi-task learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 12, pp. 5586–5609, 2021. 1

    work page 2021

  3. [3]

    Caruana, Multitask Learning

    R. Caruana, Multitask Learning. Boston, MA: Springer US, 1998, pp. 95–133. 1

    work page 1998

  4. [4]

    Multi- task learning with attention for end-to-end autonomous driving,

    K. Ishihara, A. Kanervisto, J. Miura, and V . Hautamaki, “Multi- task learning with attention for end-to-end autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2902–2911. 1

    work page 2021

  5. [5]

    Mtdiag: an effective multi-task framework for automatic diagno- sis,

    Z. Hou, Y. Cen, Z. Liu, D. Wu, B. Wang, X. Li, L. Hong, and J. Tang, “Mtdiag: an effective multi-task framework for automatic diagno- sis,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 12, 2023, pp. 14 241–14 248. 1

    work page 2023

  6. [6]

    Learning to branch for multi-task learning,

    P . Guo, C.-Y. Lee, and D. Ulbricht, “Learning to branch for multi-task learning,” in International conference on machine learning. PMLR, 2020, pp. 3854–3863. 1

    work page 2020

  7. [7]

    Multitask learning: A knowledge-based source of inductive bias,

    R. Caruana, “Multitask learning: A knowledge-based source of inductive bias,” in Proceedings of the International Conference on Machine Learning, 1993, pp. 41–48. 1

    work page 1993

  8. [8]

    Gradient surgery for multi-task learning,

    T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, “Gradient surgery for multi-task learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 5824–5836, 2020. 1, 3, 8

    work page 2020

Show all 49 references
  1. [9]

    Loss-balanced task weighting to reduce negative transfer in multi-task learning,

    S. Liu, Y. Liang, and A. Gitter, “Loss-balanced task weighting to reduce negative transfer in multi-task learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 9977–9978. 1

    2019

  2. [10]

    Which tasks should be learned together in multi-task learning?

    T. Standley, A. Zamir, D. Chen, L. Guibas, J. Malik, and S. Savarese, “Which tasks should be learned together in multi-task learning?” in International Conference on Machine Learning . PMLR, 2020, pp. 9120–9132. 1, 2, 3, 8

    2020

  3. [11]

    Reptile: a scalable metalearning algorithm,

    A. Nichol and J. Schulman, “Reptile: a scalable metalearning algorithm,” arXiv preprint arXiv:1803.02999, vol. 2, no. 3, p. 4, 2018. 1, 5

    2018 arXiv

  4. [12]

    Dynamic neural graphs based federated reptile for semi-supervised multi-tasking in healthcare applications,

    A. Thakur, P . Sharma, and D. A. Clifton, “Dynamic neural graphs based federated reptile for semi-supervised multi-tasking in healthcare applications,” IEEE Journal of Biomedical and Health Informatics, 2021. 1, 8 12

    2021

  5. [13]

    Bridging multi-task learning and meta-learning: Towards efficient training and effective adapta- tion,

    H. Wang, H. Zhao, and B. Li, “Bridging multi-task learning and meta-learning: Towards efficient training and effective adapta- tion,” in International Conference on Machine Learning. PMLR, 2021, pp. 10 991–11 002. 1, 5

    2021

  6. [14]

    Efficiently identifying task groupings for multi-task learning,

    C. Fifty, E. Amid, Z. Zhao, T. Yu, R. Anil, and C. Finn, “Efficiently identifying task groupings for multi-task learning,” in Advances in Neural Information Processing Systems, 2021. 1, 2, 3, 7, 8

    2021

  7. [15]

    Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,

    A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7482–7491. 1, 3, 8

    2018

  8. [16]

    Asymmetric multi-task learning based on task relatedness and loss,

    G. Lee, E. Yang, and S. Hwang, “Asymmetric multi-task learning based on task relatedness and loss,” in International conference on machine learning. PMLR, 2016, pp. 230–238. 1

    2016

  9. [17]

    Learning task grouping and overlap in multi-task learning,

    A. Kumar and H. Daum ´e III, “Learning task grouping and overlap in multi-task learning,” in Proceedings of the 29th International Coference on International Conference on Machine Learning , 2012, pp. 1723–1730. 1

    2012

  10. [18]

    Gradsign: Model performance inference with theoretical insights,

    Z. Zhang and Z. Jia, “Gradsign: Model performance inference with theoretical insights,” in International Conference on Learning Representations, 2021. 2, 3

    2021

  11. [19]

    Graph attention networks,

    P . Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P . Lio, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903, 2017. 2

    2017 arXiv

  12. [20]

    Cross-stitch networks for multi-task learning,

    I. Misra, A. Shrivastava, A. Gupta, and M. Hebert, “Cross-stitch networks for multi-task learning,” in Proceedings of the IEEE confer- ence on computer vision and pattern recognition , 2016, pp. 3994–4003. 3

    2016

  13. [21]

    Modeling task relationships in multi-task learning with multi-gate mixture- of-experts,

    J. Ma, Z. Zhao, X. Yi, J. Chen, L. Hong, and E. H. Chi, “Modeling task relationships in multi-task learning with multi-gate mixture- of-experts,” in Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining , 2018, pp. 1930–

    2018

  14. [22]

    Grad- norm: Gradient normalization for adaptive loss balancing in deep multitask networks,

    Z. Chen, V . Badrinarayanan, C.-Y. Lee, and A. Rabinovich, “Grad- norm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in International conference on machine learning. PMLR, 2018, pp. 794–803. 3, 8

    2018

  15. [23]

    End-to-end multi-task learn- ing with attention,

    S. Liu, E. Johns, and A. J. Davison, “End-to-end multi-task learn- ing with attention,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1871–1880. 3

    2019

  16. [24]

    Pyramidal person re-identification via multi-loss dynamic training,

    F. Zheng, C. Deng, X. Sun, X. Jiang, X. Guo, Z. Yu, F. Huang, and R. Ji, “Pyramidal person re-identification via multi-loss dynamic training,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8514–8522. 3

    2019

  17. [25]

    Dy- namic task prioritization for multitask learning,

    M. Guo, A. Haque, D.-A. Huang, S. Yeung, and L. Fei-Fei, “Dy- namic task prioritization for multitask learning,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 270–

    2018

  18. [26]

    Adaptive scheduling for multi- task learning,

    S. Jean, O. Firat, and M. Johnson, “Adaptive scheduling for multi- task learning,” arXiv preprint arXiv:1909.06434, 2019. 3

    1909 arXiv

  19. [27]

    Multi-task learning as a bargaining game,

    A. Navon, A. Shamsian, I. Achituve, H. Maron, K. Kawaguchi, G. Chechik, and E. Fetaya, “Multi-task learning as a bargaining game,” in International Conference on Machine Learning . PMLR, 2022, pp. 16 428–16 446. 3, 8

    2022

  20. [28]

    Efficient and effective multi-task grouping via meta learning on task combinations,

    X. Song, S. Zheng, W. Cao, J. Yu, and J. Bian, “Efficient and effective multi-task grouping via meta learning on task combinations,” Advances in Neural Information Processing Systems , vol. 35, pp. 37 647–37 659, 2022. 3

    2022

  21. [29]

    Towards theoretically inspired neural initialization optimization,

    Y. Yang, H. Wang, H. Yuan, and Z. Lin, “Towards theoretically inspired neural initialization optimization,” Advances in Neural Information Processing Systems, vol. 35, pp. 18 983–18 995, 2022. 3, 4

    2022

  22. [30]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009. 6, 8

    2009

  23. [31]

    Reading digits in natural images with unsupervised feature learning,

    Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011. 6, 8

    2011

  24. [32]

    Multi-class texture analysis in colorectal cancer histology,

    J. N. Kather, C.-A. Weis, F. Bianconi, S. M. Melchers, L. R. Schad, T. Gaiser, A. Marx, and F. G. Z ¨ollner, “Multi-class texture analysis in colorectal cancer histology,” Scientific reports , vol. 6, no. 1, p. 27988, 2016. 6, 8

    2016

  25. [33]

    Multixc-qm9: Large dataset of molecular and reaction energies from multi-level quantum chemical methods,

    S. Nandi, T. Vegge, and A. Bhowmik, “Multixc-qm9: Large dataset of molecular and reaction energies from multi-level quantum chemical methods,” Scientific Data, vol. 10, no. 1, p. 783, 2023. 7

    2023

  26. [34]

    Deep learning face attributes in the wild,

    Z. Liu, P . Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730–3738. 7

    2015

  27. [35]

    Mimic-iii, a freely accessible critical care database,

    A. E. Johnson, T. J. Pollard, L. Shen, H. L. Li-Wei, M. Feng, M. Ghassemi, B. Moody, P . Szolovits, L. A. Celi, and R. G. Mark, “Mimic-iii, a freely accessible critical care database,” Scientific data, vol. 3, no. 1, pp. 1–9, 2016. 7, 8

    2016

  28. [36]

    Multitask learning and benchmarking with clinical time series data,

    H. Harutyunyan, H. Khachatrian, D. C. Kale, G. V . Steeg, and A. Galstyan, “Multitask learning and benchmarking with clinical time series data,” Scientific Data, vol. 6, no. 96, pp. 1–18, 2019. 7

    2019

  29. [37]

    An Analysis of Sin- gle Layer Networks in Unsupervised Feature Learning,

    A. Coates, A. Ng, and H. Lee, “An Analysis of Sin- gle Layer Networks in Unsupervised Feature Learning,” in AISTATS, 2011, https://cs.stanford.edu/ ∼acoates/papers/ coatesleeng aistats 2011.pdf. 8

    2011

  30. [38]

    Pre-trained convo- lutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images,

    S. Rajaraman, S. K. Antani, M. Poostchi, K. Silamut, M. A. Hossain, R. J. Maude, S. Jaeger, and G. R. Thoma, “Pre-trained convo- lutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images,” PeerJ, vol. 6, p. e4568, 2018...

    2018

  31. [39]

    Wearing hat List of molecular properties/tasks used for QM9 experi- ments

  32. [40]

    α, Isotropic polarisability

  33. [41]

    ϵHOMO, Highest unoccupied molecular orbital energy

  34. [42]

    ϵLUMO, Lowest unoccupied molecular orbital energy

  35. [43]

    ∆ϵ, Gap betweenϵHOMO andϵLUMO 6)⟨R2⟩, Electronic spatial extent

  36. [44]

    ZPVE, Zero point vibrational energy

  37. [45]

    cv, Heat capavity at 298.15K

  38. [46]

    U ATOM 0 , Atomization energy at 0K

  39. [47]

    U ATOM, Atomization energy at 298.15K

  40. [48]

    HATOM, Atomization enthalpy at 298.15K

  41. [49]

    A short-term Fourier transform is performed on each signal using a segment size of 64 samples and an overlap of 32 samples to obtain spectrograms (time- frequency representation)

    GATOM, Atomization free energy at 298.15K Pre-processing ECG data MIMIC-III waveforms contain the ECG signals having a sampling rate of 256 Hz. A short-term Fourier transform is performed on each signal using a segment size of 64 samples and an overlap of 32 samples to obtain ...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.