Pith. sign in

REVIEW 5 cited by

Adapting Auxiliary Losses Using Gradient Similarity

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1812.02224 v2 pith:C5TUS6HM submitted 2018-12-05 stat.ML cs.LG

Yunshu Du , Wojciech M. Czarnecki , Siddhant M. Jayakumar , Mehrdad Farajtabar , Razvan Pascanu , Balaji Lakshminarayanan This is my paper
classification stat.MLcs.LG
keywords auxiliarylearningmaintaskapproachhelpfullosslosses
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations. However, it is not always trivial to know if an auxiliary task will be helpful for the main task and when it could start hurting. We propose to use the cosine similarity between gradients of tasks as an adaptive weight to detect when an auxiliary loss is helpful to the main loss. We show that our approach is guaranteed to converge to critical points of the main task and demonstrate the practical usefulness of the proposed algorithm in a few domains: multi-task supervised learning on subsets of ImageNet, reinforcement learning on gridworld, and reinforcement learning on Atari games.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 95 citations worldwide. Full citation record

  1. Lantern: Conflict-Aware Gradient Blending for Physics-Guided Diffusion Models in Calorimeter Simulation

    cs.LG 2026-07 conditional novelty 6.5 of 10

    GradBlend anchors diffusion updates to denoising while admitting physics auxiliaries, improving calorimeter shower FPD and CFD where PCGrad, GradNorm, IMTL-G, and ConFIG inflate FPD by 2–100×.

  2. Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control

    cs.RO 2025-09 conditional novelty 6.0 of 10

    PEGrad projects energy-minimization gradients orthogonal to task-reward gradients in RL, achieving 64% torque reduction in simulation and reduced battery draw on a Unitree Go2 without sacrificing task reward.

  3. Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

    cs.IR 2026-01 reject novelty 5.0 of 10

    GEMS proposes a multi-subspace gradient-tuning framework for unified search and recommendation in LLMs, but its null-space projection equation is inverted and its adaptive gate has no training rule.

  4. Understanding Knowledge Transferability for Transfer Learning: A Survey

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A survey that classifies transferability metrics by knowledge modality (dataset vs. model) and granularity (task vs. instance), with a theoretical primer and applications to eight learning paradigms.

  5. A Survey of State Representation Learning for Deep Reinforcement Learning

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A six-class taxonomy of state representation learning methods for model-free online deep reinforcement learning, with selection guidelines, evaluation metrics, and future directions.

Pith tools