Pith. sign in

REVIEW 3 major objections 4 minor 74 references

Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Tasks

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A shared decoder trained alternately on depth and auxiliary vision tasks improves monocular depth estimation by about 11% over the DINOv2 baseline, and allows reducing depth training data by 80-99%.

desk verdict Solid empirical study of auxiliary-task training for monocular depth, but the headline gain is undercut by a missing control: the proposed method takes twice as many optimizer steps as the baseline. read the letter →

arxiv 2501.12824 v1 pith:SZQY77TQ submitted 2025-01-22 cs.CV

Alessio Quercia , Erenus Yildiz , Zhuo Cao , Kai Krajsek , Abigail Morrison , Ira Assent , Hanno Scharr This is my paper · ORCID
classification cs.CV
keywords monoculardepthestimationmulti-tasklearningauxiliarytasksmulti-labeldenseclassificationDINOv2DPTdecoderdataefficiencysemanticsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that monocular depth estimation—predicting depth from a single image—can be improved cheaply by training a shared decoder on auxiliary vision data from other tasks, without fine-tuning the large frozen backbone. It claims that alternating depth steps with segmentation, multi-label dense classification, or reconstruction steps, weighted toward depth, improves absolute relative error by about 11% on average across six in-domain datasets over a DINOv2 baseline. It further claims that the same scheme lets one cut MDE training data by 80-99% without losing quality. The paper also argues that the improvement comes from the task itself, not merely from adding data, since using the same dataset with different auxiliary tasks gives different results.

What carries the argument

The load-bearing mechanism is an alternating gradient-step schedule with a task-focusing parameter α. A frozen DINOv2 ViT-G extracts features; a single shared DPT decoder plus two small task-specific heads produces depth and auxiliary outputs. Each global update first takes an MDE step scaled by α, then an auxiliary step scaled by 1−α, so the decoder is reshaped by both signals while depth dominates. MLDC is defined by spatially averaging the segmentation output to a multi-label vector and training with binary cross-entropy, which isolates class-presence information from precise positioning.

What would settle it

Run the exact same training scheme with MLDC fixed as the auxiliary task on all six in-domain datasets, choosing nothing per dataset, and compare average AbsRel to the DINOv2 baseline; if the average gain is near zero or negative, the headline gain depends on post-hoc task selection rather than on the training scheme.

Watch

Extended reading notes

Core claim

The central discovery is that a frozen vision foundation model can be made a better monocular depth estimator by training a single shared DPT decoder on depth and auxiliary tasks alternately, with separate task heads and a task-focusing parameter α = 0.9 that weights depth updates more heavily. On six in-domain depth datasets, the best auxiliary task per dataset reduces absolute relative error by an average of about 11% relative to the DINOv2 baseline; with the MIX6 collection of six segmentation datasets, multi-label dense classification (MLDC)—averaging the segmentation map over space and classifying which classes are present—usually gives the largest gain. The same scheme also improves the stronger Depth Anything backbone by about 2.5% on average. The paper further reports that the method preserves or improves quality while using 80-99% less MDE training data on the tested datasets.

Load-bearing premise

The load-bearing premise is that the reported ~11% average gain belongs to the method itself, although the table computes it by selecting, after the fact, whichever auxiliary task worked best for each dataset; for a new dataset there is no rule for making that choice.

Editorial extensions

If this is right

  • If the central claim holds, MDE datasets can be cut by 80-99% with no quality loss, directly lowering the cost of collecting dense depth labels.
  • Existing semantic segmentation datasets become reusable as depth-training resources, especially when converted to MLDC.
  • Auxiliary task choice matters more than raw data volume, so future systems should treat task selection as a first-class design decision.
  • The gain is not tied to one backbone: the same alternating scheme improves Depth Anything as well as DINOv2 ViT-G.
  • Out-of-domain auxiliary data can hurt, as shown by KITTI, so auxiliary data should be matched to the target scene domain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical follow-up would test whether a fixed rule such as 'always use MLDC' preserves most of the gain; the paper's ~11% is computed with the best task chosen per dataset, so a fixed procedure may give less.
  • Because image reconstruction sometimes wins on outdoor scenes, the most useful auxiliary signal may depend on scene type; dynamic task weighting could outperform any single task.
  • If class-presence information is what helps, then cheap image-level classifiers trained on large classification corpora may substitute for dense segmentation labels, potentially extending the data-efficiency result.
  • The same shared-decoder, frozen-backbone recipe may transfer to other dense prediction targets, such as surface normals or optical flow, whenever a related auxiliary dataset is available.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes an alternating training scheme for monocular depth estimation (MDE) in which a frozen DINOv2 ViT-G backbone with a shared DPT decoder is trained on MDE batches and on batches from auxiliary vision tasks (semantic segmentation, multi-label dense classification (MLDC), or image reconstruction), with the MDE update weighted by a task-focusing parameter α and the auxiliary update by 1−α. The method is evaluated on six in-domain MDE datasets using MIX6 (a union of six auxiliary datasets) and compared against the DINOv2 zero-shot baseline. The authors report an average ~11% AbsRel improvement when selecting the best auxiliary task per dataset, show that the same scheme preserves or improves quality when the MDE training set is reduced by 80–99%, and provide an additional validation with a Depth Anything backbone. They also claim that the gains come from the auxiliary task rather than from simply adding data, based on comparisons across tasks on the same auxiliary data.

Significance. If the central effect is real, this is a useful practical result: it shows that a frozen foundation model's decoder can be improved by auxiliary dense classification datasets without fine-tuning the backbone, and it identifies MLDC as an often-effective auxiliary task. The empirical effort is substantial: six in-domain MDE datasets, three auxiliary tasks, four independent runs per condition with standard errors, and ablations over the task-focusing parameter, learning-rate scaling, and backbone choice. Code release is promised upon acceptance. However, the missing control for the doubled optimization budget and the post-hoc per-dataset task selection mean that the magnitude and attribution of the headline gain are not yet established. These issues are fixable with additional experiments and a re-analysis of the reported averages.

major comments (3)
  1. [Section 3, Eqs. (4)-(5); Appendix A] The proposed global step consists of two consecutive optimizer updates: one MDE update at learning rate αη and one auxiliary update at (1−α)η, whereas the DINOv2 baseline is trained for 38400 single-update steps. Thus the proposed method performs 38400 MDE updates plus 38400 auxiliary updates on the shared decoder, i.e., twice as many total gradient updates, twice as many data batches, and a second optimizer/AdamW state. The ablations in Section 5.1 (Figure 5) vary the learning-rate scaling γ and the auxiliary scaling β, but they never include a baseline trained for the same number of total updates using only MDE data (e.g., 76800 MDE-only steps, or two MDE updates per global step). Without this control, the observed ~11% gain cannot be attributed solely to the auxiliary task; it may be partly a consequence of extra optimization steps or data presentations. This is load-bearing for the paper's central claim that improvements are not achieved by merely adding data and that the gains come from the selected auxiliary task.
  2. [Table 3; Section 4.1] The reported ~11% average gain is computed by selecting, for each MDE dataset, the best-performing auxiliary task among classification, segmentation, and reconstruction. The paper provides no rule for choosing the auxiliary task for a new dataset, and some fixed choices are much worse than the best (e.g., reconstruction on NYUv2 is worse than baseline). The 'consistent improvement' claim therefore applies to a post-hoc per-dataset oracle, not to a fixed procedure. I recommend reporting averages for each fixed auxiliary task; for instance, using MLDC fixed across all six in-domain datasets yields roughly a 9.5% average gain, which is still positive but lower than the headline 11%. The 11% figure should be described as an upper envelope over task choices, and any claim of consistency should be tied to a predetermined task-selection rule.
  3. [Section 4.2; Figure 4] The data-efficiency claim inherits the same optimization-budget confound. When comparing proposed models trained on 1–20% of the MDE data against the full-data DINOv2 baseline, the proposed models receive auxiliary updates in addition to their MDE updates. To support the '80–99% less data' statement, the comparison should include a control that uses the same total number of gradient updates on MDE data only (for example, a baseline trained for twice as many MDE-only steps, or a reduced-data baseline with the same total update budget). Without this control, part of the apparent data-efficiency gain could again be an artifact of the doubled update count rather than of the auxiliary task information.
minor comments (4)
  1. [Section 3, Eqs. (4)-(5)] The gradient notation ∇_{θ_{t-1}, φ_{t-1}} suggests an update to both θ and φ, but the equation only updates θ. Please write the θ-component explicitly (e.g., ∇_θ L_D) or clarify that the φ-component is handled by the separate task-head learning rate.
  2. [Section 4.2] The phrase 'without decreasing the performance' is ambiguous: it should explicitly state that the reference is the full-data DINOv2 baseline, not the reduced-data baseline.
  3. [Figure 4] The figure caption does not state that error bars represent standard errors; please add this for consistency with the rest of the paper.
  4. [Appendix C.2] The pseudo-labeling procedure for Taskonomy, NYUv2, Matterport3D, and DIODE is described only by reference to PolyMax; please specify the model used to generate the pseudo labels and any preprocessing, since the quality of these labels directly affects the single-source ablation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported MDE gains come from empirical comparison against a fixed DINOv2 baseline, not from a parameter fitted to the test metric.

full rationale

The derivation in Section 3 is an explicit training procedure: each joint step alternates an MDE update at rate alpha*eta (Eq. 4) with an auxiliary update at rate (1-alpha)*eta (Eq. 5), and all reported gains are measured on held-out test splits against a fixed DINOv2 baseline. The hyperparameter alpha=0.9 is chosen from validation behavior in Figure 1 and Section 5.1; it is not fitted to test metrics, and no final-model parameter is defined in terms of the test set. The average ~11% improvement is an empirical outcome of that fixed procedure, not a quantity recovered from its own inputs by construction. One caveat, which the paper itself flags in Section 7 via 'the observed task dependency highlights the significance of carefully choosing auxiliary tasks and datasets', is that Table 3 reports the gain of the best auxiliary task per dataset, so the headline average is an upper envelope rather than the expected gain of a fully automatic task-selection rule; that is a selection-bias and generalization concern, not circularity. The only reference with author overlap is the JUWELS hardware citation [26] in the acknowledgements, used solely to credit compute infrastructure, and it is not load-bearing. No 'uniqueness theorem', prior self-citation chain, or ansatz citation is used to force the method; MLDC is defined directly from segmentation outputs in Section 3, and its comparison with semantic segmentation is an empirical result on common test data. The central claim thus stands on external benchmarks and a fixed baseline rather than reducing to a fit or to self-citation.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The method introduces a new task formulation (MLDC) but that is a loss function, not an entity. The main free parameter is alpha. The key assumptions concern feature quality and domain match.

free parameters (1)
  • alpha (task-focusing parameter) = 0.9
    Scales the decoder learning rate between MDE (alpha) and the auxiliary task (1-alpha). Chosen as the best value on NYUv2 (Figure 1) and fixed for all datasets; the paper states the optimal alpha may vary per dataset.
assumptions (3)
  • domain assumption Frozen DINOv2 ViT-G features contain enough geometric and semantic information for a shared DPT decoder to transfer auxiliary task knowledge into depth.
    The backbone is never fine-tuned, so all task transfer must happen through the fixed features and the decoder (Section 3). If the frozen features were not already depth- and semantics-aware, the auxiliary signal could not help.
  • domain assumption The MIX6 auxiliary datasets (ADE20K, SUN RGBD, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context) are in-domain with the target indoor/outdoor MDE datasets.
    The method fails on KITTI (Section 4.1, Table 3), which is blamed on out-of-distribution auxiliary data; the claim of improvement is therefore scoped to datasets whose distribution is covered by MIX6.
  • domain assumption Spatially averaging segmentation logits across the image produces a training signal for multi-label classification that is useful for MDE.
    The MLDC objective is defined this way in Section 3. Its usefulness is a finding of the paper, but the specific spatial-averaging choice is assumed to be a good proxy for 'which classes are present'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Tasks." pith.science (2026). https://pith.science/paper/SZQY77TQ

@misc{pith2026250112824,
  author       = {Pith},
  title        = {Pith review of: Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZQY77TQ}},
  note         = {Machine review of arXiv:2501.12824}
}
read the original abstract

Monocular depth estimation (MDE) is a challenging task in computer vision, often hindered by the cost and scarcity of high-quality labeled datasets. We tackle this challenge using auxiliary datasets from related vision tasks for an alternating training scheme with a shared decoder built on top of a pre-trained vision foundation model, while giving a higher weight to MDE. Through extensive experiments we demonstrate the benefits of incorporating various in-domain auxiliary datasets and tasks to improve MDE quality on average by ~11%. Our experimental analysis shows that auxiliary tasks have different impacts, confirming the importance of task selection, highlighting that quality gains are not achieved by merely adding data. Remarkably, our study reveals that using semantic segmentation datasets as Multi-Label Dense Classification (MLDC) often results in additional quality gains. Lastly, our method significantly improves the data efficiency for the considered MDE datasets, enhancing their quality while reducing their size by at least 80%. This paves the way for using auxiliary data from related tasks to improve MDE quality despite limited availability of high-quality labeled data. Code is available at https://jugit.fz-juelich.de/ias-8/mdeaux.

Figures

Figures reproduced from arXiv: 2501.12824 by the authors.

Figure 1
Figure 1. (Top) NYUv2 results with MIX6 auxiliary MLDC. From [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed training pipeline. We use a frozen pre-trained DINOv2 ViT-G backbone ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Absolute Relative Error (AbsRel) of MDE on NYUv2, SUN RGBD and DIODE Outdoor using the DINOv2 baseline and our [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: AbsRel of models trained with various fractions of the dataset. The dataset sizes are reported in Table [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: AbsRel (↓) when varying the learning rate. (Left) Learning rate tuning for the DINOv2 baseline by a factor γ. (Right) Our method re-designed to do unscaled depth steps and auxiliary steps scaled by a factor β, using MIX6 MLDC task. For both plots, dashed and dotted-das…
Figure 6
Figure 6. Figure 6: Results on NYU with MIX6 auxiliary MLDC task. From left to right: image and respective ground truth, baseline and our [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Results on SUNRGBD with MIX6 auxiliary MLDC task. From left to right: image and respective ground truth, baseline and our [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Results on Matterport with MIX6 auxiliary MLDC task. From left to right: input image and respective ground truth, baseline [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Results on Taskonomy with MIX6 auxiliary MLDC task. From left to right: input image and respective ground truth, baseline [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Results on DIODE Indoor with MIX6 auxiliary MLDC task. From left to right: input image and respective ground truth, [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Results on DIODE Outdoor with MIX6 auxiliary MLDC task. Left to right: input image and ground truth; baseline and our [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 52 canonical work pages

  1. [1]

    Task2vec: Task embedding for meta-learning

    Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Ste- fano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 6430–6439,

    work page

  2. [2]

    Green- house gas equivalencies calculator, Mar 2024

    United States Environmental Protection Agency. Green- house gas equivalencies calculator, Mar 2024. 12

    work page 2024

  3. [3]

    Semantics- depth-symbiosis: Deeply coupled semi-supervised learning of semantics and depth

    Nitin Bansal, Pan Ji, Junsong Yuan, and Yi Xu. Semantics- depth-symbiosis: Deeply coupled semi-supervised learning of semantics and depth. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 5828–5839, 2023. 2

    work page 2023

  4. [4]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 1, 2

    arXiv 2023

  5. [5]

    Vision transformer adapters for generalizable multi- task learning

    Deblina Bhattacharjee, Sabine S ¨usstrunk, and Mathieu Salz- mann. Vision transformer adapters for generalizable multi- task learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19015–19026, 2023. 2

    work page 2023

  6. [6]

    Midas v3

    Reiner Birkl, Diana Wofk, and Matthias M ¨uller. Midas v3. 1–a model zoo for robust monocular relative depth estima- tion. arXiv preprint arXiv:2307.14460, 2023. 1, 2

    arXiv 2023

  7. [7]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. In Computer vision and pattern recognition (CVPR), 2018 IEEE conference on . IEEE, 2018. 5

    work page 2018

  8. [8]

    Multitask learning

    Rich Caruana. Multitask learning. Machine learning, 28:41– 75, 1997. 2, 3

    work page 1997

Show all 74 references
  1. [9]

    Depth prediction without the sensors: Leveraging structure for unsupervised learning from monocular videos

    Vincent Casser, Soeren Pirk, Reza Mahjourian, and Anelia Angelova. Depth prediction without the sensors: Leveraging structure for unsupervised learning from monocular videos. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 33, pages 8001–8008, 2019. 3

    2019

  2. [10]

    Matterport3d: Learning from rgb- d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb- d data in indoor environments. International Conference on 3D Vision (3DV), 2017. 5

    2017

  3. [11]

    Auxiliary learning with joint task and data schedul- ing

    Hong Chen, Xin Wang, Chaoyu Guan, Yue Liu, and Wenwu Zhu. Auxiliary learning with joint task and data schedul- ing. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors,Pro- ceedings of the 39th International Conference on Mac...

    2022

  4. [12]

    Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks

    Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and An- drew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In In- ternational conference on machine learning, pages 794–803. PMLR, 2018. 2, 3

    2018

  5. [13]

    MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark

    MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark. https : / / github . com / open - mmlab/mmsegmentation, 2020. 5

    2020

  6. [14]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognitio...

    2016

  7. [15]

    Efficient multi-task pro- gressive learning for semantic segmentation and disparity es- timation

    Hanz Cuevas-Velasquez, Alejandro Gal ´an-Cuenca, Robert B Fisher, and Antonio Javier Gallego. Efficient multi-task pro- gressive learning for semantic segmentation and disparity es- timation. Pattern Recognition, 154:110601, 2024. 2

    2024

  8. [16]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 12

    2009

  9. [17]

    Dery, Paul Michel, Mikhail Khodak, Graham Neu- big, and Ameet Talwalkar

    Lucio M. Dery, Paul Michel, Mikhail Khodak, Graham Neu- big, and Ameet Talwalkar. Aang: Automating auxiliary learning. ArXiv, abs/2205.14082, 2022. 3

    2022 arXiv

  10. [18]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  11. [19]

    Representation similar- ity analysis for efficient task taxonomy & transfer learning

    Kshitij Dwivedi and Gemma Roig. Representation similar- ity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12387–12396, 2019. 3

    2019

  12. [20]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) chal- lenge. International Journal of Computer Vision, 88(2):303– 338, June 2010. 5

    2010

  13. [21]

    Efficiently identifying task groupings for multi-task learning

    Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Pro- cessing Systems, 34:27503–27516, 2021. 3

    2021

  14. [22]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. Interna- tional Journal of Robotics Research (IJRR), 2013. 5

    2013

  15. [23]

    Dynamic task prioritization for multitask learning

    Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei. Dynamic task prioritization for multitask learning. In Proceedings of the European conference on com- puter vision (ECCV), pages 270–287, 2018. 2, 3

    2018

  16. [24]

    S 3 dmt-net: improving soft sharing based multi-task cnn using task-specific distillation and cross-task interactions

    Ankit Jha, Biplab Banerjee, and Subhasis Chaudhuri. S 3 dmt-net: improving soft sharing based multi-task cnn using task-specific distillation and cross-task interactions. In Pro- ceedings of the Twelfth Indian Conference on Computer Vi- sion, Graphics and Image Processing, pag...

    2021

  17. [25]

    Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491,

  18. [26]

    Juwels booster– a supercomputer for large-scale ai research

    Stefan Kesselheim, Andreas Herten, Kai Krajsek, Jan Ebert, Jenia Jitsev, Mehdi Cherti, Michael Langguth, Bing Gong, Scarlet Stadtler, Amirpasha Mozaffari, et al. Juwels booster– a supercomputer for large-scale ai research. In International Conference on High Performance Comput...

  19. [27]

    Uvim: A unified modeling approach for vision with learned guiding codes

    Alexander Kolesnikov, Andr ´e Susano Pinto, Lucas Beyer, Xiaohua Zhai, Jeremiah Harmsen, and Neil Houlsby. Uvim: A unified modeling approach for vision with learned guiding codes. Advances in Neural Information Processing Systems, 35:26295–26308, 2022. 2

    2022

  20. [28]

    Quantifying the carbon emissions of machine learning

    Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700, 2019. 12

    1910 arXiv

  21. [29]

    Efficient multi-task uncertainties for joint se- mantic segmentation and monocular depth estimation

    Steven Landgraf, Markus Hillemann, Theodor Kapler, and Markus Ulrich. Efficient multi-task uncertainties for joint se- mantic segmentation and monocular depth estimation. arXiv preprint arXiv:2402.10580, 2024. 2

    2024 arXiv

  22. [30]

    From big to small: Multi-scale local planar guidance for monocular depth estimation

    Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv preprint arXiv:1907.10326, 2019. 2

    1907 arXiv

  23. [31]

    Monocular depth esti- mation using relative depth maps

    Jae-Han Lee and Chang-Su Kim. Monocular depth esti- mation using relative depth maps. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 9167–9176, 2019. 2

    2019

  24. [32]

    Learning scribbles for dense depth: Weakly-supervised single underwater image depth es- timation boosted by multi-task learning

    Kunqian Li, Xiya Wang, Wenjie Liu, Qi Qi, Guojia Hou, Zhiguo Zhang, and Kun Sun. Learning scribbles for dense depth: Weakly-supervised single underwater image depth es- timation boosted by multi-task learning. IEEE Transactions on Geoscience and Remote Sensing, 2024. 2

    2024

  25. [33]

    Learning multi- ple dense prediction tasks from partially annotated data

    Wei-Hong Li, Xialei Liu, and Hakan Bilen. Learning multi- ple dense prediction tasks from partially annotated data. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 18879–18889, 2022. 2

    2022

  26. [34]

    Monocular depth estimation toolbox

    Zhenyu Li. Monocular depth estimation toolbox. https: / / github . com / zhyever / Monocular - Depth - Estimation-Toolbox, 2022. 5

    2022

  27. [35]

    Depthformer: Exploiting long-range correlation and local in- formation for accurate monocular depth estimation

    Zhenyu Li, Zehui Chen, Xianming Liu, and Junjun Jiang. Depthformer: Exploiting long-range correlation and local in- formation for accurate monocular depth estimation. arXiv preprint arXiv:2203.14211, 2022. 2

    2022 arXiv

  28. [36]

    Binsformer: Revisiting adaptive bins for monocular depth estimation

    Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang. Binsformer: Revisiting adaptive bins for monocular depth estimation. arXiv preprint arXiv:2204.00987, 2022. 2

    2022 arXiv

  29. [37]

    Deep con- volutional neural fields for depth estimation from a single image

    Fayao Liu, Chunhua Shen, and Guosheng Lin. Deep con- volutional neural fields for depth estimation from a single image. In Proceedings of the IEEE International Conference on Computer Vision, pages 5162–5170, 2015. 2

    2015

  30. [38]

    Meta-auxiliary learning for future depth pre- diction in videos

    Huan Liu, Zhixiang Chi, Yuanhao Yu, Yang Wang, Jun Chen, and Jin Tang. Meta-auxiliary learning for future depth pre- diction in videos. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 5756–5765, January 2023. 3

    2023

  31. [39]

    End- to-end multi-task learning with attention

    Shikun Liu, Edward Johns, and Andrew J Davison. End- to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1871–1880, 2019. 2, 3

    2019

  32. [40]

    A Survey on RGB-D Datasets

    Alexandre Lopes, Roberto Souza, and Helio Pedrini. A Survey on RGB-D Datasets. arXiv e-prints , page arXiv:2201.05761, Jan. 2022. 5

    2022 arXiv

  33. [41]

    The role of context for object detection and semantic segmentation in the wild

    Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and semantic segmentation in the wild. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. 5

    2014

  34. [42]

    Indoor segmentation and support inference from rgbd images

    Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012. 5

    2012

  35. [43]

    Real-time joint se- mantic segmentation and depth estimation using asymmetric annotations

    Vladimir Nekrasov, Thanuja Dharmasiri, Andrew Spek, Tom Drummond, Chunhua Shen, and Ian Reid. Real-time joint se- mantic segmentation and depth estimation using asymmetric annotations. In 2019 International Conference on Robotics and Automation (ICRA), pages 7101–7107. IEEE, 2019. 2

    2019

  36. [44]

    Matterport3d eula for academic use, Mar

    Matthias Niessner. Matterport3d eula for academic use, Mar

  37. [45]

    All in tokens: Unifying output space of visual tasks via soft token

    Jia Ning, Chen Li, Zheng Zhang, Chunyu Wang, Zigang Geng, Qi Dai, Kun He, and Han Hu. All in tokens: Unifying output space of visual tasks via soft token. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 19900–19910, 2023. 2

    2023

  38. [46]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, ...

    2023 arXiv

  39. [47]

    Zero-shot task transfer

    Arghya Pal and Vineeth N Balasubramanian. Zero-shot task transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2189– 2198, 2019. 3

    2019

  40. [48]

    Multi-task distributed learning using vision transformer with random patch permu- tation

    Sangjoon Park and Jong Chul Ye. Multi-task distributed learning using vision transformer with random patch permu- tation. IEEE Transactions on Medical Imaging, 2022. 2

    2022

  41. [49]

    Carbon emissions and large neural network training

    David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 , 2021. 12

    2021 arXiv

  42. [50]

    On the uncertainty of self-supervised monocular depth estimation

    Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mat- toccia. On the uncertainty of self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3227–3237, 2020. 2

    2020

  43. [51]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021. 1, 2, 3

    2021

  44. [52]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 2

    2020

  45. [53]

    Improving monocular depth esti- mation by semantic pre-training

    Peter Rottmann, Thorbj ¨orn Posewsky, Andres Milioto, Cyrill Stachniss, and Jens Behley. Improving monocular depth esti- mation by semantic pre-training. In 2021 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 5916–5923, 2021. 3

    2021

  46. [54]

    An overview of multi-task learning in deep neural networks

    Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098, 2017. 2

    2017 arXiv

  47. [55]

    Learning to relate depth and semantics for unsupervised domain adaptation

    Suman Saha, Anton Obukhov, Danda Pani Paudel, Menelaos Kanakis, Yuhua Chen, Stamatios Georgoulis, and Luc Van Gool. Learning to relate depth and semantics for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2021

  48. [56]

    Multi-task learning as multi-objective optimization

    Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural informa- tion processing systems, 31, 2018. 2

    2018

  49. [57]

    Fully convolutional networks for semantic segmentation

    Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016. 2

    2016

  50. [58]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 567–576, 2015. 5

    2015

  51. [59]

    Which tasks should be learned together in multi-task learning? In International Conference on Machine Learning, pages 9120–9132

    Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In International Conference on Machine Learning, pages 9120–9132. PMLR,

  52. [60]

    Swin- mtl: A shared architecture for simultaneous depth estimation and semantic segmentation from monocular camera images

    Pardis Taghavi, Reza Langari, and Gaurav Pandey. Swin- mtl: A shared architecture for simultaneous depth estimation and semantic segmentation from monocular camera images. arXiv preprint arXiv:2403.10662, 2024. 2

    2024 arXiv

  53. [61]

    Multi-task learning for dense prediction tasks: A survey

    Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool. Multi-task learning for dense prediction tasks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(7):3614–3633, 2021. 2

    2021

  54. [62]

    Dai, Andrea F

    Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z. Dai, Andrea F. Daniele, Moham- madreza Mostajabi, Steven Basart, Matthew R. Walter, and Gregory Shakhnarovich. DIODE: A Dense Indoor and Out- door DEpth Dataset. arXiv e-prints, page arXiv:1908.00...

    1908 arXiv

  55. [63]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 1, 2

    2017

  56. [64]

    Neural taskon- omy: Inferring the similarity of task-derived representations from brain activity

    Aria Wang, Michael Tarr, and Leila Wehbe. Neural taskon- omy: Inferring the similarity of task-derived representations from brain activity. Advances in Neural Information Pro- cessing Systems, 32, 2019. 3

    2019

  57. [65]

    Sdc-depth: Semantic divide-and-conquer net- work for monocular depth estimation

    Lijun Wang, Jianming Zhang, Oliver Wang, Zhe Lin, and Huchuan Lu. Sdc-depth: Semantic divide-and-conquer net- work for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 541–550, 2020. 2

    2020

  58. [66]

    Domain adaptive semantic segmentation with self-supervised depth estimation

    Qin Wang, Dengxin Dai, Lukas Hoyer, Luc Van Gool, and Olga Fink. Domain adaptive semantic segmentation with self-supervised depth estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8515–8525, 2021. 3

    2021

  59. [67]

    Images speak in images: A generalist painter for in-context visual learning

    Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6830–6839, 2023. 2

    2023

  60. [68]

    Structured attention guided convolutional neu- ral fields for monocular depth estimation

    Dan Xu, Wei Wang, Hao Tang, Hong Liu, Nicu Sebe, and Elisa Ricci. Structured attention guided convolutional neu- ral fields for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 3917–3925, 2018. 2

    2018

  61. [69]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. arXiv:2401.10891,

    arXiv

  62. [70]

    Polymax: General dense prediction with mask transformer

    Xuan Yang, Liangzhe Yuan, Kimberly Wilber, Astuti Sharma, Xiuye Gu, Siyuan Qiao, Stephanie Debats, Huisheng Wang, Hartwig Adam, Mikhail Sirotenko, et al. Polymax: General dense prediction with mask transformer. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications...

    2024

  63. [71]

    Taskonomy: Disentangling task transfer learning

    Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 3712–3722, 2018. 3, 5

    2018

  64. [72]

    A survey on multi-task learn- ing

    Yu Zhang and Qiang Yang. A survey on multi-task learn- ing. IEEE Transactions on Knowledge and Data Engineer- ing, 34(12):5586–5609, 2021. 2

    2021

  65. [73]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5122–5130,

    2017

  66. [2017]

    Upon acceptance, we are committed to publishing the code to facilitate trans- parency and enable other researchers to replicate our find- ings

    5 Supplemental Material Reproducibility Statement All results presented in this work are reproducible using the code associated with our research. Upon acceptance, we are committed to publishing the code to facilitate trans- parency and enable other researchers to replicate ou...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.