REVIEW 2 major objections 4 minor 26 references
MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration
T0 review · 2 major / 4 minor · reviewed 2026-07-14 · grok-4.5
Pith's one-line read Domain-supervised routing plus one temperature per domain restores the global logit comparability that independently trained LoRA experts lose, enabling true multi-domain zero-shot classification.
desk verdict Solid practical fix for multi-domain LoRA merging; supervised routing drives nearly all the +15 pt gain while temperatures are a small add-on that mainly helps under imbalance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
MED-DSLC: a mixture-of-experts whose gating network is trained with domain cross-entropy (domain identity derived from the class label) and whose final logits are rescaled by a learned per-domain temperature τ_d before the joint softmax.
What would settle it
Construct a multi-domain benchmark in which domain identity cannot be recovered from class labels (or in which logit scales vary strongly inside a domain); if domain supervision becomes impossible and accuracy falls back to unsupervised MoLE levels while cross-domain over-confidence remains, the central claim fails.
Extended reading notes
Core claim
Independently trained domain LoRAs destroy the global logit comparability that zero-shot VLMs rely on; domain-supervised gating together with a learned scalar temperature per domain is sufficient to restore that comparability, so a single mixture can classify accurately over the union of many specialized label sets without domain labels at test time.
Load-bearing premise
A single temperature number per domain is enough to correct scale mismatch between experts, and every training example’s domain can be read unambiguously from its class label.
Editorial extensions
If this is right
- Mean accuracy on the union of nine fine-grained datasets rises roughly fifteen points over simple LoRA averaging and matches or slightly exceeds oracle expert selection.
- The same gains hold when the router and temperatures are trained with only one labeled example per class.
- Cross-domain interference grows with more domains or severe class imbalance; the calibration keeps the mixture scalable.
- Within-domain ranking is preserved while cross-domain magnitudes are aligned, restoring the zero-shot property of the original VLM.
- A fragmented collection of specialist adapters can be replaced by one lightweight mixture without sacrificing in-domain accuracy.
Reading between the lines
- The same per-domain temperature idea could be applied to any multi-adapter composition (language, multi-modal, style) whenever independent fine-tuning breaks score comparability.
- When domains are hierarchical or truly open-world, an unsupervised domain-discovery stage would become necessary before the supervised router can be used.
- Logit histograms and the origin domain of top competing classes become a practical diagnostic for whether calibration has succeeded.
- At hundreds of domains, shared temperature clusters or hierarchical routing may be required to keep the number of scalars manageable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses multi-expert-domain (MED) classification for CLIP-style VLMs: independently trained LoRA experts improve in-domain accuracy but destroy global logit comparability, so that when classes from many domains are scored under a joint softmax, scale mismatches produce cross-domain interference. The authors propose MED-DSLC, a mixture-of-experts architecture that (i) trains the gating network with explicit domain supervision derived from class labels (Eq. 9–12) and (ii) learns a single positive temperature τ_d per domain that rescales logits before the joint softmax (Eq. 14). Experiments on nine fine-grained datasets (base/all splits, in-domain and cross-domain protocols) show large gains over LoRA-mean, MoLE, Phatgoose and KnOTS variants; the method matches or slightly exceeds an oracle Expert-LoRA baseline and remains effective under one-shot routing, rank/backbone sweeps and class-composition stress tests.
Significance. If the claims hold, the work supplies a lightweight, data-efficient recipe for consolidating the growing zoo of domain-specific LoRAs into a single open-vocabulary classifier without sacrificing the zero-shot property of the underlying VLM. The combination of supervised routing and output-level calibration is simple, model-agnostic (CLIP and SigLIP), and shown to scale with the number of domains and with class imbalance. The extensive ablation suite (Tables 3–7, Fig. 4) and public code make the contribution immediately usable by practitioners facing multi-domain open-set recognition.
major comments (2)
- The abstract, introduction and conclusion repeatedly present domain supervision and domain-wise logit scaling as jointly essential for the headline +15 % gain and for “restoring global logit comparability.” Table 3 (cross-domain) shows that domain supervision alone accounts for almost the entire jump (71.80 → 85.43 base / 65.85 → 71.61 all); the subsequent addition of the learned temperatures τ_d contributes only +0.08 / +0.19 points. The same pattern appears in the in-domain ablations (supplementary Table 10). While Fig. 4 demonstrates a clearer benefit of LS under extreme target-domain imbalance (k=2), the primary balanced protocol that produces the +15 % number does not support the co-essential language. The manuscript should either (a) rephrase the central claim to reflect the relative contributions or (b) supply additional primary-protocol evidence that LS is load-bearing for the re
- Section 3.4 and Eq. (14) assert that a single positive scalar temperature per domain “preserves within-domain ranking while aligning cross-domain magnitudes.” No quantitative check of within-domain rank correlation or of residual cross-domain scale mismatch after calibration is provided for the main experimental setting. Given that the primary tables show only marginal accuracy gains from LS, such diagnostics are needed to substantiate the modeling assumption that a single τ_d is sufficient.
minor comments (4)
- Figure captions and the main text occasionally use the acronyms MED-LCDS / MED-DS interchangeably with MED-DSLC; a single consistent name would improve readability.
- The filtering of overlapping Caltech101 classes is disclosed (Sec. 4) but the precise list appears only in the appendix; moving a short statement of the filtering criterion into the main experimental section would help reproducibility.
- Hyper-parameter λ that balances L_cls and L_domain is never given a concrete value or sensitivity plot; a one-line statement of the chosen value (or a short ablation) would be useful.
- Several figure panels (Figs. 1, 5–9) shade only the largest logit; adding the numerical values of the top-2 or top-3 logits would make the cross-domain interference argument easier to verify visually.
Circularity Check
No circularity: empirical MoE + temperature scaling fitted and evaluated on held-out data; accuracy gains are not forced by construction.
full rationale
MED-DSLC is an engineering method paper. Domain labels are recovered from class labels by the trivial indicator of Eq. 9 (after the authors' explicit class-filtering step), the gating network is trained with the supervised domain loss of Eq. 11, and a single scalar temperature per domain is learned by gradient descent (Eq. 14). All of these parameters are fitted on 16-shot (or 1-shot) base-split data and then evaluated on the disjoint all-split and on cross-domain unions. No equation equates a reported accuracy number to a quantity that was itself the fitting objective; the +15 pt gain over LoRA-Mean is an empirical observation, not a tautology. Self-citations are limited to standard external baselines (CLIP, LoRA, MoLE, Phatgoose, KnOTS). The skeptic's observation that domain supervision accounts for nearly all of the gain while logit scaling contributes only ~0.1 pt on the primary tables is a correctness/emphasis critique, not a circularity reduction. Consequently the derivation chain contains no self-definitional step, no fitted-input-called-prediction, and no load-bearing self-citation uniqueness claim.
Assumptions & free parameters
free parameters (3)
- domain temperatures τ_d
- loss weight λ
- gating learning rate / batch size / optimizer
assumptions (3)
- domain assumption Domain identity of a training example is uniquely recoverable from its class label via membership in the domain label sets Y_d (Eq. 9).
- ad hoc to paper A single positive temperature per domain preserves within-domain ranking while aligning cross-domain magnitudes under joint softmax.
- domain assumption Independently trained LoRA residuals can be linearly mixed by input-dependent gates without destroying the frozen experts’ specialization.
invented entities (1)
-
MED-DSLC architecture (domain-supervised MoE + domain-wise temperatures)
Cite this review
Pith. "Pith review of MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration." pith.science (2026). https://pith.science/paper/LDHOECDM
@misc{pith2026260710985,
author = {Pith},
title = {Pith review of: MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration},
year = {2026},
howpublished = {\url{https://pith.science/paper/LDHOECDM}},
note = {Machine review of arXiv:2607.10985}
}
read the original abstract
Vision-language models (VLMs) such as CLIP enable zero-shot classification by comparing image features with text prompts in a shared embedding space. A fundamental property underlying this capability is the global comparability of logits across arbitrary candidate classes. However, VLMs are often adapted to fine-grained domains using techniques such as LoRA. While this improves in-domain accuracy, out-of-domain accuracy degrades. This leads to a highly fragmented model ecosystem, with thousands of specialized models. Multi-Expert-Domain classification seeks to address this problem, by merging LoRAs trained independently on specialized domains. However, due to the independent training, the various domain experts no longer produce globally calibrated logits. As a result, when evaluating over the union of multiple domain-specific class sets, heterogeneous logit scales induce cross-domain interference and artificially high confidence for out-of-domain classes, inducing prediction errors. In this work, we identify domain supervision and cross-domain logit miscalibration as the key issue to scalable multi-domain zero-shot recognition. We propose MED-DSLC, combining domain supervised training and domain-wise logit scaling, to explicitly restore global logit comparability. MED-DSLC is a lightweight solution for MED classification, which is shown to preserve within-domain discrimination while reducing cross-domain logit interference with minimal data. Extensive experiments across diverse fine-grained benchmarks demonstrate that it substantially improves mean accuracy (+15\%), cross-domain robustness, and scalability in the size of MED classification problem. Our results show that restoring output-level calibration is essential under highly data imbalanced settings for achieving a truly zero-shot VLM under multi-domain specialization.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Alipour, M., Mohammadi Amiri, M.: Towards reversible model merging for low- rank weights (2025),https://arxiv.org/abs/2510.14163, arXiv preprint
arXiv 2025
-
[2]
Journal of Machine Learning Research (JMLR)23, 1–39 (2022)
Fedus,W.,Zoph,B.,Shazeer,N.:Switchtransformers:Scalingtotrillionparameter models with simple and efficient sparsity. Journal of Machine Learning Research (JMLR)23, 1–39 (2022)
2022
-
[3]
In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Feng, W., Hao, C., Zhang, Y., Han, Y., Wang, H.: Mixture-of-loras: An efficient multitask tuning method for large language models. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). pp. 11371–11380 (2024)
2024
-
[4]
arXiv preprint arXiv:2312.12379 (2023)
Gou, Y., Liu, Z., Chen, K., Hong, L., Xu, H., Li, A., Yeung, D.Y., Kwok, J.T., Zhang, Y.: Mixture of cluster-conditional lora experts for vision-language instruc- tion tuning. arXiv preprint arXiv:2312.12379 (2023)
arXiv 2023
-
[5]
In: International Conference on Learning Representations (ICLR) (2022)
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, W.: Lora: Low-rank adaptation of large language models. In: International Conference on Learning Representations (ICLR) (2022)
2022
-
[6]
arXiv preprint arXiv:2307.13269 (2023)
Huang, C., Liu, Q., Lin, B.Y., Pang, T., Du, C., Lin, M.: Lorahub: Efficient cross- task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269 (2023)
arXiv 2023
-
[7]
Neural Computation3(1), 79–87 (1991).https://doi.org/10.1162/ neco.1991.3.1.79
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of lo- cal experts. Neural Computation3(1), 79–87 (1991).https://doi.org/10.1162/ neco.1991.3.1.79
1991
-
[8]
In: European Confer- ence on Computer Vision (ECCV)
Jia, M., Tang, L., Chen, B.C., et al.: Visual prompt tuning. In: European Confer- ence on Computer Vision (ECCV). pp. 709–726 (2022)
2022
Show all 26 references
-
[9]
Neural Computation6(2), 181–214 (1994)
Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the em algorithm. Neural Computation6(2), 181–214 (1994)
1994
-
[10]
In: International Conference on Learning Representations (ICLR) (2021)
Lepikhin, D., Lee, H., Xu, Y., et al.: Gshard: Scaling giant models with conditional computation and automatic sharding. In: International Conference on Learning Representations (ICLR) (2021)
2021
-
[11]
arXiv preprint arXiv:2503.07137 (2025) MED-DSLC 17
Mu, S., Lin, S.: A comprehensive survey of mixture-of-experts: Algorithms, theory, and applications. arXiv preprint arXiv:2503.07137 (2025) MED-DSLC 17
2025
-
[12]
arXiv preprint arXiv:2402.05859 (2024)
Muqeeth, M., Liu, H., Liu, Y., Raffel, C.: Learning to route among specialized experts for zero-shot generalization. arXiv preprint arXiv:2402.05859 (2024)
2024 arXiv
-
[13]
In: Salakhut- dinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F
Ostapenko, O., Su, Z., Ponti, E., Charlin, L., Le Roux, N., Caccia, L., Sordoni, A.: Towards modular LLMs by building and reusing a library of LoRAs. In: Salakhut- dinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F. (eds.) Proceedings of the...
2024
-
[14]
Panariello, A., Marczak, D., Magistri, S., Porrello, A., Twardowski, B., Bagdanov, A.D., Calderara, S., van de Weijer, J.: Accurate and efficient low-rank model merg- ing in core space (2025),https://arxiv.org/abs/2509.17786, arXiv preprint
2025
-
[15]
In: Proceedings of the 31st International Conference on Computational Linguistics (Industry Track)
Prabhakar, A., Li, Y., Narasimhan, K., Kakade, S., Malach, E., Jelassi, S.: Lora soups: Merging loras for practical skill composition tasks. In: Proceedings of the 31st International Conference on Computational Linguistics (Industry Track). pp. 644–655. Association for Computa...
2025
-
[16]
In: International Conference on Machine Learning (ICML)
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, T., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning (ICML). PMLR (2021)
2021
-
[17]
European Conference on Computer Vision (ECCV) 2024, LNCS15059, 422–438 (2025)
Shah, V., Ruiz, N., Cole, F., Lu, E., Lazebnik, S., Li, Y., Jampani, V.: Ziplora: Any subject in any style by effectively merging loras. European Conference on Computer Vision (ECCV) 2024, LNCS15059, 422–438 (2025)
2024
-
[18]
In: International Conference on Learning Representations (ICLR) (2017)
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., Dean, J.: Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In: International Conference on Learning Representations (ICLR) (2017)
2017
-
[19]
In: International Conference on Learning Representations (ICLR) (2025)
Stoica, G., Ramesh, P., Ecsedi, B., Choshen, L., Hoffman, J.: Model merging with svd to tie the knots. In: International Conference on Learning Representations (ICLR) (2025)
2025
-
[20]
In: Forty-second International Conference on Machine Learning (2025),https://openreview.net/forum?id= EqoKRSR5Pa
Wei, Y., Tang, A., Shen, L., Hu, Z., Yuan, C., Cao, X.: Modeling multi-task model merging as adaptive projective gradient descent. In: Forty-second International Conference on Machine Learning (2025),https://openreview.net/forum?id= EqoKRSR5Pa
2025
-
[21]
In: International Confer- ence on Learning Representations (2024),https://openreview.net/forum?id= uWvKBCYh4S
Wu, X., Huang, S., Wei, F.: Mixture of lora experts. In: International Confer- ence on Learning Representations (2024),https://openreview.net/forum?id= uWvKBCYh4S
2024
-
[22]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
Zanella, M., Ben Ayed, I.: Low-rank few-shot adaptation of vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 1593–1603 (2024)
2024
-
[23]
In: International Conference on Learning Representations (ICLR) 2025 (2025)
Zhao, Z., Shen, T., Zhu, D., Li, Z., Su, J., Wang, X., Wu, F.: Merging LoRAs like playing lego: Pushing the modularity of lora to extremes through rank-wise clustering. In: International Conference on Learning Representations (ICLR) 2025 (2025)
2025
-
[24]
Zheng, S., Wang, H., Huang, C., Wang, X., Chen, T., Fan, J., Hu, S., Ye, P.: Decoupleandorthogonalize:Adata-freeframeworkforloramerging(2025),https: //arxiv.org/abs/2505.15875, arXiv preprint
2025 arXiv
-
[25]
In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Conditional prompt learning for vision- language models. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 16837–16847 (2022)
2022
-
[26]
In: International Journal of Computer Vision (IJCV)
Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision-language models. In: International Journal of Computer Vision (IJCV). Springer (2022) 18 Z. Zeng, D. Sridhar et al. Method Batch Size Optimizer Learning rate MED 64 Adam 2e-3 MoLE 64 Adam 2e-3 Single-LoRA 64...
2022
Reviewed July 14, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.