{"id":"eba28fc3-12c4-4b6d-8207-bf8e8383be54","arxiv_id":"2411.13787","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RouteT2I is a learned router that predicts the quality gap between edge and cloud text-to-image models from the prompt alone and sends only hard prompts to the cloud.","lead":"The paper presents RouteT2I, a system that decides, for each text-to-image request, whether to run a cheap small model on the phone or an expensive large model in the cloud. It does this by predicting how much better the cloud image would be using a learned score, and it reports cutting cloud calls by about 70% while keeping most of the quality.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 83.97% quality gain is not reproducible from Eq. (18): Table 2's values require an unstated absolute value in the numerator, which inflates the headline DeltaP.","rationale":"The reader's weakest-assumption is that the CLIP-contrastive quality metric Q is a valid proxy for human-perceived quality. That is a legitimate concern, and Appendix D's HPSv2/MPS results provide only partial reassurance (one model pair, one routing rate). However, the internal inconsistency between Eq. (18) and Table 2 is more directly load-bearing because the abstract's headline numbers are DeltaP-based. Recomputing with the printed formula gives 56.8%, not 83.97%; only an unstated absolute-value numerator yields the reported value. This is not a cosmetic typo: it changes what DeltaP means and inflates the claimed quality maintenance and cost savings. The paper's core idea and architecture remain plausible, and there is some independent support from human-preference scorers, so a reject would be too harsh. A conditional acceptance requiring (1) correction of Eq. (18) or explicit justification of the absolute value, (2) recomputation of all DeltaP and gamma values, and (3) clarification of the COCO train/test split (currently unspecified) is appropriate. Thus the reader's CONDITIONAL verdict stands.","tokens_in":30647,"tokens_out":11993,"duration_ms":163911,"concrete_test":"Recompute the RouteT2I row of Table 2 using Eq. (18) exactly as printed (signed numerator). If the result is about 56.8% rather than 83.97%, request the authors to (a) correct Eq. (18) to include an absolute value over the numerator if that was intended, and (b) explain why the absolute-value variant is a valid measure of 'relative performance improvement' when the cloud model is worse than the edge model on a given metric. If the signed formula is intended, all DeltaP and gamma values in Table 2 and Table 4 must be recomputed and the abstract's 83.97% and 70.24% claims revised accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (18) defines DeltaP = (1/N) * sum_i (mu_i(I_r) - mu_i(I_e)) / |mu_i(I_c) - mu_i(I_e)|. Recomputing Table 2's RouteT2I row (SD3 cloud, SD2.1 edge) with the printed signed numerator gives a sum of ratios equal to 5.677, i.e. DeltaP about 56.8%. To obtain the printed 83.97%, one must take absolute values in the numerator, which yields about 84.1% (the small difference is consistent with the three-decimal rounding in the table). The paper does not state an absolute value in Eq. (18). This matters directly for the headline claim: for the 'Harmony' metric the cloud average (0.5930) is below the edge average (0.5949), so a signed numerator counts a router that maintains or improves edge-level harmony as a negative or zero contribution. An absolute-value numerator instead credits any deviation away from the edge baseline as 'improvement', even when the cloud model is worse on that dimension. Because Table 4's cost-saving metric gamma is defined relative to DeltaP, the 70.24% cost-saving claim inherits the same distortion. The reader's Q-proxy concern is real, but Appendix D provides partial external support; the Eq. (18) mismatch is an internal, checkable inconsistency that directly affects the strongest quantitative claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RouteT2I, a routing framework that decides, for each text-to-image generation request, whether to invoke a large cloud model or a lightweight edge model. It defines a multi-dimensional contrastive quality metric Q based on CLIP similarity to hand-chosen positive/negative text pairs, introduces Pareto relative superiority (PRS) to quantify the quality gap between edge and cloud outputs, and trains a dual-gate token-selection mixture-of-experts model to predict PRS from the prompt text. A threshold on predicted PRS is then used to route prompts under cloud-rate constraints. Experiments on COCO 2014 with 18 model pairs report that at a 50% routing rate RouteT2I achieves 83.97% of the cloud model's average quality improvement over the edge model, and that it reduces cloud calls by 70.24% compared to random routing at a 40% relative improvement target. The appendix includes additional model-pair results, a LAION dataset check, and an external alignment check using HPSv2 and MPS.","tokens_in":30930,"tokens_out":11904,"duration_ms":99596,"significance":"If the results hold, RouteT2I addresses a timely and practically important problem: cutting the serving cost of text-to-image models without sacrificing perceived quality. The core idea of predicting a multi-metric quality gap from the prompt is coherent, and the paper's contribution includes a purpose-built router architecture, a routing strategy, and a fairly broad evaluation across 18 model pairs. The appendix's use of independent preference scorers (HPSv2, MPS) is a genuine step toward validating the quality proxy, and the reported training/inference overhead (58M parameters, 64.5ms on Jetson TX2) strengthens the practical case. The main reservations concern the semantics of the headline relative-improvement metric, the limited scope of the external validation, and missing reproducibility details.","major_comments":[{"comment":"The primary evidence for the central claim that routing maintains generation quality is circular in a narrow sense: the router is trained to predict PRS, which is derived from the same contrastive quality metric Q used to report the main results in Tables 2-4. The only external check is Appendix D, which uses HPSv2 and MPS on a single model pair (SD3 vs SD2.1) at a single routing rate. This is insufficient to establish that routing decisions generalize to human-perceived quality, especially because Section 6.2 states that \"Human evaluation ... [is] provided in Appendix\" but the appendix does not contain a study with human raters, only automated preference models. I recommend either adding a genuine human evaluation or clearly rephrasing the claim to \"automated preference-model evaluation.\"","section":"Section 6.2 / Appendix D"},{"comment":"The definition of ΔP in Eq. (18), with an absolute value in the denominator, does not measure \"the proportion of the quality improvement ... relative to the total possible improvement brought by fully using the cloud model\" when the cloud is worse than the edge on a metric. For the Harmony metric in Table 2, the cloud average (0.5930) is below the edge average (0.5949), so the denominator is the absolute value of a negative cloud-vs-edge difference. The router's harmony value (0.5975) then yields a positive ratio of 1.37, whereas a signed denominator would yield -1.37. Consequently, the reported 83.97% would fall to about 56.8% under a signed denominator, and random routing would yield roughly 50% instead of the reported 40%. The metric should either use a signed denominator with careful handling of negative improvements, or the text should explicitly acknowledge that ΔP is not a strict fraction of the cloud's advantage when the two models are not ordered per-metric.","section":"Equation (18) / Tables 2 and 4"},{"comment":"The paper omits key details needed to reproduce the routing model and strategy: the training loss used to optimize the PRS prediction, the number of experts and top-K value in the token-selection gate, the temperature Γ in Eq. (5), the metric weights w_i, and the procedure for setting the threshold α given a routing rate. Section 6.1 gives only the optimizer, learning rate, batch size, and epochs. The routing-strategy expression in Eq. (14) is also malformed: \"max_{α≤1/2} P {P RS(Ie, Ic) < α | Ie, Ic ∈ Ie, Ic} ≤ ρr\" reads as a constraint on a conditional probability whose conditioning event is unclear, and the optimization objective is missing. These gaps make it impossible to verify or build on the proposed approach from the manuscript alone.","section":"Sections 5.1-5.2 and 6.1"},{"comment":"The routing rule has a directional subtlety that should be clarified. The text states that prompts with PRS above the threshold are better handled by the edge model, and Eq. (14) routes to the cloud when predicted PRS < α with α ≤ 1/2. Because PRS is defined as the edge advantage over the cloud, this ordering is correct, but the presentation is confusing: the threshold α is described as if it directly separates \"edge-better\" from \"cloud-better,\" yet the boundary is at 0.5, not at α. Please state explicitly that the routing rule is: route to cloud if predicted PRS is below α (with α < 0.5), and explain how α is selected to meet the routing-rate constraint without solving an ill-posed optimization problem.","section":"Section 5.2"}],"minor_comments":[{"comment":"The notation Ie and Ic is used both for the edge/cloud image sets and for individual images (e.g., in the condition \"Ie, Ic ∈ Ie, Ic\"); please use different symbols for sets and elements.","section":"Section 5.2 / Eq. (14)"},{"comment":"The sentence \"the output image space is at least e^{4×10^6} times larger than the input text space\" appears to mix logarithmic and linear scales; the preceding calculation gives a log-scale ratio of about 4.36 million, so the factor should be e^{4.36×10^6} or the text should be rephrased.","section":"Appendix B"},{"comment":"The phrase \"Human evaluation, cost analysis, visual results, and more T2I pair results are provided in Appendix\" is inaccurate because the appendix contains automated preference scores, not human evaluation, and I could not locate a separate cost analysis beyond the training/inference cost in Appendix E. Please correct the pointer.","section":"Section 6.2"},{"comment":"The baselines are adapted from LLM routing by \"replacing their optimization goal with the average image quality across metrics,\" but it is not specified whether the adapted baselines use the same multi-dimensional Q or a scalar average; this matters for interpreting Table 2's per-metric values.","section":"Section 6.1"},{"comment":"The symbol σ is used for the sigmoid function in Eq. (3) and Eq. (13), and the text in Section 5.2 uses \"P {P RS...}\" with an extra opening brace; these are minor typographical issues that should be cleaned up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Eq. (18) does not land as stated: I independently recomputed Table 2's RouteT2I row using the printed Eq. (18) (absolute value only in the denominator) and obtained ΔP ≈ 84.1%, consistent with the reported 83.97%. However, the metric still has the semantic issue described in my major comment: the absolute-value denominator distorts the meaning when the cloud is worse than the edge on a metric, and the paper's interpretation as a proportion of the cloud's improvement is inaccurate. The circularity concern is real but partially mitigated by Appendix D; the bigger problem is that the paper promises human evaluation and does not deliver it. The missing reproducibility details are, in my view, the strongest reason for major revision rather than acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new and useful routing system for text-to-image generation, and the experiments are broad. But the main quantitative claim as printed is not reproducible from the paper's own formula, and that needs to be fixed before anyone should rely on the headline numbers.\n\nThe new thing here is that routing is done predictively for T2I: given a prompt, decide before generation whether to run the cheap edge model or the expensive cloud model. The architecture—dual-gate token-selection MoE with multi-head PRS prediction—is not just a repaint of LLM routers; it is designed around the cross-attention role of tokens in image generation. The paper evaluates 18 model pairs spanning different architectures and text encoders, including cross-encoder pairs, and reports edge latency numbers on Jetson hardware. That is real work, and the idea is timely.\n\nThe soft spots are real but concentrated. The biggest is Eq. (18) versus Table 2. Eq. (18) defines DeltaP with a signed numerator; redoing the arithmetic on Table 2 gives about 57 percent, not the printed 83.97 percent. To get 83.97 percent you have to take absolute values in the numerator, which the paper never states. That matters because the metric is supposed to measure how much of the cloud's quality gain over the edge is recovered. If the cloud is actually worse than the edge on one metric—Harmony is such a case—an absolute numerator counts any deviation away from edge as improvement, even when it moves toward the worse (cloud) value. The cost-saving gamma inherits the same distortion. So the headline results are not supported by the stated formula. This is fixable, but it is load-bearing in the abstract.\n\nSecond, the main evaluation metric is the same Q used to train the router. That is supervised learning, not derivation by definition, but it does mean Tables 2 through 4 mostly show how well the router predicts its own training signal. The Appendix D check with HPSv2 and MPS is the right kind of evidence, but it covers only one model pair at one routing rate. That is partial.\n\nThird, several hyperparameters are missing: the temperature Gamma, the metric weights w_i, the top-K, and the threshold alpha. Without those, reproducing the router is guesswork.\n\nAll that said, the central approach is sound. The external human-preference results point in the right direction, and the problem—cutting cloud T2I serving cost—is real. I would send this to peer review, but only with a request to fix the formula/table mismatch, report the missing hyperparameters, and broaden the external evaluation. The right reader is someone working on LLM or T2I serving systems, or anyone who uses model cascades and wants a T2I-specific router they can adapt.","headline":"Novel, useful T2I routing system with broad experiments, but the headline DeltaP numbers don't match the stated formula; fix that and it's a solid systems paper.","tokens_in":31503,"tokens_out":3063,"would_cite":true,"duration_ms":28182,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RouteT2I predicts which text-to-image model will produce a better image, and routes only the hard prompts to the expensive cloud model, capturing 83.97% of the cloud's quality advantage at a 50% routing rate.","keywords":["text-to-image generation","model routing","edge-cloud collaboration","mixture of experts","Pareto relative superiority","CLIP contrastive quality","cost-quality trade-off","generative AI serving"],"falsifier":"A direct test would take a held-out prompt set, ask human raters (or a strong human-preference model) which of the edge and cloud images is better, and compare that with the router's assignment at the 50% routing rate; the cost-saving claim holds only if per-prompt agreement stays well above chance, and it collapses if prompt-level agreement with human preference is near random while the paper's own score is satisfied.","tokens_in":2065,"feed_emoji":"🖼️","tokens_out":2288,"duration_ms":975510,"temperature":0.7,"pith_summary":"The paper claims that image-generation requests can be routed between a cheap edge model and an expensive cloud model by predicting, from the prompt text alone, which model would generate the better image. It builds a ten-dimensional quality measure, each dimension a CLIP contrast between a positive and a negative text description, and defines a Pareto relative superiority score that compares edge and cloud output quality across those dimensions. A router with a dual-gate token-selection mixture-of-experts architecture predicts this score from the prompt, and a threshold-based strategy sends only prompts with a large predicted cloud advantage to the cloud. If correct, RouteT2I delivers 83.97% of the cloud model's quality improvement at a 50% routing rate, and cuts cloud calls by 70.24% compared to random routing at a 50% quality-improvement target. The proposal matters because cloud text-to-image serving costs scale with model size, and most earlier routing work addresses text-to-text models, where input and output spaces are comparable in scale.","feed_headline":"Router keeps 84% of cloud image quality while halving cloud calls","feed_subtitle":"RouteT2I predicts from the prompt alone which model wins, slashing serving cost without sacrificing quality.","key_machinery":"The central object is the Pareto relative superiority (PRS) score, a weighted sum over ten quality metrics of normalized sigmoid distances between edge and cloud image qualities, where the quality of an image on metric $i$ is $q(I, m_i) = \\sigma(\\mathrm{CLIP}(I, m_i^+) - \\mathrm{CLIP}(I, m_i^-))$, the contrast between CLIP similarities to positive and negative text. This score turns an unobservable multi-metric quality comparison into a scalar regression target that can be computed before generation. The router is a Transformer whose linear layers are replaced by a dual-gate token-selection MoE: a positive gate and a negative gate each select the top-$K$ tokens most relevant to each quality metric, expert projections factor into positive and negative low-rank matrices sharing one score matrix, and the contrast $\\sigma(T^+[t] - T^-[t])$ decides each token's predominant influence. Multiple prediction heads then output per-metric quality predictions, and the routing strategy thresholds the predicted superiority at a rate set by the cost budget.","core_discovery":"RouteT2I treats text-to-image routing as a prediction problem: from a user prompt, predict whether the edge model's image would be as good as or better than the cloud model's, and only send to the cloud when the predicted gap is large. The prediction target is the Pareto relative superiority between edge and cloud outputs, built from a ten-metric quality vector in which each metric is the sigmoid of the difference between CLIP similarity to a positive text and a negative text. A Transformer-based router with a dual-gate token-selection mixture-of-experts architecture — a set of specialized submodules, each aligned to a quality metric, that select the tokens most relevant to them — identifies prompt tokens with strong positive or negative influence on each quality metric and contrasts the two influences to form a per-token signal. Under a preset cloud routing rate, the routing strategy thresholds the predicted superiority so that only prompts where the cloud clearly wins are routed upward. The paper reports that at a 50% routing rate the routed images capture 83.97% of the cloud model's quality improvement over the edge model, and that reaching 50% of that improvement requires 70.24% fewer cloud calls than random routing, with results across 18 model pairs on COCO 2014 and a LAION subset.","pith_inferences":["A direct extension would retrain the router on a human-preference-aligned reward instead of the ten hand-chosen CLIP-contrast metrics; if those metrics understate what users prefer for certain prompt classes, the routing decisions and reported savings would shift accordingly.","The same dual-gate token-selection mechanism could apply to other generation tasks with large output spaces, such as text-to-video or text-to-3D, where a predicted-quality threshold would decide between a local and a remote model.","Because the paper reports routing gains only in aggregate, a per-prompt calibration test would be informative: check whether the prompts the router sends to the cloud at the 50% rate are the same ones a human panel rates as substantially better from the cloud, since systematic disagreement would reveal prompt classes where routing is misdirected.","The cost-quality evaluation is presented at fixed routing-rate targets; a natural operational extension is to price the two models and solve the Lagrangian form of the budget-constrained objective, yielding one routing threshold per price ratio."],"forward_implications":["At a fixed 50% routing rate, RouteT2I attains 83.97% of the quality gain that fully using the cloud model would give over the edge model, surpassing adapted LLM-routing baselines by at least 6 percentage points.","To reach 50% of the cloud model's quality improvement, RouteT2I needs 70.24% fewer cloud calls than random routing, and savings stay above 50% for the other tested quality targets.","Routing decisions are made before generation, so the unselected model is never invoked, keeping edge-device routing overhead to roughly 64.5 ms on a Jetson TX2 and 131.3 ms on a Jetson Nano.","The method transfers across diffusion, diffusion-Transformer, and autoregressive architectures and across CLIP, OpenCLIP, and T5 text encoders, with the largest gains on model pairs with large quality gaps.","For some edge-cloud pairs the relative performance improvement exceeds 100%, meaning the routed mixture can outperform full cloud usage on the multi-metric score."],"supporting_citations":[{"why":"Supplies the RouteLLM predictive-routing baseline that RouteT2I is compared against in the main tables.","marker":"[23]"},{"why":"Supplies the Hybrid LLM threshold-routing baseline adapted to the image-routing setting.","marker":"[8]"},{"why":"Supplies the ZOOTER baseline, which distills routing decisions from an external quality scorer.","marker":"[21]"},{"why":"Together with T2I-Scorer and VisionPrefer, sources the ten quality metrics and their positive/negative text pairs.","marker":"[14]"},{"why":"CLIP is the backbone of the contrastive quality metric that turns each metric into a text-image similarity contrast.","marker":"[25]"},{"why":"Defines the Stable Diffusion edge models (SD1.5, SD2.1) used in the experiments.","marker":"[28]"},{"why":"Defines Stable Diffusion 3, the main cloud model whose quality gap must be predicted.","marker":"[9]"},{"why":"Provides the COCO 2014 captions used as user-prompt training and evaluation data.","marker":"[19]"},{"why":"HPSv2 human-preference scorer used in Appendix D to check alignment of routing gains with human judgment.","marker":"[38]"},{"why":"MPS human-preference scorer used as the second alignment check in Appendix D.","marker":"[42]"}],"fun_headline_variants":["Smart router cuts cloud calls by 70% with minimal quality loss","AI router saves 70% cloud calls while keeping image quality","Predict then route: edge can beat cloud for many prompts","Cloud or edge? This router decides before generating","Cut cloud costs 70% by predicting when edge suffices"],"cache_read_input_tokens":33536,"weakest_assumption_plain":"The load-bearing premise is that the ten hand-written positive and negative text pairs, compared through CLIP, measure the image quality users actually care about, since the router is trained on this score and the headline cost-saving results are measured with the same score.","fun_headline_variants_meta":{"raw":{"variants":["Smart router cuts cloud calls by 70% with minimal quality loss","AI router saves 70% cloud calls while keeping image quality","Predict then route: edge can beat cloud for many prompts","Cloud or edge? This router decides before generating","Cut cloud costs 70% by predicting when edge suffices"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000885,"raw_usage":{"total_tokens":3857,"prompt_tokens":1013,"completion_tokens":2844,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":2761}},"tokens_in":629,"tokens_out":2844,"duration_ms":29851,"temperature":1.0,"reasoning_tokens":2761,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:53:17.530645+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would take a held-out prompt set, ask human raters (or a strong human-preference model) which of the edge and cloud images is better, and compare that with the router's assignment at the 50% routing rate; the cost-saving claim holds only if per-prompt agreement stays well above chance, and it collapses if prompt-level agreement with human preference is near random while the paper's own score is satisfied.","supporting_citations":[{"cited_title":"Holis- tic evaluation of text-to-image models","cited_arxiv_id":null,"evidence_quote":"Together with T2I-Scorer and VisionPrefer, sources the ten quality metrics and their positive/negative text pairs."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the Stable Diffusion edge models (SD1.5, SD2.1) used in the experiments."},{"cited_title":"Scaling recti- fied flow transformers for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Defines Stable Diffusion 3, the main cloud model whose quality gap must be predicted."},{"cited_title":"Learning multi- dimensional human preference for text-to-image generation","cited_arxiv_id":null,"evidence_quote":"MPS human-preference scorer used as the second alignment check in Appendix D."}],"review_version":1}