{"id":"653d00d5-b04f-488d-ac5e-446fb3c65dda","arxiv_id":"2607.08103","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":5,"one_line_summary":"Stochastic Order Learning associates each instance with multiple plausible ranks and trains embeddings via complementary discriminative and stochastic-order losses that remain robust to ordinal label noise.","lead":"SOL reformulates rank estimation under noisy ordinal labels as a stochastic ordering problem and learns embeddings with a discriminative loss plus a probabilistic pairwise order loss. The method improves accuracy on age estimation, aesthetics, bone-age, and text-quality tasks when training ranks are corrupted.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper cleanly reformulates noisy rank estimation as stochastic ordering, derives the two complementary losses and the centroid update under that view, and supplies extensive multi-domain, multi-noise empirical support plus released code. The Gaussian modeling choice identified by the reader is the most natural soft spot, yet the mathematics never hard-codes Gaussian moments beyond the definition of ps, and the experiments already cover the natural counter-examples. Consequently the central claim stands and the ACCEPT verdict needs no adjustment.","tokens_in":32175,"tokens_out":390,"duration_ms":4610,"concrete_test":"Re-train SOL on MORPH II (κ=0.3 Gaussian) after replacing the fixed Gaussian ps with a non-parametric histogram estimated once from the empirical residuals of a preliminary GOL run; if MAE rises by more than 0.05 relative to the published 2.663, the parametric form is more critical than claimed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (parametric discrete-Gaussian noise in Eq. 2) is real but not load-bearing for the central claim. The desideratum (4), monotonicity (5), discriminative loss (8)–(10), stochastic order probabilities (12)–(14), and centroid rule (18) are all written for a general discrete distribution {ps}; the Gaussian is only a convenient unimodal symmetric choice. Appendix D.2 and Tables 1–4 already stress-test Laplacian, uniform, skewed, and input-dependent noise, and the real-world WMT2020 setting, with consistent gains. The optional outlier-relabeling step (19)–(21) is likewise shown to be non-essential (Tables 18–19). No internal inconsistency or untested critical assumption remains that would overturn the claim that SOL produces reliable rank estimates under diverse noise.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper reformulates rank estimation under noisy ordinal labels as a stochastic ordering problem: each instance is associated with a distribution over plausible ranks rather than a single deterministic label. It introduces Stochastic Order Learning (SOL), which learns an embedding via a discriminative loss that attracts/repels instances relative to rank centroids under a stochastic dissimilarity Dh (Eqs. 3–10) and a stochastic order loss that enforces probabilistic pairwise ordering relations (Eqs. 12–17). Centroids are updated in closed form (Eq. 18); an optional outlier detection and relabeling step (Eqs. 19–21) is also provided. Inference selects the rank minimizing expected distance (Eq. 6). Experiments on MORPH II, CLAP2015, AADB, RSNA, and WMT2020 under Gaussian, Laplacian, uniform, skewed, and real-world noise show consistent gains over noise-robust classification/regression and order-learning baselines.","tokens_in":32434,"tokens_out":1029,"duration_ms":9555,"significance":"If the results hold, SOL supplies a clean, generalizable framework for ordinal prediction under structured label noise—an important practical setting that standard classification noise methods and clean-label order learning both handle poorly. Strengths include: (i) explicit derivations of the monotonicity constraint (Appendix A) and closed-form centroid rule (Appendix B); (ii) complementary losses that are not tautological restatements of the evaluation metric; (iii) broad empirical coverage across five datasets, four synthetic noise families, multiple κ levels, real-world WMT2020 noise, ablations of each loss, multi-seed stability, and alternative relabeling schemes; and (iv) released source code. The free parameters (T, τ, γ, β, σ_test) are documented and analyzed. The contribution is incremental relative to prior order learning (GOL, MWR) but the stochastic reformulation and the resulting robustness gains are concrete and useful.","major_comments":[{"comment":"Main-result Tables 1–4 report only point estimates of MAE/CS with no standard deviations or multi-seed statistics. Appendix D.12 later supplies five-seed means±std on MORPH II only. Because the central claim is consistent outperformance under diverse noise, the main tables should either include error bars (or multi-seed means) for all five datasets or explicitly state that the Appendix D.12 stability analysis is representative; otherwise the magnitude of the reported gains (often 0.02–0.1 MAE) cannot be assessed for statistical reliability.","section":null},{"comment":"Section 3.1 and Eq. (2) introduce a discrete Gaussian for the noise model, and the desideratum, losses, and centroid rule are written with general {ps}. The paper correctly stress-tests Laplacian/uniform/skewed/input-dependent noise and real WMT2020 annotations (Tables 1–5, Appendix D.2). However, the main text still presents the Gaussian as the default generative model without a short, explicit statement that the algorithmic objects remain well-defined for any unimodal discrete {ps}. Adding one clarifying sentence (and, if space permits, a one-line general-ps derivation note) would remove residual ambiguity about how load-bearing the parametric form is.","section":null}],"minor_comments":[{"comment":"Figure 1 caption and surrounding text correctly contrast nominal vs. ordinal noise severity; a brief pointer to the later quantitative tables would help readers who skip the figure.","section":null},{"comment":"Hyperparameter defaults (Table 9) and sensitivity analyses (Appendix D.1) are thorough; moving a one-sentence summary of recommended defaults into the main Implementation subsection would improve reproducibility for readers who do not open the appendix.","section":null},{"comment":"Notation: the same symbol e is used both for the random noise variable and for individual errors ex; a minor typographic distinction would avoid momentary confusion in Section 3.1.","section":null},{"comment":"Impact Statement correctly flags demographic bias risk on facial datasets; a parallel one-sentence note for the medical (RSNA) setting would be consistent.","section":null},{"comment":"A few typographical slips remain (e.g., “order-identitiy” in Related Work; occasional missing spaces around citations). A final proofreading pass is warranted.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a solid, well-executed incremental contribution that fits a methods-oriented ML venue. The reader’s and skeptic’s assessments that the Gaussian assumption is not load-bearing are correct on my reading; the two major comments above are presentation/completeness issues rather than correctness failures. I see no novelty or citation-pattern concerns that would require editorial attention beyond ordinary review."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing: this is a genuine technical step past GOL. They stop treating each instance as tied to a single rank and instead associate it with a distribution over neighboring ranks, then derive a discriminative loss (instance–centroid attraction/repulsion under that distribution) plus a pairwise stochastic-order loss. The math in Appendices A–B is clean; the centroid rule is closed-form; the desideratum is explicit rather than hand-waved.\n\nWhat they do well is the evaluation. Five datasets (age, aesthetics, bone age, real-world WMT text), four noise families plus input-dependent and real annotation noise, ablations of each loss, multi-seed numbers, and code. Gains over GOL, MWR, and the usual noise-robust regression baselines are consistent and not tiny. The optional outlier-relabeling loop is shown to be helpful but not load-bearing. That is more thorough than most ordinal papers I see.\n\nSoft spots are real but secondary. The derivation is written for a general discrete {ps}; they just instantiate a discrete Gaussian for convenience. They already stress-test Laplacian, uniform, skewed, and input-dependent noise, so the parametric choice is not a hidden failure mode. There are the usual free knobs (T, τ, γ, β, σ_test), but the defaults are stable and they report sensitivity. Main tables lack error bars in places, though the multi-seed appendix helps. None of this overturns the claim.\n\nThis is for people who already ship ordinal pipelines (age, medical scores, aesthetic or quality regression) and keep hitting structured label noise. It is not a foundational rewrite of ranking theory, but it is a solid, citable engineering advance with reproducible code. I would send it to referees without hesitation; the work is coherent, the evidence matches the claims, and the literature engagement is honest. Worth reading and, for anyone in the sub-area, worth citing.","headline":"Clean, usable extension of geometric order learning to noisy ordinal labels; the stochastic reformulation is real and the empirics are broad enough to trust the gains.","tokens_in":32989,"tokens_out":478,"would_cite":true,"duration_ms":25680,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Noisy ranks are treated as probabilistic associations with several neighboring ranks, and an embedding is learned that respects those associations.","keywords":["rank estimation","ordinal regression","label noise","order learning","stochastic ordering","embedding space","noisy supervision"],"falsifier":"On a large ordinal dataset whose annotation errors are known to be strongly skewed or multi-modal, replace the Gaussian ps with the true error frequencies; if SOL then loses its advantage over ordinary order-learning and noise-robust regression baselines, the Gaussian modeling assumption is essential rather than merely convenient.","tokens_in":33107,"feed_emoji":"📊","tokens_out":812,"duration_ms":10194,"temperature":0.7,"pith_summary":"Rank labels in age estimation, aesthetics, medical scoring and similar tasks are rarely exact; nearby ranks are far more likely than distant ones, so ordinary classification or regression that treats every wrong label the same is poorly matched to the problem. The paper reframes the task as stochastic ordering: each training example is allowed to relate, with decaying probability, to a whole neighborhood of ranks rather than to a single correct one. From that view it builds Stochastic Order Learning, which places instances and rank centroids in a shared embedding so that expected squared distances obey a monotonicity condition. Two losses cooperate to enforce the condition: a discriminative loss that attracts each point toward plausible centroids and repels it from implausible ones, and a stochastic order loss that keeps relative orderings between pairs consistent with the noise model. Optional outlier detection and mild relabeling further clean extreme errors. Experiments on facial age, aesthetic scores, bone-age radiographs and translation quality scores show that the resulting estimator remains accurate under Gaussian, Laplacian, uniform and real-world annotation noise.","feed_headline":"Noisy ranks become probabilistic neighborhoods, not single labels","feed_subtitle":"An embedding that respects those neighborhoods keeps age, score and bone-age estimates accurate under real annotation noise","key_machinery":"Stochastic dissimilarity Dh(x,r) = sum_s ps ||h(x) - mu_{r+s}||^2, whose minimization under a discrete-Gaussian noise model produces the monotonicity constraint that the discriminative and stochastic-order losses jointly enforce.","core_discovery":"When ordinal labels are noisy, each instance should be treated as stochastically associated with multiple neighboring ranks; an embedding that minimizes the expected squared distance to the corresponding rank centroids, together with pairwise probabilistic order constraints, yields rank estimates that stay reliable across noise types and severities.","pith_inferences":["If annotator-specific or input-dependent noise distributions can be estimated on the fly, the same Dh formulation could absorb them without redesigning the losses.","The monotonicity constraint may transfer to continuous ordinal regression by replacing discrete centroids with a smooth rank manifold.","Because the method already models structured uncertainty, it is a natural candidate for active learning of the most ambiguous ranks."],"forward_implications":["Noise-robust rank estimators can be obtained without requiring clean reference labels or explicit noise-rate estimation.","The same stochastic-order losses apply unchanged to vision, medical imaging and text regression once an appropriate encoder is chosen.","Outlier detection and mild relabeling become natural by-products of the learned embedding rather than separate heuristics.","Inference reduces to finding the rank whose expected squared distance is smallest, avoiding expensive nearest-neighbor search at test time."],"fun_headline_variants":["Noisy ordinal ranks treated as multi-rank stochastic neighborhoods","Embeddings learn probabilistic order to fix noisy rank estimates","SOL: instance-centroid distances plus order constraints under label noise","Stochastic ordering turns rank noise into reliable embeddings","Rank estimation stays accurate when labels become probabilistic orders"],"cache_read_input_tokens":23040,"weakest_assumption_plain":"The method derives its losses and centroid rule from a discrete Gaussian model of label errors; if real annotation noise is strongly asymmetric or multi-modal, those derived objectives may no longer match the data.","fun_headline_variants_meta":{"raw":{"variants":["Noisy ordinal ranks treated as multi-rank stochastic neighborhoods","Embeddings learn probabilistic order to fix noisy rank estimates","SOL: instance-centroid distances plus order constraints under label noise","Stochastic ordering turns rank noise into reliable embeddings","Rank estimation stays accurate when labels become probabilistic orders"]},"model":"grok-4.5","effort":"low","cost_usd":0.004058,"raw_usage":{"total_tokens":1145,"prompt_tokens":655,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":40580000,"prompt_tokens_details":{"text_tokens":655,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":412,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":655,"tokens_out":78,"duration_ms":4309,"temperature":1.0,"reasoning_tokens":412,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-10T13:01:36.864298+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a large ordinal dataset whose annotation errors are known to be strongly skewed or multi-modal, replace the Gaussian ps with the true error frequencies; if SOL then loses its advantage over ordinary order-learning and noise-robust regression baselines, the Gaussian modeling assumption is essential rather than merely convenient.","supporting_citations":[],"review_version":1}