{"id":"c961d249-eace-469f-a8e2-1d589c28285f","arxiv_id":"2412.12031","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RepFace combines auxiliary-sample noise filtering, three-way sample splitting, memory-bank label fusion, and smoothed label correction to improve face recognition under closed-set label noise.","lead":"This paper proposes RepFace, a training framework that sorts face-recognition images into clean, ambiguous, and noisy groups and applies different label-handling strategies, including auxiliary-sample filtering, memory-bank label fusion, and smoothed label correction. On standard face benchmarks with 10% and 20% of training labels randomly flipped, RepFace reports small accuracy gains over prior noise-robust training methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on single-run accuracy differences (e.g., +0.19 avg at 10% noise, +0.04 at 20% on MS1Mv2) that are within expected run-to-run variance; without error bars or multiple seeds, the claimed improvement is not distinguishable from stochasticity.","rationale":"The reader's weakest assumption targets external validity: synthetic random label flips may not mimic real closed-set noise. That is a legitimate limitation, but it is about transfer of the method beyond the benchmarks. The paper's central claim as stated is about performance on those benchmarks under the synthetic noise protocol. The more immediate load-bearing condition is that the reported numerical advantages are real and reproducible. The differences are small, the comparisons are single-run, and hyperparameter tuning on the evaluation sets can inflate apparent gains. A multi-seed experiment would settle this directly. If the advantage survives with non-overlapping confidence intervals, the SOTA claim is supported; if not, the paper reduces to a 'competitive with BoundaryFace' claim. I therefore recommend keeping the conditional verdict, with the added condition that the authors report variance or release code that reproduces the exact numbers. The loss-equation ambiguity noted by the reader is also real, but it is secondary: it affects reproducibility, whereas the missing error bars affect whether the claimed effect exists at all.","tokens_in":15354,"tokens_out":9850,"duration_ms":89743,"concrete_test":"Run RepFace and BoundaryFace with the exact Table 5 protocol on the 10% and 20% closed-set-noise CASIA-WebFace splits, using at least five random seeds that vary data shuffling, auxiliary-sample selection, and initialization; report mean and standard deviation for LFW, AgeDB, CFP-FP, CALFW, SLLFW, and the four RFW subgroups. Compute the paired mean difference and its 95% confidence interval for the average verification accuracy. If the interval includes zero in either noise condition, the claimed SOTA improvement over BoundaryFace is not supported by the reported evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline result is that RepFace 'improves the previous BoundaryFace by 0.19 on the avg' at 10% noise and remains competitive at 20%. In Table 5, the 10% average gain is driven by AgeDB (+0.45) and three RFW subgroups (+1.32, +1.33, +1.42) while CFP-FP drops by -0.43; at 20% noise on CASIA-WebFace the average is exactly tied with BoundaryFace (95.76 vs. 95.76). On MS1Mv2 with 20% noise (Table 6), the average gain over BoundaryFace is +0.04 and the IJB-C TAR@FAR=1e-4 gain is +0.07. These effect sizes are comparable to or smaller than the typical run-to-run variation of face recognition training on 0.5M-5.8M image datasets (often 0.1-0.3% accuracy), and the paper reports no standard deviations, no seeds, and no significance test. Additionally, the key hyperparameters tau, alpha, and beta are selected on the same benchmarks that are later used for the SOTA comparison (Fig. 3, Tables 1-2), so the reported differences may reflect selection of the best hyperparameter setting rather than a robust advantage. The central claim therefore requires the assumption that a single favorable run represents the method's true performance; that assumption is the least secure link in the argument.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes RepFace, a training framework for face recognition under closed-set label noise. The method consists of three components: Auxiliary Sample Cleaning (ASC), which generates randomly labeled auxiliary samples in each mini-batch to set a threshold for filtering noisy samples during early training; a tri-partition of samples into clean, ambiguous, and noise groups based on the difference between the cosine similarity to the nearest negative class center and the positive class center; Label Robust Fusion (LRF), which fuses memory-bank-accumulated model predictions with ground-truth labels for ambiguous samples; and Smoothing Label Correction (SLC), which smooths the labels of noise samples between the original label and the nearest negative class. The method is evaluated on CASIA-WebFace and MS1Mv2 with synthetic closed-set noise at 10% and 20% ratios, reporting improvements over several baselines including BoundaryFace and claiming state-of-the-art results.","tokens_in":15733,"tokens_out":7118,"duration_ms":58037,"significance":"The paper addresses an important practical problem, label noise in large-scale face recognition datasets, and the idea of splitting samples into three categories with separate handling of ambiguous and noisy samples is sensible. The auxiliary-sample cleaning mechanism for stabilizing early training is also novel. However, the technical presentation has a critical flaw in the loss formulation (Eq. 9), the empirical evidence is weakened by single-run evaluations, hyperparameters are selected on the same benchmark sets used for the final comparison, and the reported improvements over BoundaryFace are often within typical run-to-run variance. These issues mean the central state-of-the-art claim is not convincingly supported by the presented experiments.","major_comments":[{"comment":"The loss for closed-set noise samples is written as -I(cos theta_yi) * q_s * log P_yj, but q_s is a soft label vector while P_yj is defined as the scalar probability of the positive class y_i computed with N2 smoothing in the denominator. A scalar log-probability cannot be multiplied by a label vector to yield a valid cross-entropy. The correct loss should be a cross-entropy over the smoothed distribution, e.g., -(1-k) log P(y_i) - k log P(y_j), using probabilities computed under the corrected label. As written, the equation is internally inconsistent and the training objective for the SLC module is undefined.","section":"Methodology, Eq. (9)"},{"comment":"The ASC threshold eta is defined as the average cosine similarity of M randomly labeled auxiliary samples to their assigned labels plus a constant alpha. The paper provides no theoretical or empirical argument that this statistic separates clean from noisy samples, and the constant alpha is tuned on the evaluation benchmarks (Table 1). This makes the noise-filtering criterion appear arbitrary rather than derived from the decision-boundary analysis that motivates the rest of the method.","section":"Methodology, Eqs. (1)-(2)"},{"comment":"All accuracy numbers are reported from a single run without error bars, multiple seeds, or significance tests. The central claim that the method improves BoundaryFace by 0.19 on average at 10% noise is contradicted at 20% noise, where the average is exactly equal (95.76 vs. 95.76) and CFP-FP is 0.45 points lower (93.44 vs. 93.89). On MS1Mv2 at 20% noise, the average gain over BoundaryFace is +0.04 and the IJB-C gain is +0.07, both within typical run-to-run variation for large-scale face recognition training. The state-of-the-art claim is therefore not supported by the reported evidence.","section":"Experiments, Tables 5-6"},{"comment":"The hyperparameters tau, alpha, and beta are selected by evaluating on LFW, AgeDB, CFP-FP, CALFW, and RFW, which are the same datasets used for the SOTA comparison in Table 5. Since the final results are obtained after tuning on these test sets, the reported improvements may reflect selection bias rather than a robust advantage of the method.","section":"Experiments, Fig. 3 and Tables 1-2"},{"comment":"The evaluation is limited to synthetic closed-set noise generated by randomly flipping labels. Real-world closed-set noise in face datasets is often structured, such as consistent identity confusion between confusable identities or repeated mislabels from the same source. Without experiments on naturally noisy datasets (e.g., MS1Mv3, WebFace260M, or a manually validated noisy subset), the generalizability of the proposed method to realistic noise conditions is not established.","section":"Experiments"}],"minor_comments":[{"comment":"The paper states that tau is a 'learned threshold,' but in the ablation study (Fig. 3) it is manually set to 0.2 after comparing four fixed values; this wording is misleading.","section":"Methodology, Sample Splitting"},{"comment":"The definitions of P_yj and P_yi are confusing; both use the same numerator exp(s * T(cos theta_yi)), so P_yj as written is not the probability for the nearest negative class. Please clarify the notation and ensure the denominator matches the intended target class.","section":"Methodology, Eq. (9)"},{"comment":"The same beta is used for the exponential moving average of the memory bank (Eq. 4) and for the label fusion weight (Eq. 5); the paper should explain whether this is intentional and how the two uses are related.","section":"Methodology, Eqs. (4)-(5)"},{"comment":"The text says alpha = 0.05 is chosen based on Table 1, but the table also shows that alpha = 0.07 gives better CFP performance (93.89 vs. 93.44), which is not discussed.","section":"Experiments, Hyperparameter Analysis"},{"comment":"The start epoch T_s is used in the algorithm but is never defined in the main text; please specify its value and how it is chosen.","section":"Supplementary Material, Algorithm 1"},{"comment":"There are several typos and inconsistent notations, including 'tree categories' instead of 'three categories' (Methodology), 'CIAIA-WebFace' instead of 'CASIA-WebFace' (Experiments), 'Tabel' instead of 'Table' (Ablation), 'Afican' instead of 'African' (Table 5), and inconsistent use of 'close-set' and 'closed-set' throughout.","section":"Typos and notation"},{"comment":"The comparison set is limited to methods up to 2022 (BoundaryFace, AdaFace, RVFace); more recent noise-robust face recognition methods from 2023-2024 are not included, which limits the strength of the state-of-the-art claim.","section":"Related Work and Comparisons"},{"comment":"The supplementary discussion of label correction (Figure 5) reports recall, precision, and correction accuracy only graphically; quantitative values would strengthen the claims about noise detection and correction accuracy.","section":"Supplementary Material, Label Correction Discussion"}],"recommendation":"major_revision","confidential_remarks":"The manuscript builds heavily on BoundaryFace, which shares a co-author, and the comparison baseline is that same method. This is not inherently problematic, but the reported improvements are small and sometimes absent (e.g., equal average at 20% noise on CASIA-WebFace). Given the single-run evaluation and the hyperparameter tuning on the evaluation benchmarks, the authors should be asked to provide multi-seed results with variance estimates and to validate on a naturally noisy dataset before the state-of-the-art claim can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: RepFace is a plausible three-module training recipe for closed-set label noise in face recognition, and the ablation study is more thorough than most. But the central loss equation as written is internally inconsistent, and the reported gains over BoundaryFace are small enough that, without error bars, they could easily be run-to-run variance. The paper is worth refereeing, not worth accepting as is.\n\nWhat's new: the specific combination — auxiliary random-label samples to calibrate a noise threshold (ASC), a three-way clean/ambiguous/noise split by distance to positive vs nearest negative center, memory-bank label fusion for ambiguous samples (LRF, borrowed from SELC), and smoothed label correction (SLC). The individual pieces come from BoundaryFace, SELC, and MV-Softmax, but the assembly and the dynamic threshold are not in prior work. The ablations in Tables 3–4 are genuinely informative; the supplementary also tests on a different loss (AdaFace) and on mixed open+closed noise, which is more than many papers do.\n\nThe soft spots are real. First, Eq. 9 is wrong as written. For the d_i > τ branch, the corrected label is q_s (a blend of true and nearest-negative one-hots), but the cross-entropy term uses log P_yj whose numerator is e^{s·T(cos θ_yi)} — the original positive class — not the corrected class. That doesn't compute the loss the text describes. Also the algorithm only applies the ASC indicator I(·) in the T_k < T_s branch, while the text says the indicator multiplies the final loss throughout. Either the equation or the algorithm is wrong; this needs fixing before the method can be implemented as written.\n\nSecond, the empirical case is weak. On CASIA at 20% noise, RepFace ties BoundaryFace on average (95.76 both). On MS1Mv2 at 20%, the average gain is +0.04 and the IJB-C TAR@FAR=1e-4 gain is +0.07. The 10% gain of +0.19 is driven by three RFW subgroups; CFP-FP drops by 0.43. No seeds, no error bars, no significance test. Given typical run-to-run variation of 0.1–0.3%, these numbers don't establish SOTA.\n\nThird, τ, α, and β are selected on the same benchmarks that appear in the final SOTA tables (Fig. 3, Tables 1–2). That's selection on the test set; it makes the reported advantage optimistic.\n\nFourth, the noise model is only random label flips. Real closed-set noise in web datasets is not necessarily uniform random flips. The paper limits its claim to closed-set noise, which is fair, but the transfer to real structured noise is untested.\n\nOverall: this is a serious paper with a plausible mechanism and honest ablations, but the internal equation inconsistency and the lack of statistical support for the headline claim mean the SOTA assertion is not credible as written. The authors should fix the loss, report multiple seeds, and ideally evaluate on a real noisy face dataset or at least a structured noise model.\n\nMy recommendation: send it to review rather than desk reject — there is enough substance and the issues are fixable — but the acceptance decision should be conditional on the equations being corrected and the empirical claims supported by error bars.","headline":"A plausible three-module recipe for closed-set label noise with a thorough ablation, but the central loss equation is internally inconsistent and the SOTA gains are within run-to-run variance.","tokens_in":16200,"tokens_out":5160,"would_cite":false,"duration_ms":42186,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RepFace: refining closed-set label noise in face recognition with progressive label correction.","keywords":["face recognition","closed-set label noise","label correction","noise-robust training","sample splitting","memory bank label fusion","label smoothing","auxiliary sample cleaning"],"falsifier":"Run RepFace against BoundaryFace on a web-scraped face dataset with naturally occurring label noise, measured by TAR@FAR=1e-4 on IJB-C; if the gain disappears or reverses, the synthetic-noise claim fails. A sharper test: construct structured closed-set noise by consistently confusing similar-looking identities (same wrong label repeated per image, rather than random flips) at 10-20% and check whether RepFace still beats BoundaryFace by roughly 0.19 points.","tokens_in":15160,"feed_emoji":"👤","tokens_out":9518,"duration_ms":75930,"temperature":0.7,"pith_summary":"This paper sets out to show that closed-set label noise — images labeled with an identity that exists elsewhere in the training set but is not the correct person — can be handled during face-recognition training, rather than by costly manual cleaning. It proposes RepFace, a framework that sorts every training sample into clean, ambiguous, or noisy using the gap between its similarity to its assigned class center and to the nearest other class center, then trains each group with a different strategy. The claimed result is that models trained with this scheme match or beat earlier noise-tolerant face-recognition methods on benchmark evaluations: at 10% synthetic closed-set noise it improves the previous best method by 0.19 points on average, and at 20% noise it stays competitive while degrading far less than some alternatives. If the claim holds, web-scraped face datasets could be used more effectively, and the three-way split plus label fusion could transfer to any recognition task suffering from closed-set noise.","feed_headline":"RepFace beats prior face recognition under label noise by 0.19","feed_subtitle":"A three-branch scheme for clean, fuzzy, and wrong samples surpasses the previous best on 10% closed-set noise.","key_machinery":"The load-bearing object is the signed distance $d_i = \\cos\\theta_{\\max_j} - \\cos\\theta_{y_i}$, which separates a sample's similarity to its assigned identity center from its similarity to the nearest competing identity center. One threshold $\\tau$ turns this single scalar into a three-way split: clean ($d_i < 0$), ambiguous ($0 < d_i < \\tau$), and noisy ($d_i > \\tau$). The same $d_i$ feeds the smoothing weight $k = \\mathrm{Sigmoid}(10 d_i)$ used in label correction, making the correction more aggressive as the evidence of noise grows. Two auxiliary mechanisms make the split reliable when the model is still weak: the Auxiliary Sample Cleaning module, which generates its own random-label samples and thresholds on their average cosine similarity, and the Label Robust Fusion memory bank, which accumulates prediction logits by exponential moving average and blends them into the ground-truth label for ambiguous samples. The loss in Eq. 9 routes each group to its own supervision, so the framework is one coherent training scheme rather than three independent tricks.","core_discovery":"RepFace claims that the central obstacle to handling closed-set noise is not the noise itself but the early training phase and the ambiguous zone near the decision boundary. To keep early training clean, the framework injects random auxiliary noisy samples into each mini-batch and uses their average cosine similarity to the injected wrong labels as a dynamic threshold; any real sample scoring below that threshold is suppressed from the loss. Once the model can discriminate, each sample is scored by $d_i = \\cos\\theta_{\\max_j} - \\cos\\theta_{y_i}$, the difference between its cosine similarity to the nearest negative class center and to its assigned positive center. With a fixed threshold $\\tau$, samples with $d_i < 0$ are treated as clean, samples with $0 < d_i < \\tau$ as ambiguous, and samples with $d_i > \\tau$ as closed-set noise. Ambiguous samples receive a fused label: a memory bank stores an exponentially moving average of the model's maximum cosine logits, normalized into soft labels and blended with the one-hot ground truth. Noisy samples receive a smoothed correction: the label is shifted partway toward the nearest negative class, with mixing weight $k = \\mathrm{Sigmoid}(10 d_i)$, and the hard-sample-mining cosine matrix is smoothed by the same weight. Trained with a margin-based softmax loss, this framework produces the reported accuracy gains on CASIA-WebFace and MS1MV2 with 10% and 20% synthetic noise.","pith_inferences":["Beyond the paper: the self-generated auxiliary-sample threshold is a calibration trick that needs no clean labels, so it could be dropped into other recognition tasks with closed-set noise (e.g., person re-identification, speaker verification) to stabilize early training.","Beyond the paper: the authors report in their supplementary material that the recall of noise detection is not very high, because some noise is absorbed into the ambiguous group; a natural follow-up test is whether the Label Robust Fusion module alone, without explicit noise correction, accounts for most of the resilience.","Beyond the paper: the smoothing schedule $k=\\mathrm{Sigmoid}(10 d_i)$ depends only on the cosine gap; an untested variant is to make the 10x factor increase during training, which might correct high-confidence noise earlier while leaving near-boundary labels intact longer.","Beyond the paper: the reported generalization to ethnic subgroups (larger gains for Asian and African groups on RFW) suggests that label-fusion training may help underrepresented classes; a direct comparison on quality-balanced or fairness benchmarks would test this."],"forward_implications":["Training on noisy web-scale face data should improve without manual label cleaning, since RepFace reports best average verification accuracy on both CASIA-WebFace and MS1MV2 under synthetic closed-set noise.","A model trained with RepFace at 20% noise drops only about 0.5 points from its clean-data accuracy on CASIA-WebFace, whereas some strong baselines drop more than 1.5 points, so the framework should tolerate higher noise loads.","Samples that sit near the decision boundary are no longer discarded or blindly corrected; they keep contributing to training through the fused soft labels, increasing usable data.","The gains persist when RepFace is combined with a different hard-sample-mining loss (AdaFace), so the framework composes with existing losses rather than replacing them.","On large-scale MS1MV2 with 20% noise, RepFace achieves the best IJB-C TAR@FAR=1e-4 among compared methods, indicating the improvement is not limited to small training sets."],"supporting_citations":[{"why":"Supplies the BoundaryFace baseline and the decision-boundary perspective that RepFace extends with three-way splitting and smoothing; the 0.19-point gain is measured against it.","marker":"Wu and Gong 2022"},{"why":"Provides the MV-Softmax hard-sample-mining loss used as the baseline training loss and the margin formulation $T(\\cos\\theta_{y_i})=\\cos(\\theta_{y_i}+m)$.","marker":"Wang et al. 2020"},{"why":"Supplies the ArcFace additive angular margin and the training recipe (112x112 crops, IR-50, scale $s=64$, margin $m=0.5$) used in RepFace.","marker":"Deng et al. 2019"},{"why":"Supplies the self-ensemble label correction idea that the Label Robust Fusion memory-bank module adapts.","marker":"Lu and He 2022"},{"why":"Supplies the CASIA-WebFace training set on which the 10% and 20% synthetic closed-set noise experiments are run.","marker":"Yi et al. 2014"},{"why":"Supplies the MS1MV2 training set used for the generalizability and IJB-B/IJB-C evaluations.","marker":"Guo et al. 2016"},{"why":"Documents the roughly 9.3-13% noise in CASIA-WebFace, which motivates the 10% and 20% synthetic noise ratios.","marker":"Wang et al. 2018a"}],"fun_headline_variants":["RepFace beats label noise by triaging samples into three groups","RepFace uses synthetic noise to teach early training resilience","RepFace: clean, ambiguous, noisy split for better face recognition","RepFace progressively corrects labels, beating closed-set noise","RepFace's three-way sample split beats prior label-noise work"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that real closed-set label noise behaves like randomly flipping a fixed fraction of each identity's samples to random other identities; the method's thresholds, tuning, and the claimed gains are established only on that synthetic noise model.","fun_headline_variants_meta":{"raw":{"variants":["RepFace beats label noise by triaging samples into three groups","RepFace uses synthetic noise to teach early training resilience","RepFace: clean, ambiguous, noisy split for better face recognition","RepFace progressively corrects labels, beating closed-set noise","RepFace's three-way sample split beats prior label-noise work"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00072,"raw_usage":{"total_tokens":3303,"prompt_tokens":1088,"completion_tokens":2215,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":704,"completion_tokens_details":{"reasoning_tokens":2131}},"tokens_in":704,"tokens_out":2215,"duration_ms":13957,"temperature":1.0,"reasoning_tokens":2131,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:20:54.479971+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RepFace against BoundaryFace on a web-scraped face dataset with naturally occurring label noise, measured by TAR@FAR=1e-4 on IJB-C; if the gain disappears or reverses, the synthetic-noise claim fails. A sharper test: construct structured closed-set noise by consistently confusing similar-looking identities (same wrong label repeated per image, rather than random flips) at 10-20% and check whether RepFace still beats BoundaryFace by roughly 0.19 points.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MV-Softmax hard-sample-mining loss used as the baseline training loss and the margin formulation $T(\\cos\\theta_{y_i})=\\cos(\\theta_{y_i}+m)$."}],"review_version":1}