{"id":"1631a975-9b4e-4e54-81aa-80c1cd44b32e","arxiv_id":"2501.12020","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Filtering face recognition test data so male and female images share specific non-demographic attributes such as hairstyle and facial hair makes measured gender accuracy gaps nearly disappear, but the causal interpretation is not supported.","lead":"Face recognition systems show large accuracy gaps between male and female users. This paper argues the gap is not about gender itself, but about appearance attributes like hairstyle and facial hair, and reports that the gap nearly vanishes when male and female test images are matched on such attributes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The greedy search optimizes iGARBE, so finding fair attribute combinations is partly by construction; without a random-combination baseline, the vanishing gap does not support the paper's causal conclusion.","rationale":"The reader's verdict is REJECT, with the weakest assumption being that filtering to attribute-matched subsets allows causal attribution. My analysis converges on the same load-bearing concern: the search explicitly optimizes the fairness metric, so the existence of fair subsets is expected and cannot support the conclusion that gender bias is an illusion or that biology is irrelevant. I agree with the reader's framing and add technical specificity by pointing to Algorithm 1 and Equation 8, and to the absence of a null distribution. The concrete test—a random-combination baseline and/or held-out validation—would settle whether the vanishing gap is an artifact of selection. If random combinations also achieve high iGARBE, the paper's central claim is reduced to a trivial existence statement; if they do not, the claim gains some support. Either way, the current manuscript does not include this control, so the REJECT verdict is appropriate. I see no reason to adjust the reader's verdict, hence UNCHANGED.","tokens_in":24247,"tokens_out":2885,"duration_ms":33131,"concrete_test":"Sample 10,000 random assignment combinations of cardinality equal to the reported top combinations (e.g., 4–6 assignments) from the 27 decorrelated clusters, apply the same equalization and sampling pipeline (Section III-B2), and compute iGARBE at FMR=1e-3 for both ArcFace and FaceNet. Compare the distribution of random iGARBE values to the top-10 scores (0.9974–0.9997). If a substantial fraction (e.g., >5%) of random combinations also exceed 0.99, the vanishing gap is not specific to the selected attributes and the causal interpretation collapses. As a complementary check, split the comparison database in half, run the greedy search on one half, and evaluate the selected combinations on the other half; if iGARBE drops markedly, the results are overfitted to the search.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that sharing non-demographic attributes makes the gender gap vanish, implying bias is social rather than biological—rests on the top combinations reported in Tables IIIa and IIIb. But those combinations are not passively observed; they are the output of a search (Algorithm 1, Section III-B) whose ranking metric R_f (Equation 8) explicitly rewards increasing iGARBE, and whose tree expansion keeps only assignments that improve fairness relative to baseline. The procedure therefore guarantees that reported combinations have high iGARBE; the empirical 'vanishing gap' is partly an artifact of the optimization objective. The paper provides no null comparison: it does not report the distribution of iGARBE over random attribute combinations of the same cardinality, nor does it validate the selected combinations on a held-out split of the data. Without such a baseline, observing that the top-10 selected combinations reach iGARBE ~0.99 is expected under selection, and does not establish that the original gender gap is caused by the selected attributes rather than by any sufficiently specific subpopulation. This is especially pertinent because the attributes include clearly biological traits (e.g., facial hair, receding hairline, big lips), so even if the gap vanishes on matched subsets, the conclusion that 'biology' is irrelevant does not follow. The causal claim in Section VII is therefore unsupported by the experimental design.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an automated search over combinations of non-demographic facial attributes to explain the gender gap in face recognition accuracy. It introduces three components: a clustering-based decorrelation toolchain for MAAD-Face attributes, an inverse GARBE (iGARBE) fairness metric together with a contextualized fairness measure (CoFair), and a greedy search algorithm that forms attribute combinations whose use as filter predicates yields nearly equal error rates for male and female subjects. With ArcFace and FaceNet on VGGFace2/MAAD-Face, the authors report top combinations with iGARBE near 0.99 and conclude that the gender gap vanishes once male and female subjects share these attributes, interpreting this as evidence that gender bias is a matter of social definition of appearance rather than biology.","tokens_in":24522,"tokens_out":7491,"duration_ms":81110,"significance":"If the causal claim were supported, the paper would be a meaningful contribution to the fairness-in-biometrics literature: it extends prior single-attribute analyses to a large set of 40 attributes, makes an explicit attempt to reduce attribute correlation, evaluates on two widely used face recognition models, and reports large genuine-sample counts. The toolchain and search framework are potentially reusable for exploratory fairness analysis. However, the central causal conclusion is not established by the current experimental design; the paper presently reads as an optimization study that finds fair subpopulations, not as a causal explanation of the gender gap.","major_comments":[{"comment":"The near-perfect iGARBE scores reported in Tables IIIa and IIIb are an expected consequence of the optimization procedure, not an empirical discovery. The ranking metric R_f = σ(ω(f_i − f_0)) explicitly rewards increases in iGARBE, and Algorithm 1 discards every candidate with FAIRNESS(m_Al) ≤ f0 (line 10) and branches only into the top-n assignments (line 14). The paper provides no null distribution over random attribute combinations of the same cardinality and no held-out evaluation, so the selected combinations' high scores cannot be used to attribute the original gender gap to those attributes. A random-combination baseline and an out-of-sample replication are necessary before the causal reading in Section VII is warranted.","section":"Section III-B3, Eq. (8); Algorithm 1"},{"comment":"The paper reports no uncertainty or validation for the selected combinations. Although Section III-B2 draws γ=3 sample sets, Tables II and III present only single point estimates, without standard deviations, confidence intervals, or repeated-selection results. Since the same VGGFace2/MAAD-Face data are used both to select and to evaluate the combinations, the reported iGARBE values may reflect selection overfitting; the large genuine-sample counts reduce sampling noise but do not address this issue. The cross-model results (ArcFace combinations evaluated on FaceNet and vice versa) are informative but are still on the same underlying data.","section":"Section V-C and Tables IIIa/IIIb"},{"comment":"The conclusion that 'Gender bias in FRS is likely no issue of biology but of the social definition of gender-specific appearance' is not supported by the experimental design. The selected combinations include attributes that are at least partly biological or morphological, such as Receding Hairline, Frontal Facial Hair, Big Lips, and Corpulent in Tables IIIa and IIIb. Filtering on these attributes removes biological variation together with social appearance, so the vanishing gap is also consistent with a morphological or biological explanation. To support the social-definition claim, the authors would need to demonstrate that the gap vanishes when biological/morphological factors are controlled for, or that purely social attributes alone are sufficient; neither is shown.","section":"Section VII"},{"comment":"CoFair is presented as a core metric, but its estimate rests on a kernel density over a very small number of scores. The KDE uses only the 41 single-attribute iGARBE values from Table II, with a bandwidth adjustment factor of 0.5, and the CoFair values in Table III (e.g., 0.9997) are tail probabilities of that smoothed 41-point distribution. The metric is therefore not robust enough to support statements such as 'higher than 99.9% of single attributes'; the paper should either report the sensitivity of CoFair to bandwidth and sample size or substantially weaken the contextual claims.","section":"Section III-A2 and Section V-C1"}],"minor_comments":[{"comment":"Calling Algorithm 1 'unsupervised' is misleading, since it optimizes an explicit fairness objective; 'objective-guided search' or 'automated search' would be more accurate.","section":"Section I and Section III-B"},{"comment":"The term 'decorrelated' overstates the outcome; the clustering result still has a mean inter-cluster correlation of about 0.21 and a maximum of about 0.67, so the attribute set is less correlated rather than decorrelated.","section":"Section V-A"},{"comment":"There are several typos and stylistic issues: 'seperately' in the Related Work section, 'assigment' in Section III-B3, and 'Expanability' in the Index Terms should be corrected.","section":"Throughout"},{"comment":"The notation 'Tγ S(∩∅) g = ∅' is not defined; clarify that it refers to the disjointness of the γ genuine-sample sets.","section":"Section III-B2"},{"comment":"The first data row in Tables IIIa and IIIb appears to be the unfiltered baseline but is not labeled as such; add an explicit 'baseline' label.","section":"Table III"},{"comment":"The 90% threshold for 'strongly correlated' is introduced only in the text; state it in the caption or define it before referencing the figure.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important problem, and the authors' effort to move beyond single-attribute analyses is commendable. However, the gap between the exploratory search results and the strong causal conclusion in Section VII is wide. I would require either a substantial reanalysis with random-combination baselines, held-out validation, and appropriate uncertainty quantification, or a careful reframing of the claim as an optimization/exploration result rather than a causal explanation, before considering it for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the toolchain: decorrelation-by-clustering, the CoFair metric, and a greedy search over combinations of 40 attributes rather than single attributes. That is a useful contribution to the biometrics fairness literature, and the scale of the evaluation on MAAD-Face/VGGFace2 is a plus. The empirical observation that some attribute-matched subsets show near-equal error rates between men and women is plausible and worth reporting; it has direct implications for how test sets should be built. The cross-model transfer results (ArcFace combinations evaluated on FaceNet and vice versa) give some independence and are a good instinct.\n\nNow the soft spots, and they are substantial. The search explicitly ranks combinations by a metric whose R_f term rewards higher iGARBE (Equation 8), and the tree expansion keeps only assignments that improve fairness. So the top-10 combinations reaching iGARBE around 0.99 is expected under selection, not evidence that those attributes cause the original gap. The paper has no random-combination baseline, no confidence intervals, and no held-out split. That is the load-bearing flaw in the causal claim. The conclusion that bias is social rather than biological does not follow from the data: attributes like facial hair are themselves biological traits, and filtering on any sufficiently specific subpopulation can shrink a gap without identifying the cause. Even the paper's own limitations section does not address selection bias. The 'unsupervised' framing is also a bit generous; the search is unsupervised with respect to attribute labels, but it is optimizing fairness.\n\nThe prior literature is handled honestly, and the self-citations are appropriate because the same group did the earlier individual-attribute work. The conceptual conclusion overlaps with Albiero and Bowyer's 'gendered hairstyles and biology' argument, so the novelty is in the method, not the headline. The paper is coherent on its own terms; the authors just overreach from an optimized search to a biological claim.\n\nWho is this for? The biometrics fairness community will want to know about the decorrelation and combination-search tools, and the empirical finding that attribute-balanced test sets shrink the gap is useful for evaluation methodology. It deserves a serious referee: the method is worth publishing, but the conclusion needs to be narrowed to an evaluation-methodology claim and the design needs a null baseline. Send it to review, but expect heavy revision.","headline":"The combination search is a real methodological contribution, but the causal claim that gender bias is an illusion is not supported by the design: the search optimizes iGARBE, so finding near-perfect fairness scores on selected subsets is partly by construction.","tokens_in":25066,"tokens_out":1477,"would_cite":false,"duration_ms":18554,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that the gender gap in face recognition is an illusion: it vanishes once male and female test images share the same non-demographic attributes such as hairstyle, facial hair, and accessories.","keywords":["gender bias","face recognition","fairness","non-demographic attributes","attribute decorrelation","iGARBE","CoFair","MAAD-Face"],"falsifier":"Take the full comparison database and sample many random attribute combinations of the same size as the reported fair combinations; if a large fraction of them also reach iGARBE values above 0.99 under the same sampling procedure, the vanishing gap is a selection effect rather than evidence about specific attributes.","tokens_in":24017,"feed_emoji":"🎭","tokens_out":4974,"duration_ms":46394,"temperature":0.7,"pith_summary":"The paper aims to show that the well-documented gender gap in face recognition accuracy is not caused by gender itself, but by non-demographic appearance attributes socially correlated with gender. It introduces an unsupervised search that decorrelates 40 facial attributes, forms clusters, and looks for combinations that, when used to filter a test set so that both genders share them, push the fairness score toward perfect. Across two recognition models, the top combinations reach fairness scores above 0.99, and the error difference between male and female subjects becomes negligible. If correct, this means fairness evaluations should control for these shared appearance attributes, and mitigations should target robustness to hairstyle, facial hair, and occluding accessories rather than gender per se.","feed_headline":"Gender gap in face recognition vanishes when attributes match","feed_subtitle":"Filtering tests so men and women share hairstyle, facial hair, and accessories erases the accuracy gap.","key_machinery":"The method runs on three components. A decorrelation-by-clustering toolchain groups the 40 MAAD-Face non-demographic attributes into 27 clusters so that correlated attributes are treated as one unit. Two new metrics quantify fairness: iGARBE, a fairness score bounded in [0,1] with 1 meaning perfect fairness, and CoFair, which contextualizes a fairness score as a percentile relative to scores of other attribute sets. The search itself is a greedy breadth-first expansion that repeatedly adds the n most fairness-increasing attribute-label assignments to combinations, using a ranking metric that balances sample retention, verification error, and fairness. Applying a combination as a filter predicate to the comparison dataset, requiring both compared images to carry the same attribute labels, is what the authors call equalizing, and this operation is what makes the gender gap collapse.","core_discovery":"The central claim is that once male and female subjects share specific non-demographic attributes, the gender gap in recognition accuracy vanishes. The paper reports that this holds for both tested face recognition models, ArcFace and FaceNet, across all experimental setups, with the top attribute combinations yielding iGARBE fairness scores of 0.997 to 0.9997. On that basis the authors interpret gender bias in face recognition as likely originating from non-demographic attributes associated with gender, such as hairstyles, facial hair, and occluding accessories, rather than from gender itself. The vanishing gap is presented not as a tweak to the metric but as evidence that the apparent bias is an artifact of how appearance differs between the male and female groups in existing test data.","pith_inferences":["Editorial inference: because the search explicitly optimizes the fairness metric, finding high-scoring combinations is partially by construction; a stronger test of the causal claim would compare the best combinations against a null distribution of randomly chosen combinations of the same size.","Editorial inference: the 'social definition' interpretation predicts that the specific attribute combinations that erase the gap will shift as gender-specific appearance norms change, which is a testable prediction on newer datasets.","Editorial inference: the same decorrelation-and-search pipeline could be transferred to other protected attributes such as age or ethnicity, but the attribute set and correlation structure would need to be re-derived for each case."],"forward_implications":["If the claim holds, fairness evaluations of face recognition should report results on test sets balanced not only by gender but also by the relevant non-demographic attributes.","The gender gap can be erased by controlling for a small number of appearance categories, so a fair FRS is attainable through targeted test-set design or robustness to these attributes.","The finding supports the conclusion that balancing training data by gender alone is insufficient, since the gap is driven by appearance attributes rather than gender representation.","Mitigation efforts should shift from removing gender information to making models invariant to hairstyle, facial hair, and occluding accessories."],"supporting_citations":[{"why":"Supplies the MAAD-Face database with 40 non-demographic attribute annotations used for all filtering and clustering.","marker":"[46]"},{"why":"Provides VGGFace2, the underlying image dataset whose templates are compared in the experiments.","marker":"[47]"},{"why":"Defines the GARBE fairness model that the paper inverts to create the iGARBE metric.","marker":"[44]"},{"why":"Established that balancing training data by gender does not remove the gender gap, motivating the search for non-demographic causes.","marker":"[14]"},{"why":"Prior comprehensive analysis of 47 attributes individually showing strong correlations, which this work extends to decorrelated combinations.","marker":"[10]"},{"why":"Showed that balancing on scalp hairstyle reduces the gender gap for genuine pairs, a direct precursor to the attribute-equalization approach.","marker":"[9]"},{"why":"ArcFace is one of the two face recognition models on which the vanishing-gap results are demonstrated.","marker":"[49]"},{"why":"FaceNet is the second face recognition model used to confirm that the findings generalize across models.","marker":"[50]"}],"fun_headline_variants":["Face recognition gender gap vanishes with matched attributes","Attribute sharing erases face recognition gender bias","Gender bias in face ID is really about appearance","Face recognition fairness: shared attributes close the gap","Study: face recognition bias tied to hairstyle, not gender"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that filtering the test data to subsets where both genders share the same attribute labels reveals those attributes as the cause of the original gender gap, rather than simply selecting subsets that happen to have smaller bias.","fun_headline_variants_meta":{"raw":{"variants":["Face recognition gender gap vanishes with matched attributes","Attribute sharing erases face recognition gender bias","Gender bias in face ID is really about appearance","Face recognition fairness: shared attributes close the gap","Study: face recognition bias tied to hairstyle, not gender"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000626,"raw_usage":{"total_tokens":2879,"prompt_tokens":911,"completion_tokens":1968,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":1896}},"tokens_in":527,"tokens_out":1968,"duration_ms":16870,"temperature":1.0,"reasoning_tokens":1896,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:36:37.059485+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the full comparison database and sample many random attribute combinations of the same size as the reported fair combinations; if a large fraction of them also reach iGARBE values above 0.99 under the same sampling procedure, the vanishing gap is a selection effect rather than evidence about specific attributes.","supporting_citations":[{"cited_title":"MAAD-Face: A Mas- sively Annotated Attribute Dataset for Face Images,","cited_arxiv_id":null,"evidence_quote":"Supplies the MAAD-Face database with 40 non-demographic attribute annotations used for all filtering and clustering."},{"cited_title":"VGGFace2: A Dataset for Recognising Faces across Pose and Age,","cited_arxiv_id":null,"evidence_quote":"Provides VGGFace2, the underlying image dataset whose templates are compared in the experiments."},{"cited_title":"Evaluating Proposed Fairness Models for Face Recognition Algorithms,","cited_arxiv_id":null,"evidence_quote":"Defines the GARBE fairness model that the paper inverts to create the iGARBE metric."},{"cited_title":"How Does Gender Balance In Training Data Affect Face Recog- nition Accuracy?","cited_arxiv_id":null,"evidence_quote":"Established that balancing training data by gender does not remove the gender gap, motivating the search for non-demographic causes."},{"cited_title":"A Com- prehensive Study on Face Recognition Biases Beyond Demographics,","cited_arxiv_id":null,"evidence_quote":"Prior comprehensive analysis of 47 attributes individually showing strong correlations, which this work extends to decorrelated combinations."},{"cited_title":"The gender gap in face recognition accuracy is a hairy problem,","cited_arxiv_id":null,"evidence_quote":"Showed that balancing on scalp hairstyle reduces the gender gap for genuine pairs, a direct precursor to the attribute-equalization approach."},{"cited_title":"ArcFace: Additive Angular Margin Loss for Deep Face Recog- nition,","cited_arxiv_id":null,"evidence_quote":"ArcFace is one of the two face recognition models on which the vanishing-gap results are demonstrated."},{"cited_title":"FaceNet: A unified embedding for face recognition and cluster- ing,","cited_arxiv_id":null,"evidence_quote":"FaceNet is the second face recognition model used to confirm that the findings generalize across models."}],"review_version":1}