{"id":"e780c549-3322-474d-8cdd-530de9102732","arxiv_id":"2508.06357","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A classifier trained on the ranks of additional enrolled images of the rank-one match can detect out-of-gallery probes more accurately than score thresholds.","lead":"This paper proposes a new way to tell whether the top match in a face search is truly in the gallery or not, by looking at how many of that person's other photos also rank highly. The method could reduce wrongful arrests from face recognition, but its real-world value depends on gallery structure and on avoiding data leakage.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Train/test split is not shown to be identity-disjoint; if identities overlap across splits, reported accuracy may reflect memorized gallery identities rather than generalization to unseen probes.","rationale":"The reader's weakest_assumption focuses on the core premise that Out-of-gallery rank-one identities are selected by incidental similarities, so their additional images are displaced. That is a meaningful conceptual risk, and the paper provides only limited empirical support (Figure 3, one demographic). However, the evaluation-protocol issue is more immediately damaging: if the train/test split is not identity-disjoint, every reported accuracy could be inflated, and the central claim of feasibility for unknown probes would be unsubstantiated regardless of whether the premise holds. The paper's own wording ('disjoint 80-20 training and testing sets') is ambiguous, and the reader correctly flagged the missing identity-disjoint specification in the rationale. This is a concrete, fixable issue that should be resolved before the results are relied upon. I therefore maintain the CONDITIONAL verdict, with the condition being a clean identity-disjoint evaluation. Credit is due for a genuinely new feature source and for evaluating multiple matchers and degradations, but those contributions do not remove the need for a valid generalization test.","tokens_in":792,"tokens_out":740,"duration_ms":84965,"concrete_test":"Re-run the MORPH and MFAD experiments with an identity-disjoint split: assign all images of each identity exclusively to training or test; build galleries for training and test separately from their respective identity sets; regenerate In/OOG rank vectors; retrain the MLP; compare per-demographic/per-condition accuracy to Tables 1 and 2. A substantial drop (e.g., >5 points on original probes) would indicate the original split allowed leakage and the feasibility claim is not established. The authors should also report the exact splitting code or random seed to make the protocol reproducible.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing concern is the evaluation protocol, not the conceptual premise. Section 5 states the curated dataset is 'split into disjoint 80-20 training and testing sets' but never specifies that the split is identity-disjoint. Each probe identity generates two samples: one In-gallery (rank-one correct) and one Out-of-gallery (rank-one false). A random sample-level split can place the same identity's In sample in training and its OOG sample in test, and more importantly can place the same identities in the galleries used to compute training and test rank vectors. Since rank features are computed against the gallery, any identity present in both train and test galleries lets the MLP memorize identity-specific rank behavior rather than learn a generalizable In/OOG signal. The paper's headline accuracy (up to 98.58% on original probes) therefore may not reflect performance on truly unseen identities, undermining the central claim that the method can determine In/OOG for unknown probes. This is concrete and testable, whereas the 'incidental similarities' premise is at least partially supported by Figure 3 for the tested setting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a threshold-free method for deciding whether the rank-one identity returned by a one-to-many face search is actually in the gallery or is an out-of-gallery false match. The signal is the vector of ranks of additional enrolled images of the rank-one identity: for a true in-gallery match these additional images tend to rank close behind rank-one, while for a false match they are displaced. The authors train a small MLP on this rank vector, using synthetic in-gallery/out-of-gallery training data constructed from MORPH and ND-MFAD by including or excluding the probe identity's own images from the gallery. Experiments with FaceNet, ArcFace, AdaFace, and TransFace under original, blurred, downsampled, turbulence-distorted, and sunglasses-wearing probes report accuracies up to 98.58%, with strong performance for margin-based modern matchers and degraded performance for FaceNet. The method is compared favorably against score thresholding, mean/median classifiers, and gallery-image fusion.","tokens_in":12934,"tokens_out":4921,"duration_ms":56950,"significance":"If the results are valid, the paper offers a genuinely new signal for open-set face identification that does not require score threshold calibration and leverages the multi-image enrollment already common in operational galleries. Strengths include evaluation on two datasets with different acquisition conditions, four matchers spanning pre-margin and margin-based architectures, an explicit degradation analysis, a demographic breakdown, and comparison to several baselines. The core weakness is that the evaluation protocol does not guarantee the training and test sets are identity-disjoint, which is essential for supporting the claim that the classifier generalizes to genuinely unknown probe identities. The paper is clearly written and the idea is well motivated, but the headline accuracy figures must be re-derived under an identity-disjoint split before the central claim can be accepted.","major_comments":[{"comment":"The split into 'disjoint 80-20 training and testing sets' is not specified as identity-disjoint. Each probe identity contributes an In-gallery sample (its images in the gallery) and an Out-of-gallery sample (its images removed). If the same identity appears in both training and testing galleries, the MLP may learn identity-specific rank patterns rather than a generalizable In/Out signal, and the reported accuracies (e.g., 98.58% in Table 1) may not reflect performance on unseen identities. This directly affects the central claim about 'unknown probe image'. Please re-run the evaluation with an identity-disjoint split (all samples of a given identity restricted to either train or test) and report the resulting accuracies, or explicitly justify why sample-level disjointness is sufficient.","section":"Section 5, 'Train-Test Rank Data Curation'"},{"comment":"Only a single accuracy value is reported per condition and demographic group, with no variance, confidence intervals, or number of runs. The paper concludes that accuracy is 'similar across demographics' based on differences of a few percentage points (e.g., 98.58 vs 97.24 for original probes, AdaFace). Without variance estimates or repeated-seed statistics, these differences are not interpretable. Please report mean and standard deviation across cross-validation folds or multiple random seeds, and state the number of test samples per group so the reader can assess the precision of the demographic comparison.","section":"Section 6, Tables 1 and 2"}],"minor_comments":[{"comment":"The argument that a two-dimensional output requires at least three input dimensions to avoid 'trivial mappings' is handwavy. Figure 5 is cited as the empirical justification, but the figure caption does not describe the experimental setup or whether the plateau is statistically robust. Please clarify what is plotted and add error bars.","section":"Section 5, 'Rank Cardinality for Effective Learning'"},{"comment":"The feature permutation augmentation is reasonable if the three additional ranks are exchangeable, but this should be stated explicitly. If the enrolled images have a fixed order (e.g., chronological), permuting would destroy a potentially meaningful signal.","section":"Section 5, 'Data Augmentation for Robust Classification'"},{"comment":"The paper claims the method 'naturally generalizes' to variable enrollment sizes, but all experiments use exactly four enrolled images. The proposed per-enrollment-size classifiers and repeated-sampling procedure are not evaluated. This is a practical claim that needs at least a small experiment or should be phrased as future work.","section":"Section 8, 'Handling Variable Image Enrollment Conditions'"},{"comment":"Typo: 'Florida Insitute of Technology' should be 'Florida Institute of Technology'.","section":"Author affiliation, title page"},{"comment":"No code or model weights are provided. Given the simplicity of the classifier, releasing code would aid reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and presents a novel, well-motivated approach. The main risk is the evaluation protocol: an identity-disjoint split is essential for the claimed generalization to unknown probes. If the authors can show that the accuracy holds under such a split, the paper would be a solid contribution. The demographic-restricted gallery design is a deliberate choice, but the absence of mixed-gallery experiments should at least be discussed as a limitation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the idea is real. Using the ranks of additional enrolled images of the rank-one identity as a feature to classify in-gallery vs. out-of-gallery is genuinely new relative to the threshold, EVT, and fusion lines. The results are consistent across two datasets and four matchers, and the degradation experiments are a real plus. The paper also deserves credit for comparing fairly against thresholding, mean/median classifiers, and feature fusion, and for honestly showing that older matchers like FaceNet fall apart under downsampling. That last point supports their claim that the signal depends on margin-based training, which is a useful finding in itself.\n\nThe main soft spot is the evaluation protocol. The paper says the curated dataset was split into disjoint 80-20 training and testing sets, but it never says the split is identity-disjoint. That matters because each probe identity produces one in-gallery sample and one out-of-gallery sample, and the rank features are computed against a gallery. A random sample-level split can place the same identity in both train and test galleries, letting the MLP memorize identity-specific rank behavior rather than learn a generalizable signal. If that happened, the 98.58% headline number would not reflect performance on unseen identities. This is concrete, testable, and not resolved by anything in the text. It is the load-bearing concern, not the \"incidental similarities\" premise; that premise is at least partially supported by Figure 3 for the tested setting.\n\nOther issues are minor by comparison. Single test accuracies are reported with no variance or error bars. Galleries are restricted to same-demographic groups, so the demographic claims are narrower than the framing suggests. The comparison table is run only on the African-American male subset. And the variable-enrollment handling is described but not demonstrated. None of these kill the idea; the central argument holds up if the split is clean, but the paper as written does not establish that.\n\nWho is this for? People working on open-set face identification, especially anyone concerned with false arrests and 1-to-many search failure modes. It deserves a serious referee, but the referee should require the authors to specify and verify an identity-disjoint split and to report variance across folds or random seeds. With that fixed, this would be a solid applied contribution.","headline":"Genuinely novel rank-vector feature for in/out-of-gallery detection; results are consistent and useful, but the missing identity-disjoint split must be clarified before the headline accuracy is credible.","tokens_in":13322,"tokens_out":2264,"would_cite":true,"duration_ms":29018,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the ranks of additional enrolled images of the rank-one match—rather than a similarity-score threshold—can tell whether a probe is in the gallery, reaching up to 98.58% accuracy on mugshot-quality probes.","keywords":["open-set face recognition","1-to-many identification","in-gallery / out-of-gallery detection","rank-based features","face embedding networks","margin-based losses","probe image degradation","demographic analysis"],"falsifier":"Take a probe known to be out-of-gallery whose highest-scoring false identity has several enrolled photos that look similar to the probe (same expression, beard, lighting). If those additional photos consistently rank near the top, and a classifier trained on this data cannot beat chance, the premise fails. Concretely: measure the rank of the second enrolled image of the rank-one identity for known out-of-gallery probes; if a large fraction of false hits have additional images inside the top ranks, the signal disappears.","tokens_in":12605,"feed_emoji":"🪪","tokens_out":6670,"duration_ms":74744,"temperature":0.7,"pith_summary":"The paper tries to establish that whether a one-to-many face search has hit a real enrolled identity or a false positive can be read from the ranks of the other photos of the top-matched identity. Instead of tuning a similarity-score threshold, it trains a small classifier on a three-dimensional rank vector: where the second, third, and fourth enrolled images of the rank-one identity land in the search result. The paper reports that this signal works on two datasets and four face matchers, with up to 98.58% accuracy on clean mugshot probes, and remains strong under blur, low resolution, atmospheric turbulence, and sunglasses for modern matchers trained with margin-based losses. If correct, the result offers a threshold-free, per-search confidence estimate that could reduce false identifications in open-set face recognition deployments.","feed_headline":"A face match's extra photos reveal whether it is real","feed_subtitle":"No score thresholds: a small classifier reads ranks of extra enrolled photos to tell gallery matches from impostor hits.","key_machinery":"The central object is the rank query vector: for a probe search, take the rank-one identity and record the ranks of up to three additional enrolled images of that identity. For an in-gallery probe these ranks are concentrated near the top, while for an out-of-gallery probe they scatter, because the false rank-one identity was selected by incidental similarity. A small multi-layer perceptron with feature-permutation augmentation learns the non-linear boundary between these two rank patterns, using $R_{\\mathrm{din}}=3$ ranks as input in all experiments.","core_discovery":"The paper's central claim is that a probe is in-gallery precisely when the additional enrolled images of the rank-one identity keep ranking near the top: for a true match these images cluster at low ranks, whereas for a false rank-one match they drift far down because that identity was selected by incidental similarity. The authors build a small nonlinear classifier on the ranks of up to three additional images, trained on paired searches with and without the probe's own images, and report that this classifies in-gallery vs. out-of-gallery probes with 98.58% accuracy on original MORPH probes using AdaFace, stays above 86% for blurred, downsampled, turbulent, and sunglasses probes with modern","pith_inferences":["An untested extension is combining the rank-pattern classifier with a traditional score threshold; thresholding and rank patterns may be complementary, and a fused decision could raise precision in near-boundary cases.","The contrast between FaceNet and margin-based matchers suggests a diagnostic: the method's accuracy should track how tightly the embedding loss packs same-identity images in rank space; one could measure rank displacement across models trained with increasing angular margins.","The same idea—using the ranks of the top match's other enrolled samples—should transfer to other multi-sample biometrics or to re-identification, but the paper only demonstrates faces; a test on fingerprints or person re-identification would show whether the signal is general.","A cautious deployment reading: the classifier gives a continuous confidence score, so it could be used to flag low-confidence hits for human review rather than as a binary arrest trigger; the paper reports classification accuracy, not downstream decision policy."],"forward_implications":["Open-set face identification can be performed without hand-tuned score thresholds; the rank pattern of the top identity's extra photos supplies the decision signal.","With margin-based-loss matchers such as AdaFace and TransFace, the method keeps high accuracy on degraded probes, so it can be used in surveillance-style settings where blur and low resolution are common; older embeddings like FaceNet are not sufficient.","Accuracy stays within a few points across the four demographic groups tested, so the approach does not introduce a large demographic disparity in this In/Out decision.","In-gallery and out-of-gallery training data can be generated automatically from any multi-image-per-identity gallery by running each probe twice, once with and once without its own images in the gallery.","The method needs only about four enrolled images per identity; variable enrollment sizes can be handled by sampling to a fixed rank-vector size and averaging repeated samples."],"supporting_citations":[{"why":"Documents that real-world identification galleries store multiple images per identity, the precondition for rank-based features.","marker":"[18]"},{"why":"Supplies MORPH, the primary demographic-labeled dataset used for training and evaluation.","marker":"[33]"},{"why":"Supplies MFAD, the second dataset with sunglasses probes used to test generalization.","marker":"[42]"},{"why":"Provides ArcFace embeddings, one of the four matchers and a margin-based-loss representative.","marker":"[13]"},{"why":"Provides TransFace embeddings, the ViT-based matcher that maintains accuracy under degradation.","marker":"[12]"},{"why":"Provides FaceNet embeddings, the older matcher used to show the method fails without margin-based training.","marker":"[38]"},{"why":"Supplies the blur and downsampling degradation settings and the same-demographic sampling motivation.","marker":"[29]"},{"why":"Supplies the atmospheric turbulence simulator used for the distorted-probe condition.","marker":"[28]"}],"fun_headline_variants":["Facial ID: extra photo ranks detect out-of-gallery","No score thresholds: extra photo ranks reveal true match","In-gallery? See if extra photos stay high in rank","Rank of extra gallery photos tells if match is real","Out-of-gallery detection via ranks of extra enrolled images"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The method assumes that a false top match is picked because of some incidental similarity, so the other photos of that wrong identity will rank far down the list; if those other photos also rank near the top, the classifier has no signal to work with.","fun_headline_variants_meta":{"raw":{"variants":["Facial ID: extra photo ranks detect out-of-gallery","No score thresholds: extra photo ranks reveal true match","In-gallery? See if extra photos stay high in rank","Rank of extra gallery photos tells if match is real","Out-of-gallery detection via ranks of extra enrolled images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1235,"prompt_tokens":828,"completion_tokens":407,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":340}},"tokens_in":572,"tokens_out":407,"duration_ms":4940,"temperature":1.0,"reasoning_tokens":340,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:44:52.264745+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a probe known to be out-of-gallery whose highest-scoring false identity has several enrolled photos that look similar to the probe (same expression, beard, lighting). If those additional photos consistently rank near the top, and a classifier trained on this data cannot beat chance, the premise fails. Concretely: measure the rank of the second enrolled image of the rank-one identity for known out-of-gallery probes; if a large fraction of false hits have additional images inside the top ranks, the signal disappears.","supporting_citations":[{"cited_title":"Face recog- nition vendor test (FRVT) part 2: Identification","cited_arxiv_id":null,"evidence_quote":"Documents that real-world identification galleries store multiple images per identity, the precondition for rank-based features."},{"cited_title":"Morph: A longitudinal image database of normal adult age-progression","cited_arxiv_id":null,"evidence_quote":"Supplies MORPH, the primary demographic-labeled dataset used for training and evaluation."},{"cited_title":"Impact of sunglasses on one-to-many facial identi- fication accuracy","cited_arxiv_id":null,"evidence_quote":"Supplies MFAD, the second dataset with sunglasses probes used to test generalization."},{"cited_title":"Arcface: Additive angular margin loss for deep face recognition","cited_arxiv_id":null,"evidence_quote":"Provides ArcFace embeddings, one of the four matchers and a margin-based-loss representative."},{"cited_title":"Transface: Calibrating trans- former training for face recognition from a data-centric per- spective","cited_arxiv_id":null,"evidence_quote":"Provides TransFace embeddings, the ViT-based matcher that maintains accuracy under degradation."},{"cited_title":"Facenet: A unified embedding for face recognition and clus- tering","cited_arxiv_id":null,"evidence_quote":"Provides FaceNet embeddings, the older matcher used to show the method fails without margin-based training."},{"cited_title":"Analyzing the impact of de- mographic and operational variables on 1-to-many face id search","cited_arxiv_id":null,"evidence_quote":"Supplies the blur and downsampling degradation settings and the same-demographic sampling motivation."},{"cited_title":"Accel- erating atmospheric turbulence simulation via learned phase- to-space transform","cited_arxiv_id":null,"evidence_quote":"Supplies the atmospheric turbulence simulator used for the distorted-probe condition."}],"review_version":1}