{"id":"d22e0e66-decb-40c4-ac58-c4a6c27270b5","arxiv_id":"2505.01335","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A PCA-based balanced logistic regression classifier achieves the highest recall for radio-loud quasars (0.52) among the tested models, but with low precision (0.11) and lower overall accuracy than Random Forest.","lead":"This paper tests a PCA-based balanced logistic regression method for separating radio-loud from radio-quiet quasars in SDSS data, comparing it with Random Forest, XGBoost, and SMOTE-based classifiers. It reports that the PCA-logistic classifier recovers more radio-loud quasars (0.52 recall) than the others, but with low precision (0.11) and limited overall accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The target is defined by raw radio flux, not the standard radio-loudness ratio; all reported recall comparisons may measure a different dichotomy.","rationale":"The reader's weakest assumption is exactly the load-bearing issue I identify: the radio-loud label is defined by raw radio flux rather than by the standard flux ratio. This is the first step in the pipeline, and every reported performance number inherits it. If the label is wrong, the classifier comparison is not about radio-loud versus radio-quiet quasars, and the paper's central claim would not stand. The concern is concrete and testable: recompute the labels with the standard ratio and re-evaluate. I agree with the conditional verdict because the qualitative observation, that class rebalancing in a low-dimensional logistic regression improves minority-class recall at the cost of precision, is plausible and broadly consistent with known class-imbalance behavior. The paper's reported numbers, however, cannot be accepted until the label definition is corrected or justified. I therefore leave the reader's conditional verdict unchanged rather than escalating to rejection, since the proposed check could plausibly preserve the main qualitative finding if the label issue is fixed without destroying the apparent recall advantage.","tokens_in":12662,"tokens_out":2867,"duration_ms":31851,"concrete_test":"Recompute the labels using the standard radio-loudness parameter R = log10(f_radio / f_optical) > 1, deriving the optical flux density from the SDSS i-band AB magnitude with appropriate K-correction, or using the catalog's official radio-loud flag. Then rerun the four classifiers (RFC, PBC, XGBoost, SMOTE+RFC) under the same train/test split and compare the radio-loud recall values in Tables 3, 4, and 6. If PBC's recall advantage over XGBoost (0.52 vs 0.39) and SMOTE+RFC (0.52 vs 0.25) shrinks or reverses, the central claim is an artifact of the mislabeled target; if the advantage persists with the corrected labels, the label concern is not fatal to the qualitative conclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PBC raises radio-loud recall to 0.52 depends entirely on what 'radio-loud' means. Section 2 states: 'we applied a logarithmic transformation to the radio flux and labeled sources with log10(radio flux) > 1 as radio-loud and the rest as radio-quiet [10].' This is not the standard definition of radio-loudness, which is the ratio R = log10(f_radio / f_optical) > 1 (Kellermann et al. 1989, reference [10] in the paper). Using raw radio flux alone conflates intrinsic radio-loudness with optical luminosity: a luminous quasar with, say, 8 mJy of radio flux can have a high ratio and be truly radio-loud yet be labeled radio-quiet, while a faint quasar with 12 mJy can have a low ratio and be labeled radio-loud. Since the input features include optical magnitudes, the classifiers may partly be learning optical brightness rather than the physical radio-loud/radio-quiet dichotomy. Every metric in Tables 3, 4, 5, and 6, including the headline recall of 0.52, is computed against this potentially mislabeled target. If the labels are wrong, the comparison is not about radio-loud versus radio-quiet quasars, and the central methodological conclusion is unsupported. This is not merely a matter of convention: the paper itself cites Kellermann et al. (1989), which defines radio-loudness via the ratio, so the implementation is internally inconsistent with its own reference. A secondary concern is that the XGBoost threshold in Table 5 appears to be selected on the test set, which can inflate reported performance, but the label definition is upstream and more fundamental.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Joshi and Shinde present a comparative study of classifiers for distinguishing radio-loud and radio-quiet quasars in the SDSS-DR3 catalog. They apply PCA to the five SDSS optical magnitudes, find that two components capture 97% of the variance, and train a balanced logistic regression classifier on these components (PBC). They compare PBC against a random forest (RFC), an XGBoost classifier with threshold tuning, and RFC with SMOTE. The headline result is that PBC achieves a recall of 0.52 for the minority radio-loud class, far higher than RFC's 0.04, at the cost of lower overall accuracy (0.62 vs 0.92). The paper also reports redshift distribution analysis and a Lyman-alpha forest effect at high redshift.","tokens_in":13022,"tokens_out":6093,"duration_ms":56267,"significance":"If the label definition and evaluation-protocol issues are resolved, the paper would provide a useful, reproducible comparison of simple and interpretable baselines (PCA + balanced logistic regression) against more complex methods on an imbalanced astrophysical classification task. The inclusion of the code and data repository is a strength. However, the methodological novelty is limited: balanced logistic regression on PCA components is a standard technique, and the main contribution is the application-specific comparison. The central recall claim is therefore conditional on fixing the target-variable definition and the evaluation protocol.","major_comments":[{"comment":"The classification target is defined by the raw radio flux (log10(radio flux) > 1) rather than the standard radio-loudness parameter R = log10(f_radio/f_optical), even though reference [10] (Kellermann et al. 1989) defines radio-loudness via this ratio. This is internally inconsistent with the cited reference and misclassifies quasars whose radio flux is low but whose optical flux is even lower, or whose radio flux is high but whose optical flux is higher. Since the input features include optical magnitudes, the classifiers may be learning optical brightness rather than the physical dichotomy. All reported metrics in Tables 3-6, including the headline recall of 0.52, are computed against this potentially incorrect target. The authors should either recompute the labels using the standard ratio (possible with the available optical photometry) or provide a strong astrophysical justification for the raw-flux cutoff and discuss how it differs from the standard definition.","section":"Section 2, label definition"},{"comment":"The XGBoost decision threshold appears to be selected by evaluating performance on the test set for thresholds from 0.10 to 0.90, and the threshold of 0.50 is then reported as 'optimal' along with the resulting metrics. This constitutes test-set leakage: the reported accuracy, recall, and F1 scores for XGBoost at that threshold are fitted to the evaluation data rather than representing a clean out-of-sample prediction. The threshold should be chosen on a validation set or via cross-validation, and only then should the final model be evaluated once on the held-out test set. Without this correction, the XGBoost comparison in Table 6 is optimistically biased.","section":"Section 3.6, Table 5"},{"comment":"It is unclear whether the StandardScaler and the PCA transformation are fitted on the full dataset before the train/test split or on the training split only. If the former, the test set contributes to the estimated mean, variance, and principal components, which is a form of data leakage that can inflate performance estimates. The authors should fit the scaler and PCA on the training split and apply the fitted transform to the test split, and state this explicitly in the methodology.","section":"Section 2 and 3.5, preprocessing"}],"minor_comments":[{"comment":"The abstract reports XGBoost accuracy as 72%, but Table 6 lists 0.78 for the same configuration; the discrepancy should be corrected.","section":"Abstract vs Table 6"},{"comment":"The sentence 'We also identify outliers from the data set with i >19 by simply by separating the outliers from i< 19 dataset' is unclear and contains duplicated wording; clarify whether the identified outliers are removed before classification or only used in visualizations, and report the final sample size after any removal.","section":"Section 2, outlier handling"},{"comment":"There is a typo in 'fiur approaches'; it should be 'four approaches'.","section":"Section 3.5"},{"comment":"There is a typo in 'Diagnostic plots as sbown above'; it should be 'shown'.","section":"Section 3.4"},{"comment":"The model described as 'PCA based balanced linear regression classifier' is implemented as logistic regression (Table 3); use consistent terminology throughout the paper.","section":"General terminology"},{"comment":"The claim that 'the distribution of the RL and RQ quasars in the upper quadrant is significantly different' is not supported by any quantitative test; consider adding a density contour plot or a statistical comparison.","section":"Figure 10 and surrounding text"}],"recommendation":"major_revision","confidential_remarks":"The abstract inconsistency between 72% and 78% accuracy for XGBoost, together with the label definition issue, suggests that the manuscript needs careful checking before publication. The paper's scope is appropriate for an astronomical journal but the novelty is modest; the acceptance decision should hinge on whether the authors can validate the results under a corrected label definition and leakage-free evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Ram, quick take on arXiv:2505.01335. The central claim—that a PCA + balanced logistic regression ('PBC') lifts radio-loud recall to 0.52 versus 0.04 for RFC—does not hold up on inspection because the target itself is defined by log10(radio flux)>1, not the standard radio-loudness ratio R = log10(f_radio/f_optical)>1. The paper cites Kellermann et al. 1989 for that cutoff, but Kellermann defines radio-loudness via the ratio. Using raw flux alone conflates intrinsic radio-loudness with optical luminosity. Since the input features are optical magnitudes, the classifier can partly learn brightness instead of the physical dichotomy. Every table metric is computed against this potentially mislabeled target. That is the load-bearing flaw.\n\nWhat the paper does well: it releases code and data on GitHub, compares four classifiers on a real imbalanced quasar sample, and gives a plausible interpretation of the PCA components (PCA1 = overall brightness, PCA2 = UV excess). The qualitative result—class-weighting raises minority recall at the cost of precision and accuracy—is standard but clearly demonstrated. The t-test on redshift distributions, while peripheral, is honest and reproducible.\n\nOther soft spots:\n- XGBoost threshold tuning: Section 3.6 selects the decision threshold by scanning 0.1–0.9 on the test set and then reports the 'optimal' metrics at 0.50. That is in-sample selection; the 0.60 AUC confirms weak separability.\n- Abstract vs Table 6: abstract says XGBoost accuracy is 72%, Table 6 says 0.78. Needs correction.\n- Novelty claim: 'novel PCA based balanced regression classifier' is just PCA followed by logistic regression with class_weight='balanced' in scikit-learn. The paper's own text mostly calls it 'balanced logistic regression.' The title overstates.\n\nNone of these destroy the qualitative trade-off story, but the reported numbers are unsupported until the label is fixed. If the authors redo the analysis with the standard radio-loudness ratio (and ideally discuss the SDSS flux limit), the recall comparison might survive in direction, not in magnitude.\n\nWho is this for? Astronomers working on rare-class photometric classification might find the code and the trade-off discussion useful, but the paper needs major revision before being a reliable reference. I would send it to peer review—the empirical question is real and the data/code are available—but with a strong request to fix the label definition and the threshold selection. I would not cite it in its current form.","headline":"The headline recall claim rests on a mislabeled target: radio-loudness defined by raw flux instead of the standard ratio, so the paper's central comparison may be measuring something else; still worth refereeing for its clean code/data and clear demonstration of the precision-recall trade-off.","tokens_in":13555,"tokens_out":3196,"would_cite":false,"duration_ms":31663,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A PCA-based balanced logistic regression, using just two principal components of SDSS photometry, raises recall for rare radio-loud quasars to 0.52, far above the 0.04 to 0.39 range of standard classifiers.","keywords":["radio-loud quasars","radio-quiet quasars","principal component analysis","balanced logistic regression","class imbalance","SDSS-DR3","random forest","XGBoost"],"falsifier":"Recompute the labels with the standard radio-loudness ratio $R = \\log_{10}(f_{\\rm radio}/f_{\\rm optical}) > 1$ on the same SDSS data, rerun the same train/test protocol, and see whether PBC still reaches roughly 0.52 minority recall; a drop toward the baselines would show the reported gain is an artifact of the flux-only label.","tokens_in":12426,"feed_emoji":"🔭","tokens_out":10767,"duration_ms":88259,"temperature":0.7,"pith_summary":"The paper tries to establish that a deliberately simple pipeline—logistic regression on the first two principal components of five SDSS optical magnitudes, with class weights balanced to offset the 1:14 imbalance—can find radio-loud quasars far more reliably than standard classifiers. On the SDSS-DR3 sample it reports 52% recall for the radio-loud minority, versus 4% for random forest, 39% for threshold-tuned XGBoost, and 25% for SMOTE-augmented random forest. A sympathetic reader would care because radio-loud quasars are the rare class that previous accuracy-driven classifiers almost entirely miss, so this is a concrete candidate baseline for completeness-oriented searches. The comparison is made under the paper's own labeling rule, $\\log_{10}(\\mathrm{radio\\,flux}) > 1$, and the gains come at the price of low precision (0.11) and overall accuracy (0.62).","feed_headline":"Balanced PCA regression finds 52% of rare radio-loud quasars","feed_subtitle":"A two-component PCA + balanced logistic regression finds more radio-loud quasars than XGBoost or SMOTE.","key_machinery":"The central object is the PCA-based balanced linear regression classifier (PBC): z-score the five SDSS optical magnitudes, project onto the first two principal components, and fit a logistic regression with class weights set to offset the 1:14 imbalance between radio-quiet and radio-loud quasars. The first component loads uniformly across bands (overall brightness, 87.6% of variance) and the second is dominated by the u-band (UV-sensitive spectral variation, 9.7%), together reaching 97.3% cumulative variance. The regression uses these two scores as a single linear decision boundary in a plane where the paper reports that the radio-loud and radio-quiet projections occupy different regions, letting the balanced weights push the boundary toward higher minority recall.","core_discovery":"On its own terms, the paper establishes a performance comparison on the SDSS-DR3 quasar catalog: using the first two principal components of the u, g, r, i, z magnitudes (together 97% of the variance) as features, a balanced logistic regression labels 52% of test radio-loud quasars correctly, against 4% for a random forest on the original five magnitudes, 39% for threshold-tuned XGBoost, and 25% for SMOTE-augmented random forest. The same model reaches an F1 of 0.19 for the minority class, sandwiched between XGBoost's 0.22 and the random forest's 0.08. The paper interprets this as evidence that balancing class weights in a regression on PCA components is an effective way to recover rare radio-loud quasars, at the price of precision (0.11) and overall accuracy (0.62).","pith_inferences":["Because the label is raw flux rather than the radio-to-optical ratio, part of the apparent separability may be distance or luminosity information; a fair test using $R = \\log_{10}(f_{\\rm radio}/f_{\\rm optical}) > 1$ would likely shrink the gap between PBC and the baselines.","The paper's own caveat that PCA maximizes variance, not class separation, suggests the two-component projection may discard the color information most tied to jet activity; projecting onto PCA of color indices rather than raw magnitudes is a direct testable extension.","The reported numbers rest on a single train/test split; a repeated stratified cross-validation with a leakage check on the SDSS-DR3 catalog would show whether the 0.52 recall is stable or a selection artifact.","The recipe—balance the weights, regress on two principal components—generalizes to any rare-object catalogue problem where completeness beats purity, such as high-redshift quasar hunting, though the paper only demonstrates it for radio loudness."],"forward_implications":["The reported recall of 0.52 means a survey that uses PBC as a pre-filter would catch more than half of the radio-loud quasars that a random forest misses, making it a candidate first stage for completeness-driven radio-loud quasar searches.","Because precision for the radio-loud class is only 0.11, the same pipeline would flood a follow-up sample with false positives; for purity-driven studies the random forest or XGBoost remains the better choice, a tradeoff the paper states explicitly.","The two-component representation (brightness plus UV-sensitive variation) explains 97% of the photometric variance, so the classification itself needs only two numbers per quasar, a large practical reduction in feature cost.","The significant redshift difference between the two classes (t-test p = 0.00003) means redshift carries some of the separation the classifiers use, so future work should test whether recall holds when redshift is controlled for.","The extreme imbalance (about one radio-loud for every 14 radio-quiet in the sample) is itself quantified by the comparison: every method, including PBC, sacrifices precision when it raises minority recall."],"supporting_citations":[{"why":"Supplies the SDSS-DR3 quasar catalog whose photometric and radio measurements form the dataset.","marker":"[26]"},{"why":"Cited as the source of the radio-loudness cutoff applied to the radio flux, so it fixes the class labels the whole comparison depends on.","marker":"[10]"},{"why":"Provides the PCA methodology and variance-explained framework used for dimensionality reduction.","marker":"[4]"},{"why":"Precedent for combining PCA with logistic regression on imbalanced data, which the paper adapts to the binary quasar case.","marker":"[8]"},{"why":"Defines the Random Forest baseline whose poor minority-class recall (0.04) motivates the balanced-regression approach.","marker":"[5]"},{"why":"Defines the XGBoost algorithm used as the threshold-tuned comparison model.","marker":"[19]"}],"fun_headline_variants":["PCA regression recalls 52% of radio-loud quasars","Balanced PCA beats XGBoost and SMOTE on rare quasar recall","Two PCA components lift quasar minority recall to 52%","PCA-based regression maximizes recall for radio-loud quasars"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that radio-loud and radio-quiet quasars are correctly separated by the cutoff $\\log_{10}(\\mathrm{radio\\,flux}) > 1$, applied to raw flux rather than the standard radio-to-optical ratio; if that labeling is incorrect, the reported recalls measure a different dichotomy.","fun_headline_variants_meta":{"raw":{"variants":["PCA regression recalls 52% of radio-loud quasars","Balanced PCA beats XGBoost and SMOTE on rare quasar recall","Two PCA components lift quasar minority recall to 52%","PCA-based regression maximizes recall for radio-loud quasars"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1447,"prompt_tokens":1104,"completion_tokens":343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":269}},"tokens_in":720,"tokens_out":343,"duration_ms":4090,"temperature":1.0,"reasoning_tokens":269,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:20:28.596714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the labels with the standard radio-loudness ratio $R = \\log_{10}(f_{\\rm radio}/f_{\\rm optical}) > 1$ on the same SDSS data, rerun the same train/test protocol, and see whether PBC still reaches roughly 0.52 minority recall; a drop toward the baselines would show the reported gain is an artifact of the flux-only label.","supporting_citations":[{"cited_title":"The Sloan Digital Sky Survey Quasar Catalog III. Third Data Release","cited_arxiv_id":"astro-ph/0503679","evidence_quote":"Supplies the SDSS-DR3 quasar catalog whose photometric and radio measurements form the dataset."},{"cited_title":"Multiclass Logistic Regression Classification with PCA for Imbalanced Medical Datasets,","cited_arxiv_id":null,"evidence_quote":"Precedent for combining PCA with logistic regression on imbalanced data, which the paper adapts to the binary quasar case."}],"review_version":1}