{"id":"cea96b2c-cda3-4b00-83b9-b740413db350","arxiv_id":"2506.04241","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A probabilistic logic layer over interpretable concepts improves out-of-distribution detection when multiplied into existing detector scores.","lead":"This paper combines Markov logic networks with existing out-of-distribution detectors, using logical rules over human-understandable concepts like color and shape to flag unusual inputs. The combined method improves detection on traffic sign and face attribute benchmarks, and the learned rules provide a human-readable explanation for each flag.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) drops the MLN partition function before multiplication by pD, which changes combined-score rankings; the claimed invariance is false.","rationale":"The reader's weakest assumption concerns concept classifier accuracy on OOD data; that is a plausible empirical limitation, but the most load-bearing issue is internal: the combination rule in Eq. (8) is not invariant to the omitted partition function, contrary to the paper's explicit claim. Because pD(D(x)) varies across inputs, adding the constant log Z before multiplication changes the score by an input-dependent amount, which can alter the detector's ranking. This is a mathematical error in the derivation of the central method, not merely a missing experiment. The empirical results may still stand as a heuristic, but the paper's framing as probabilistic MLN reasoning is unsupported. Since the issue is fixable by either including Z (at additional computational cost) or explicitly redefining the score as a heuristic and removing the invariance claim, the appropriate verdict remains conditional acceptance pending revision. I therefore do not change the reader's CONDITIONAL verdict, but I identify a different and more concrete correctness risk than the one emphasized in the reader's weakest_assumption.","tokens_in":15156,"tokens_out":9611,"duration_ms":108831,"concrete_test":"Recompute the combined scores using D*_M(x) = (DM(x) + log Z) * pD(D(x)) for the GTSRB and CelebA experiments, where Z is the exact partition function obtained by enumerating the finite semantic space (for CelebA, 2^14 possible worlds; for GTSRB, the product of class, color, and shape categories). Then reevaluate AUROC and FPR95 for MLN+Ensemble and MLN+Mahalanobis in Tables 1 and 3. If any AUROC shifts by more than a small epsilon or the ranking of any pair flips relative to the reported unnormalized scores, the omitted partition function is load-bearing. Also verify the specific two-input counterexample above using the learned weights from the published code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines the combined score as D'_M(x) = DM(x) * pD(D(x)), with DM(x) = -Σ_i w_i φ_i(x) from Eq. (6). The paper claims the partition function can be omitted without affecting the overall ranking of outlier scores because it is a constant. This is correct for the standalone MLN ranking, but not for the product. Writing the properly normalized negative log-probability as -log PM(x) = DM(x) + log Z, the combined score would be (DM(x) + log Z) * pD(D(x)). Since pD varies across inputs by design (near 1 for ID, near 0 for OOD), the term log Z * pD(x) is input-dependent, so rankings under the normalized and unnormalized products can reverse. For example, with log Z = 10, an ID input with DM = -2, pD = 0.9 and an OOD input with DM = -5, pD = 0.1 yield D'_M scores -1.8 vs -0.5 (OOD higher) but normalized scores 7.2 vs 0.5 (ID higher). Thus the reported combined detector is not derived from the MLN posterior; performance depends on the arbitrary omission of the partition function. This is a concrete error in the central fusion formula and undermines the probabilistic-reasoning interpretation of the experiments.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes augmenting existing out-of-distribution (OOD) detectors with a Markov logic network (MLN) defined over human-interpretable concepts. A DNN-based interpretation maps each input to a semantic vector; the MLN assigns an outlier score as the negative weighted sum of violated logical constraints. This MLN score is then combined with a baseline detector by multiplying it with the survival function of the baseline score on in-distribution data. The authors also introduce a greedy constraint-search algorithm for automatically learning such logical constraints from data. Experiments on GTSRB and CelebA, with multiple detectors, backbones, ten random seeds, and several ablations, report consistent AUROC/FPR95 improvements over the baselines and moderate computational overhead.","tokens_in":15472,"tokens_out":7488,"duration_ms":81605,"significance":"If the claims hold, the paper makes a useful neuro-symbolic contribution to OOD detection by showing that semantic constraints over human-understandable concepts can complement purely representation-based detectors. The empirical study is extensive: ten seed replicates, multiple detectors and backbones, an open-source implementation, runtime measurements, and ablations covering constraint count, normalization family, parameter sharing, and the influence of the OOD source used during constraint search. The explicit discussion of potential bias in learned constraints is also a strength. However, the probabilistic justification of the fusion rule contains a mathematical error, and the reliability of the concept classifiers on OOD data is not assessed; these issues need to be resolved before the contribution can be fully credited.","major_comments":[{"comment":"The statement that omitting the partition function Z does not affect the overall ranking of outlier scores is incorrect for the product rule. Since pD(D(x)) varies across inputs, adding log Z to DM(x) before multiplication can change the ranking. For example, with log Z = 10, an ID input with DM = -2 and pD = 0.9 and an OOD input with DM = 2 and pD = 0.1 yield unnormalized product scores -1.8 and 0.2 (OOD ranked higher), while the normalized product scores are 7.2 and 1.2 (ID ranked higher). The omission of Z is benign for the standalone MLN ranking in Eq. (6), but not for the combined score in Eq. (8). The authors should either define the combination using the properly normalized MLN probability or explicitly present Eq. (8) as a heuristic and remove the invariance claim.","section":"Section 3.2, Eq. (8)"},{"comment":"The DNNs that serve as the interpretation I for the logical predicates and functions are trained only on in-distribution data, but the paper does not measure or report their accuracy or calibration on OOD inputs. If these classifiers produce semantically meaningless labels for out-of-distribution images, the constraint violations detected by the MLN reflect classifier artifacts rather than genuine semantic anomalies, which would undermine the paper's explainability and 'semantic OOD' claims. This is a load-bearing gap because the method's mechanism depends on the semantic labels being meaningful for both ID and OOD inputs. Please report per-attribute accuracy or calibration on OOD test data, or provide systematic qualitative evidence beyond the few examples in Fig. 7, and discuss how interpretation errors propagate to the final score.","section":"Section 3, 'Connecting DNNs and FOL Semantics'; Section 5"},{"comment":"The prose and the pseudocode for the greedy constraint search are not fully aligned. The text says a candidate is drawn from the 'remaining pool' and that the search requires only |T| evaluations, but Algorithm 2 iterates over all constraints in T without any removal of accepted constraints, and each loop calls 'Train detector with φ′ on Dtrain' without specifying how the MLN weights are set (e.g., optimized via Eq. (11) or simply kept at the initial value -1). The reader cannot tell whether the detector is retrained from scratch at each step, whether accepted constraints are excluded from later consideration, or what the actual computational cost is. This ambiguity affects the reproducibility of a central contribution and should be clarified by aligning the text with a precise search procedure.","section":"Section 4, Algorithm 2"}],"minor_comments":[{"comment":"The table reports only point estimates without standard errors, despite the paper's emphasis on statistical replication; adding standard deviations or error bars would make the comparisons more informative.","section":"Table 3"},{"comment":"The |φ| axis in Figure 4 appears garbled in the submitted version, with repeated digits that make the number of identified constraints impossible to read; the figure should be regenerated with a standard axis.","section":"Figure 4"},{"comment":"The survival function pD is defined as P(D(X) ≥ D(x)) over ID samples; please clarify whether this is estimated empirically on the validation set and how ties between scores are handled.","section":"Eq. (7)"},{"comment":"The runtime comparison would benefit from error bars or a confidence band, since the text states that inference time is averaged over 100 batches but no variability information is provided.","section":"Section 5.4, 'Computational Overhead'"},{"comment":"The paper notes that weights can be negative, which would make a constraint violation decrease the score; however, the interpretation in terms of 'violation increases the score' is used throughout the text. It would be helpful to report whether the learned weights were always positive in practice and, if not, how negative weights should be interpreted.","section":"Section 3.1, Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution, but the fusion-rule issue and the unvalidated concept classifiers on OOD data need to be addressed before publication. I see no grounds for rejection, as these issues appear fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Real contribution, with a real flaw in the fusion math that needs fixing, but the empirical core is solid enough to referee.\n\nWhat's new: first application of MLNs to OOD detection, plus a greedy constraint-learning algorithm. The experiments are unusually thorough for this subfield: ten seeds, multiple detectors, backbones, ablations, and an ablation on OOD validation set sensitivity. The gains, especially on CelebA where the MLN alone jumps from ~51 AUROC for Logic to 84, are credible. The code is public. Credit where due: the constraint search is simple and seems to work, and the paper is honest about the sensitivity of discovered constraints to the OOD validation source.\n\nSoft spots, in proportion. The stress-test note is correct. Eq. (8) multiplies the unnormalized MLN score D_M by p_D. The partition function is not a constant factor in this product; it's an additive term in log-probability, and multiplying the constant log Z by p_D makes it input-dependent. Rankings can reverse. So the combined detector is not a principled combination of the MLN posterior and the base detector; it's a heuristic that happens to work. That's a real overclaim in Section 3.2 and needs to be fixed, either by using the normalized score or by re-framing the fusion as heuristic. The empirical results don't collapse because of this, but the theoretical justification does.\n\nOther issues are minor: several tables lack error bars (Tabs 2–5), the text and Alg. 2 disagree about whether weights are retrained per candidate, and the concept classifiers are never validated on OOD inputs, which matters because the whole semantic signal depends on them. None of these are load-bearing.\n\nBottom line: the empirical case is strong enough that the paper deserves a serious referee. The fusion error is fixable with an honest rewrite. If I were handling it, I'd send it out with a request to address the partition-function point.","headline":"A genuinely useful neuro-symbolic OOD method with a real but fixable error in the fusion formula; deserves reviewing.","tokens_in":15954,"tokens_out":2037,"would_cite":true,"duration_ms":20514,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that adding a Markov logic network that scores inputs against weighted logical constraints over human-understandable concepts improves out-of-distribution detection across a range of existing detectors, backbones, and…","keywords":["out-of-distribution detection","Markov logic networks","neuro-symbolic reasoning","logical constraints","explainability","constraint learning","semantic concepts","AUROC"],"falsifier":"Run the GTSRB or CelebA concept classifiers on OOD images and measure their accuracy against human-provided semantic labels; if accuracy on OOD data is near chance while the MLN still improves detection, the claimed semantic signal is not doing the work. Alternatively, randomly permute the MLN's concept predictions during inference on OOD data while keeping the base detector fixed: if the combined AUROC stays the same, the semantic layer is irrelevant.","tokens_in":14965,"feed_emoji":"🧠","tokens_out":2929,"duration_ms":35544,"temperature":0.7,"pith_summary":"This paper proposes a neuro-symbolic layer for out-of-distribution (OOD) detection: a Markov logic network (MLN) reasons over semantic concepts extracted by neural networks, such as color, shape, age, or gender, and assigns each input a plausibility score based on weighted logical constraints. The authors argue that this semantic signal is complementary to the statistical pattern-based signals used by existing OOD detectors, and that multiplying the two scores yields a strictly better detector. They demonstrate the claim on traffic-sign and face-attribute benchmarks, showing that the combined detector beats both the base detector and the MLN alone, and that a supervised variant with auxiliary outliers performs best. They also introduce a greedy algorithm that automatically learns useful constraints from data, and show that the approach generalizes across detectors, backbones, and OOD datasets.","feed_headline":"Probabilistic logic sharpens OOD detection","feed_subtitle":"Adding Markov-logic constraints over image concepts lifts detection accuracy across detectors and datasets.","key_machinery":"The core mechanism is the MLN outlier score $D_M(x) = -\\sum_i w_i \\varphi_i(x)$, which sums the weights of all constraints satisfied by the input's semantic representation and negates the sum; this ranking is identical to using the full MLN probability because the partition function is constant across inputs. This score is combined with a base OOD detector by first converting the base score into a survival function $p_D(D(x)) = P(D(X) \\ge D(x))$, estimated on in-distribution data with a generalized extreme value distribution, and then multiplying: $D'_M(x) = D_M(x) \\times p_D(D(x))$. A greedy constraint-search algorithm (Alg. 2) builds a set of logical formulas by adding candidates that improve validation AUROC by at least a threshold $\\delta_{\\min}$, trading rule-set complexity against detection performance.","core_discovery":"The central claim is that an MLN-based semantic score, defined as the negative weighted sum of violated logical constraints over a low-dimensional, human-interpretable semantic representation, can be combined with any existing OOD detector to improve detection. The combination works by normalizing the base detector's scores with an estimated survival function and multiplying the normalized value by the MLN score. Experiments on GTSRB and CelebA show that the combined detector outperforms both the base detector and the standalone MLN, with AUROC gains on nearly every tested detector (for example, MLN+Mahalanobis on CelebA raises AUROC from 95.12 to 96.01 and cuts FPR95 by roughly 20 percent), and that a supervised variant that learns an in-distribution predicate yields further gains. The paper also claims that a greedy search over candidate constraints, using AUROC on a validation set as the selection criterion, discovers interpretable and effective rules, and that the MLN's constraint-based scores provide explainability because each violation contributes a traceable, weighted penalty.","pith_inferences":["A natural extension would be to test whether the MLN layer helps most when the base detector is weak on semantically-grounded anomalies (e.g., a blue stop sign that still yields high softmax confidence); the paper's CelebA results suggest that detectors with poor baseline AUROC benefit the most, but this pattern is not stated explicitly.","The failure case with Gaussian noise as the OOD source during constraint search hints that the semantic layer may add little when OOD inputs are completely uninterpretable; a reader could investigate whether the MLN is only useful when concept classifiers produce meaningful outputs on OOD data.","The authors do not evaluate concept-classifier accuracy on OOD inputs; a testable prediction of the paper's own logic is that the MLN's contribution should vanish or become negative when the concept classifiers are systematically wrong on OOD data, because the constraint violations would then reflect classifier artifacts rather than genuine semantic anomalies.","One could extend the framework to temporal or relational data by moving from unary predicates to multi-object predicates, which the paper explicitly leaves out, and test whether the same survival-function combination still holds."],"forward_implications":["If the central claim holds, any existing OOD detector can be upgraded by appending an MLN layer, and the upgrade yields consistent AUROC and FPR95 improvements without retraining the base detector.","The semantic constraints provide a built-in explanation mechanism: when an input is flagged OOD, the specific violated rules and their weights tell a human which concepts were implausible.","The greedy constraint search means that prior knowledge is not strictly required; useful, human-readable rules can be mined directly from a dataset with ID and OOD examples.","The approach's modularity means that supervised outlier exposure can be added simply by learning an extra predicate, without altering the rest of the detection pipeline.","Because the MLN score is cheap to evaluate (no partition function), the combined detector keeps inference overhead small even for large batch sizes."],"supporting_citations":[{"why":"Defines Markov logic networks, the probabilistic logical formalism that the proposed method is built on.","marker":"(Richardson & Domingos, 2006)"},{"why":"Presents LogicOOD, the strict logical-reasoning baseline that this work softens with weighted constraints.","marker":"(Kirchheim et al., 2024)"},{"why":"Provides the MSP baseline detector and the standard formulation of OOD detection as a score thresholding problem.","marker":"(Hendrycks & Gimpel, 2017)"},{"why":"Introduces the Mahalanobis latent-representation detector that is one of the main base detectors combined with the MLN.","marker":"(Lee et al., 2018)"},{"why":"Provides the deep-ensemble detector that serves as the strongest pattern-based baseline and a primary combination target.","marker":"(Lakshminarayanan et al., 2017)"},{"why":"Introduces ViM, a virtual-logit matching detector used in the ablation and normalization experiments.","marker":"(Wang et al., 2022)"}],"fun_headline_variants":["Logic constraints sharpen OOD detection","Markov logic boosts outlier detection","Semantic rules improve OOD detectors","MLN scores enhance detection accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The neural networks used to interpret predicates and functions (such as color, shape, age, and gender) provide accurate semantic labels for both in-distribution and out-of-distribution inputs; the paper does not measure concept-classifier accuracy on OOD data, so if those classifiers fail on OOD inputs, the MLN's constraint violations would reflect model artifacts instead of genuine semantic anomalies.","fun_headline_variants_meta":{"raw":{"variants":["Logic constraints sharpen OOD detection","Markov logic boosts outlier detection","Semantic rules improve OOD detectors","MLN scores enhance detection accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1322,"prompt_tokens":872,"completion_tokens":450,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":402}},"tokens_in":488,"tokens_out":450,"duration_ms":5195,"temperature":1.0,"reasoning_tokens":402,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:11:19.835948+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the GTSRB or CelebA concept classifiers on OOD images and measure their accuracy against human-provided semantic labels; if accuracy on OOD data is near chance while the MLN still improves detection, the claimed semantic signal is not doing the work. Alternatively, randomly permute the MLN's concept predictions during inference on OOD data while keeping the base detector fixed: if the combined AUROC stays the same, the semantic layer is irrelevant.","supporting_citations":[{"cited_title":"and Domingos, P","cited_arxiv_id":null,"evidence_quote":"Defines Markov logic networks, the probabilistic logical formalism that the proposed method is built on."},{"cited_title":"Out-of-distribution detection with logical reasoning","cited_arxiv_id":null,"evidence_quote":"Presents LogicOOD, the strict logical-reasoning baseline that this work softens with weighted constraints."},{"cited_title":"and Gimpel, K","cited_arxiv_id":null,"evidence_quote":"Provides the MSP baseline detector and the standard formulation of OOD detection as a score thresholding problem."},{"cited_title":"A simple unified framework for detecting out-of-distribution samples and adversarial attacks","cited_arxiv_id":null,"evidence_quote":"Introduces the Mahalanobis latent-representation detector that is one of the main base detectors combined with the MLN."},{"cited_title":"Vim: Out-of-distribution with virtual-logit matching","cited_arxiv_id":null,"evidence_quote":"Introduces ViM, a virtual-logit matching detector used in the ablation and normalization experiments."}],"review_version":1}