{"id":"dcc9df30-a58d-48bf-bde4-ba2c9c27e705","arxiv_id":"2502.05843","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SymbolicDet adds an LLM-guided evolutionary search over object detector outputs to produce interpretable rules for event recognition, reporting large AUROC gains across fishing, safety, and crowd benchmarks.","lead":"A training-free-looking framework called SymbolicDet uses an LLM to guide evolutionary search for logical rules over object detector outputs, turning detections into event predictions on fishing, safety, and crowd datasets. The paper reports large AUROC gains over raw detectors, but closer inspection shows the rules are fitted to labeled event data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training-free claim is contradicted by the paper's own supervised objective and per-event labeled datasets; the central claim is unsupported.","rationale":"The reader's verdict is CONDITIONAL, with a weakest assumption about the expressiveness of symbolic features. I read the paper in good faith and found an even more fundamental problem: the headline claim of 'training-free' operation is contradicted by the paper's own methodology. Equation (5) is a supervised risk minimization over labeled examples, and the experiments use large per-event labeled training sets. This is not a matter of consensus or framing; it is an internal inconsistency between the abstract and introduction on one hand and the method and experiments on the other. The strongest claim as quoted by the reader, 'without requiring additional training,' is thus not supported. If the authors intend 'training-free' to mean only that the object detector's weights are not updated, that is a legitimate but much weaker claim that must be stated explicitly, and the assertion 'requiring no additional labeled data beyond what the underlying detector was trained on' (Section 1) must be removed. The proposed concrete test, an audit of the code to verify label usage plus a label-withheld reproduction, would settle the issue. I agree with the reader's overall conditional recommendation, but I locate the load-bearing weakness in the training-free claim rather than in feature expressiveness, so agreement is partial. The verdict should remain CONDITIONAL, meaning acceptance should require the authors to reframe their contribution as a supervised symbolic classifier and provide the promised code and data so the label dependence can be verified.","tokens_in":16395,"tokens_out":8675,"duration_ms":86629,"concrete_test":"Audit the released code and data to confirm that the fitness function in Equation (5) receives ground-truth event labels from a training split for each dataset, such as the 15,000 Multi-rods images and 7,571 Helmet-Mac images; if it does, reproduce the Multi-rods pipeline with labels withheld from the search (e.g., using only LLM-generated or random rules) and compare AUROC, and a drop to chance would confirm that the method depends on labeled data, falsifying the training-free claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract, Section 1) is that SymbolicDet reveals complex events 'without requiring additional training' and with 'no additional labeled data beyond what the underlying detector was trained on.' This is internally inconsistent with the method's own optimization objective: Equation (5) minimizes a supervised loss over labeled image-event pairs, and Section 4.1.1 reports per-event labeled training sets of 15,000 images for Multi-rods Fishing and 7,571 images for Helmet-Mac. The evolutionary search is therefore a supervised classifier fit, not a training-free procedure. Since every reported result is obtained after fitting f* to event labels, the paper has not demonstrated that detector outputs alone suffice; it has demonstrated a supervised symbolic classifier. If 'training-free' is intended to mean only 'no detector fine-tuning,' the paper must state this explicitly and remove the claim of no additional labels. This is the load-bearing issue because the abstract's promise of plug-and-play generalization to new events without task-specific data is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SymbolicDet, a framework that uses an open-vocabulary object detector to extract object counts and simple relational features, then applies an LLM-guided evolutionary symbolic search to discover logical expressions that classify whether a target event is present in an image. The method is evaluated on event recognition benchmarks including ERA, Helmet-Mac, Multi-rods Fishing, UCSD Ped2, and USED, using APE, GLIP, and YOLO-World as detectors. The authors report AUROC improvements over the raw detectors and claim that the framework is training-free, requires no additional labeled data beyond the detector's pretraining, and provides interpretable symbolic rules.","tokens_in":16612,"tokens_out":3385,"duration_ms":37359,"significance":"The core idea — that pretrained open-vocabulary detectors plus interpretable symbolic search can produce event classifiers — is interesting and potentially useful. The paper's strengths include its detector-agnostic design, the cross-architecture experiments in Table 1, the interpretability of the discovered expressions, the introduction of a new dataset, and the stated intent to release code. However, the central 'training-free / no additional labels' claim is not supported by the method as written, because the symbolic expression is fit to event labels through a supervised objective. If the claims are reframed honestly as 'no detector fine-tuning' and the evaluation is made more rigorous, the contribution could be valuable; in its current form, the significance is contingent on that reframing.","major_comments":[{"comment":"The central claim that SymbolicDet is 'training-free' and requires 'no additional labeled data beyond what the underlying detector was trained on' (Abstract and Section 1) is contradicted by the method's own objective. Equation (5) explicitly minimizes sum_i L(f(X_i), y_i) over labeled image-event pairs, and Section 4.1.1 reports 15,000 training images for Multi-rods Fishing and 7,571 training images for Helmet-Mac. The evolutionary search is therefore a supervised fit of a symbolic classifier to event labels, not a training-free procedure. This is load-bearing because the abstract and introduction promise plug-and-play generalization to new events without task-specific supervision. The authors should either remove the 'training-free / no additional labels' claims or redefine them precisely as 'no detector fine-tuning' and adjust all downstream statements (including Findings 1 and 2 and Table 2 comparisons) accordingly.","section":"Section 3.1, Eq. (5); Section 4.1.1"},{"comment":"The evaluation protocol is under-specified for the public benchmarks. For ERA, the paper lists image counts for categories but does not state how the training and test splits are formed, how the symbolic search uses event labels, or whether the reported AUROC numbers are on the same images used for search. For USED, Table 3 reports a single score with no description of the subset, splits, or event definitions, and the supplementary Table 7 reports accuracy on only three subsets without a clear protocol. There are also no error bars or multiple-seed results anywhere in the paper. Without this information, the reported gains cannot be independently assessed or compared with prior work.","section":"Section 4.1.2, Section 4.2, Table 3"},{"comment":"The paper claims broad applicability to 'complex events' but the experiments do not cover temporal or motion-dependent events, and the feature representation in Eq. (4) and the discovered rules in Eq. (6) are limited to object counts and simple static relations. Section 4.1.1 explicitly lists 'personnel loitering and intrusion' as part of the Multi-Event Dataset, yet no loitering or intrusion results are reported. The absence of any temporal feature or evaluation on such events means the claim that the framework 'unlocks complex visual understanding' is demonstrated only for a narrow class of count- and co-occurrence-based events. The paper should state this limitation explicitly and temper the general claim, or add experiments on at least one temporal event.","section":"Section 4.1.1, Section 3.1"}],"minor_comments":[{"comment":"The first line of Figure 1 ('PHHWCSHGScaffoldingS >>V') appears to be garbled or corrupted; the symbolic pattern and its explanation should be rendered cleanly.","section":"Figure 1"},{"comment":"Equation (6) uses the symbol E inside the feature functions while the preceding notation uses X; it also leaves the index set {p,d} and the meaning of each feature function undefined in the main text. Please unify the notation.","section":"Section 3.1, Eq. (6)"},{"comment":"Equation (3) defines the objective with G_LLM and S, but the actual optimization in Eq. (5) does not include these terms. The relationship between the two formulations should be clarified.","section":"Section 3, Eq. (3)"},{"comment":"The Multi-Event Dataset description says the multi-rods scenario contains '12,000 images' and then states '15,000 training images' for the same scenario; this inconsistency should be resolved.","section":"Section 4.1.1"},{"comment":"The table header and row markers are confusing: the column labeled 'Lora Prompt Our' is not clearly separated, and the checkmarks are not defined in a legend. Please reformat so each configuration is identifiable.","section":"Table 2"},{"comment":"The table title says 'computational overhead of each part' but the table reports accuracy values; retitle it appropriately.","section":"Supplementary Table 7"},{"comment":"Reference [45] for ERA is cited as 'in press' without a year or venue details; please provide the full citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's novelty rests on combining LLM-guided symbolic search with open-vocabulary detectors, but the presentation overstates the training-free nature of the method. The use of APE [56], a detector from the same group, as the primary detector is not itself problematic, but the reader should be aware that the reported AUROC gains partly reflect supervised fitting of symbolic rules to event labels. I recommend requesting a revision that honestly reframes the claims, tightens the experimental protocol, and acknowledges the static-feature limitation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is sensible: take an open-vocabulary detector, extract object counts and simple spatial relations, and run an LLM-guided evolutionary search for short logical rules that classify images by event type (illegal fishing, helmet compliance, sports/crowd scenes). The symbolic rules are genuinely human-readable, the method works across three detector backbones, and the new Helmet-Mac dataset is a real resource for construction safety. The ablation shows LLM guidance speeds up convergence, which is a nice practical touch.\n\nThe soft spot is the one the stress-test flags, and it is load-bearing. The paper repeatedly calls the method \"training-free\" and says it needs \"no additional labeled data,\" but Equation (5) minimizes a loss over event labels y_i, and Section 4.1.1 describes per-event training sets of 15,000 and 7,571 images. The evolutionary search is a supervised classifier fit, not a zero-label procedure. The contribution should be reframed as \"no detector fine-tuning\" or \"interpretable symbolic classifier with modest labeled data.\" As written, the central promise of plug-and-play zero-shot event recognition is unsupported.\n\nOther soft spots are smaller. The evaluation lacks error bars, and the train/validation/test splits for ERA and USED are unclear; the paper should report them. The UCSD Ped2 comparison pits a single-frame method against video anomaly detectors that use temporal information, so the near-parity is actually a decent result, but the comparison needs a clearer scope statement. The supplementary search-scale experiments show performance rising with the fraction of labeled data used in the search, which reinforces the supervised nature of the method.\n\nCredit where due: the circularity concern is minor; this is fitting, not p-hacking or circular evaluation. The self-citation of APE is a mild concern, but they also evaluate GLIP and YOLO-World, so it is not disqualifying.\n\nThis paper deserves a serious referee. It needs a major revision for framing and evaluation detail, but the dataset and the LLM-guided symbolic search idea are worth publishing once the claims are scaled back. Treat it as a useful, moderately novel application paper, not a field-reorganizing result.","headline":"A useful interpretable symbolic layer on top of open-vocabulary detectors, but the paper's 'training-free' claim is contradicted by its own supervised objective and labeled per-event training sets.","tokens_in":17117,"tokens_out":2783,"would_cite":true,"duration_ms":28160,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A plug-and-play symbolic layer turns any open-vocabulary detector into an interpretable event recognizer without extra training.","keywords":["object detection","event recognition","symbolic regression","large language models","neuro-symbolic reasoning","training-free","interpretability","open-vocabulary detection"],"falsifier":"Take a surveillance event defined purely by motion, such as loitering or intrusion, run SymbolicDet on single frames with the same settings, and check AUROC. If no discovered expression over per-frame object counts and positions exceeds chance level, the claim that detectors' outputs carry enough information for event recognition fails for temporal events; equally, construct a pair of image sets with identical class-count distributions but opposite event labels to show the feature set cannot separate them.","tokens_in":16240,"feed_emoji":"🧩","tokens_out":3924,"duration_ms":36270,"temperature":0.7,"pith_summary":"This paper claims that a standard open-vocabulary object detector already contains enough information to recognize complex events, if its outputs are interpreted through short logical expressions. The authors propose SymbolicDet, a plug-and-play layer that searches for such expressions — combining evolutionary symbolic regression with LLM guidance — and applies them to detector outputs for event classification. The point is to get event understanding without task-specific training, with rules a human can read. If right, any off-the-shelf detector can be upgraded to an interpretable event recognizer at the cost of a symbolic search.","feed_headline":"Symbolic logic turns object detectors into event recognizers","feed_subtitle":"A plug-and-play LLM-guided search writes human-readable rules that lift detection to event understanding, no fine-tuning needed.","key_machinery":"The machinery is a symbolic expression space over arithmetic operators {+, −, ×, ÷, max, min} and logical operators {∧, ∨, ¬}, applied to structured features extracted from detector outputs — chiefly class counts and simple relational features. An evolutionary algorithm (symbolic regression) searches this space for the expression minimizing a fitness loss plus a complexity penalty, while an LLM, prompted with scene context, chain-of-thought, and feedback from past search scores, proposes candidate expressions that steer the population. The LLM's role is not perception but search guidance, keeping the final expression human-readable and the detector untouched.","core_discovery":"The central claim is that detectors are not just entity recognizers but visual sensors whose outputs, when run through a discovered logical rule over object counts and simple relational features, can separate images containing a target event from those that do not. The paper demonstrates this on illegal multi-rod fishing, construction safety helmet compliance, and crowd anomalies, where the discovered rules — e.g., a violation when the count of heads exceeds helmets in a person-and-workshop context — lift AUROC by 8 to 39 points across APE, GLIP, and YOLO-World, all without fine-tuning. On UCSD Ped2 the training-free rules reach 98.7% AUROC, close to specialized trained systems.","pith_inferences":["The same recipe should transfer to other symbolic targets, such as scene graph relations or action heuristics, wherever the event can be stated as a count or threshold condition over detected entities.","A natural test the paper does not run is temporal events such as loitering or intrusion; if a rule over per-frame detections fails there, the claim would narrow to spatial and static events rather than events in general.","The observation that LLM guidance mostly accelerates convergence rather than changing the final rule suggests the method could run with weaker LLMs or even keyword-based priors for low-resource deployments.","Because the rules are fit on a search set, the method may overfit to dataset-specific object-name vocabularies; robustness across different detector prompt phrasings remains to be checked."],"forward_implications":["Any open-vocabulary detector can be wrapped with SymbolicDet to recognize a new event from a handful of labeled images or none, because the rule search replaces fine-tuning.","Event decisions come with a symbolic explanation, such as 'head count greater than helmet count in a workshop', which makes compliance auditing and failure analysis straightforward.","Performance scales with the search budget and with the quality of the underlying detector, so future detector improvements transfer directly to event recognition.","On static-frame benchmarks like Helmet-Mac and multi-rod fishing, the training-free rules match or approach fine-tuned detectors, suggesting that much of event semantics is already latent in object class distributions."],"supporting_citations":[{"why":"Supplies the primary open-vocabulary detector (APE) whose outputs the symbolic rules operate on.","marker":"[56]"},{"why":"GLIP detector used to show architecture-agnostic transfer of the symbolic layer.","marker":"[30]"},{"why":"YOLO-World detector used to demonstrate the framework works across real-time open-vocabulary architectures.","marker":"[7]"},{"why":"ERA dataset provides the aerial event benchmarks (BALL, PersonCrowd, Sport) used for evaluation.","marker":"[45]"},{"why":"UCSD Ped2 benchmark is the comparison point against state-of-the-art anomaly detection methods.","marker":"[61]"},{"why":"USED dataset supplies the social event recognition evaluation (SPORT, CONCERT, PROTEST).","marker":"[1]"},{"why":"LoRA fine-tuning serves as the parameter-efficient baseline the training-free method is compared against.","marker":"[21]"}],"fun_headline_variants":["Object detectors learn events via LLM-guided symbolic rules","No-training event recognition from detector outputs","Symbolic rules plus LLM turn detectors into event sensors","Detectors see events: LLM-guided symbolic reasoning","From objects to events without fine-tuning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes the target event can be decided by a short logical expression over object counts and simple spatial features taken from a single static image; events that need motion, temporal context, or fine-grained attributes like which person holds which rod have no representable feature to exploit.","fun_headline_variants_meta":{"raw":{"variants":["Object detectors learn events via LLM-guided symbolic rules","No-training event recognition from detector outputs","Symbolic rules plus LLM turn detectors into event sensors","Detectors see events: LLM-guided symbolic reasoning","From objects to events without fine-tuning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1330,"prompt_tokens":945,"completion_tokens":385,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":324}},"tokens_in":561,"tokens_out":385,"duration_ms":4094,"temperature":1.0,"reasoning_tokens":324,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:42:40.368868+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a surveillance event defined purely by motion, such as loitering or intrusion, run SymbolicDet on single frames with the same settings, and check AUROC. If no discovered expression over per-frame object counts and positions exceeds chance level, the claim that detectors' outputs carry enough information for event recognition fails for temporal events; equally, construct a pair of image sets with identical class-count distributions but opposite event labels to show the feature set cannot separate them.","supporting_citations":[{"cited_title":"Aligning and prompting everything all at once for univer- sal visual perception","cited_arxiv_id":null,"evidence_quote":"Supplies the primary open-vocabulary detector (APE) whose outputs the symbolic rules operate on."},{"cited_title":"Grounded language-image pre-training","cited_arxiv_id":null,"evidence_quote":"GLIP detector used to show architecture-agnostic transfer of the symbolic layer."},{"cited_title":"Yolo-world: Real-time open-vocabulary object detection","cited_arxiv_id":null,"evidence_quote":"YOLO-World detector used to demonstrate the framework works across real-time open-vocabulary architectures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ERA dataset provides the aerial event benchmarks (BALL, PersonCrowd, Sport) used for evaluation."},{"cited_title":"Anomaly detection in crowd scene","cited_arxiv_id":null,"evidence_quote":"UCSD Ped2 benchmark is the comparison point against state-of-the-art anomaly detection methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"USED dataset supplies the social event recognition evaluation (SPORT, CONCERT, PROTEST)."}],"review_version":1}