Pith. sign in

REVIEW 4 major objections 4 minor 75 references

SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SymDiag reframes LLM reasoning verification as structured failure diagnosis, translating chains-of-thought into symbolic constraints to localize failing steps and produce verifiable evidence.

desk verdict SymDiag's detection pipeline is worth a look, but the repair gains are likely inflated because the counterexample feedback can hand over the gold answer, and the evaluation lacks error bars and public artifacts. read the letter →

arxiv 2608.08786 v1 pith:KXKUCWF2 submitted 2026-08-09 cs.AI

Wenyao Cui , Huaping Zhang , Yongyi Huang , Qiuchi Li , Jian Xu , Cheng-Lin Liu , Chunxiao Gao , Juan Wang
show 1 more author
Baohua Zhang
This is my paper · ORCID
classification cs.AI
keywords neuro-symbolicverificationchain-of-thoughtfaithfulnessreasoningdiagnosisself-auditingPrologexplainableAILLMrepairstep-levelentailment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that verifying an LLM's chain-of-thought should be a diagnosis, not a score: translate each reasoning step into symbolic constraints, run satisfiability and entailment checks, and report exactly which step fails and why. It introduces SymDiag, which compiles natural-language steps into two independent Prolog encodings and uses a Self-Auditor to separate genuine reasoning defects from translation noise. On a manually audited set of 240 reasoning traces spanning mathematics, logic, science, and general reasoning, SymDiag detects unfaithful chains-of-thought with an F1 of 70.7, and its evidence-grounded feedback produces faster and larger repair gains than answer matching, LLM-as-judge, a reward model, and a theorem-prover-based logic reward baseline. If true, the contribution is a feedback signal that is checkable rather than subjective: counterexamples, inconsistency witnesses, and missing-premise flags that a model can act on across domains.

What carries the argument

The load-bearing mechanism is dual symbolic encoding plus step-level satisfiability and entailment verification. Two branches produce independent Prolog programs, a formal translation and a critical restatement, and the Self-Auditor compares fact sets, constraints, and entailment targets to decide whether an apparent violation is translation noise or a genuine reasoning defect. The verifier then tests $SAT(P_i\land C_i)$ and $UNSAT(P_{i-1}\land C_{i-1}\land\neg\varphi_i)$; the second check yields a counterexample when it succeeds. The faithfulness decision is conservative: a trace is faithful only if all steps pass in at least one approved branch.

What would settle it

A concrete test is to feed SymDiag traces that are correct-but-unfaithful with the same flawed step translated consistently in both branches; if the Self-Auditor passes them as faithful or localizes the wrong step relative to human annotations, the diagnosis is not tracking the actual reasoning.

Watch

Extended reading notes

Core claim

SymDiag's central claim is that reasoning verification can be reframed as structured failure diagnosis. The system represents each chain-of-thought step as a state $S_i=\{P_i,I_i,C_i\}$ of premises, intended inference, and constraints, and it verifies each step by checking satisfiability of $P_i\land C_i$ and entailment of the step's claim $\varphi_i$ from the prior state, generating counterexample assignments whenever the entailment fails. Failures are labeled with a domain-agnostic taxonomy (premise omission, invalid inference, constraint neglect, rule misuse, arithmetic error, type mismatch) and attributed to TranslationError versus ReasoningError by a Self-Auditor that compares a formal translation branch against a critical restatement branch. The paper reports that the full system reaches 70.7 overall F1 for faithfulness detection on its manually audited gold set and outperforms four baselines in multi-round repair, with the largest ablation drop when step-level symbolic verification is removed.

Load-bearing premise

The whole diagnosis stands on the assumption that the two independently generated symbolic encodings faithfully represent the actual reasoning of the chain-of-thought, and that the Self-Auditor's consistency check can reliably tell a translation slip from a real reasoning mistake.

Editorial extensions

If this is right

  • Unfaithful-but-correct chains of thought become detectable: on the paper's gold set, 25% of instances are correct yet unfaithful, and SymDiag catches them rather than letting the final answer mask the flaw.
  • Feedback becomes actionable: instead of a scalar reward or a subjective critique, a repair model receives counterexample assignments, inconsistency witnesses, and missing-premise indicators, which explains the reported faster and larger multi-round accuracy gains.
  • The error taxonomy gives a common vocabulary across mathematics, logic, science, and general reasoning, and the paper's analysis shows systematic failure-mode shifts with model scale: small models err on arithmetic and premise omission, larger models on rule hallucination and type mismatch.
  • The Self-Auditor makes the pipeline more reliable in practice: iterative auditing drops translation errors from 20.1% to near zero by round 3 and raises the step pass rate from 53.4% to 81.4%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same dual-encoding consistency check could be used as a lightweight faithfulness probe without a solver: two independent formalizations that disagree flag ambiguity even before satisfiability checking is run.
  • If the symbolic encodings are faithful, the localized counterexamples could be recycled as weak supervision for training diagnosis-aware reward models, which the paper names as future work rather than demonstrating.
  • The Prolog backend avoids domain libraries, but for mathematics the same step-level entailment checks could be ported to stronger provers or SMT solvers; whether the gain survives that port is not tested here.
  • The headline number is computed on a 240-instance human-audited set; scaling the diagnostic benchmark to a larger corpus would reveal whether the 70.7 F1 advantage persists across harder or noisier traces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SymDiag, a neuro-symbolic pipeline that compiles LLM chains-of-thought into two independent Prolog programs, performs step-level satisfiability/entailment checks, attributes failures to TranslationError versus ReasoningError via a Self-Auditor, and converts localized diagnoses into feedback for iterative repair. Experiments on a manually audited 240-instance set spanning mathematics, logic, science, and general reasoning report F1 70.7 for faithfulness detection, ablations showing that all components contribute, and repair curves claiming faster and larger gains than answer matching, LLM-as-Judge, reward model, and LogicReward baselines.

Significance. If the detection results are robust, the diagnosis-level framing is a useful step beyond scalar or outcome-only verification: SymDiag produces checkable evidence such as counterexamples and localizes failing steps, and the dual-branch symbolic compilation with a Self-Auditor is a sensible response to translation noise. The manually audited gold set, the cross-domain benchmark construction, and the ablation design are strengths. However, the repair-effectiveness claim is not currently established because the feedback shown in the running example appears to leak the gold answer, and the error attribution is validated only internally against the system's own consistency heuristic. With proper controls and statistical rigor, the central contribution would be solid and publishable.

major comments (4)
  1. [§3.3; Figure 2; Figure 4] The repair-effectiveness claim (Q2) is confounded by answer leakage. In the running example, the problem asks for the least positive integer m such that m^4+1 is divisible by 17^2, and the SymDiag feedback shown in Figure 2 states "Counterexample found: m = 110 satisfies...", where 110 is exactly the gold answer in the repaired trace. For any minimization or maximization problem, a witness that beats the proposed solution directly reveals the final answer, so the faster repair curves in Figure 4 may reflect the symbolic solver having found the gold answer rather than the value of step-level localization or TranslationError/ReasoningError attribution. The paper never reports how often the produced evidence entails the gold answer, nor does it compare against a baseline that receives the same witness without the diagnostic wrapper. This control is necessary before the "substantially more effective feedback" claim can be accepted.
  2. [§3.2.4; §3.2.5; Figure 5] The Self-Auditor's TranslationError/ReasoningError attribution is validated only internally. Section 3.2.4 defines TranslationError as failures that disappear under minimal canonical rewrites, and Section 3.2.5 then verifies only "Approved" states; there is no independent gold standard for error labels and no human or external audit of attribution accuracy. The ablation in Figure 5 measures only trace-level F1, so it cannot tell whether the Self-Auditor correctly separates translation artifacts from genuine defects or merely discards cases where the two branches happen to differ. In particular, if both branches contain the same translation error, cross-branch consistency will not detect it, and the step-level entailment checks in Section 3.2.5 would then verify the generated program rather than the original reasoning. The paper should report attribution-level agreement against human judgments and quantify the shared-error scenario.
  3. [§5.1; Table 1; Figure 4] All headline comparisons are point estimates without confidence intervals or significance tests. Table 1 reports F1 per dataset on n=30 instances and an overall F1 of 70.7 on 240 instances, yet the text repeatedly uses "significantly more effective" (Section 5.1) and "consistently achieves faster and larger gains" (Section 5.2). Figure 4 plots accuracy curves with no error bars or repeated-seed variation. Given the small gold set and the modest absolute gaps on several datasets, permutation tests or bootstrap confidence intervals are needed to establish that the observed differences are not noise.
  4. [§3.4] The construction of the gold evaluation set is underspecified. Section 3.4 states that 240 instances were "manually verified" and that correct-answer instances were filtered by "conservative multi-judge voting", but it does not report the number of annotators, the annotation instructions, inter-annotator agreement, or the adjudication procedure. Since the faithfulness-detection result in Section 5.1 rests entirely on these labels, the absence of audit metadata makes the stated F1 difficult to interpret or reproduce. The authors should report label statistics and agreement, and ideally make the annotation protocol available.
minor comments (4)
  1. [Abstract; §4.1.2] There are typographical spacing errors in the abstract ("proposeSymDiag") and in Section 4.1.2 ("GPTOSS-120Bas the base model"); these should be corrected.
  2. [Figure 4] The y-axis ranges in Figure 4 are truncated (e.g., AIME24 from 73.5 to 76.5, MMLU from 87.5 to 90.5), which visually exaggerates the size of the repair gains; the axes should be shown from a common baseline or with full ranges.
  3. [§3.2.4] The notion of a "minimal canonical rewrite" is not formally defined; specifying the allowed rewrite set would make the TranslationError criterion reproducible.
  4. [Figure 2] The repaired CoT panel in Figure 2 contains a stray "OR" after Step 4, and the transition from diagnosis feedback to the repaired trace is not clearly separated; the figure should be cleaned up.

Circularity Check

1 steps flagged · score 6.0 of 10

Repair-gain claim is partially circular: SymDiag's counterexample feedback can contain the gold answer (Fig. 2's m=110), so faster repair may reflect answer leakage; faithfulness detection is independently benchmarked and not circular.

  1. other [Section 3.3 (Figure 2) and Section 4.1.5 / Figure 4 (repair evaluation)]
    "Find the least positive integer $m$ such that $m^{4}+1$ is divisible by $p^{2}$. ... **SymDiag Feedback** CRITIQUE_FAIL detected by symbolic execution. Counterexample found: **m = 110** satisfies ( m^4 + 1 \equiv 0 \pmod{17} )."

    For any 'find the least/smallest' problem, a smaller valid witness returned by the solver is the target answer itself. In Figure 2, adversarial_test_2 enumerates M upward from 1, so the first returned witness (110) is the minimal valid m, i.e., the gold answer used to score repair accuracy. The feedback embeds this value verbatim, so 'repair' can succeed by copying the witness from the feedback; the measured gain in Figure 4 is then forced by the information content of the evidence rather than by the diagnostic localization/attribution that Q2 claims to evaluate. No baseline is given the same solver-produced witness without the diagnostic wrapper, so the repair comparison cannot separate diagnostic quality from answer leakage on such instances.

full rationale

The faithfulness-detection claim (Q1) is not circular: the 240-instance gold set is manually audited (Section 3.4), and SymDiag's verdicts come from satisfiability/entailment checks (Section 3.2.5), not from fitting to those labels. The TranslationError/ReasoningError attribution is operationally defined by the Self-Auditor's consistency and rewrite tests (Section 3.2.4); that is a heuristic limitation rather than a circular derivation. The repair-claim (Q2), however, has a concrete by-construction reduction: the solver's 'counterexample' for extrema problems can equal the gold answer, and repair accuracy is scored on that same answer. The paper's own running example (Figure 2) exhibits this exactly: the feedback contains m=110, the least valid m and the gold answer. Since no control baseline receives the same witness without the diagnostic framing, the central repair-gain result is partially circular, warranting score 6. There is no load-bearing self-citation, imported uniqueness, or ansatz-smuggling via citation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim does not rely on numeric fitted parameters. It rests on four soft assumptions: faithful LLM-to-Prolog translation, consistency-as-translation-fidelity, Prolog validity as a proxy for reasoning validity, and correctness of the gold labels. The first two are the most fragile because they can in principle be wrong in both branches simultaneously. The paper provides no independent measurement of translation accuracy, so these assumptions are the main source of correctness risk.

assumptions (4)
  • domain assumption Natural-language CoT steps can be faithfully compiled into Prolog Horn-clause constraints by an LLM.
    Section 3.2.2 assumes a two-branch generator produces accurate symbolic encodings; if translation misses or distorts the intended meaning, step-level checks verify the translation, not the reasoning.
  • ad hoc to paper Cross-branch consistency of two LLM-generated encodings is a reliable indicator that an apparent violation is not a translation artifact.
    Section 3.2.4 defines the Self-Auditor using consistency between two neural branches, with no external ground truth for translation correctness.
  • domain assumption Step-level satisfiability and entailment in Prolog capture human-relevant logical validity of chains of thought.
    Section 3.2.5 uses SWI-Prolog as the backend, which can only check formalized constraints; the mapping from formal validity to natural-language reasoning validity is assumed.
  • domain assumption Gold labels on the 240-instance set (Faithful or Unfaithful) are correct.
    Section 3.4 describes labels built from LLM sampling, conservative multi-judge voting, and manual audit, but the annotation protocol and inter-annotator agreement are not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification." pith.science (2026). https://pith.science/paper/KXKUCWF2

@misc{pith2026260808786,
  author       = {Pith},
  title        = {Pith review of: SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXKUCWF2}},
  note         = {Machine review of arXiv:2608.08786}
}
read the original abstract

Large language models (LLMs) increasingly serve as data-driven reasoners, yet their chains-of-thought (CoT) can be unfaithful even when final answers are correct. Most existing ``verification'' signals are not diagnostic: answer matching observes only the outcome, LLM-as-judge provides subjective and non-verifiable critiques, and scalar rewards (e.g., PRMs/RMs) offer little insight into where a multi-step derivation fails.We propose \textbf{SymDiag}, a neuro-symbolic framework that \textbf{reframes reasoning verification as structured failure diagnosis}. SymDiag translates natural-language CoT into symbolic constraints and performs step-level satisfiability/entailment checks to (i) localize failing steps and (ii) produce verifiable diagnostic evidence, including counterexamples, inconsistency witnesses, and missing-premise indicators. A central challenge is that apparent ``logic violations'' can be caused either by genuine reasoning defects or by neural-to-symbolic translation noise. SymDiag therefore incorporates a Self-Auditor that disentangles TranslationError from ReasoningError via dual symbolic encodings consistency checks, enabling robust diagnosis under partial observability. Across diverse mathematical, logical, scientific, and general reasoning benchmarks, SymDiag improves detection of unfaithful reasoning and provides substantially more effective feedback for multi-round reasoning repair than outcome-only verification and LLM-based judging, offering a principled foundation for trustworthy and scalable reasoning diagnosis.

Figures

Figures reproduced from arXiv: 2608.08786 by the authors.

Figure 1
Figure 1. Paradigms for evaluating chain-of-thought (CoT) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SymDiag overview. Stage I (Diagnosis): a neuro-symbolic generator produces (i) a formal translation and (ii) a critical [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Core experimental dataset composition. We manu [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Diagnosis-guided reasoning repair curves across datasets. Each subplot reports task accuracy after each repair round [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Ablation results on overall faithfulness detection [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: Progressive reduction of error types in the SymDiag [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Effect of base model scale on overall faithfulness [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

75 extracted references · 50 canonical work pages

  1. [1]

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925(2025)

    arXiv 2025

  2. [2]

    Chengkun Cai, Haoliang Liu, Xu Zhao, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, John Lee, Jenq-Neng Hwang, and Lei Li. 2025. Bayesian Optimization for Controlled Image Editing via LLMs. InProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)

    work page 2025

  3. [3]

    Chengkun Cai, Xu Zhao, Haoliang Liu, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, Jenq-Neng Hwang, and Lei Li. 2025. The Role of Deductive and Inductive Reasoning in Large Language Models. InProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)

    work page 2025

  4. [4]

    Diego Calanzone, Stefano Teso, and Antonio Vergari. 2024. Logically consistent language models via neuro-symbolic integration.arXiv preprint arXiv:2409.13724 (2024)

    arXiv 2024

  5. [5]

    Linbo Cao and Jinman Zhao. 2025. Pretraining on the Test Set Is No Longer All You Need: A Debate-Driven Approach to QA Benchmarks. InSecond Conference on Language Modeling

    work page 2025

  6. [6]

    Michael K Chen. 2025. A comparative study of neurosymbolic ai approaches to interpretable logical reasoning.arXiv preprint arXiv:2508.03366(2025)

    arXiv 2025

  7. [7]

    Shen Chen, Jiale Zhou, and Lei Li. 2025. Dense Point Clouds Matter: Dust-GS for Scene Reconstruction from Sparse Viewpoints. InICASSP 2025 – 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5

    work page 2025

  8. [8]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168(2021)

    arXiv 2021

Show all 75 references
  1. [9]

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025. Retool: Reinforcement learning for strategic tool use in llms.arXiv preprint arXiv:2504.11536(2025)

    2025 arXiv

  2. [10]

    Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Junjie Hu, Tianyu Liu, and Baobao Chang

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  4. [12]

    Ruocheng Gu, Sen Jia, Yule Ma, Jinqin Zhong, Jenq-Neng Hwang, and Lei Li

  5. [13]

    Yunchuan Guan, Yu Liu, Ke Zhou, Hui Li, Sen Jia, Zhiqi Shen, Ziyang Wang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, et al. 2025. Learning an Efficient Op- timizer via Hybrid-Policy Sub-Trajectory Balance.arXiv preprint arXiv:2511.00543 (2025)

    2025

  6. [14]

    InProceedings of the 33rd ACM International Conference on Multimedia

    MoCount: Motion-Based Repetitive Action Counting. InProceedings of the 33rd ACM International Conference on Multimedia. 9026–9034

  7. [15]

    Yunchuan Guan, Yu Liu, Ke Zhou, Zhiqi Shen, Jenq-Neng Hwang, and Lei Li

  8. [16]

    Yunchuan Guan, Yu Liu, Ke Zhou, Zhiqi Shen, Jenq-Neng Hwang, Serge Belongie, and Lei Li. 2025. Is Meta-Learning Out? Rethinking Unsupervised Few-Shot Classification with Limited Entropy. InProceedings of the IEEE/CVF International Conference on Computer Vision. 4188–4197

    2025

  9. [17]

    Jilin Hu, Jianyu Zhang, Yongwang Zhao, and Talia Ringer. 2025. HybridProver: Augmenting Theorem Proving with LLM-Driven Proof Synthesis and Refinement. arXiv preprint arXiv:2505.15740(2025)

    2025 arXiv

  10. [18]

    arXiv preprint arXiv:2502.01117(2025)

    Learning to Learn Weight Generation via Local Consistency Diffusion. arXiv preprint arXiv:2502.01117(2025)

    2025 arXiv

  11. [19]

    Michael Hersche, Giacomo Camposampiero, Roger Wattenhofer, Abu Sebastian, and Abbas Rahimi. 2024. Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning.arXiv preprint arXiv:2412.05586(2024)

    2024 arXiv

  12. [20]

    Ryo Kamoi, Yusen Zhang, Nan Zhang, Sarkar Snigdha Sarathi Das, and Rui Zhang

  13. [21]

    Sen Jia, Ning Zhu, Jinqin Zhong, Jiale Zhou, Huaping Zhang, Jenq-Neng Hwang, and Lei Li. 2026. RAM: Recover Any 3D Human Motion in-the-Wild.arXiv preprint arXiv:2603.19929(2026)

    2026 arXiv

  14. [22]

    Wenyao Cui et al

    Aditya Kalyanpur, Kailash Karthik Saravanakumar, Victor Barres, CJ McFate, Lori Moon, Nati Seifu, Maksim Eremeev, Jose Barrera, Abraham Bautista-Castillo, KDD 2026, August 9–13, 2026, Jeju Island, Republic of Korea. Wenyao Cui et al. Eric Brown, et al . 2024. Multi-step Infere...

    2024 arXiv

  15. [23]

    Lei Li, Sen Jia, and Jenq-Neng Hwang. 2026. Multiple Human Motion Under- standing. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 6297–6305

    2026

  16. [24]

    Lei Li, Sen Jia, Jianhao Wang, Zhongyu Jiang, Feng Zhou, Ju Dai, Tianfang Zhang, Zongkai Wu, and Jenq-Neng Hwang. 2025. Human Motion Instruction Tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  17. [25]

    Tian Lan, Jinyuan Xu, Xue He, Jenq-Neng Hwang, and Lei Li. 2025. Attention Consistency for LLMs Explanation. InFindings of the Association for Compu- tational Linguistics: EMNLP 2025. Association for Computational Linguistics, Suzhou, China, 1736–1750

    2025

  18. [26]

    Lei Li. 2024. Image Semantic Segmentation via Chain-of-Thought Prompts. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV)

    2024

  19. [27]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s Verify Step by Step. InInternational Conference on Learning Representations

    2024

  20. [28]

    Libin Liu, Shen Chen, Sen Jia, Jingzhe Shi, Zhongyu Jiang, Can Jin, Zongkai Wu, Jenq-Neng Hwang, and Lei Li. 2024. Graph Canvas for Controllable 3D Scene Generation.arXiv preprint arXiv:2412.00091(2024)

    2024 arXiv

  21. [29]

    Feng Liang, Weixin Zeng, Runhao Zhao, and Xiang Zhao. 2025. NeSTR: A Neuro- Symbolic Abductive Framework for Temporal Reasoning in Large Language Models.arXiv preprint arXiv:2512.07218(2025)

    2025

  22. [30]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. InThe Twelfth International Conference on Learning Representations

    2023

  23. [31]

    Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, and Roger Levy. 2023. LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers. InProceedings of the Conference on Empirical Metho...

    2023

  24. [32]

    Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. 2023. Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning. InFindings of the Association for Computational Linguistics: EMNLP

    2023

  25. [33]

    Shinobu Miya. 2025. Eidoku: A Neuro-Symbolic Verification Gate for LLM Reasoning via Structural Constraint Satisfaction.arXiv preprint arXiv:2512.20664 (2025)

    2025

  26. [34]

    Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi, and Yasuhiro Sogawa. 2024. Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus. InAdvances in Neural Information Processing Systems

    2024

  27. [35]

    Xin Quan, Marco Valentino, Danilo Carvalho, Dhairya Dalal, and André Freitas

  28. [36]

    Xin Quan, Marco Valentino, Louise Dennis, and Andre Freitas. 2024. Enhanc- ing ethical explanations of large language models through iterative symbolic refinement. InProceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Vo...

    2024

  29. [37]

    Hyejin Park, Junhyuk Kwon, Suha Kwak, and Jungseul Ok. 2026. VIRO: Ro- bust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension.arXiv preprint arXiv:2601.12781(2026)

    2026

  30. [38]

    Pengcheng Qiu, Chaoyi Wu, Shuyu Liu, Yanjie Fan, Weike Zhao, Zhuoxia Chen, Hongfei Gu, Chuanjin Peng, Ya Zhang, Yanfeng Wang, et al. 2025. Quantifying the reasoning abilities of LLMs on clinical cases.Nature Communications16, 1 (2025), 9799

    2025

  31. [39]

    Jingzhe Shi, Qinwei Ma, Hongyi Liu, Hang Zhao, Jenq-Neng Hwang, and Lei Li

  32. [40]

    InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)

    Peirce: Unifying material and formal reasoning via llm-driven neuro- symbolic refinement. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). 11–21

  33. [41]

    Pengcheng Shi, Jiawei Chen, Jiaqi Liu, Xinglin Zhang, Tao Chen, and Lei Li. 2025. Medal S: Spatio-Textual Prompt Model for Medical Segmentation. InCVPR 2025: Foundation Models for 3D Biomedical Image Segmentation

    2025

  34. [42]

    Dennis, and Andre Freitas

    Xin Quan, Marco Valentino, Louise A. Dennis, and Andre Freitas. 2023. Verifica- tion and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics

    2023

  35. [43]

    Albert Sadowski and Jarosław A Chudziak. 2025. Explainable rule application via structured prompting: A neural-symbolic approach.Procedia Computer Science 270 (2025), 2166–2175

    2025

  36. [45]

    InThe Fourteenth International Conference on Learning Representations

    Intrinsic Entropy of Context Length Scaling in LLMs. InThe Fourteenth International Conference on Learning Representations

  37. [46]

    Jingzhe Shi, Qinwei Ma, Huan Ma, and Lei Li. 2024. Scaling Law for Time Series Forecasting. InAdvances in Neural Information Processing Systems (NeurIPS)

    2024

  38. [47]

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-Shepherd: Verify and Reinforce LLMs Step-by- step without Human Annotations. InProceedings of the Annual Meeting of the Association for Computational Linguistics. 9426–9439

    2024

  39. [48]

    Oren Sultan, Eitan Stern, and Dafna Shahaf. 2025. Towards Reliable Proof Gener- ation with LLMs: A Neuro-Symbolic Approach.arXiv preprint arXiv:2505.14479 (2025)

    2025 arXiv

  40. [49]

    Haonan Tong, Ke Liu, Chuang Zhang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, and Lei Li. 2025. PAMN: Multi-phase Correlation Modeling for Contrast- Enhanced 3D Medical Image Retrieval. InFindings of the Association for Compu- tational Linguistics: EMNLP 2025. 3456–3467

    2025

  41. [50]

    Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao, and Jun Liu. 2023. Symbol-LLM: Towards Foundational Symbol-centric Interface for Large Language Models. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics

    2023

  42. [51]

    Francis Song, Noah Y

    Jonathan Uesato, Nate Kushman, Ramana Kumar, H. Francis Song, Noah Y. Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving Math Word Problems with Process- and Outcome-Based Feedback.arXiv preprint arXiv:2211.14275(2022)

    2022 arXiv

  43. [52]

    Ruslan Idelfonso Magana Vsevolodovna and Marco Monti. 2025. Enhancing Large Language Models through Neuro-Symbolic Integration and Ontological Reasoning.arXiv preprint arXiv:2504.07640(2025)

    2025

  44. [53]

    Jundong Xu, Hao Fei, Huichi Zhou, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, and Wynne Hsu. 2025. Train- ing LLMs with LogicReward for Faithful and Rigorous Reasoning.arXiv preprint arXiv:2512.18196(2025)

    2025

  45. [54]

    Wenguan Wang, Yi Yang, and Fei Wu. 2024. Towards data-and knowledge-driven AI: a survey on neuro-symbolic computing.IEEE Transactions on Pattern Analysis and Machine Intelligence(2024)

    2024

  46. [55]

    Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Hoo-Chang Shin, Felipe Soares, Alexander Bukharin, Ellie Evans, Yi Dong, and Oleksii Kuchaiev. 2025. HelpSteer3- Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages. arXiv:2505.11475 [cs.CL] https://ar...

    2025

  47. [56]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    2025 arXiv

  48. [57]

    Jundong Xu, Hao Fei, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, and Wynne Hsu. 2025. Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework. In Proceedings of the 63rd Annual Meeting of the Associa...

    2025

  49. [58]

    Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu

  50. [59]

    Wenhao Yang, Jianguo Wei, Wenhuan Lu, and Lei Li. 2025. You Only Speak Once to See. InIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    2025

  51. [60]

    Xiao-Wen Yang, Jie-Jing Shao, Lan-Zhe Guo, Bo-Wen Zhang, Zhi Zhou, Lin-Han Jia, Wang-Zhou Dai, and Yu-Feng Li. 2025. Neuro-symbolic artificial intelligence: Towards improving the reasoning abilities of large language models.arXiv preprint arXiv:2508.13678(2025)

    2025 arXiv

  52. [61]

    Qiaoyi Xu, Afzan Adam, Azizi Abdullah, Tao Chen, Xinglin Zhang, Adam Shep- hard, Patsy Ng Pei Sze, Noraidah Masir, Lei Li, and Reena Rahayu. 2026. Robust Multi-Domain Digital Pathology Image Segmentation via Joint Balancing Repre- sentation Learning.Expert Systems with Applica...

    2026

  53. [62]

    Ziyang Yan, Yihua Shao, Minwen Liao, Siyu Chen, Nan Wang, Muyuan Lin, Jenq- Neng Hwang, Hao Zhao, Fabio Remondino, and Lei Li. 2026. 3DSceneEditor: Controllable 3D Scene Editing with Gaussian Splatting. InProceedings of the IEEE/CVF Winter Conference on Applications of Compute...

    2026

  54. [63]

    Xin Zhang, Shen Chen, Jiale Zhou, and Lei Li. 2026. PSGS: Text-driven Panorama Sliding Scene Generation via Gaussian Splatting.arXiv preprint arXiv:2602.00463 SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification KDD 2026, August 9–13, 2026, Jeju Isl...

    2026

  55. [64]

    Kaiyu Yang, Aidan Swope, Alex Gu, Rahul Chalamala, Peiyang Song, Shixing Yu, Saad Godil, Ryan J Prenger, and Animashree Anandkumar. 2023. Leandojo: Theorem proving with retrieval-augmented language models.Advances in Neural Information Processing Systems36 (2023), 21573–21612

    2023

  56. [65]

    Sen Yang, Xin Li, Leyang Cui, Lidong Bing, and Wai Lam. 2025. Neuro-symbolic integration brings causal and reliable reasoning proofs. InFindings of the Associ- ation for Computational Linguistics: NAACL 2025. 5732–5744

    2025

  57. [66]

    Jinman Zhao, Erxue Min, Hui Wu, Ziheng Li, Zexu Sun, Hengyi Cai, Shuaiqiang Wang, Xu Chen, and Gerald Penn. 2026. Beyond Step Pruning: Information Theory Based Step-level Optimization for Self-Refining Large Language Models. Proceedings of the AAAI Conference on Artificial Int...

    2026 doi

  58. [67]

    Jinman Zhao and Xueyan Zhang. 2024. Large Language Model is not a (Mul- tilingual) Compositional Relation Reasoner. InFirst Conference on Language Modeling

    2024

  59. [68]

    Ziyu Yao, Xuxin Cheng, Zhiqi Huang, and Lei Li. 2025. CountLLM: Towards Gen- eralizable Repetitive Action Counting via Large Language Model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2025

  60. [69]

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2025. Generative Verifiers: Reward Modeling as Next-Token Prediction. InInternational Conference on Learning Representations

    2025

  61. [71]

    Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao, and Yining Wang. 2025. UORA: Uniform Orthogonal Reinitialization Adap- tation in Parameter Efficient Fine-Tuning of Large Models. InProceedings of the 63rd Annual Meeting of the Association for Computa...

    2025

  62. [72]

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. The Lessons of Developing Process Reward Models in Mathematical Reasoning. InFindings of the Association for Computational Linguistics: ACL 2025. 10495–10516

    2025

  63. [75]

    Jinman Zhao, Xueyan Zhang, Jiaru Li, Jingcheng Niu, Yulan Hu, Erxue Min, and Gerald Penn. 2025. Tiny Budgets, Big Gains: Parameter Placement Strategy in Parameter Super-Efficient Fine-Tuning. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Process...

    2025

  64. [76]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems36 (2023), 46595–46623. A Erro...

    2023

  65. [2024]

    Faithful Logical Reasoning via Symbolic Chain-of-Thought.arXiv preprint arXiv:2405.18357(2024)

    2024 arXiv

  66. [2025]

    InFindings of the Association for Computational Linguistics: ACL 2025

    LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathemat- ical Verifier with Natural Language Feedback. InFindings of the Association for Computational Linguistics: ACL 2025. 14588–14604

    2025

  67. [2026]

    InInternational Conference on Learning Representations

    Generalizable Process Reward Models via Formally Verified Training Data. InInternational Conference on Learning Representations

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.