{"id":"a6707a71-f481-4570-aee1-12bba3859347","arxiv_id":"2505.00562","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A GNN-encoded flow matching model learns to generate STL-satisfying trajectories across five robot simulation domains, with a 200K-specification dataset, reporting best-of-1024 satisfaction rates.","lead":"TeLoGraF trains a graph neural network and flow-matching model to generate robot trajectories that satisfy signal temporal logic (STL) specifications, using a newly collected dataset of 200K STL examples. The authors report faster inference than classical planners, but the evaluation selects the best of 1,024 generated trajectories and the dataset is not yet released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'general STL' claim is unsupported: the four templates are a narrow fragment, and the paper's own Fig. 7 shows low validation satisfaction on the complex templates in Linear/Dubins/Panda even with best-of-1024 selection.","rationale":"The reader correctly identifies the template representativeness and post hoc interval fitting as weak spots. I agree partially, but the more direct problem is internal: even within the four templates, the model's validation satisfaction is low for the complex categories in three of five environments. This is stated by the authors in Sec. 5.4 and visible in Fig. 7. Because the abstract and Sec. 4.1 make an unqualified claim that the model produces satisfiable trajectories for general STL, the paper overstates its result. A conditional acceptance is still appropriate because the dataset, the GNN-encoded flow matching formulation, and the relative improvement over baselines are plausible contributions; however, the authors must narrow the claims and provide per-template, non-best-of-K results. My concern strengthens the reader's verdict rather than changing it.","tokens_in":18144,"tokens_out":10137,"duration_ms":101583,"concrete_test":"Run the released model and code on a held-out set of STL formulas drawn from the full grammar of Eq. (1) that do not match any of the four templates (e.g., nested Until F[0,10](A and F[5,15]B), disjunction of sequential tasks, G[0,5](C or D) with reach/avoid predicates), in the Linear and Franka Panda environments. Report the best-of-1024 satisfaction rate and the single-sample satisfaction rate with 95% CIs over at least 5 seeds. If the best-of-1024 rate drops below the template-validation rates (e.g., from about 0.45 in Dubins to under 0.1), the 'general STL' claim is not supported and the paper should be revised to claim capability on the four templates only. Additionally, recompute Fig. 7 with single-sample (no best-of-K) satisfaction to determine whether the model genuinely solves templates II-IV.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract; Sec. 4.1) that TeLoGraF takes a 'general STL' specification and produces trajectories satisfying it is not supported by the evaluation. The training grammar (Eq. 3) covers only four templates (single-goal, multi-goal, sequential, partial) with atomic propositions restricted to reach/avoid; this is a small fragment of the STL grammar in Eq. (1), which includes arbitrary predicates, nested Until, and unbounded combinations. More importantly, the paper's own Sec. 5.4 and Fig. 7 state that all models' performance on templates II-IV is 'relatively low' on Linear, Dubins, and Franka Panda. For Dubins, TeLoGraF's validation satisfaction is 0.45 even with best-of-1024 selection (Sec. 5.2, Fig. 6); per-template rates for sequential/partial are lower. Thus the operational claim 'trajectories sampled from p_theta satisfy the STL' fails for the complex templates that the method is supposed to handle, independent of any out-of-distribution concern. The expressiveness argument in Sec. 4.4 (GNN can distinguish syntax trees) only speaks to encoding, not to generating satisfying trajectories. The maze results are additionally weakened by the post hoc time-interval inference in Appendix A.1.3, where intervals are fitted to trajectories rather than fixed a priori, so those experiments do not measure satisfaction of user-specified time constraints. The paper acknowledges the limitation in the Limitations paragraph but the abstract and intro retain the 'general STL / satisfiable solutions' claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TeLoGraF, a conditional generative model for trajectory planning under Signal Temporal Logic (STL) specifications. STL formulas are encoded as syntax-tree graphs and processed by a GNN, and a flow-matching model generates trajectories conditioned on the STL embedding and initial state. The authors identify four STL templates (single-goal, multi-goal, sequential, partial), generate roughly 200K specifications with paired demonstration trajectories in five simulation environments (Linear, Dubins, PointMaze, AntMaze, Franka Panda), and compare TeLoGraF against classical planners (Grad, CEM) and learning-based baselines (CTG, LTLDoG). The paper claims state-of-the-art STL satisfaction rates, 10-100x faster inference than classical methods, and the ability to handle 'general STL specifications.'","tokens_in":18463,"tokens_out":3830,"duration_ms":40509,"significance":"If the central claims were fully supported, TeLoGraF would be a valuable contribution: it is among the first learned models to take STL syntax as input and generate trajectories, and the GNN encoding of STL syntax trees is a sensible and well-motivated design choice. The dataset of 200K STL-conditioned demonstrations, if released, could be a useful benchmark for the community. The paper also provides a theoretical argument for why GNNs can distinguish STL syntax trees and runs a systematic encoder comparison. However, the evidence falls short of the 'general STL' claim: the four templates are a narrow fragment of the STL grammar, the per-template satisfaction on complex templates is low by the paper's own figures, the best-of-1024 evaluation protocol without variance estimates is optimistic, and the maze demonstrations are generated by fitting time intervals to pre-planned trajectories rather than respecting fixed user-specified deadlines. The core methodology is defensible for the four templates, but the headline claims need substantial revision or additional evidence.","major_comments":[{"comment":"The paper's central claim that TeLoGraF handles 'general STL specifications' (Abstract; Sec. 4.1) is not supported by the presented grammar and results. Equation (3) covers only four templates whose atomic propositions are restricted to reach/avoid predicates, a small fragment of the STL grammar in Eq. (1) that includes arbitrary predicates, nested Until, and unbounded combinations. More importantly, the paper's own Fig. 7 shows that validation satisfaction for templates II-IV is 'relatively low' on Linear, Dubins, and Franka Panda; for Dubins, Sec. 5.2 reports a validation satisfaction rate of 0.45 for TeLoGraF even with best-of-1024 selection, and per-template rates for sequential and partial are lower. The expressiveness argument in Sec. 4.4 only shows that a GNN can distinguish different syntax trees, not that the generated trajectories satisfy the STL formula. The abstract and introduction should be revised to claim handling of 'four common templates' rather than 'general STL,' or the evaluation should include formulas outside these templates.","section":"Sec. 4.2, Eq. (3); Sec. 5.4, Fig. 7; Sec. 4.1"},{"comment":"The maze data pipeline undermines the evaluation of time constraints. Appendix A.1.3 states that, after planning a trajectory, the authors 'randomly infer the possible time intervals for the goal-reaching sub-formulas and run post-verification to ensure the trajectory satisfies this STL.' This means the STL formulas in PointMaze and AntMaze are constructed to be feasible for already-generated trajectories, so the model is never tested on user-specified deadlines that are fixed in advance. Consequentially, the reported high satisfaction rates in maze environments do not measure the model's ability to satisfy exact temporal requirements. The experiments should either fix time intervals before planning or report maze results separately with this feasibility-by-construction caveat.","section":"Appendix A.1.3"},{"comment":"The evaluation protocol is optimistic and lacks statistical grounding. The paper states that for each STL, 1024 trajectories are sampled and the one with the highest STL score is selected as the final trajectory. Reporting best-of-1024 satisfaction without error bars, multiple seeds, or any statistical comparison makes the claimed superiority over baselines fragile; for instance, the differences between TeLoGraD, TeLoGraF, TreeLSTM, and the guidance-based baselines in Fig. 4 could be within sampling noise under this protocol. The authors should report mean and variance over multiple random seeds, report both mean and best-over-sample statistics, and describe how many STL specifications and seeds are used in each reported number.","section":"Sec. 5, Metrics"}],"minor_comments":[{"comment":"There is a typo: 'integrating the predicted vector field over over time' should read 'over time.'","section":"Sec. 3.2"},{"comment":"The last sentence says 'The loss is similar to Eq. (8),' but Eq. (8) is the Dubins dynamics; the intended reference appears to be Eq. (7), the gradient-based loss.","section":"Appendix A.1.2"},{"comment":"The section heading reads 'AntMaze,' but Figures 31-37 are labeled 'PointMaze environment'; this mismatch should be corrected.","section":"Appendix C.4"},{"comment":"The explanation of phi_reach's stay condition says the robot stays for the time interval [ta + tc, tb + td], but the grammar only defines G[tc,td] after F[ta,tb]; the relationship between the two intervals should be clarified or corrected.","section":"Eq. (3) and surrounding text"},{"comment":"Figure 6 reports training and validation satisfaction for different ODE steps without confidence intervals; adding error bars would make the claim that performance 'does not drop until 10 steps' more robust.","section":"Sec. 5, Metrics and Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful core idea and a substantial dataset contribution, but the headline 'general STL' claim is significantly overstated relative to the four-template grammar and the paper's own low per-template satisfaction numbers on complex formulas. The maze data-generation protocol (post-hoc interval inference) is a serious correctness concern for the time-constraint evaluation, and the best-of-1024 metric without variance estimates makes the quantitative comparisons hard to interpret. I would require the authors to either substantially broaden the evaluation or carefully scope the claims, redo or clearly caveat the maze experiments, and provide statistically sound evaluation numbers before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TeLoGraF is worth engaging with, but the paper's central claim is broader than what it actually demonstrates. The genuinely new pieces are the GNN encoder over STL syntax trees and the 200K-spec, five-environment dataset; that combination hasn't been done before, and the ablations suggest graph encoders beat sequence encoders for this kind of structure. The flow-matching backbone is a sensible choice and the runtime speedups over gradient and CEM baselines are real where they hold.\n\nThe soft spots are in the evaluation and the surrounding claims. The abstract and Sec. 4.1 say 'general STL,' but Eq. (3) defines only four templates with reach/avoid atomic propositions. That's a narrow fragment, and the paper's own Fig. 7 shows validation satisfaction on templates II-IV is relatively low in Linear, Dubins, and Franka Panda; for Dubins the overall validation number is 0.45 even with best-of-1024 selection. So the operational claim that sampled trajectories satisfy the specification fails on exactly the complex templates that are supposed to justify the 'general' label. The maze experiments are weaker evidence because Appendix A.1.3 fits time intervals to already-planned trajectories, which guarantees feasibility and doesn't test user-specified timing constraints. Also, the evaluation uses best-of-1024 with no multiple seeds or error bars, which inflates the headline numbers.\n\nNone of this kills the contribution. The simple-template results are solid, the dataset is a new resource, and the authors are upfront in the Limitations section that performance degrades on complex syntax. The fixes are clear: release the code and dataset with a hash, report single-sample and best-of-K satisfaction with variance, and either redo the maze protocol or report those environments as a separate feasibility check. Then scale the claims back to what the templates actually cover.\n\nThis is a paper I'd send to a serious referee. The core idea is novel enough and the resource useful enough to deserve a proper review cycle; the overreach is correctable.","headline":"A useful STL-conditioned generative planner and dataset, but the 'general STL' claim and the maze data protocol need to be tightened before publication.","tokens_in":19041,"tokens_out":2316,"would_cite":true,"duration_ms":23091,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single learned generative model can take any Signal Temporal Logic (STL) specification and an initial state and output a trajectory that satisfies it—the first general STL-conditioned planner.","keywords":["signal temporal logic","flow matching","graph neural network","trajectory planning","conditional generative model","temporal logic planning","robot motion planning","STL satisfaction"],"falsifier":"Build a validation set of STL formulas whose syntax trees go deeper than the four templates—for example, nested $\\Box_{[a,b]} \\Diamond_{[c,d]}$ combinations over conjunctions of reach and avoid predicates—and measure the satisfaction rate of TeLoGraF's sampled trajectories in one benchmark environment. If the rate collapses to the level of the sequence-encoder baselines, the claim that graph encoding generalizes across STL structure is refuted; if satisfaction already drops when time windows are narrowed by a small margin below training values, the model's scope is narrower than the 'general STL' claim.","tokens_in":17863,"feed_emoji":"🤖","tokens_out":11247,"duration_ms":91191,"temperature":0.7,"pith_summary":"TeLoGraF claims to be the first learned model that takes a general Signal Temporal Logic (STL) specification as input and produces trajectories that satisfy it, without retraining per formula. The paper identifies four common STL task patterns—single-goal, multi-goal, sequential, and partial-order—and builds a dataset of 200K specifications with paired demonstration trajectories across five simulated robot systems. The method encodes each STL formula as a graph and uses flow matching to generate trajectories conditioned on that graph embedding and the initial state. This matters because classical STL planners are slow and struggle in high dimensions, while earlier learned methods handled only fixed or parameterized formulas, so a working general model would be the first of its kind and 10 to 100 times faster at inference than classical planners.","feed_headline":"One model turns any temporal-logic spec into a valid robot path","feed_subtitle":"Graph-encoded flow matching answers Signal Temporal Logic queries 10 to 100 times faster than classical planners.","key_machinery":"The load-bearing object is the STL syntax tree recast as a directed graph with child-to-parent edges, where each node holds an 8-dimensional feature vector (operator type; start and end times, with a default of -1; object coordinates and radius or side length; and a binary flag marking a left child of an Until operator). A multi-layer GNN with a permutation-invariant aggregation performs message passing over this graph and reads out a fixed-dimension embedding; that embedding, concatenated with the initial state, conditions a temporal U-Net trained with the flow-matching loss $\\mathcal{L}_{\\mathrm{FM}} = \\mathbb{E}_{t,X_0,X_1}\\|H_\\omega(F_\\theta(G), x_0, t, X_t) - (X_1 - X_0)\\|^2$, where $X_t = tX_1 + (1-t)X_0$. Generation integrates the learned velocity field $\\frac{d}{dt}\\psi(t,X) = H_\\omega(F_\\theta(G), x_0, t, \\psi(t,X))$ by Euler steps starting from Gaussian noise, and the ODE solution at $t=1$ is the planned trajectory. The expressiveness argument rests on the Weisfeiler–Leman test being complete for trees, which the paper uses to conclude that distinct STL syntax trees receive distinct embeddings.","core_discovery":"The central claim is that an STL formula's syntax tree can be turned into a directed graph—each operator and atomic proposition becomes a node carrying features for operator type, time interval, object geometry, and Until-child role—and a graph neural network embedding of that graph suffices to condition a flow-matching trajectory generator, so that trajectories drawn from $p_\\theta(\\tau \\mid x_0, \\phi)$ satisfy the specification, $\\tau, 0 \\models \\phi$. The authors justify the encoding by noting that the Weisfeiler–Leman graph test is complete for trees, so a GNN can in principle distinguish distinct STL syntax trees. Empirically, the GNN-conditioned model posts the highest STL satisfaction rates against guidance-based learning baselines and classical gradient and sampling planners across five environments, and with only ten ODE integration steps it retains most of its satisfaction while running over a hundred times faster than classical baselines in the Franka Panda environment. The paper further claims that graph encoding generalizes better than sequence encoders to unseen and synthetically augmented STL structures.","pith_inferences":["If the four templates capture most practical STL tasks, the bottleneck shifts to demonstration collection: the model is only as broad as the solver that produced its training trajectories, so better offline datasets could extend the same graph-conditioned generator to new environments.","The maze demonstrations were made by planning a path and then inferring time intervals that make it satisfy the formula, so a testable consequence is that the model learns comfortable time margins and satisfaction should degrade sharply when queried with time windows narrower than any seen in training.","The graph encoder is agnostic to the generative backbone and the operator vocabulary, so the same conditioning could steer diffusion or consistency samplers, and the operator set could be extended past reach and avoid to predicates over continuous signals.","Coupling TeLoGraF's fast proposals with a classical STL verifier as a filter would yield an anytime planner with soundness guarantees on accepted trajectories, addressing the paper's stated lack of guarantees."],"forward_implications":["A single trained model covers the four STL templates and formulas built from them, so a user can query new specifications at test time without retraining.","Because generation is a short ODE solve rather than an optimization loop, inference is fast enough for real-time use; the paper reports 10 to 100 times speedups and, with ten flow steps, roughly 123.6 times faster planning than the gradient baseline on the 7-DoF Franka Panda arm at a higher satisfaction rate.","The approach does not require differentiable dynamics, so it applies to maze navigation and other non-differentiable systems as long as demonstration trajectories can be collected.","Graph-based encoding propagates time intervals and temporal ordering better than sequence encoders, which is where the satisfaction gains over GRU and Transformer baselines concentrate, namely sequential and partial-order templates."],"supporting_citations":[{"why":"Defines the robustness score ρ that measures and optimizes STL satisfaction, the objective the learned trajectories are judged against.","marker":"(Donzé & Maler, 2010)"},{"why":"Supplies the gradient-based planning method (Grad) used as a baseline and as the demonstration collector for differentiable environments.","marker":"(Dawson & Fan, 2022)"},{"why":"Supplies the CEM sampling-based baseline compared for satisfaction and runtime.","marker":"(Kapoor et al., 2020)"},{"why":"Introduces the linear/rectified flow-matching objective that TeLoGraF's trajectory generator trains on.","marker":"(Liu et al., 2022)"},{"why":"Provides the temporal U-Net architecture used as the trajectory-generation backbone.","marker":"(Janner et al., 2022)"},{"why":"Establishes that GNN expressiveness is bounded by the Weisfeiler–Leman test, the theoretical basis for the graph encoder.","marker":"(Xu et al., 2018)"},{"why":"Supplies the result that the Weisfeiler–Leman test is complete for trees, justifying that the GNN can distinguish different STL syntax trees.","marker":"(Kiefer, 2020)"},{"why":"Provides the PointMaze and AntMaze benchmarks used for the non-differentiable navigation experiments.","marker":"(Fu et al., 2020)"},{"why":"Provides the CTG guidance-based diffusion baseline that TeLoGraF outperforms without guidance at inference.","marker":"(Zhong et al., 2023)"},{"why":"Provides the LTLDoG classifier-guidance baseline, adapted to STL, that TeLoGraF is compared against.","marker":"(Feng et al., 2024)"}],"fun_headline_variants":["GNN-flow matches any temporal logic spec, 100x faster","TeLoGraF: graph-encoded flow beats classical planners","From syntax tree to path: 10-100x faster STL planning","One graph-encoding model for all temporal logic tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim of generality rests on the four procedural STL templates (single-goal, multi-goal, sequential, partial) standing in for all practically useful STL specifications, so formulas outside these patterns, or with time constraints tighter than the demonstration distribution, may not be satisfiable by the learned model.","fun_headline_variants_meta":{"raw":{"variants":["GNN-flow matches any temporal logic spec, 100x faster","TeLoGraF: graph-encoded flow beats classical planners","From syntax tree to path: 10-100x faster STL planning","One graph-encoding model for all temporal logic tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000164,"raw_usage":{"total_tokens":1265,"prompt_tokens":984,"completion_tokens":281,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":208}},"tokens_in":600,"tokens_out":281,"duration_ms":3653,"temperature":1.0,"reasoning_tokens":208,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:39:34.605457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a validation set of STL formulas whose syntax trees go deeper than the four templates—for example, nested $\\Box_{[a,b]} \\Diamond_{[c,d]}$ combinations over conjunctions of reach and avoid predicates—and measure the satisfaction rate of TeLoGraF's sampled trajectories in one benchmark environment. If the rate collapses to the level of the sequence-encoder baselines, the claim that graph encoding generalizes across STL structure is refuted; if satisfaction already drops when time windows are narrowed by a small margin below training values, the model's scope is narrower than the 'general STL' claim.","supporting_citations":[{"cited_title":"and Fan, C","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-based planning method (Grad) used as a baseline and as the demonstration collector for differentiable environments."},{"cited_title":"Power and limits of the Weisfeiler-Leman algorithm","cited_arxiv_id":null,"evidence_quote":"Supplies the result that the Weisfeiler–Leman test is complete for trees, justifying that the GNN can distinguish different STL syntax trees."},{"cited_title":"Guided conditional diffusion for controllable traffic simulation","cited_arxiv_id":null,"evidence_quote":"Provides the CTG guidance-based diffusion baseline that TeLoGraF outperforms without guidance at inference."},{"cited_title":"LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning","cited_arxiv_id":"2405.04235","evidence_quote":"Provides the LTLDoG classifier-guidance baseline, adapted to STL, that TeLoGraF is compared against."}],"review_version":1}