{"id":"a3ede0c9-3c48-4cbf-ba4a-bf3fd1dacefb","arxiv_id":"2504.17355","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"TCTO, a graph-based multi-agent RL method that prunes and backtracks over feature transformation paths, outperforms existing automated feature engineering methods on most of 25 tabular datasets.","lead":"This paper presents TCTO, a multi-agent reinforcement learning framework that automatically creates new features from tables by representing transformations as paths on an evolving graph. The method reports higher scores than ten existing feature transformation tools on most of 25 datasets, with a traceable record of how each new feature was built.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section III-D pruning as written can leave surviving nodes whose ancestor was pruned, invalidating the roadmap's traceability; the paper never specifies how descendants of removed nodes are handled.","rationale":"The reader identified essentially the same load-bearing concern: pruning can break the validity of transformations that depend on removed nodes. I agree with that core worry, and I add that the paper is even more ambiguous than the reader stated: it says both \"select top-K nodes\" and \"removes low-correlation nodes,\" so even the number of retained versus removed nodes is unclear. The concern is load-bearing because the paper's novelty is the traceable roadmap with pruning/backtracking; if the roadmap becomes an invalid DAG after pruning, the central claim of full traceability collapses. The empirical comparison could still be correct, and the case study (Table III) shows traceable features at the end of training, but no graph snapshots are provided to show that the graph remains coherent throughout training. A code inspection or small re-implementation would settle the issue. I therefore keep the reader's CONDITIONAL verdict: the paper is acceptable only if the authors clarify the graph-repair semantics or release working code that demonstrates a valid roadmap after pruning. This is not a rejection because the ambiguity might be a documentation lapse rather than a fundamental flaw, and the empirical results, if reproducible, may stand. However, the concern is concrete enough that it must be addressed.","tokens_in":19173,"tokens_out":5509,"duration_ms":57147,"concrete_test":"Obtain the released code (the paper promises public Dropbox access) and run TCTO on Housing Boston or another small dataset. After each node-wise pruning step, traverse the roadmap and check whether any surviving node has a pruned ancestor on its root-to-node path. If such nodes exist, the implementation must be doing something not described (e.g., cascading deletion, retaining feature columns outside the graph, or reconnecting edges). If code is unavailable, re-implement Section III-D minimally and log the graph after pruning; if any surviving descendant remains, the algorithm as written is not executable. Alternatively, inspect the code for a topological repair or descendant-deletion step; its absence confirms the gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of TCTO is that a traceable, evolving transformation roadmap enables safe node-wise pruning and step-wise backtracking. Section III-D describes node-wise pruning as selecting \"top-K nodes\" by mutual information and \"removes low-correlation nodes,\" but it never states what happens to descendants of a removed node. In the roadmap, each node's \"path back to the root node represents a transformation pathway\" (Section II-A). If a pruned node is an ancestor of a surviving node, that surviving node's transformation path includes a deleted node, so its feature column is undefined and D' = G(D) is not well-formed. The RGCN aggregation (Eq. 4) would also reference nonexistent neighbors. Step-wise backtracking suffers a related gap: \"rolling back to the previous optimal transformation roadmap\" is never defined operationally (what is \"optimal,\" and how are feature columns restored?). The paper's claimed advantages--full traceability, safe pruning, and backtracking--therefore rest on an unspecified graph-repair rule. The reader's summary states the strategy deletes the K lowest-MI nodes, but the text is ambiguous (it first says keep top-K, then says remove low-correlation nodes); this ambiguity further confirms the mechanism is under-specified. This is an internal-consistency concern, not a disagreement with community consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TCTO, a collaborative multi-agent reinforcement learning framework for automated feature transformation. TCTO maintains an evolving directed graph ('transformation roadmap') whose nodes are feature states and edges are mathematical operations, and uses graph clustering, an RGCN state encoder, and three sequential agents (head cluster, operation, operand cluster) to generate new features. Two roadmap maintenance strategies are introduced: node-wise pruning based on mutual information with the label, and step-wise backtracking to a previous optimal roadmap. The paper claims that TCTO outperforms ten existing automated feature transformation methods on 16 classification and 9 regression datasets (Section V-A, Table I), and includes ablations, scalability experiments, robustness checks across downstream models, and a case study showing reuse of high-value sub-transformations.","tokens_in":19433,"tokens_out":3841,"duration_ms":37760,"significance":"If the empirical claims hold, TCTO would be a useful contribution to automated feature engineering: the idea of representing transformation history as a traceable graph and reusing high-utility subgraphs is well motivated, and the ablations in Section V-B support the value of the roadmap and clustering components. The paper is also honest about the time bottleneck (downstream evaluation) and includes a robustness check across multiple ML models. However, the central empirical claim of 'superior performance' is currently under-supported because baseline methods are reported without error bars, several reported margins are within one standard deviation of TCTO's own variance, and two important hyperparameters (pruning ratio and reward weight) are tuned on only two datasets and then applied universally. The pruning/backtracking mechanism is also under-specified, which matters because traceability is the paper's headline advantage. These issues are fixable within the manuscript's scope.","major_comments":[{"comment":"The node-wise pruning mechanism is underspecified in a way that threatens the paper's central claim of full traceability. The text first says the strategy 'will entail the identification of K nodes that show the greatest relevance to labels' and 'select top-K nodes by the score', but then states that it 'removes low-correlation nodes'—it is unclear whether the top-K are kept or removed. More importantly, the manuscript never specifies what happens to descendant nodes whose ancestor is pruned. Since each node's path back to the root is defined as its transformation pathway (Section II-A) and the roadmap generates a dataset via D' = G(D), deleting an ancestor leaves descendant feature columns undefined and breaks the claimed traceability. The RGCN aggregation in Eq. (4) would also reference non-existent neighbors if a pruned node is in the neighborhood. The paper needs to specify the graph-repair rule: are descendants recomputed from surviving ancestors, invalidated and removed, or retained with a broken path? Without this, the claimed advantages of safe node-wise pruning and step-wise backtracking are not established.","section":"Section III-D, Eq. (8)"},{"comment":"The central empirical claim of superior performance over ten baselines is not supported by the evidence as presented. Only TCTO is reported with a standard deviation (from 5 runs); all baselines are single numbers. For several datasets the margin over the second-best method is zero or within TCTO's own standard deviation (e.g., Openml 589: TCTO 0.606±0.003 vs FastFT 0.606; Ionosphere: TCTO 0.971±0.001 vs GRFG 0.971 and FastFT 0.971; ALBERT: TCTO 0.681±0.004 vs TTG 0.681). Without error bars or statistical significance tests for the baselines, the reported 'superior performance' cannot be distinguished from noise. Please provide variance estimates for all methods (or at least the strongest baselines) and, where possible, paired significance tests.","section":"Section V-A, Table I"},{"comment":"The global hyperparameters are selected on individual datasets and then applied to all 25 datasets without evidence of transferability, which weakens the generalizability claim. Section V-D sets the node-wise pruning ratio to 30% based on experiments on Airfoil and PimaIndia, and Section V-F sets the performance/complexity reward weight to 1:1 based on a 'preliminary experiment' on Airfoil only. These choices are then fixed for every dataset in Table I. If the hyperparameters are tuned on a subset and the evaluation is on the same datasets, the reported gains may not reflect the method's performance under a properly held-out hyperparameter selection. Please provide a sensitivity analysis across datasets or justify why the selected values are universally appropriate (e.g., by showing the trend is consistent across a broader set of datasets).","section":"Sections V-D and V-F"}],"minor_comments":[{"comment":"The sentence 'During step-wise pruning, we utilize the k most importance features' is unclear: k was previously defined as the number of clusters, and the pruning trigger is called K (capital). Please use distinct notation and define what 'most importance' means here.","section":"Section IV-D"},{"comment":"The split between node-wise and step-wise pruning is described inconsistently: Section III-D says 'first 30% of the exploration period' and 'remaining 70%', while Section IV-D says 'first 30% epochs' and 'remaining epochs'. Please align the wording.","section":"Section III-D"},{"comment":"The notation V = {v_i}_{i=1}^m and E = {e_i}_{i=1}^n uses m and n, while n is already used for the number of input features; this can confuse readers. Please use distinct indices or clarify the counts.","section":"Section II-A"},{"comment":"The footnote for ALBERT and Newsgroups says an asterisk indicates large datasets, but the table note only defines the '-' entries. Please define the asterisk explicitly in the caption.","section":"Table I"},{"comment":"The claim that TCTO 'consistently achieved the highest performance' is overstated: for Housing Boston with MLP, TCTO ties with FastFT at 0.310, and for Messidor with KNB, TCTO ties with FastFT at 0.587. Please soften the claim or note the ties.","section":"Section V-G, Table II"},{"comment":"The statement 'Our codes and data are publicly accessible via Dropbox' does not include a URL or repository identifier. For reproducibility, please provide a permanent link (e.g., GitHub, Zenodo).","section":"Section I"},{"comment":"In the time complexity analysis, 'Given a generative feature number k' reuses k from the clustering section; this is confusing because k already denotes the number of clusters. Please use a different symbol.","section":"Section III-E"}],"recommendation":"major_revision","confidential_remarks":"The paper is built heavily on the authors' own prior work (GRFG, FastFT, and related references [22]–[24], [9]), which is fine methodologically, but it also means the strongest baselines are internal comparisons; the lack of error bars on those baselines is concerning and should be a priority for the revision. The under-specified pruning mechanism is the main technical gap; I would not recommend acceptance until the graph-repair rule is clearly defined, since full traceability is the paper's headline contribution. The Dropbox claim without a link is also a reproducibility red flag for a journal submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TCTO is a solid incremental extension of the authors' earlier group-wise RL feature transformation line, with broad empirical coverage, but the pruning mechanism has a genuine underspecification that needs fixing before I'd fully trust the traceability claims.\n\nThe genuinely new parts are the mutual-information-based node pruning and step-wise backtracking on top of the traceable transformation roadmap. I give the authors credit for evaluating on 25 datasets, testing robustness across multiple downstream models, and showing a concrete case study where high-utility subgraphs are reused. The roadmap idea is useful, and the paper is clearly written. On most datasets TCTO beats the baselines, and the gains over GRFG and FastFT look consistent, though modest.\n\nThe soft spots are real but uneven. The biggest one, flagged by the stress-test, is Section III-D. The text says the framework 'selects top-K nodes by the score' and then 'removes low-correlation nodes,' which is ambiguous, and it never states what happens to descendants of a pruned node. Since each node's path to the root defines its feature column, deleting an ancestor makes surviving descendants undefined. The RGCN aggregation also assumes neighbors exist. This needs clarification: either pruning removes entire subtrees, or features are recomputed, or the roadmap is reconnected somehow. As written, the central traceability claim is not well-defined. That's a fixable but load-bearing issue.\n\nSecond, the baselines in Table I are reported without error bars; only TCTO has standard deviations. That makes it hard to know if the 0.01–0.02 improvements are noise. Third, the 30% node-wise pruning ratio and the 1:1 reward weight are tuned on Airfoil and PimaIndia, so the reported gains on those two datasets are not independent. The paper calls this 'preliminary,' but it still weakens the generality claim.\n\nAlso, the code link is a Dropbox URL and the reader reports it is not accessible. That is a practical hit to reproducibility. Novelty is incremental, with heavy self-citation, but that is not a flaw per se; the method does differ from their prior work in the pruning/backtracking mechanisms.\n\nOverall, this is a useful contribution to the automated feature engineering subfield. It deserves a real peer review, but with requests for major revision: specify the pruning semantics precisely, add error bars or significance tests for baselines, and make the code accessible. I'd be cautiously optimistic after those fixes.","headline":"TCTO is a credible incremental extension of the authors' own group-wise RL feature transformation work, with broad experiments and a genuine pruning underspecification that needs fixing; worth sending to review with major revision.","tokens_in":19994,"tokens_out":2745,"would_cite":true,"duration_ms":27301,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TCTO models feature transformation as an evolving, prunable roadmap navigated by three collaborating reinforcement-learning agents, and reports outperforming ten existing automated feature transformation methods across most of 16…","keywords":["automated feature transformation","multi-agent reinforcement learning","feature transformation roadmap","tabular data","graph pruning","backtracking","traceability"],"falsifier":"Run TCTO on any Table I dataset with an instrumentation that records, after each node-wise pruning event, whether any surviving feature's ancestry includes a deleted node. If even one surviving column cannot be recomputed from the pruned roadmap, or its trace path references a pruned ancestor, the traceability claim is refuted on that run. Separately, rerun the Table I benchmarks with node-wise pruning disabled; if the margin over FastFT disappears, pruning is the active cause rather than the roadmap memory.","tokens_in":18978,"feed_emoji":"🧭","tokens_out":9096,"duration_ms":80816,"temperature":0.7,"pith_summary":"Feature transformation—turning raw columns into informative combinations like a body-mass index from height and weight—is usually done manually or by search procedures that treat each transformation as an isolated step. TCTO instead keeps a persistent, evolving directed graph, called a feature transformation roadmap, where nodes are feature states and edges are the operations that created them. Three reinforcement-learning agents act on this roadmap in sequence: one chooses a group of features to transform, one chooses the mathematical operation, and one chooses the operand group; the roadmap is then updated, and low-value nodes or bad paths are pruned or rolled back. The paper reports that this graph-driven search outperforms ten established automated feature transformation methods on most of 16 classification and 9 regression benchmarks, while keeping every generated feature traceable to its construction path. If the claim holds, automated feature engineering becomes both more effective and more inspectable, with less manual effort and a clear record of how each feature was built.","feed_headline":"Evolving graph roadmap beats ten automated feature engineering baselines","feed_subtitle":"Three collaborating RL agents share a traceable roadmap to improve predictive accuracy over prior feature engineering methods.","key_machinery":"Feature transformation roadmap G: an evolving directed graph whose nodes are feature states and whose edges are the operations that generated them, with each node embedded from descriptive statistics of its feature column. This graph is the load-bearing object of the paper: it supplies the state representation for the agents, the history that lets the framework reuse valuable subtransformations, the structure that spectral clustering and a relational graph convolutional network operate on, and the target of the two pruning strategies (node-wise mutual-information pruning and step-wise backtracking to previous optimal roadmaps).","core_discovery":"The central discovery is that a feature transformation roadmap—a directed graph whose nodes are concrete feature states and whose edges are the unary or binary mathematical operations that produced them—can serve as the shared memory and search space for reinforcement-learning agents. On this roadmap, TCTO clusters nodes by their statistical and structural similarity, embeds them with a relational graph convolutional network, and lets three agents (head cluster, operation, operand cluster) propose new feature crossings. The paper argues that keeping the historical graph gives three advantages: transformations can be applied to features from any earlier stage, high-utility subgraphs can be reused, and the graph can be pruned (by mutual information) or rolled back (by step-wise backtracking) to keep exploration stable. Across 25 datasets, the reported F1 and 1-RAE scores place TCTO ahead of the ten baselines in most cases, with each generated feature described as an explicit combination of original and intermediate features.","pith_inferences":["If the roadmap stays coherent after pruning, TCTO's traceability could be extended into a full audit log: every generated column could be exported as a closed-form expression of the original inputs, which would matter for clinical or financial models where feature provenance is required.","The same three-agent choreography—pick a group, pick an operation, pick a partner group—could transfer to other sequential construction problems with reusable intermediate states, such as symbolic regression or query plan generation, wherever a persistent dependency graph can be maintained.","A clean way to isolate what causes the reported gains is to compare TCTO with the roadmap kept but node-wise pruning disabled; if the margin over FastFT disappears, pruning rather than graph memory is the active ingredient.","The description leaves the fate of descendant features unspecified when an ancestor node is pruned; a testable extension would make invalidation explicit—recompute, remove, or mark descendants—and measure how each choice changes downstream performance."],"forward_implications":["If the roadmap claim is right, feature search no longer starts from scratch after each step: proven intermediate features are kept in the graph and can be reused by later transformations instead of being rediscovered.","The two-phase schedule—node-wise pruning for the first 30% of training, then step-wise backtracking—gives a concrete recipe for balancing broad exploration with stable convergence in RL-driven search over structured state spaces.","The reward that penalizes deep transformation chains means TCTO is explicitly biased toward shallow, human-readable feature formulas, so the features it finds are not just predictive but also cheap to explain.","Traceable transformation paths turn every generated feature into an explicit formula over original inputs, which would let practitioners audit exactly how a high-value column was constructed.","The paper also observes that on very large-sample datasets feature transformation adds little, implying TCTO's practical value is concentrated on small- and medium-sized tabular problems."],"supporting_citations":[{"why":"FastFT is one of the ten baselines; TCTO's reported advantages are measured against its novelty-reward exploration.","marker":"[9]"},{"why":"FETCH is the single-agent RL baseline that motivates TCTO's multi-agent design and historical-memory claim.","marker":"[20]"},{"why":"GRFG is the cascading RL baseline whose group-wise crossing TCTO extends to a roadmap with pruning and backtracking.","marker":"[24]"},{"why":"TTG is the earlier graph-based transformation method that TCTO contrasts with by adding step-wise backtracking.","marker":"[26]"},{"why":"DIFER is the differentiable AutoML baseline that TCTO compares against as an unstable, non-traceable alternative.","marker":"[27]"},{"why":"OpenFE is the expansion-reduction baseline that represents the class of methods TCTO claims to surpass through learned exploration.","marker":"[28]"},{"why":"Spectral clustering supplies the eigenvalue-based clustering method TCTO uses to group roadmap nodes for group-wise transformation.","marker":"[33]"},{"why":"Relational graph convolutional networks provide the node-embedding mechanism TCTO uses to represent roadmap state for its agents.","marker":"[34]"},{"why":"NFS is the reinforcement-learning sequence-generation baseline that TCTO argues ignores historical transformation dependencies.","marker":"[40]"}],"fun_headline_variants":["Graph roadmap guides multi-agent feature engineering","Three RL agents share feature roadmap, beat ten baselines","Evolving graph prunes and reuses for better features","TCTO: collaborative agents optimize feature transformations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"After a node is pruned from the roadmap, every surviving feature built from it must still have a well-defined value and a reconstructible transformation history; if that fails, the roadmap's coherence and traceability collapse.","fun_headline_variants_meta":{"raw":{"variants":["Graph roadmap guides multi-agent feature engineering","Three RL agents share feature roadmap, beat ten baselines","Evolving graph prunes and reuses for better features","TCTO: collaborative agents optimize feature transformations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1293,"prompt_tokens":884,"completion_tokens":409,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":500,"tokens_out":409,"duration_ms":4280,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:42:31.307086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TCTO on any Table I dataset with an instrumentation that records, after each node-wise pruning event, whether any surviving feature's ancestry includes a deleted node. If even one surviving column cannot be recomputed from the pruned roadmap, or its trace path references a pruned ancestor, the traceability claim is refuted on that run. Separately, rerun the Table I benchmarks with node-wise pruning disabled; if the margin over FastFT disappears, pruning is the active cause rather than the roadmap memory.","supporting_citations":[{"cited_title":"Fastft: Accelerating reinforced feature transformation via advanced exploration strategies,","cited_arxiv_id":null,"evidence_quote":"FastFT is one of the ten baselines; TCTO's reported advantages are measured against its novelty-reward exploration."},{"cited_title":"Learning a data-driven policy network for pre-training automated fea- ture engineering,","cited_arxiv_id":null,"evidence_quote":"FETCH is the single-agent RL baseline that motivates TCTO's multi-agent design and historical-memory claim."},{"cited_title":"Traceable group-wise self-optimizing feature transformation learning: A dual optimization perspective,","cited_arxiv_id":null,"evidence_quote":"GRFG is the cascading RL baseline whose group-wise crossing TCTO extends to a roadmap with pruning and backtracking."},{"cited_title":"Feature engineering for predictive modeling using reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"TTG is the earlier graph-based transformation method that TCTO contrasts with by adding step-wise backtracking."},{"cited_title":"Difer: differentiable automated feature engineering,","cited_arxiv_id":null,"evidence_quote":"DIFER is the differentiable AutoML baseline that TCTO compares against as an unstable, non-traceable alternative."},{"cited_title":"OpenFE: Automated feature generation with expert-level performance,","cited_arxiv_id":null,"evidence_quote":"OpenFE is the expansion-reduction baseline that represents the class of methods TCTO claims to surpass through learned exploration."},{"cited_title":"Neural feature search: A neural architecture for automated feature engineering,","cited_arxiv_id":null,"evidence_quote":"NFS is the reinforcement-learning sequence-generation baseline that TCTO argues ignores historical transformation dependencies."}],"review_version":1}