SiriusDeliver: Automating Data Warehouse Delivery at Tencent

Debiao He; Haining Xie; Jiaming Yang; Jiawei Jiang; Jie Jiang; Peng Chen; Siqi Shen; Tengyue Xu; Xiaokai Zhou; Xiao Yan

REVIEW 2 major objections 6 minor 38 references

SiriusDeliver: Automating Data Warehouse Delivery at Tencent

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SiriusDeliver claims full automation of data warehouse delivery, raising offline success from 71.5% to 86.0% and cutting median delivery time from 228 to 23 minutes in production, all while keeping final delivery quality steady.

desk verdict Credible industrial agent system for warehouse delivery; offline evidence is well controlled, but the production A/B test needs tighter measurement before the cost-reduction claims are accepted. read the letter →

arxiv 2608.09185 v1 pith:AD3ZNYF4 submitted 2026-08-10 cs.DB cs.AIcs.SE

Haining Xie , Xiaokai Zhou , Jiaming Yang , Siqi Shen , Ziwei Wang , Yifeng Zheng , Tengyue Xu , Yipeng Shi

show 7 more authors

Zefang Zong Yang Li Peng Chen Jie Jiang Debiao He Xiao Yan Jiawei Jiang

This is my paper · ORCID

classification cs.DBcs.AIcs.SE

keywords datawarehousetaskdeliveryautomationagentLLMagentshierarchicalskillorchestrationartifactlifecyclecontroltrace-drivenevolutionproductiondeploymentOps

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SiriusDeliver is a system that attempts to automate the entire enterprise data warehouse delivery process—turning a business request into a validated, submittable bundle of workflow definition, code, and configuration—rather than merely generating SQL. The paper argues that production delivery needs three elements that general coding agents lack: a hierarchy of reusable warehouse skills composed step by step by an agent, a lifecycle control stage that verifies artifacts before submission and diagnoses failures after execution, and an offline mechanism that converts past delivery trajectories into bounded, human-reviewed skill updates. The reported evidence is a 14.5-point offline gain in end-to-end delivery success over the strongest skill-augmented coding-agent baseline, an 87.2% success rate and 73.5% autonomous submission rate across 18,240 production sessions, and an A/B test in which median delivery time fell from 228 to 23 minutes and engineer effort from 95 to 11 minutes with final success essentially unchanged. If true, the most labor-intensive part of data engineering—coordinating context, dependencies, permissions, and schedules—could be handed to an agent that keeps improving with use.

What carries the argument

The load-bearing object is the artifact bundle $\langle G, C, P \rangle$—workflow specification, executable artifacts, and task configurations treated as jointly validated units rather than free-standing code—together with a four-layer skill hierarchy (scenario, context, artifact, and platform skills) that the delivery agent composes in a closed loop with a manager, an executor, and a memory. Lifecycle control carries the reliability argument: pre-execution diagnosis combines deterministic platform evidence (metadata, dependencies, pre-compilation results) with LLM reasoning over defined diagnosis items, and post-execution diagnosis converts platform feedback into extracted failure signals plus retrieved diagnostic knowledge before any revision is generated. Trace-driven skill evolution carries the adaptation argument: trajectories grouped by skill and outcome are compressed into structured evidence, LLM updates are bounded by an update budget, validated by offline replay, and gated by human review with rollback.

What would settle it

Instrument the manual-workflow control arm with the same automated session-timestamp logging used for the agent arm—start-of-request through final submission—and log engineer effort by the same interaction-tracking method across a comparable set of delivery tasks. If matched-instrumentation medians come out well below 228 minutes for delivery time or well below 95 minutes for effort, the headline A/B reductions would not survive; a complementary check is evaluating the three components on the paper's public companion benchmark, which omits the scheduling, permission, and lifecycle workflows of the private platform.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that warehouse delivery automation fails when it is modelled as code generation and succeeds when it is modelled as a closed loop over four layers of skills—scenario, context, artifact, and platform—governed by a delivery state, verified before and after platform execution, and maintained over time from recorded trajectories. The unit of delivery is the artifact bundle $\langle G, C, P \rangle$ (workflow specification, executable artifacts, task configurations), which is produced by the orchestration loop, passes through pre-execution diagnosis (evidence grounding, hybrid verification, candidate revision), is submitted, and, if it fails, is repaired by post-execution diagnosis that extracts failure signals, retrieves diagnostic knowledge, and generates a grounded revision. Trace-driven skill evolution then groups trajectories by skill and outcome, applies bounded LLM updates for failure repair and success compression, validates them on held-out trajectories, and releases them only after human review with rollback. The reported result is that offline end-to-end delivery success rises from 71.5% to 86.0% over the strongest baseline while token use falls 30%, and in production the system attains an 87.2% end-to-end success rate with 73.5% of submissions fully autonomous, with the A/B test showing delivery time down 89.9% and engineer effort down 88.4% at comparable final quality (97.6% versus 98.1%).

Load-bearing premise

The A/B test's headline reductions assume that 'engineer effort' (95 to 11 minutes) and 'delivery time' (228 to 23 minutes) were measured with the same method in both arms, but Section 4.3 reports the manual-workflow control figures without defining how they were logged, so if manual effort was self-estimated or tracked differently from the agent's session timestamps, the 89.9% and 88.4% reductions could be measurement artifacts.

Editorial extensions

If this is right

Warehouse delivery can be largely automated without sacrificing final quality: the paper's production figures put end-to-end success at 87.2% with 73.5% of sessions submitted without manual artifact modification, and the A/B test holds final success at 97.6% versus 98.1%.
The gains are architectural rather than model-specific: paired tests across four LLM backbones show 12.0 to 16.0 percentage-point success gains, with the weakest backbone gaining the most, implying structured orchestration substitutes for raw reasoning ability.
Lifecycle control converts most execution failures into single-shot repairs: 73.0% of failed executions were repaired within one diagnosis-guided revision and 75.9% of real artifact issues were caught before submission, at latencies of 24.2 seconds and 6.1 seconds respectively.
The system becomes cheaper as it is used: trace-driven skill evolution cut token use by 25.5% and reasoning turns by 17.5% on held-out cases, and produced 128 human-approved skill updates during two months of production.
The largest single lever is hierarchical skill orchestration: removing it drops offline success by 11.5 points, more than removing lifecycle control or skill evolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same three-component recipe—hierarchical skill orchestration, pre- and post-artifact verification, and trace-driven skill updates—should transfer to other approval-gated production workflows such as CI/CD pipeline configuration, infrastructure-as-code changes, and ML model deployment, which share the generate-submit-verify-repair shape; this is an extrapolation, not a claim the paper makes.
The backbone results suggest that structured orchestration and deterministic verification act as a capability floor, so a testable corollary is that the per-backbone gain should persist on newer models released after this evaluation.
The A/B test's 21% residual manual intervention rate and the offline 78% success on offline computation tasks jointly suggest that the remaining automation ceiling is the hardest task class; whether skill evolution erodes that ceiling over many months is an open question the two-month window cannot settle.
Because the private 200-case benchmark cannot be released, the portability of the diagnosis components cannot be verified publicly; building the pre-execution partition-scan and post-execution log-diagnosis checks on a public warehouse schema would test whether the gain generalizes beyond the system's home platform.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid industrial systems paper, and the strongest evidence is the offline paired evaluation, not the A/B test. The headline production numbers are believable but less rigorously supported than they look.

What's new: applying agent techniques—hierarchical skill orchestration, pre/post-execution artifact validation, trace-driven skill evolution—to the warehouse delivery problem at Tencent scale. Individually the components resemble existing agent patterns (Reflexion, Voyager, etc.), but the combination for DW delivery, with a concrete skill hierarchy and artifact lifecycle control, is genuinely engineered and evaluated at a scale that matters. The offline evaluation is well controlled: shared skill library, frozen snapshots, disjoint evolution trajectories, per-product paired comparisons, and ablations that isolate each component. The 14.5-point success gain over the strongest baseline is credible. The backbone study showing consistent gains across different models is a nice touch.

The soft spots are real but concentrated. First, the A/B test randomizes 70 engineers, not tasks. Treatment users interact with a web interface and can pick which requests to route through the agent, so task difficulty may differ between arms. The paper reports no task-complexity covariates, and stratified randomization by team and task type doesn't fix that. Second, engineer effort in the control arm is never defined: is it self-estimated, logged, or derived from timestamps? Without that, the 95-to-11 minute effort reduction is hard to interpret. These don't invalidate the offline results, but they weaken the production cost claims. Third, the benchmark is internal and proprietary; the public DataClawEval companion is useful but doesn't test the actual system. Success adjudication is human verification, which is reasonable.

I'd send this to peer review. The architecture is coherent, the offline methodology is above the usual bar for systems papers, and the production deployment gives the community real grounding. The A/B test needs a major revision—task comparability analysis, effort measurement definition, ideally a causal or at least stratified analysis—but that's fixable, not fatal.

Referee Report

2 major / 6 minor

Summary. SiriusDeliver is an end-to-end agent for automating production data warehouse task delivery at Tencent. The paper proposes three components: a hierarchical delivery agent that orchestrates warehouse skills, an artifact lifecycle control module that verifies and revises artifacts before and after platform execution, and a trace-driven skill evolution mechanism that converts delivery trajectories into bounded skill updates. The evaluation has three parts: an offline benchmark of 200 real production cases across four warehouse task types, a two-month production deployment serving 3,600 monthly active users and 18,240 delivery sessions, and a one-month A/B test with 70 engineers. The paper claims a 14.5-point improvement in offline end-to-end delivery success over the strongest skill-augmented baseline (71.5% to 86.0%), an 87.2% end-to-end success rate and 73.5% autonomous submission rate in deployment, and, from the A/B test, reductions in median delivery time from 228 to 23 minutes, engineer effort from 95 to 11 minutes, and manual intervention from 100% to 21%, while final delivery success remains comparable (97.6% vs. 98.1%).

Significance. If the reported results hold, SiriusDeliver would provide strong evidence that production data warehouse delivery can be largely automated without sacrificing final quality. The offline evaluation is unusually well controlled: within each product pair the skill library, platform tools, execution environment, and refinement budget are fixed; evaluation snapshots are frozen; skill-evolution trajectories are disjoint from evaluation cases; and success is adjudicated by the platform and manual verification. The ablation study and the multi-backbone paired evaluation strengthen the internal validity of the offline claims. The production deployment numbers are large and the A/B test addresses a relevant practical question. However, the internal validity of the A/B test is not established as reported, and this A/B test is the load-bearing evidence for the headline cost-reduction claims. The paper also provides a public companion benchmark, though it is explicitly not used to produce the reported results, which limits external reproducibility. On balance, the contribution is significant if the A/B test concerns can be resolved.

major comments (2)

[§4.3 (Online Deployment, A/B test, Figure 8)] The A/B test is randomized at the user level, not at the task level, and treatment-group engineers choose which requests to route through the SiriusDeliver web interface. The 718 treatment tasks and 682 control tasks are therefore not shown to be comparable in difficulty; no task-complexity covariates (e.g., number of dependencies, table count, artifact length, or historical manual delivery time for equivalent requests) are reported for either arm. Under these conditions, the reported 89.9% reduction in median delivery time and 88.4% reduction in engineer effort, as well as the near-equal final success rates (97.6% vs. 98.1%), could reflect systematic differences in workload composition rather than a causal effect of SiriusDeliver. Please provide evidence that the task mix is balanced across arms, or analyze the outcomes within predefined task-difficulty strata, or redesign the experiment so that tasks themselves are randomly assigned.
[§4.3 (Online Deployment, Figure 8)] The 'engineer effort' metric is not defined. The paper reports median engineer effort of 95 minutes in the control group and 11 minutes in the treatment group, but it does not state how effort was collected for engineers performing manual delivery (e.g., self-reported estimates, activity logs, or time tracking) or what 'net' excludes. If the two arms used different measurement instruments, the 88.4% reduction is not interpretable. The same ambiguity applies to 'time to first executable artifact' and 'interaction rounds.' Please provide precise definitions and the data-collection protocol for each cost metric, ideally with a demonstration that the measurement methods are consistent across arms.

minor comments (6)

[§4.3 (Online Deployment)] The 'manual intervention rate' is 100% in the control group by construction, so the reduction to 21% is not informative; please clarify what counts as an intervention and report absolute rates for each intervention type (clarification, artifact edit, manual submission, manual repair, and so on).
[§4.1 (Dataset) and Appendix A.2] The public companion benchmark DataClawEval is explicitly not used for any reported result and does not model WeData-specific workflows; the paper should state more prominently that the main offline results are not externally reproducible with the released artifact, or provide additional released components that can reproduce a subset of the claims.
[§4.2 (End-to-end performance, Table 2)] The per-backbone success rates in Table 2 are reported as integers without confidence intervals; given the sample size of 200 cases, the 12-to-16-point gains should be accompanied by uncertainty estimates or a note that the comparisons are descriptive rather than inferential.
[Figure 6 and §4.2 (Effect of skill evolution)] The skill-evolution evaluation reports token usage and reasoning turns for the original and evolved skill libraries, but it does not report end-to-end success rates on the held-out set for both conditions; please include success rates (or an explicit statement that they are unchanged) so the efficiency gains can be interpreted without concern that evolution achieves lower cost by failing more often.
[Appendix A.3 (LLM-as-Judge Protocol)] The LLM-as-judge protocol for artifact quality uses three judges and position swapping, but the paper does not report the per-judge agreement rate or the number of 'inconsistent' outcomes; a high inconsistency rate would weaken the claim that evolved artifacts are preferred or comparable in 79.2% of cases, so please report these quantities.
[Throughout] There are several small presentation issues: 'RT Sync' and 'Off. Sync' are not expanded in Table 1; the sentence in §4.2 beginning 'these results show that lifecycle control Artifact lifecycle control improves' contains a duplicated phrase; and the phrase 'one-month A/B test' in the abstract is not consistent with the deployment timeline described in §4.3, so please clarify the relationship between the two.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are supported by external benchmarks, production logs, and an A/B test, not by construction or self-citation.

full rationale

SiriusDeliver is an empirical systems paper whose claimed benefits are measured against a frozen 200-case offline benchmark, large-scale production logs, and a one-month A/B test. The offline metric 'end-to-end delivery success rate' is defined independently of the system's own judgments: a case succeeds only if the artifact bundle is platform-accepted, executable, and passes manual verification by three engineers with a predefined checklist. The baselines share the same skill library, platform tools, and refinement budgets, and the reported gains are obtained by comparing configurations rather than by deriving results from the system's own definitions. The skill-evolution evaluation uses a stated 50% training / 50% held-out split, and artifact quality is judged by three independent LLM judges with position swapping and majority voting, which is not a self-referential scoring protocol. The only self-citation, reference [21] (SiriusHelper), is used in Section 3.2 as background for log-grounded diagnosis alongside external works [4,27]; it is not invoked as a uniqueness theorem, as the source of the evaluation protocol, or as evidence that SiriusDeliver works, so it is not load-bearing. The A/B test's user-level randomization raises a legitimate internal-validity concern about task-mix comparability between the 718 treatment and 682 control tasks, but that is an empirical measurement concern, not a circularity reduction: no metric in the paper is defined in terms of the outcome it is used to predict, and no fitted parameter is renamed as a prediction. The central derivation chain therefore remains self-contained with respect to its inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on engineering assumptions rather than fitted parameters. No numbers are tuned to make a derivation work; the reported rates are measured outcomes. The paper assumes LLMs can generate and repair warehouse artifacts when grounded in the right context, that platform acceptance plus in-house manual verification is a valid success measure, and that historical delivery trajectories are representative enough for skill evolution. No invented unobserved entities are required.

assumptions (3)

domain assumption LLMs can produce delivery-submittable artifacts when orchestrated with sufficient context and verification.
The whole system's artifact generation and repair rely on LLM capabilities; Sections 3.1 and 3.2 assume grounded generation plus hybrid verification is enough to yield executable bundles.
domain assumption Platform acceptance plus manual engineer verification is a valid proxy for warehouse delivery success.
Success adjudication uses a predefined checklist and three engineers (Appendix A.2), but the checklist is unpublished and the evaluators are within Tencent.
domain assumption Historical delivery trajectories are representative of future delivery requests and can be used for skill evolution without leakage.
Trace-driven skill evolution and evaluation assume disjointness and stability of task distributions; Sections A.1 and Figure 6 rely on this premise.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SiriusDeliver: Automating Data Warehouse Delivery at Tencent." pith.science (2026). https://pith.science/paper/AD3ZNYF4

@misc{pith2026260809185,
  author       = {Pith},
  title        = {Pith review of: SiriusDeliver: Automating Data Warehouse Delivery at Tencent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AD3ZNYF4}},
  note         = {Machine review of arXiv:2608.09185}
}

read the original abstract

Enterprise data warehouses (DWs) support business-critical analytics, but warehouse task delivery remains a complicated production process involving context retrieval, workflow configuration, code generation, platform submission, and failure diagnosis. Although large language models (LLMs) and coding agents have improved software development, they are insufficient for production DW delivery, which requires dependency-aware orchestration, lifecycle-aware artifact control, and continuous adaptation to evolving platform practices. We present SiriusDeliver, an end-to-end delivery automation agent for production warehouse task submission. SiriusDeliver integrates three components: a hierarchical delivery agent that orchestrates warehouse skills, an artifact lifecycle control module that verifies and revises artifacts before and after platform execution, and a trace-driven skill evolution mechanism that maintains reusable skills from delivery trajectories. We evaluate SiriusDeliver through offline datasets and large-scale production deployment on Tencent Cloud WeData. Offline experiments on real-world warehouse delivery cases show that SiriusDeliver improves delivery success and automation efficiency over representative baselines. During a two-month deployment across 6 business teams and 4 warehouse task types, SiriusDeliver served 3,600 monthly active users and supported 18,240 delivery sessions, achieving an 87.2% end-to-end success rate and a 73.5% autonomous submission rate. A one-month A/B test shows that SiriusDeliver reduces median delivery time from 228 to 23 minutes and engineer effort from 95 to 11 minutes, while maintaining comparable final delivery success.

Figures

Figures reproduced from arXiv: 2608.09185 by the authors.

**Figure 2.** Overview of key techniques in SiriusDeliver. • We evaluate SiriusDeliver on 200 real-world cases and in largescale production deployment, achieving an 87.2% end-to-end success rate while reducing median delivery time and engineer effort by 89.9% and 88.4%, respectively. 2 Related Work Data warehouse platforms. Modern cloud-native data platforms (e.g., Snowflake [5], BigQuery [13], and Delta Lake [3]) provide highp… view at source ↗

**Figure 3.** Orchestration loop of the delivery automation agent. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Artifact lifecycle control. Artifacts undergo pre [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Trace-driven skill evolution. Raw trajectories are [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Effect of trace-driven skill evolution. Left: token [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 8.** Figure 8: Online A/B test comparing the manual workflow [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

**Figure 9.** Figure 9: Case study of pre-execution diagnosis for an ex [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

**Figure 11.** Figure 11: Case study of trajectory-driven skill refinement [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

**Figure 12.** Figure 12: Case study of end-to-end delivery of an offline computation task. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]

**Figure 13.** Figure 13: Production web interface of SiriusDeliver [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 32 canonical work pages

[1]

Anthropic. 2024. Introducing Claude 3.5 Sonnet. https://www.anthropic.com/ index/claude-3-5-sonnet. Accessed: 2026-06-30

work page 2024
[2]

Anthropic. 2026. Claude Code overview. https://docs.anthropic.com/en/docs/ claude-code/overview. Accessed: 2026-06-30

work page 2026
[3]

Michael Armbrust, Tathagata Das, Sameer Paranjpye, Reynold Xin, Shixiong Zhu, Ali Ghodsi, Burak Yavuz, Mukul Murthy, Joseph Torres, Liwen Sun, Peter Boncz, Mostafa Mokhtar, Herman Van Hovell, Adrian Ionescu, Alicja Luszczak, Michal Switakowski, Takuya Ueshin, Xiao Li, Michal Szafranski, Pieter Senster, and Matei Zaharia. 2020. Delta Lake: High-Performance...

work page 2020
[4]

Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, et al. 2024. Automatic root cause analysis via large language models for cloud incidents. InProceedings of the Nineteenth European Conference on Computer Systems. 674–688

work page 2024
[5]

Lee, Ashish Motivala, Abdul Q

Benoît Dageville, Thierry Cruanes, Marcin Zukowski, Vadim Antonov, Artin Avanes, Jon Bock, Jonathan Claybaugh, Daniel Engovatov, Martin Hentschel, Jiansheng Huang, Allison W. Lee, Ashish Motivala, Abdul Q. Munir, Steven Pelley, Peter Povinec, Greg Rahn, Spyridon Triantafyllis, and Philipp Unterbrunner. 2016. The Snowflake Elastic Data Warehouse. InProceed...

work page 2016
[6]

Databricks. 2026. Databricks assistant: your context-aware ai assistant. https: //www.databricks.com/product/databricks-assistant. Accessed: 2026-06-30

work page 2026
[7]

DeepSeek-AI. 2026. DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning. arXiv:2501.12948 [cs.CL] doi:10.1038/s41586-025-09422-z

arXiv 2026
[8]

Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jin- gren Zhou. 2023. Text-to-sql empowered by large language models: a benchmark evaluation.Proceedings of the VLDB Endowment(2023)

work page 2023

Show all 38 references

[9]

Gemini Team. 2024. Gemini 1.5: unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530 [cs.CL] https://arxiv.org/abs/ 2403.05530

2024 arXiv
[10]

GitHub. 2026. GitHub Copilot: your AI pair programmer. https://github.com/ features/copilot. Accessed: 2026-06-30

2026
[11]

Google Cloud. 2026. Write queries with gemini Aassistance in bigquery. https: //cloud.google.com/bigquery/docs/write-sql-gemini. Accessed: 2026-06-30

2026
[12]

Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, et al. 2025. Data interpreter: an llm agent for data science. InFindings of the Association for Computational Linguistics: ACL 2025. 19796–19821

2025
[13]

Sergey Melnik, Andrey Gubarev, Jing Jing Long, Geoffrey Romer, Shiva Shiv- akumar, Matt Tolton, and Theo Vassilakis. 2010. Dremel: Interactive Analysis of Web-Scale Datasets. InProceedings of the VLDB Endowment, Vol. 3. 330–339

2010
[14]

OpenAI. 2024. GPT-4 technical report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

2024 arXiv
[15]

OpenAI. 2025. OpenAI codex. https://openai.com/codex/. Accessed: 2026-06-30

2025
[16]

Patil, Tianjun Zhang, Xin Wang, and Joseph E

Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2024. Gorilla: Large Language Model Connected with Massive APIs.International Conference on Neural Information Processing Systems(2024)

2024
[17]

Mohammadreza Pourreza and Davood Rafiei. 2023. Din-sql: decomposed in- context learning of text-to-sql with self-correction.International Conference on Neural Information Processing Systems(2023)

2023
[18]

Bo Qiao, Liqun Li, Xu Zhang, Shilin He, Yu Kang, Chaoyun Zhang, Fangkai Yang, Hang Dong, Jue Zhang, Lu Wang, Minghua Ma, Pu Zhao, Si Qin, Xiaoting Qin, Chao Du, Yong Xu, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang
[19]

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ToolLLM: facilitating large language model...

2024
[20]

Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: language models can teach themselves to use tools.International Conference on Neural Information Processing Systems(2023)

2023
[21]

Yu Shen, Shiyang Liu, Qihang He, Yihang Cheng, Haining Xie, Zhiming He, Huahua Fan, Xianzhi Tan, Teng Ma, Shaoquan Zhang, et al. 2026. SiriusHelper: an LLM agent-based operations assistant for big data platforms.arXiv preprint arXiv:2605.00043(2026)

2026 arXiv
[22]

Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning.International Conference on Neural Information Processing Systems(2023)

2023
[23]

Snowflake. 2026. Cortex analyst. https://docs.snowflake.com/en/user-guide/ snowflake-cortex/cortex-analyst. Accessed: 2026-06-30

2026
[24]

Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. Chess: contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755(2024)

2024 arXiv
[25]

Tencent Cloud. 2026. Tencent cloud code assistant CodeBuddy: product overview. https://www.codebuddy.cn. Accessed: 2026-06-30

2026
[26]

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An Open-Ended Embodied Agent with Large Language Models.Transactions on Machine Learning Research (2023)

2023
[27]

Weiyuan Xu, Juntao Luo, Tao Huang, Kaixin Sui, Jie Geng, Qijun Ma, Isami Akasaka, Xiaoxue Shi, Jing Tang, and Peng Cai. 2025. LogSage: an LLM-based framework for CI/CD failure detection and remediation with industrial validation. IEEE/ACM International Conference on Automated ...

2025
[28]

Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: agent-computer interfaces enable automated software engineering. InInternational Conference on Neural Information Processing Systems

2024
[29]

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev
[30]

Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. Au- toCodeRover: autonomous program improvement. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis

2024
[31]

etl_task.py

Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM agents are experiential learners.Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence(2024). Conference’17, July 2017, Washington, DC, USA Xie et al. A Exp...

2024
[34]

•Generate a daily detail table for cross-day Ray notebook executions

Business Intent and Task RoutingBusiness Requirement: Task Router:New offline SQL task using Spark-compatible SuperSQL. •Generate a daily detail table for cross-day Ray notebook executions. Split each execution at natural-day boundaries, calculate instance runtime, code runtim...
[35]

ArtifactGenerationand ConfigurationExecutable Artifact:SuperSQL is generated to split cross-day intervals, aggregate execution durations, resolve service-to-pod mappings, and join minute-level GPU metrics. Task Configuration:A new single-node offline task is configured with th...
[36]

Pre-execution DiagnosisEvidence Grounding:The generated code is checked against table schemas, preview data, partition semantics, and the task configuration.Hybrid Verification:Syntax, referenced fields, target mapping, and submission settings are jointly verified.Decision:Pas...
[37]

Post-execution DiagnosisFailure Extraction:Platform feedback reports a column-count mismatch and invalid arithmetic on STRING values.Knowledge Retrieval:Relevant historical cases, diagnostic rules, and troubleshooting knowledge are retrieved. Root-cause-based Revision:Remove d...
[38]

Context Planning and Skill OrchestrationContext Acquisition:Schemas and previews are retrieved for five source tables and the target table. Field definitions and example values are combined into Light Schemas.Skill Orchestration:The delivery agent invokes metadata retrieval, S...
[2018]

InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

Spider: a large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3911–3921

2018
[2023]

Taskweaver: a code-first agent framework.arXiv preprint arXiv:2311.17541 (2023)

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Anthropic. 2024. Introducing Claude 3.5 Sonnet. https://www.anthropic.com/ index/claude-3-5-sonnet. Accessed: 2026-06-30

work page 2024

[2] [2]

Anthropic. 2026. Claude Code overview. https://docs.anthropic.com/en/docs/ claude-code/overview. Accessed: 2026-06-30

work page 2026

[3] [3]

Michael Armbrust, Tathagata Das, Sameer Paranjpye, Reynold Xin, Shixiong Zhu, Ali Ghodsi, Burak Yavuz, Mukul Murthy, Joseph Torres, Liwen Sun, Peter Boncz, Mostafa Mokhtar, Herman Van Hovell, Adrian Ionescu, Alicja Luszczak, Michal Switakowski, Takuya Ueshin, Xiao Li, Michal Szafranski, Pieter Senster, and Matei Zaharia. 2020. Delta Lake: High-Performance...

work page 2020

[4] [4]

Yinfang Chen, Huaibing Xie, Minghua Ma, Yu Kang, Xin Gao, Liu Shi, Yunjie Cao, Xuedong Gao, Hao Fan, Ming Wen, et al. 2024. Automatic root cause analysis via large language models for cloud incidents. InProceedings of the Nineteenth European Conference on Computer Systems. 674–688

work page 2024

[5] [5]

Lee, Ashish Motivala, Abdul Q

Benoît Dageville, Thierry Cruanes, Marcin Zukowski, Vadim Antonov, Artin Avanes, Jon Bock, Jonathan Claybaugh, Daniel Engovatov, Martin Hentschel, Jiansheng Huang, Allison W. Lee, Ashish Motivala, Abdul Q. Munir, Steven Pelley, Peter Povinec, Greg Rahn, Spyridon Triantafyllis, and Philipp Unterbrunner. 2016. The Snowflake Elastic Data Warehouse. InProceed...

work page 2016

[6] [6]

Databricks. 2026. Databricks assistant: your context-aware ai assistant. https: //www.databricks.com/product/databricks-assistant. Accessed: 2026-06-30

work page 2026

[7] [7]

DeepSeek-AI. 2026. DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning. arXiv:2501.12948 [cs.CL] doi:10.1038/s41586-025-09422-z

arXiv 2026

[8] [8]

Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jin- gren Zhou. 2023. Text-to-sql empowered by large language models: a benchmark evaluation.Proceedings of the VLDB Endowment(2023)

work page 2023