Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

OpRAG: A Resource-Deterministic Runtime for GPU-Backed Multi-Stage RAG Workflows

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new runtime models RAG stages as schedulable operators and shows that fixing orchestration overhead—not the decoding kernel—delivers 16% end-to-end speedups on GPU-backed retrieval-augmented generation.

desk verdict The speedup is real but entirely embedding-stage; the resource-deterministic runtime's contribution goes unproven, though the operator model is worth discussing. read the letter →

arxiv 2608.08340 v1 pith:MX2YAX5K submitted 2026-08-08 cs.DC

Arup Kumar Sarker , Mills Staylor , Aymen Alsaadi , Gregor von Laszewski , Shantenu Jha , Geoffrey Fox This is my paper · ORCID
classification cs.DC
keywords retrieval-augmentedgenerationdistributedruntimeGPUorchestrationoperatorgraphzero-copydataplaneboundedqueuesCPU-GPUoverlapRAGpipelineoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OpRAG argues that in modern retrieval-augmented generation, the slow part is not the language model decoding on the GPU but the orchestration around it: tokenizing, embedding, retrieving, building context, updating indexes, and moving data between CPUs and GPUs. The paper's proposal is a runtime that treats these stages as first-class operators with deterministic execution semantics, pairing them with persistent workers, bounded queues, a zero-copy data plane, CPU tokenizer prefetching, length-bucketed GPU embedding, and overlapped retrieval/generation. In end-to-end tests on two A100 GPUs with Llama3-8B and Mistral-7B over 32K chunks, OpRAG reports 16.16% and 15.66% improvements over the nearest competitor, about 20.6% over a distributed task-parallel baseline, and 52.48–53.55% lower latency in a query-serving scenario while holding Recall@5 at 1.0. The authors attribute these gains to reducing the overhead term $\Omega$ in a simple batch-stage cost model, not to any change in the decoding kernel. If the attribution is right, RAG system builders can get large end-to-end speedups by scheduling the dataflow around the model rather than by optimizing the model itself.

What carries the argument

The central object is the operator graph with resource-deterministic execution semantics: each RAG stage is an operator $O_i=(I_i,O_i,f_i,P_i,R_i)$ carrying input/output schemas, a function, a communication pattern, and a resource domain. Dynamic behavior is handled by compiling agent-chosen workflow segments $W_t$ into graphs $G_t=\mathrm{Compile}(W_t,S_t)$ that run deterministically, so the runtime never plans an entire conversation. The cost argument rests on the batched-stage model $T_{\mathrm{stage}}\approx \frac{N}{bP}(\alpha+\beta b)$ and the total $T\approx \frac{N\beta}{P}+\frac{N\alpha}{bP}+\Omega$, where $\Omega$ is the scheduler, serialization, and synchronization overhead that OpRAG claims to reduce through zero-copy exchange, persistent workers, bounded queues, and overlapped execution whose steady-state critical path approaches $\max(T_{\mathrm{cpu\_prep}},T_{\mathrm{gpu\_embed}},T_{\mathrm{retrieve}}+T_{\mathrm{context}},T_{\mathrm{generate}},T_{\mathrm{upsert}})+\Omega$.

What would settle it

Run the same two-GPU, 32K-chunk workload with OpRAG's length-bucketed batching and compiled embedding path disabled, or add those embedding optimizations to the nearest baseline; if the end-to-end gap collapses, the orchestration mechanisms are not carrying the reported gains, and if it persists, the resource-deterministic runtime is the source of the improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that multi-stage RAG is a distributed systems problem, and that the orchestration layer's overhead—captured as $\Omega$ in the cost model $T\approx \frac{N\beta}{P}+\frac{N\alpha}{bP}+\Omega$—is a major bottleneck that can be attacked without changing the LLM decoding kernel. OpRAG formalizes embedding, retrieval, reasoning, memory, and upsert as operators with explicit schemas, communication patterns, and resource domains, compiles dynamic agent decisions into deterministic execution segments, and executes them over a zero-copy data plane with persistent workers, bounded queues, and CPU/GPU overlap. The reported steady-state pipeline times drop from roughly 152–153 seconds of embedding plus orchestration in baselines to 126–128 seconds in OpRAG, with generation time nearly unchanged, which the authors take as direct evidence that the gains come from the orchestration path rather than from faster decoding.

Load-bearing premise

The measured speedups are caused by the runtime's orchestration mechanisms, such as bounded queues, persistent workers, zero-copy exchange, and CPU/GPU overlap, rather than by the embedding-stage optimizations that any baseline could adopt on its own.

Editorial extensions

If this is right

  • End-to-end GPU RAG pipelines can improve by roughly 16% over the closest non-operator baseline and by over 20% against a distributed task-parallel baseline, with the LLM decoding kernel untouched.
  • The dominant non-model cost in these workloads is the embedding stage; length-bucketed batching and a compiled, CUDA-graph-friendly embedding path transfer the bulk of the saving and appear to carry over across different generative models.
  • Query-serving latency can be cut by roughly 59% for hybrid retrieval and 52–53% for retrieval-plus-generation scenarios while holding Recall@5 at 1.0, because retrieval becomes an operator with vectorized dense scoring, postings-based BM25, and partial top-k selection.
  • The same runtime mechanisms (bounded queues, persistent workers, operator-level scheduling) generalize to CPU-only ingestion, where the paper reports a 4.64× pipeline improvement and stable weak scaling to 1024 physical workers.
  • Workloads dominated almost entirely by long generation will see smaller relative gains, since the orchestration terms being reduced contribute little to a generation-bound critical path.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the reported savings concentrate in the embedding stage, the strongest test of the paper's orchestration claim is an ablation that disables length-bucketed batching and the compiled embedding path while keeping the operator runtime intact; until that ablation exists, the headline numbers could be substantially explained by embedding-batching improvements that any basel
  • Editorial inference: the results suggest a concrete portability claim the paper does not make: adding length-bucketed, CUDA-graph-friendly embedding batching to an existing agent framework or task-parallel runtime should recover most of the observed speedup, which would separate the batching contribution from the deterministic-runtime contribution.
  • Editorial inference: OpRAG's operator model is a natural substrate for co-scheduling with memory-aware retrieval and vector-index upserts; the 0.03 ms/query memory overhead suggests that stateful RAG memory can be made nearly free on the critical path, which future work could verify under longer multi-turn workloads with retention policies.
  • Editorial inference: since the runtime is complementary to model-serving engines, combining OpRAG-style orchestration with a production serving backend that already optimizes decoding should yield roughly additive gains, a testable extension the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes OpRAG, a resource-deterministic distributed runtime for GPU-backed multi-stage RAG. It models embedding, retrieval, reasoning, memory, and upsert as first-class operators, lowers workflow segments into communication-aware execution graphs, and combines an Arrow/Cylon zero-copy data plane with persistent workers, bounded queues, tokenizer prefetching, length-bucketed GPU embedding, and CPU/GPU overlap. The evaluation on two A100 GPUs with Llama3-8B and Mistral-7B reports end-to-end improvements of 16.16% and 15.66% over the nearest competitor (HigressRAG), 20.57% and 20.71% over RayScalableRAG, 17.77% and 17.48% over the best framework baseline, and 52.48--53.55% latency reduction in Higress-style generation scenarios, with 100% Recall@5. CPU strong/weak scaling experiments are also presented. The central claim is that the orchestration layer around LLM decoding, not the decoding kernel itself, is a major performance bottleneck in GPU-backed multi-stage RAG.

Significance. If the reported results hold, the paper makes a useful systems contribution: it provides evidence that operator-level scheduling, batching, and CPU/GPU overlap can reduce non-model overhead in GPU-backed RAG without touching the LLM decoding kernel. The controlled-comparison design (same model, corpus, chunk count, generation parameters, and vector insertion semantics across baselines) is the right methodological choice, and the descriptive cost model in Eqs. (1)--(5) gives a clear framework for thinking about per-batch overhead and the residual runtime term Omega. The CPU strong/weak scaling experiments add useful evidence that the operator-runtime design scales to hundreds of physical workers. However, the current evaluation does not isolate the resource-deterministic runtime mechanisms from the embedding-path optimizations that produce the dominant measured saving, and Section 7.1 explicitly defers repeated runs and confidence intervals, which weakens the quantitative claims.

major comments (4)
  1. [§5.3, Table 1; §4.3] The entire end-to-end advantage over HigressRAG in Table 1 appears in the Embedding row: 126.832 s vs 151.979 s for Llama3-8B and 128.287 s vs 152.776 s for Mistral-7B, while Load, Transform, and Generate are nearly identical across systems. Section 4.3 attributes this saving to length-bucketed batching, a compiled embedding path, and CUDA-graph-friendly shapes. Because no ablation separates these embedding-specific optimizations from the resource-deterministic operator runtime (bounded queues, persistent workers, zero-copy data plane), the reported 16.16%/15.66% improvements are also consistent with the hypothesis that any baseline could obtain the same gain by adopting length-bucketed CUDA-graph embedding. Please provide an ablation that runs OpRAG without the embedding-specific optimizations, or a baseline with the same embedding optimizations enabled, to isolate the orchestration-layer contribution.
  2. [§7.1] Section 7.1 states that repeated runs and confidence intervals are future work. Given that the headline differences are 16--21% and the dominant stage takes roughly 126--128 s, the absence of run-to-run variance makes it impossible to establish that the observed differences exceed noise, especially on shared GPU infrastructure. Please report at least three to five repeated runs per configuration with means and standard deviations (or confidence intervals) for Tables 1--3 and for the CPU scaling totals in Figures 4--6.
  3. [§5.1] The controlled-comparison claim in Section 5.1 fixes model, corpus, chunk count, generation settings, embedding semantics, and vector insertion semantics, but it does not state whether the baselines were permitted to use the same batching policy, length bucketing, CUDA graphs, tokenizer prefetching, or persistent workers. Since Section 4.3 identifies exactly these mechanisms as the source of the main gain, the evaluation needs to disclose baseline configuration details (batch sizes, embedding loop implementation, use of GPU vs CPU FAISS, etc.) and ideally include a best-effort baseline with equivalent embedding batching. Without this, the comparison may reflect implementation effort in the embedding path rather than a fundamental limitation of the existing orchestration models.
  4. [§4.3; §5.3] The experimental configuration deliberately uses the generative 8B/7B model as the embedder, which makes Embedding the dominant stage (about 152 s) while generation is only about 4 s. This stress test is legitimate, but the paper's broader conclusion that the orchestration layer is a major bottleneck in GPU-backed multi-stage RAG should be scoped to this configuration. In production settings that use a small encoder model for embedding and longer generation workloads, the relative contribution of the orchestration layer could be much smaller. Please state this scope limitation explicitly in the conclusions and, if feasible, add sensitivity results with a standard embedding model or a longer-generation workload.
minor comments (4)
  1. [§3.5, Eq. (5)] The notation 'T_retrieve + T_context' inside the max expression is ambiguous because the surrounding text emphasizes overlap; please clarify whether retrieval and context construction are assumed to be sequential within the critical path or are themselves overlapped with other stages.
  2. [§4.1] The phrase 'CUDA graph repeated shape strategy' is not a standard or defined term; consider replacing it with a concrete description of how repeated-shape CUDA graphs are captured and replayed in the embedding path.
  3. [§5.7, Table 5] Table 5 reports Top-1 accuracy of 0.9560 for OpRAG versus 0.0010 for HigressRAG on follow-up queries, but the comparison is explicitly stateless versus memory-augmented; the table caption and surrounding text should make clear that this is a capability demonstration of the memory operator rather than a head-to-head retrieval-quality benchmark.
  4. [§4.4] The memory-cost estimate of approximately 3 KB per 768-dimensional vector and 30 MB for 10K summaries counts raw float32 storage only; because the text mentions FAISS and metadata overhead separately, please label this estimate as a lower bound on total memory footprint.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: reported gains are external measurements; cost model is descriptive; self-citations are not load-bearing.

full rationale

The reported improvements in this paper are measured against external baselines (RayScalableRAG, DaskScalableRAG, LangChain, LangGraph, CrewAI, AutoGen, and HigressRAG) under a controlled comparison, not derived from the paper's cost model. Equations (1)-(5) are descriptive accounting identities with no fitted parameters used to construct the reported speedups. The dominant saving in Table 1 is in the Embedding stage, which Section 4.3 attributes to length-bucketed batching, a compiled embedding path, and CUDA-graph-friendly shapes. This raises an attribution question because no ablation isolates those embedding-path optimizations from the resource-deterministic operator runtime, and Section 7.1 admits that a microarchitectural breakdown is left for future work. However, this is a limitation in causal attribution, not circularity: the embedding optimizations are independent, implementable changes to batch formation, and the measured total is not computed from the cost model. The paper's self-citations to prior Cylon/Arrow work and to Radical-Pilot/Parsl provide prior software and a pattern classification, but the operator definitions in Section 3.2 independently justify the communication patterns (broadcast/reduction for retrieval, shuffle for upsert), and no uniqueness theorem or fitted parameter is imported from those citations. The acknowledged limitations in Section 7.1—missing confidence intervals, uncharacterized memory growth, and lack of microbenchmarks—are gaps in evidence, not circular steps. Therefore no load-bearing step reduces to its own input by construction; the score reflects residual attribution ambiguity rather than demonstrable circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two things the reader is asked to take on faith: that the five-operator decomposition is the right abstraction, and that the baselines are fair. The embedded optimization knobs (batch sizes, length buckets, top-128 candidate pool, fusion weights) are real free parameters that could affect the headline numbers. No new physical entities are introduced.

free parameters (5)
  • Embedding batch size b_e = not reported
    Controls GPU embedding throughput; OpRAG batches chunks, but the paper does not state the chosen batch size or how it was selected.
  • Upsert batch size b_u = not reported
    Coalesces vector writes; the value is not specified.
  • Dense-first candidate count (top 128) = 128
    Dense-first hybrid retrieval uses a top-128 candidate pool before lexical reranking; chosen to keep Recall@5 at 1.0 in the 1024-query benchmark, but no sensitivity analysis is given.
  • Dense/lexical fusion weights w_d and w_l = not reported
    Used in Higress-style score fusion; the values are never given.
  • Length-bucket grouping policy = not reported
    Chunks are grouped by similar token length, but the number and size of buckets are unspecified.
assumptions (5)
  • domain assumption RAG workflows decompose into the five operators Op_embed, Op_retrieve, Op_reason, Op_memory, and Op_upsert
    Section 3.2; this decomposition is the core modeling choice, and it places preprocessing partly outside the operator set.
  • domain assumption The cost model T = N*beta/P + N*alpha/(bP) + Omega captures runtime overhead additively
    Eq. 3 in Section 3.5; used to attribute gains to Omega, but the model is descriptive and not validated by separate measurements of alpha, beta, and Omega.
  • ad hoc to paper Deterministic execution semantics are preserved by segment compilation and bounded queues
    Sections 3.3 and 3.5; determinism is asserted, but no formal semantics or verification is supplied.
  • domain assumption Embedding a corpus with the generative LLM (Llama3-8B or Mistral-7B) is a fair and equivalent semantic workload across all baselines
    Section 4.3; the paper deliberately uses the same model family across systems, but this does not match typical production setups that use smaller encoders.
  • domain assumption Baselines were competently configured with the same batching and embedding semantics
    Section 5.1; comparisons are only meaningful if each baseline embedding loop was not left in an unoptimized default, but configuration details are not provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpRAG: A Resource-Deterministic Runtime for GPU-Backed Multi-Stage RAG Workflows." pith.science (2026). https://pith.science/paper/MX2YAX5K

@misc{pith2026260808340,
  author       = {Pith},
  title        = {Pith review of: OpRAG: A Resource-Deterministic Runtime for GPU-Backed Multi-Stage RAG Workflows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MX2YAX5K}},
  note         = {Machine review of arXiv:2608.08340}
}
read the original abstract

Agentic retrieval-augmented generation (RAG) systems combine preprocessing, embedding, retrieval, memory access, context construction, generation, and vector-index updates. Although LLM decoding is GPU-bound, the surrounding orchestration layer can still limit end-to-end performance through serialization overhead, fragmented scheduling, inefficient batching, and CPU--GPU pipeline stalls. Existing frameworks provide flexible control flow, while distributed runtimes provide scalable task parallelism, but neither exposes RAG stages as resource-aware operators with deterministic execution semantics. We present OpRAG, a resource-deterministic distributed runtime for GPU-backed multi-stage RAG workflows. OpRAG models embedding, retrieval, reasoning, memory, and upsert as first-class operators and lowers them into communication-aware execution graphs. It combines an Arrow zero-copy data plane, persistent workers, bounded queues, CPU tokenizer prefetching, batched GPU embedding, and overlapped retrieval/generation execution to reduce non-model overhead around LLM inference. We evaluate OpRAG using Llama3-8B and Mistral-7B with FlashAttention~2, BF16 execution, and 32K RAG chunks. In end-to-end GPU pipeline experiments, OpRAG improves over the nearest competitor by 16.16% for Llama3-8B and 15.66% for Mistral-7B, and over RayScalableRAG by 20.57% and 20.71%, respectively. Against LangChain, LangGraph, CrewAI, and AutoGen, OpRAG is 17.77% and 17.48% faster than the best framework baseline. In Higress-style query serving, OpRAG reduces hybrid retrieval latency by 59.20--59.62% and generation-scenario latency by 52.48--53.55%, while preserving 100% Recall@5. These results show that optimizing the distributed orchestration layer can substantially improve GPU-backed multi-stage RAG without modifying the LLM decoding kernel.

Figures

Figures reproduced from arXiv: 2608.08340 by the authors.

Figure 1
Figure 1. End-to-end OpRAG component flow with memory [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Layered OpRAG design. Dynamic Multi-Stage de [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Dynamic agent execution in OpRAG. The Workflow [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Strong scaling behavior for Load, Transform, Em [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Weak scaling behavior for Load, Transform, Em [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Total runtime for CPU strong and weak scaling. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control

    cs.DC 2026-08 conditional novelty 6.0 of 10

    Exact deadline packing raises the GPU-eligible share of agent control events from 30.19% to 43.00% in a frozen trace replay, and keeping a GPU decision on device beats host round trips by 1.19x to 2.39x.

Reference graph

Works this paper leans on

61 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vibhatha Abeykoon, Pulasthi Wickramasinghe, Supun Kamburugamuve, Hasala Maithree, Chathura Widanage, Niranda Perera, Thejaka Amila Kanewala, Ahmet Uyar, Gurhan Gunduz, and Geoffrey Fox. 2023. High Performance Dataframes from Parallel Processing Patterns. InParallel Processing and Applied Mathematics: 14th International Conference, PPAM 2022, Gdansk, Polan...

    work page 2023

  2. [2]

    Alibaba Cloud and the Higress Authors. 2023. Higress. https://github.com/aliba ba/higress

    work page 2023

  3. [3]

    Aymen Alsaadi, Mason Hooten, Mariya Goliyad, Andre Merzky, Andrew Shao, Mikhail Titov, Tianle Wang, Yian Chen, Maria Kalantzi, Kent Lee, et al . 2025. RHAPSODY: Execution of Hybrid AI-HPC Workflows at Scale.arXiv preprint arXiv:2512.20795(2025). https://arxiv.org/abs/2512.20795

    arXiv 2025

  4. [4]

    Aymen Alsaadi, Logan Ward, Andre Merzky, Kyle Chard, Ian Foster, Shantenu Jha, and Matteo Turilli. 2022. Radical-Pilot and Parsl: Executing heterogeneous workflows on HPC platforms. In2022 IEEE/ACM Workshop on Workflows in Support of Large-Scale Science (WORKS). IEEE, 27–34. doi:10.1109/WORKS56498 .2022.00009

    work page arXiv 2022

  5. [5]

    Katz, Ben Clifford, Rohan Kumar, Lukasz Lacinski, Ryan Chard, Justin M

    Yadu Babuji, Anna Woodard, Zhuozhao Li, Daniel S. Katz, Ben Clifford, Rohan Kumar, Lukasz Lacinski, Ryan Chard, Justin M. Wozniak, Ian Foster, Michael Wilde, and Kyle Chard. 2019. Parsl: Pervasive Parallel Programming in Python. InProceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing(Phoenix, AZ, USA)(HPDC ...

    arXiv 2019

  6. [6]

    Paul Barham, Aakanksha Chowdhery, Jeff Dean, Sanjay Ghemawat, Steven Hand, Daniel Hurt, Michael Isard, Hyeontaek Lim, Ruoming Pang, Sudip Roy, et al. 2022. Pathways: Asynchronous distributed dataflow for ML.Proceedings of Machine Learning and Systems4 (2022), 430–449. https://proceedings.mlsys.org/paper_fil es/paper/2022/file/37385144cac01dff38247ab11c119...

    arXiv 2022

  7. [7]

    Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache flink: Stream and batch processing in a single engine.The Bulletin of the Technical Committee on Data Engineering38, 4 (2015). https://asterios.katsifodimos.com/assets/publications/flink-deb.pdf

    work page 2015

  8. [8]

    2024.Optimize Vector Databases, Enhance RAG- Driven Generative AI

    Malini Bhandaru Cathy Zhang. 2024.Optimize Vector Databases, Enhance RAG- Driven Generative AI. Technical Report. Intel. https://medium.com/intel-tech/o ptimize-vector-databases-enhance-rag-driven-generative-ai-90c10416cb9c

    work page 2024

Show all 61 references
  1. [9]

    Rishabh Kumar Choudhary. [n. d.]. How to Achieve 10x Performance with Vector Database for LLM using LanceDB and PyArrow. https://www.rishabhxchoudhar y.com/blog/How_to_Achieve_10x_Performance_with_Vector_Database_for_L LM_using_LanceDB_and_PyArrow

  2. [10]

    2026.Kickoff Crew Asynchronously

    CrewAI Team. 2026.Kickoff Crew Asynchronously. https://docs.crewai.com/en/l earn/kickoff-async

    2026

  3. [11]

    Lisandro Dalcin, Rodrigo Paz, and Mario Storti. 2005. MPI for Python.J. Parallel and Distrib. Comput.65, 9 (1 Sept. 2005), 1108–1115. doi:10.1016/j.jpdc.2005.03.010

    2005 doi

  4. [12]

    Tri Dao. 2023. FlashAttention-2: Faster Attention with Better Paral- lelism and Work Partitioning.arXiv preprint arXiv:2307.08691(2023). arXiv:2307.08691 [cs.LG]

    2023 arXiv

  5. [13]

    Emily Davis. 2024. Building Custom AI Workflows Using LangChain Tools. ThinkTide Global Research Journal5, 4 (2024), 54–62. https://thinktidejournal.c om/index.php/TGRJ/article/view/53/63

    2024

  6. [14]

    2024.LangGraph: Stateful Multi-Agent Workflows

    LangChain Developer. 2024.LangGraph: Stateful Multi-Agent Workflows. Techni- cal Report. LangChain Inc. https://blog.langchain.com/langgraph-multi-agent- workflows

    2024

  7. [15]

    2023.Pinecone: Scalable vector database for machine learning applications

    Pinecone Developers. 2023.Pinecone: Scalable vector database for machine learning applications. Technical Report. Pinecone Systems Inc. https://www.pinecone.io/

    2023

  8. [16]

    Zhihua Duan and Jialin Wang. 2024. Exploration of llm multi-agent application implementation based on langgraph+ crewai.arXiv preprint arXiv:2411.18241 (2024). https://doi.org/10.32388/R27SW4

    2024 arXiv

  9. [17]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. 2024. The Llama 3 Herd of Models.arXiv preprint arXiv:2407.21783 (2024). arXiv:2407.21783 [cs.AI]

    2024 arXiv

  10. [18]

    Mathieu Dugré, Valérie Hayot-Sasson, and Tristan Glatard. 2023. Performance comparison of Dask and Apache Spark on HPC systems for neuroimaging. e7635 pages. doi:10.1002/cpe.7635

    2023 doi

  11. [19]

    2023.Gloo: Collective Communications Library with various primitives for multi-machine training

    Facebookincubator. 2023.Gloo: Collective Communications Library with various primitives for multi-machine training. Technical Report. Facebook. https: //github.com/facebookincubator/gloo"

    2023

  12. [20]

    Yuanshuang Fu, Dan Liu, Bonan Zhang, Zhuotong Jiang, Haibo Mei, and Jiajin Guan. 2025. Cue RAG: Dynamic multi-output cue memory under H framework for retrieval-augmented generation.Neurocomputing639 (2025), 130235. doi:10 .1016/j.neucom.2025.130235

    2025

  13. [21]

    Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. 2025. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models. InForty-second International Conference on Machine Learning. https: //openreview.net/forum?id=LWH8yn4HS2

    2025

  14. [22]

    Higress Documentation. 2026. AI Cache Plugin. https://higress.cn/en-us/docs/pl ugins/ai/ai-cache

    2026

  15. [23]

    2025.Context Rot: How Increasing Input Tokens Impacts LLM Performance

    Kelly Hong, Anton Troynikov, and Jeff Huber. 2025.Context Rot: How Increasing Input Tokens Impacts LLM Performance. Technical Report. Chroma. https: //trychroma.com/research/context-rot

    2025

  16. [24]

    2025.Generative Benchmarking

    Kelly Hong, Anton Troynikov, Jeff Huber, and Morgan McGuire. 2025.Generative Benchmarking. Technical Report. Chroma. https://trychroma.com/research/gen erative-benchmarking

    2025

  17. [25]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023 arXiv

  18. [26]

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-Scale Similarity Search with GPUs.IEEE Transactions on Big Data7, 3 (2021), 535–547. doi:10.110 9/TBDATA.2019.2921572

    2021

  19. [27]

    Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. DSPy: Compiling Declarative Language Model Calls into Self...

    2024

  20. [28]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Principl...

    2023

  21. [29]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. InProceedings o...

    2020

  22. [30]

    Hao Liang, Xiaochen Ma, Zhou Liu, Zhen Hao Wong, Zhengyang Zhao, Zimo Meng, Runming He, Chengyu Shen, Qifeng Cai, Zhaoyang Han, et al . 2025. SOCC, November 18–20, 2026, Singapore Arup Kumar Sarker, Mills Staylor, Gregor von Laszewski, Geoffrey Fox, Aymen Alsaadi, and Shantenu...

    2025

  23. [31]

    Weixi Lin. 2025. Higress-RAG: A Holistic Optimization Framework for Enterprise Retrieval-Augmented Generation via Dual Hybrid Retrieval, Adaptive Routing, and CRAG.arXiv preprint arXiv:2602.23374(2025). https://arxiv.org/abs/2602.2 3374

    2025

  24. [32]

    LlamaIndex. [n. d.]. Simplify your RAG application architecture with LlamaIndex + PostgresML. https://www.llamaindex.ai/blog/simplify-your-rag-application- architecture-with-llamaindex-postgresml

  25. [33]

    Wei Ma, Yixiao Yang, Qiang Hu, Shi Ying, Zhi Jin, Bo Du, Zhenchang Xing, Tianlin Li, Junjie Shi, Yang Liu, et al. 2025. Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol. doi:10.48550 /arXiv.2508.20737

    2025 doi arXiv

  26. [34]

    Merzky, M

    A. Merzky, M. Turilli, M. Titov, A. Al-Saadi, and S. Jha. 2022. Design and Perfor- mance Characterization of RADICAL-Pilot on Leadership-Class Platforms.IEEE Transactions on Parallel and amp; Distributed Systems33, 04 (apr 2022), 818–829. doi:10.1109/TPDS.2021.3105994

    2022

  27. [35]

    Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I Jordan, et al. 2018. Ray: A distributed framework for emerging {AI} applications. In 13th USENIX Symposium on Operating Systems Desi...

    2018

  28. [36]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2025. Generative representational instruc- tion tuning. InInternational Conference on Learning Representations, Vol. 2025. 45544–45613

    2025

  29. [37]

    Niranda Perera, Arup Kumar Sarker, Kaiying Shan, Alex Fetea, Supun Kambu- rugamuve, Thejaka Amila Kanewala, Chathura Widanage, Mills Staylor, Tianle Zhong, Vibhatha Abeykoon, Gregor von Laszewski, and Geoffrey Fox. 2024. Su- percharging distributed computing environments for h...

    2024

  30. [38]

    Niranda Perera, Arup Kumar Sarker, Mills Staylor, Gregor von Laszewski, Kaiy- ing Shan, Supun Kamburugamuve, Chathura Widanage, Vibhatha Abeykoon, Thejaka Amila Kanewela, and Geoffrey Fox. 2023. In-depth analysis on paral- lel processing patterns for high-performance Dataframe...

    2023 doi

  31. [39]

    Maximilian Petersohn, Stephen Macke, Doris Xin, William Ma, J. K. Wittenauer, Stephen Hoyer, Ryan Marcus, Matei Zaharia, and Benjamin Recht. 2020. Towards Scalable Dataframe Systems.Proceedings of the VLDB Endowment (PVLDB)13, 12 (2020), 2033–2046. doi:10.14778/3407790.3407807

    2020

  32. [40]

    Hongjin Qian, Zheng Liu, Peitian Zhang, Kelong Mao, Defu Lian, Zhicheng Dou, and Tiejun Huang. 2025. MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation. InProceedings of the ACM on Web Conference 2025(Sydney NSW, Australia)(WWW ’25). A...

    2025 doi

  33. [41]

    Matthew Rocklin. 2015. Dask: Parallel computation with blocked algorithms and task scheduling. InProceedings of the 14th python in science conference, Vol. 130. Citeseer, 136. https://proceedings.scipy.org/articles/Majora-7b98e3ed-013.pdf

    2015

  34. [42]

    Arup Kumar Sarker, Aymen Alsaadi, Alexander James Halpern, Prabhath Tan- gella, Mikhail Titov, Niranda Perera, Mills Staylor, Gregor von Laszewski, Shantenu Jha, and Geoffrey Fox. 2025. Deep RC: A Scalable Data Engineering and Deep Learning Pipeline. InJob Scheduling Strategie...

    2025 doi

  35. [43]

    Arup Kumar Sarker, Aymen Alsaadi, Niranda Perera, Mills Staylor, Gregor von Laszewski, Matteo Turilli, Ozgur Ozan Kilic, Mikhail Titov, Andre Merzky, Shantenu Jha, et al. 2024. Radical-Cylon: A Heterogeneous Data Pipeline for Scientific Computing. InJob Scheduling Strategies f...

    2024 doi

  36. [44]

    Arup Kumar Sarker, Alexander James Halpern, Mills Staylor, Aymen Alsaadi, Gregor von Laszewski, Yue Cheng, and Geoffrey Charles Fox. 2026. [AAFLOW+] Stateful Operator Abstraction with Zero-Copy Distributed KV Cache Orches- tration for Multi-Agent Workflows.arXiv preprint arXiv...

    2026 arXiv

  37. [45]

    Arup Kumar Sarker and Felix Xiaozhu Lin. 2022. Incremental perception on real time 3D data. InProceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications(Tempe, Arizona)(HotMobile ’22). Association for Computing Machinery, New York, NY, U...

    2022

  38. [46]

    Arup Kumar Sarker, Mills Staylor, Aymen Alsaadi, Gregor von Laszewski, Shantenu Jha, and Geoffrey Fox. 2026. AAFLOW: Scalable Patterns for Agentic AI Workflows.arXiv preprint arXiv:2605.02162(2026)

    2026 arXiv

  39. [47]

    Graham Lopez, Matthew B

    Pavel Shamis, Manjunath Gorentla Venkata, M. Graham Lopez, Matthew B. Baker, Oscar Hernandez, Yossi Itigin, Mike Dubman, Gilad Shainer, Richard L. Graham, Liran Liss, Yiftah Shahar, Sreeram Potluri, Davide Rossetti, Donald Becker, Dun- can Poole, Christopher Lamb, Sameer Kumar...

    2015 doi

  40. [48]

    Kaiying Shan, Niranda Perera, Damitha Lenadora, Tianle Zhong, Arup Ku- mar Sarker, Supun Kamburugamuve, Thejaka Amila Kanewela, Chathura Widan- age, and Geoffrey Fox. 2022. Hybrid Cloud and HPC Approach to High- Performance Dataframes. In2022 IEEE International Conference on B...

    2022

  41. [49]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. InProceedings of the 37th International Conference on Neural Information Processing Systems(New Orleans, LA, USA)(NIPS ’23). ...

    2023

  42. [50]

    Mills Staylor, Arup Kumar Sarker, Gregor von Laszewski, Geoffrey Fox, Yue Cheng, and Judy Fox. 2026. Combining Serverless and High-Performance Com- puting Paradigms to support ML Data-Intensive Applications.Frontiers in High Performance Computing(2026)

    2026

  43. [51]

    2025.What is LlamaIndex ?Technical Report

    Erika Russi Vanna Winland. 2025.What is LlamaIndex ?Technical Report. IBM. https://www.ibm.com/think/topics/llamaindex

    2025

  44. [52]

    Baiqiang Wang, Dongfang Zhao, Nathan R Tallent, and Luanzheng Guo. 2025. On The Reproducibility Limitations of RAG Systems. https://doi.org/10.48550/a rXiv.2509.18869

    2025 doi

  45. [53]

    Qinglin Wang, Jie Liu, Xiantuo Tang, Feng Wang, Guitao Fu, and Zuocheng Xing. 2014. Accelerating embarrassingly parallel algorithm on Intel MIC. In2014 IEEE International Conference on Progress in Informatics and Computing. 213–218. doi:10.1109/PIC.2014.6972327

    2014

  46. [54]

    Jiale Wei, Shuchi Wu, Ruochen Liu, Xiang Ying, Jingbo Shang, and Fangbo Tao

  47. [55]

    Chathura Widanage, Niranda Perera, Vibhatha Abeykoon, Supun Kamburuga- muve, Thejaka Amila Kanewala, Hasara Maithree, Pulasthi Wickramasinghe, Ahmet Uyar, Gurhan Gunduz, and Geoffrey Fox. 2020. High performance data engineering everywhere. In2020 IEEE International Conference ...

    2020

  48. [56]

    White, Doug Burger, and Chi Wang

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang (Eric) Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Ahmed Awadallah, Ryen W. White, Doug Burger, and Chi Wang. 2024. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. InCOLM 2024. https:...

    2024

  49. [57]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations. https://open review.net/pdf?id=WE_vluYUL-X

    2023

  50. [58]

    Jinhui Yuan, Xinqi Li, Cheng Cheng, Juncheng Liu, Ran Guo, Shenghang Cai, Chi Yao, Fei Yang, Xiaodong Yi, Chuan Wu, et al. 2021. Oneflow: Redesign the distributed deep learning framework from scratch.arXiv preprint arXiv:2110.15032 (2021). https://arxiv.org/abs/2110.15032

    2021 arXiv

  51. [59]

    Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J

    Matei Zaharia, Reynold S. Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J. Franklin, Ali Ghodsi, Joseph Gonzalez, Scott Shenker, and Ion Stoica. 2016. Apache Spark: a unified engine for big data pro...

    2016 doi

  52. [60]

    Gonzalez, Clark Barrett, and Ying Sheng

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: efficient execution of structured language model programs. InProceedings of the 38th In...

    2024

  53. [2025]

    https://arxiv.org/abs/2503.16071

    Tuning LLMs by RAG Principles: Towards LLM-native Memory.arXiv preprint arXiv:2503.16071(2025). https://arxiv.org/abs/2503.16071

    2025 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.