Pith. sign in

REVIEW 4 major objections 5 minor 27 references

The paper claims that wrapping a probabilistic LLM in a deterministic state-governed runtime yields reproducible, auditable decisions without sacrificing creativity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 14:53 UTC pith:GL2LN3TZ

load-bearing objection Honest, well-scoped systems report with real artifacts, but the determinism claim is overstated and the headline cache metric is largely circular — read it for the architecture, not the numbers. the 4 major comments →

arxiv 2607.18246 v1 pith:GL2LN3TZ submitted 2026-05-04 cs.AI cs.SE

Phionyx: A Deterministic AI Runtime Architecture with Structured State Management and Pre-Response Governance

Ali Toygar Abak This is my paper
classification cs.AI cs.SE
keywords deterministic AI runtimeLLM governancestructured state vectorpre-response safetysemantic timeimpact-weighted cache evictionnoisy sensor modelstate-evolution equations
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the core obstacle to deploying LLMs in regulated settings—non-determinism—can be removed not by making the model deterministic but by putting a deterministic envelope around it. Phionyx reclassifies LLM output as noisy sensor measurement, feeds it through a fixed 46-block pipeline, and derives reproducible control signals from a structured state vector governed by deterministic evolution equations. On top of this, a pre-response safety layer gates dangerous output before generation, and a semantic-time memory system evicts cache entries by cognitive impact rather than recency. If true, the architecture would give auditors a reproducible decision trail while preserving the model's generative variability. The evidence is scoped to single-instance, single-node runs: 100 hash-verified repetitions with zero variance, ~31% CPU reduction against a post-hoc filtering baseline, and 24%/72% retention improvements over LRU/FIFO.

Core claim

The paper's central claim is a separation of concerns: let the LLM be probabilistic, but treat each output as a noisy measurement M(t) and run it through a deterministic function g that updates a structured state vector S(t) and emits control signals C(t). The state vector—entropy, amplitude, valence, rates, semantic time—evolves under fixed update rules, so identical inputs produce identical control signals even when LLM text varies. A canonical 46-block pipeline executes in fixed order, and a safety layer damps response amplitude before a response is built. The paper also claims measurable efficiency gains: pre-response gates reduce post-hoc filtering cost, and impact-weighted eviction ret

What carries the argument

The load-bearing object is the structured state vector S(t) = [H, A, V, dotA, dotV, t_local, t_global] plus derived metrics Phi and R, updated by deterministic state-evolution equations such as S(t+dt)=S(t)+alpha*beta*dt - S(t)*gamma*dt and Phi(t+dt)=Phi(t)*exp(-lambda*dt). The noisy-sensor model M(t) = f_LLM(I,S)+N(t) separates probabilistic generation from deterministic evaluation: control signals are computed as C(t) = g_deterministic(M(t), S(t)). The 46-block pipeline (B1–B46) is the concrete executor of g, organized into six macro-groups covering ingress, perception, cognition, state evolution, fusion, and audit; safety gates and cache eviction read directly from these state metrics.

Load-bearing premise

The architecture is deterministic and auditable only if the numbers it computes from the state vector—especially entropy from zlib compression and Phi from exponential decay—actually measure the behavioral stability and information value they are said to govern; the paper acknowledges this proxy validity has not been confirmed by external user studies.

What would settle it

Run the system on a corpus of paraphrased distress-language or identity-assertion outputs that evade the CEP engine's regex and TF-IDF thresholds (e.g., low pronoun density, novel phrasings); if any such output passes the pre-response gate and reaches the response builder, the 100% detection claim in the paper's safety validation is falsified. Alternatively, an ablation that disables the entropy gate and measures no change in externally rated response quality would falsify the claim that these metrics influence output.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, regulated deployments could keep using existing LLMs while gaining a deterministic, hash-verifiable audit trail of every control decision.
  • Pre-response governance implies unsafe or policy-violating content can be damped before it is materialized, reducing reliance on post-hoc output filters.
  • Impact-weighted eviction means memory systems can be tuned by semantic significance rather than recency, with measurable retention gains in RAG workloads.
  • Deterministic recovery from injected failures (entropy overflow, coherence violation, ethics escalation, state corruption) suggests restart-based failover can be replaced by local state restoration.
  • The separation of probabilistic generation from deterministic control is a general pattern that could extend to other stochastic AI components, not just text LLMs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitation notes that entropy is measured via zlib compression ratio and Phi via exponential decay; a fair reader would infer that the entire governance chain stands or falls on whether these proxies track genuine risk and value, which the paper has not yet validated in user studies.
  • The ~31% CPU savings assume a 30% unsafe-input ratio; in low-threat environments the savings would shrink, while in high-threat environments pre-response gating could dominate, so the economics are environment-dependent.
  • The same deterministic-envelope pattern could be applied to other probabilistic generators (speech, vision, code models) by defining appropriate state metrics, which the paper does not discuss but follows directly from the noisy-sensor abstraction.
  • A concrete testable extension: run the pipeline with two different LLM backends on the same inputs and check whether control signals and audit logs converge, which would confirm the sensor abstraction is backend-agnostic.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Phionyx, a three-layer AI runtime that treats LLM outputs as noisy sensor measurements and routes them through a canonical 46-block deterministic evaluation pipeline. The architecture combines a structured state vector with deterministic update equations, a pre-response safety/governance layer, and a semantic-time memory system with impact-weighted cache eviction. The authors claim approximately 31% CPU reduction versus post-hoc filtering, 24%/72% high-value retention improvements versus LRU/FIFO, zero variance across 100 hash-verified repeated runs, and zero unplanned restarts in single-instance testing. All claims are explicitly scoped to single-instance, sequential, non-adversarial deployments, with distributed and multi-tenant generalization deferred to future work.

Significance. If the central architectural claim held, Phionyx would offer a useful governance envelope for LLM-based applications: a runtime that makes control decisions and state evolution deterministic while preserving the generative variability of the underlying model. The paper has several commendable features: it ships a public test suite (1,137 tests), a reproducibility pack with hashes and benchmark JSON, and it is unusually candid in Appendix C about the scope of each measurement. The conflict of interest is declared. However, the two load-bearing pillars of the paper—deterministic decision-making over stochastic LLM outputs and the meaningfulness of the impact-weighted eviction metric—are not established as written. The determinism claim is internally inconsistent with the formalization in Eqs. (5)–(6), and the retention improvement is tautologically defined by the paper's own impact metric. These issues undermine the abstract's headline promises and require substantive revision.

major comments (4)
  1. [§4.2, Eqs. (5)–(6), Appendix C.1] The core determinism claim is internally inconsistent. Eq. (5) defines M(t) = f_LLM(I,S(t)) + N(t) with N(t) stochastic, and Eq. (6) defines C(t) = g_deterministic(M(t), S(t)). Since C is a function of M, C(t) is a random variable unless g is constant in its first argument or N(t) is removed by aggregation. The paper claims in §4.2 that this produces 'reproducible control signals C(t) regardless of LLM variance,' which is not supported by the equations. Appendix C.1 verifies zero variance only under a fixed seed; that demonstrates seeded reproducibility, not invariance to LLM sampling noise. The abstract's unqualified 'enabling reproducible behavior' is therefore too strong. The architecture must either specify a deterministic reduction of M (e.g., fixed-seed sampling, consensus over multiple samples, or a deterministic feature extraction) or explicitly scope all determinism claims to fi
  2. [§6.3–6.4, Eqs. (24)–(25), Table 3] The headline memory-retention improvement is circular. Eq. (24) defines w_impact as an exponentially decaying weight based on semantic time, and Eq. (25) evicts blocks with w_impact < θ. The benchmark then labels data as 'high-value' using exactly this same w_impact measure. An eviction policy that prioritizes high w_impact will of course retain entries that the metric itself calls high-value, so the reported 24% vs LRU and 72% vs FIFO improvements are by construction rather than an empirical finding about information value. An independent value label—for example, downstream task performance, human relevance judgments, or a held-out retrieval metric—is needed to make the claim meaningful.
  3. [§7.4, Table 3, §8.3] The resource-efficiency evidence is weaker than the presentation suggests. The ~31% CPU reduction is explicitly a simulated cost model with a sensitivity range of 7–50% depending on threat environment; no raw measurements or confidence intervals are reported. The safety metrics in Table 4 (100% detection rate, zero contamination) are based on automated test scenarios only, and Appendix C.3 states that adversarial testing and independent audits have not been performed. These caveats appear in the limitations, but the abstract and contribution list state the numbers as results. I recommend rephrasing the abstract and Section 7 to present the 31% figure as a model-based estimate and to make the synthetic nature of the safety results equally prominent.
  4. [§7.1, §7.2, §8.3] All governance decisions rely on proxy metrics whose external validity is unestablished. Entropy is measured by zlib compression ratio and treated as 'behavioral stability,' and Φ is an exponential decay of the primary state; these values drive safety gates, amplitude damping, and cache eviction. Section 8.3 concedes that the influence of these metrics on actual response quality has not been validated through external user studies. Without evidence connecting the proxies to real risk or value, the paper's claim that Phionyx 'governs' LLM behavior is not yet supported. At minimum, the paper should explicitly label the proxy metrics as conjectural and describe an external validation plan.
minor comments (5)
  1. [Abstract] Typo: 'outputs asnoisy sensor measurements' should read 'as noisy'. Also, the abstract claims 'up to 24% improvement in high-value data retention vs. LRU (72% vs. FIFO)' without noting that the improvement is measured on the paper's own impact metric; this should be clarified.
  2. [§4.3] The parameters α, β, γ, and λ are defined as 'free parameters' but no ranges, default values, or sensitivity analysis are given. Since the state-evolution equations are central, adding a table with parameter values and the effect of their variation would strengthen reproducibility.
  3. [Figures 2 and 4] Figure 2's macro-group labels (G1–G6) are introduced in the text but the figure does not show which block numbers fall in each group; a table listing block ranges per group would be clearer. Figure 4 is a schematic without units; the axes or magnitudes of H(t), Φ(t), and w_impact are undefined.
  4. [Appendix C.1] The determinism verification table reports '100 identical runs, fixed seed (hash-verified)' but does not specify what is hashed (the full control signal? final state? all intermediate block outputs?) or whether the hash includes the LLM outputs. This detail is essential for interpreting the result.
  5. [References] The 'Echoism' framework is mentioned in the abstract and §1.2 but no reference or detailed description is provided. The paper would benefit from a citation or a brief independent explanation, since the architecture is claimed to be derived from it.

Circularity Check

2 steps flagged

High-value retention gain is by construction under the paper's own impact-weight metric; deterministic-control evidence is a fixed-seed tautology, not the claimed variance-free property.

specific steps
  1. self definitional [Section 6.3 Eq. (24), Section 6.4 Eq. (25), supported by Section 7.4 and Appendix C.2]
    "wimpact(t) = wbase · e^{−δ·tlocal} · (1−ϵ·tglobal) ... evict = {m : wimpact(m) < θ} ... This process removes low-impact entries from the cache, improving high-value data retention by up to 24% vs. LRU and 72% vs. FIFO (benchmark-verified, same cache capacity)."

    The eviction rule is literally 'evict entries with low w_impact'. If the benchmark's 'high-value data retention' labels entries by the same w_impact score, then a policy that preferentially keeps high-w_impact entries will outperform LRU/FIFO on that metric by construction. The paper supplies no independent definition of 'high-value' (e.g., future-access frequency), so the headline 24%/72% retention improvement is a restatement of the policy's own scoring rule rather than an empirical result.

  2. other [Section 4.2 Eqs. (5)-(6), Figure 3 caption, Appendix C.1 Table 2]
    "M(t) = fLLM(I,S(t)) + N(t) ... C(t) = gdeterministic(M(t),S(t)) ... producing reproducible control signals C(t) regardless of LLM variance. [Table 2:] 100 identical runs, fixed seed (hash-verified); Result Zero variance across all 100 runs"

    Eq. 6 makes C(t) a deterministic function of a random M(t), so C(t) is random unless N(t) is removed by seeding or g ignores M. The only zero-variance evidence is 100 runs with a fixed seed; that demonstrates seeded-run reproducibility, which is an input assumption, not the architecture's headline property of reproducible behavior 'regardless of LLM variance'. The verification is self-fulfilling: the fixed seed eliminates the very noise Eq. 5 introduces.

full rationale

No load-bearing self-citation chain exists: the paper's references are external and the only self-referential element is the disclosed single-author/company conflict of interest, which is not a circular argumentative step. The memory-retention claim is the clearest circular reduction: Eq. (24) defines impact weights, Eq. (25) evicts low-impact entries, and Sections 6.4/7.4 present 24%/72% 'high-value data retention' improvements without defining 'high-value' independently of w_impact; if the benchmark's high-value label is the same w_impact, the result is forced by construction. The determinism verification is also not independent: Eqs. (5)-(6) propagate LLM noise into control signals, yet the zero-variance result comes from fixed-seed runs (Appendix C.1), which only proves that a seeded deterministic pipeline is reproducible; the paper's own Section 8.3 concedes the tests used controlled inputs. Other quantitative claims are scoped rather than circular: the 31% CPU figure is a simulated comparison against an explicit post-hoc baseline, zero restarts are scoped to single-instance testing, and safety detection is limited to automated scenarios. Overall, one central reported improvement reduces by construction and the determinism evidence is self-fulfilling under seeding, giving partial circularity rather than full reduction of the architecture to its inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 3 invented entities

The central design rests on the assumption that LLM variance is separable noise, that compression-based entropy and exponential-decay 'resonance' are meaningful proxies, and that semantic-time impact weights capture information value. None of these are derived or externally calibrated; coefficients were tuned in 291 internal experiments against a vendor-defined quality score. The cache benchmark uses the same impact-weight definition as the eviction policy, so the memory improvement is partly self-justifying.

free parameters (4)
  • Entropy evolution coefficients α, β, γ = not reported (66 params tuned internally; CQS 0.862)
    Eq. (8)-(9) define ΔS_increase=αβ dt and ΔS_decay=Sγ dt; no values are given, and Section 7.3 reports 291 experiments over 66 state-evolution parameters.
  • Phi decay coefficient λ = not reported
    Eq. (10)/(23) define Φ(t+Δt)=Φ(t)e^{-λ dt}; λ is a free constant controlling the derived cognitive-resonance metric.
  • Impact-weight decay rates δ, ε and baseline w_base = not reported
    Eq. (24) defines w_impact=w_base e^{-δ t_local}(1-ε t_global); these determine cache eviction and thus the 24%/72% retention claims.
  • Eviction threshold θ and CEP thresholds τ_sr/τ_nov = not reported
    Eq. (25) evicts when w_impact<θ; CEP mechanisms use thresholds for self-reference and novelty; values are unspecified in the paper.
axioms (3)
  • domain assumption LLM output can be decomposed into signal plus additive noise: M(t)=f_LLM(I,S(t))+N(t).
    Eq. (5) is the basis for the whole 'noisy sensor' design; the paper does not test whether LLM variance is additive, benign, or separable.
  • domain assumption zlib compression-based entropy H(t) is a valid proxy for behavioral stability.
    Section 7.1 uses 'entropy (H, zlib compression-based)' as the core stability metric; no evidence links compression ratio to behavioral or risk properties.
  • ad hoc to paper Semantic time components (t_local, t_global) and the decay form in Eq. (24) capture the cognitive significance of information.
    Section 6.1-6.3 introduce the semantic time vector and exponential/linear decay without derivation or external calibration; these drive both cache eviction and state decay.
invented entities (3)
  • Cognitive resonance Φ (and resonance score R) no independent evidence
    purpose: Derived metric computed from primary state; used for drift detection and cache impact; never persisted.
    Eq. (2)/(10)/(23) define Φ via exponential decay of primary state; no external benchmark or falsifiable prediction ties Φ to observed behavior.
  • Semantic time vector T=[dt,t_local,t_global] no independent evidence
    purpose: Replaces chronological timestamps as the driver of decay and eviction.
    Section 6.1 introduces T as a modeling choice; no independent data shows these components improve prediction of memory value.
  • Cognitive envelopes no independent evidence
    purpose: Message containers validated by 'cognitive integrity' for inter-agent communication.
    Section 5.2 defines envelopes with Φ, S, parent trace, TTL, and signature; the validation function Eq. (17) is not specified beyond existence, and no external test is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 11152 in / 16450 out tokens · 153357 ms · 2026-08-02T14:53:41.201713+00:00 · methodology

0 comments
read the original abstract

We present Phionyx, a deterministic AI runtime architecture derived from the broader Echoism interaction framework that introduces a governance-first approach to AI engineering: treating large language model (LLM) outputs as noisy sensor measurements rather than direct decisions. Unlike probabilistic agents, Phionyx enforces deterministic state evolution via a structured state vector governed by deterministic state-evolution equations, enabling reproducible behavior in applications requiring auditability and governance. The architecture integrates three layers: (1) a deterministic evaluation kernel processing noisy sensor measurements through a canonical 46-block pipeline, (2) a unified safety layer providing pre-response control and architectural privacy enforcement, and (3) a semantic time-based memory system implementing impact-weighted cache eviction. Experimental validation on single-instance deployments demonstrates approximately 31% reduction in computational overhead vs. post-hoc filtering (at 30% unsafe input ratio, simulated cost model) and up to 24% improvement in high-value data retention vs. LRU (72% vs. FIFO, same cache capacity, benchmark-verified), deterministic execution verified across 100 repeated runs with zero variance in control signals (hash-verified), and zero unplanned restarts in single-instance deployment testing (see Appendix C for methodology and scope). This paper presents the architecture, its analytic structure, and scoped experimental evidence; generalization to distributed or multi-tenant deployments remains future work.

Figures

Figures reproduced from arXiv: 2607.18246 by Ali Toygar Abak.

Figure 1
Figure 1. Figure 1: Three-layer Phionyx architecture: (1) Deterministic Cognitive [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Canonical pipeline (46 blocks, v3.8.0) organized into six functional [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The noisy sensor separation: LLM outputs ( [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Semantic time-based memory system: Time vector [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 10 linked inside Pith

  1. [1]

    R. S. Michalski, J. G. Carbonell, and T. M. Mitchell,Machine Learning: An Artificial Intelligence Approach. Springer, 2013

    2013

  2. [2]

    Newell and H

    A. Newell and H. A. Simon,Human Problem Solving. Prentice-Hall, 1972

    1972

  3. [3]

    Tsang,Foundations of Constraint Satisfaction

    E. Tsang,Foundations of Constraint Satisfaction. Academic Press, 1993

    1993

  4. [4]

    C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, ”On Calibration of Modern Neural Networks,”International Conference on Machine Learning (ICML), pp. 1321–1330, 2017. 21

    2017

  5. [5]

    Kadavath et al., ”Language Models (Mostly) Know What They Know,”arXiv preprint arXiv:2207.05221, 2022

    S. Kadavath et al., ”Language Models (Mostly) Know What They Know,”arXiv preprint arXiv:2207.05221, 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Bai et al., ”Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback,”arXiv preprint arXiv:2204.05862, 2022

    Y. Bai et al., ”Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback,”arXiv preprint arXiv:2204.05862, 2022

    Pith/arXiv arXiv 2022

  7. [7]

    Askell et al., ”A General Language Assistant as a Laboratory for Alignment,”arXiv preprint arXiv:2112.00861, 2021

    G. Askell et al., ”A General Language Assistant as a Laboratory for Alignment,”arXiv preprint arXiv:2112.00861, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Bai et al., ”Constitutional AI: Harmlessness from AI Feedback,” arXiv preprint arXiv:2212.08073, 2022

    Y. Bai et al., ”Constitutional AI: Harmlessness from AI Feedback,” arXiv preprint arXiv:2212.08073, 2022

    Pith/arXiv arXiv 2022

  9. [9]

    Inan et al., ”Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations,”arXiv preprint arXiv:2312.06674, 2023

    H. Inan et al., ”Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations,”arXiv preprint arXiv:2312.06674, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, ”Generative Agents: Interactive Simulacra of Human Be- havior,”ACM Symposium on User Interface Software and Technology (UIST), 2023

    2023

  11. [11]

    Weston, S

    J. Weston, S. Chopra, and A. Bordes, ”Memory Networks,”arXiv preprint arXiv:1410.3916, 2014

    Pith/arXiv arXiv 2014

  12. [12]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber, ”Long Short-Term Memory,”Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  13. [13]

    Graves, G

    A. Graves, G. Wayne, and I. Danihelka, ”Neural Turing Machines,” arXiv preprint arXiv:1410.5401, 2014

    Pith/arXiv arXiv 2014

  14. [14]

    Sukhbaatar, A

    S. Sukhbaatar, A. Szlam, J. Weston, and R. Fergus, ”End-To-End Mem- ory Networks,”Advances in Neural Information Processing Systems, vol. 28, 2015

    2015

  15. [15]

    Vinyals et al., ”Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning,”Nature, vol

    O. Vinyals et al., ”Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning,”Nature, vol. 575, no. 7782, pp. 350–354, 2019

    2019

  16. [16]

    R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, ”Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Envi- ronments,”Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  17. [17]

    H. B. McMahan et al., ”Communication-Efficient Learning of Deep Net- works from Decentralized Data,”International Conference on Artificial Intelligence and Statistics (AISTATS), 2017

    2017

  18. [18]

    Kairouz et al., ”Advances and Open Problems in Federated Learn- ing,”Foundations and Trends in Machine Learning, vol

    P. Kairouz et al., ”Advances and Open Problems in Federated Learn- ing,”Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021. 22

    2021

  19. [19]

    Chase, ”LangChain: Building Applications with LLMs through Composability,” GitHub, 2022

    H. Chase, ”LangChain: Building Applications with LLMs through Composability,” GitHub, 2022

    2022

  20. [20]

    Wu et al., ”AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation,”arXiv preprint arXiv:2308.08155, 2023

    Q. Wu et al., ”AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation,”arXiv preprint arXiv:2308.08155, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    LangChain Inc., ”LangSmith: LLM Observability and Evaluation Plat- form,” 2024

    2024

  22. [22]

    Arize AI, ”Phoenix: AI Observability and Evaluation,” 2023

    2023

  23. [23]

    Rebedea et al., ”NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails,”arXiv preprint arXiv:2310.10501, 2023

    G. Rebedea et al., ”NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails,”arXiv preprint arXiv:2310.10501, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Guardrails AI, ”Guardrails: Adding Guardrails to Large Language Models,” GitHub, 2023

    2023

  25. [25]

    Beurer-Kellner, M

    L. Beurer-Kellner, M. Fischer, and M. Vechev, ”Prompting Is Pro- gramming: A Query Language for Large Language Models,”ACM SIGPLAN Conference on Programming Language Design and Imple- mentation (PLDI), 2023

    2023

  26. [26]

    Khattab et al., ”DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines,”arXiv preprint arXiv:2310.03714, 2023

    O. Khattab et al., ”DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines,”arXiv preprint arXiv:2310.03714, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    Microsoft, ”Guidance: A Guidance Language for Controlling Large Language Models,” GitHub, 2023. A Canonical Pipeline Block Sequence The 46-block canonical pipeline (v3.8.0) executes in the following order: 1.kill switch gate- Emergency shutdown gate (fail-closed) 2.time update sot- Time semantics update 3.input safety gate- Combined input gate and safety ...

    2023