Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Prompt Orchestration Markup Language

T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read POML claims prompt formatting should be a separate, testable layer: on WikiTQ, style changes move GPT-3.5 Turbo accuracy from 6% to 62% and Phi-3 Medium by 4,450%.

desk verdict A well-built prompt engineering artifact, but the TableQA study overstates what it proves: it shows format sensitivity, not that POML improves accuracy. read the letter →

arxiv 2508.13948 v1 pith:QA37IJOR submitted 2025-08-19 cs.HC cs.AIcs.CLcs.PL

Yuge Zhang , Nan Chen , Jiahang Xu , Yuqing Yang This is my paper
classification cs.HCcs.AIcs.CLcs.PL
keywords promptengineeringPOMLmarkuplanguageLLMformatsensitivitystylesheetseparationstructuredpromptingdataintegrationdevelopertooling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces POML, a markup language for writing LLM prompts as structured components (roles, tasks, examples) with dedicated tags for embedding documents, tables, and images. Its central proposal is to separate prompt content from prompt presentation through a CSS-like stylesheet system, so developers can change formatting without rewriting the prompt's logic. The paper argues this matters because LLMs are extremely sensitive to formatting: on a subset of WikiTableQuestions, choosing the best of 100 sampled styles instead of the worst raised GPT-3.5 Turbo's accuracy from 6% to 61.8% and Phi-3 Medium's from 0.7% to 32.2%. POML's value would be that it makes such variation systematically explorable — one 30-line base prompt plus generated stylesheets produced roughly 74,000 distinct prompt styles — and manageable in real applications. A two-day iOS agent prototype and a seven-participant user study support the broader claim that the language and its IDE tooling improve the prompt-engineering workflow.

What carries the argument

The load-bearing mechanism is POML's styling system: a CSS-like layer where a stylesheet (JSON or inline stylesheet tag) controls presentation attributes — overall syntax, table serialization format, caption style and visibility, example layout (chat vs introducer vs plain), list markers — independently of semantic markup. A three-pass rendering pipeline (parse into components, build an intermediate representation, serialize through a target writer) makes the decoupling operational, allowing one base prompt to be rendered into tens of thousands of styles and into different output formats (Markdown, JSON, plain text) from the same representation.

What would settle it

Render the same POML source under two stylesheets and record the exact final strings sent to the model. If two styles that produce identical strings nevertheless lead to different answers, the styling system is not the causal variable; if two styles that are meant to differ only in presentation change which content is visible (e.g., a hidden caption or a collapsed table drops a fact the model needs), the decoupling premise fails. A direct test would compare model accuracy on style pairs whose rendered strings differ only in whitespace or order versus pairs that delete or reorder content.

Watch

Extended reading notes

Core claim

POML's core claim is that prompt presentation can be engineered as a first-class, content-independent layer. Using HTML-like components for intention (role, task, output-format) and data (document, table, image, folder, conversation), and a JSON stylesheet system controlling syntax (Markdown, JSON, XML, HTML), captions, list styles, chat-versus-block layout, and verbosity, a single POML source can be rendered into many presentation variants without touching the underlying content. The empirical heart is the TableQA case study: 100 randomly sampled styles applied to one base prompt on 283 WikiTQ samples produced accuracy swings of 16% to 4,450%, with optimal styles differing by model; some mo

Load-bearing premise

The load-bearing premise is that changing a prompt's presentation never changes its meaning: POML assumes stylesheets can alter syntax, captions, and layout while the model still sees the same underlying content.

Editorial extensions

If this is right

  • If styling is decoupled, a single prompt's content can be re-rendered for different target models, and the TableQA results indicate the optimal rendering is model-specific rather than universal.
  • Prompt teams can treat formatting as a tunable search dimension: the paper demonstrates 73,926 styles generated from one 30-line base POML file, so systematic experiments can replace hand-maintained parallel copies.
  • Format-sensitivity findings imply that published prompt-engineering results should specify or control presentation, since a style change can swing accuracy more than many content changes.
  • The PomLink 'Auto' table format shows the same data can be serialized differently per backend model, making style adaptation a runtime deployment decision rather than a manual rewrite.
  • Because POML files are text and modular, they fit standard version-control workflows, easing collaboration on prompt changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the paper leaves implicit is that prompt-styling search can be automated: the stylesheet representation is machine-readable, so an optimizer could sample and score styles the way the paper did manually, and the model-specific rankings suggest the objective function differs per model.
  • The decoupling premise implies a testable boundary: if a style change ever alters token order, emphasis, or which content is visible, it is no longer pure presentation. That boundary is worth probing because hidden captions and collapsed tables can delete information.
  • A testable extension is to apply the same three-pass rendering idea beyond single-turn text prompts — for example, to agent system prompts or multimodal contexts, where the content tree stays stable but the target writer changes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces POML (Prompt Orchestration Markup Language), an HTML-like markup language for authoring LLM prompts. POML provides structural components (<role>, <task>, <example>), data components for documents/tables/images, a CSS-inspired styling system that decouples content from presentation, and a templating engine. The authors also describe an IDE extension (VSCode IntelliSense, live preview, diagnostics) and Python/Node SDKs. Validation consists of two case studies: PomLink, an iOS agent prototype built in two days using POML, and TableQA, a study of 100 randomly sampled prompt styles applied to 283 WikiTQ samples across 8 LLMs. In addition, a user study with 7 participants assesses usability across five tasks. The paper claims that POML reduces formatting sensitivity and improves accuracy performance, and that its tooling improves developer workflow.

Significance. If the empirical claims were fully supported, POML would be a meaningful contribution to prompt-engineering tooling: the design is comprehensive, the implementation appears substantial (14.8k lines, 37 components, 115 test cases), and the detailed appendices (style search space, three-pass rendering architecture, IR specifications) are valuable. The PomLink case study and the qualitative user-study feedback suggest that POML's data components and live preview are genuinely useful in practice. However, the central quantitative claim—that POML's styling system improves accuracy or mitigates format sensitivity—is not established by the TableQA experiment as presented. The experiment shows, at best, that LLM performance is sensitive to prompt formatting, a known phenomenon, and that POML can generate many formatting variants. The paper overstates this as POML's impact on accuracy. The user study, while informative, is small and lacks a comparison baseline. With targeted revisions to the evaluation and a more measured interpretation, the paper could be a solid systems/HCI contribution.

major comments (5)
  1. [§7.2, Table 2] The TableQA experiment has no baseline. The reported 'accuracy improvements' of 929% and 4450% are relative differences between the worst and best of 100 randomly sampled styles, not improvements of POML over plain-text prompts or existing markup tools. The abstract and contribution (3) claim POML 'demonstrates its impact on accuracy performance,' but the design cannot support that. To support such a claim, the paper must include a baseline condition (e.g., the same TableQA task written as plain text or in an existing format such as ChatML/PromptML) and compare POML-rendered prompts against it. Without this, the paper should be reworded to claim only that styling variations affect LLM accuracy, not that POML improves it.
  2. [§7.2, Table 1 and §7.1 'Auto' option] The 'optimal' styles in Table 1 are selected post hoc on the same 283 samples used to compute all accuracy numbers. There is no held-out evaluation of the selected styles. This matters because the PomLink 'Auto' option (§7.1) consumes these fitted results to choose table formats for new prompts. The self-correlation metric (Table 2) is a split-half stability measure, not a validation of the top style. The authors should split the data, select styles on a training subset, and evaluate on a held-out subset; they should also report whether the 'Auto' choice outperforms a fixed default or random style on unseen data.
  3. [Table 2, Phi-3 Medium row] The extreme relative improvements are statistically fragile. Phi-3 Medium's minimum accuracy is 0.007, which on 283 samples corresponds to roughly 2 correct answers, so the denominator is indistinguishable from zero. Reporting a 4450% improvement from such a baseline is misleading. The paper should report absolute differences with confidence intervals (e.g., Wilson intervals) and should avoid presenting min/max extremes from 100 random draws as evidence of POML's benefit. Similarly, Table 6 performs many Mann-Whitney U tests without multiple-comparison correction; the p-values should be adjusted (e.g., Benjamini-Hochberg) and effect sizes reported.
  4. [§8 and §9.2] The user study's central claims are limited by the acknowledged small sample (N=7), the restricted 90-minute sessions, and the authors' own admission that most participants 'did not deeply engage with complex stylesheets' (§9.2). The paper nevertheless describes the study as a 'formal user study' and uses it to support POML's effectiveness. I would ask the authors to either temper the language (e.g., 'pilot study') or add complementary evidence, such as a comparison task with plain-text prompts or with another markup tool. The current design cannot distinguish POML's effect from the Hawthorne effect of a new tool in a lab setting.
  5. [§4.3, DG3] The design goal DG3 assumes that prompt presentation can be decoupled from content without changing the underlying meaning the model sees. The paper operationalizes this through stylesheets, but does not verify content invariance: changing syntax, caption styles, or list styles also changes tokenization, ordering, and emphasis, which could alter semantics. This is not necessarily a fatal flaw, but it is a load-bearing assumption for the claim that POML 'reduces formatting sensitivity' rather than simply shifting it. The TableQA study could partially address this by comparing styles that differ only in presentation while holding the exact words constant, but the current analysis does not isolate this. I recommend adding an explicit test or at least a careful discussion of this limitation.
minor comments (6)
  1. [Table 2 caption] The term 'Self-corr.' is used without a definition in the main text. It is defined only later in Appendix E. Please define it in the caption or in §7.2.
  2. [Figure 6] In Figure 6, the diagnostic message shows 'captionStyle should be one of header, bold, plain, hidden, not "head"' but the list of valid values is inconsistent with the text in §4.3, which mentions 'header, bold, plain, hidden'. This is a small documentation inconsistency that could confuse users.
  3. [Figure 10] The style search space diagram (Figure 10) is dense and hard to read, especially the labels for options (a) through (e). A table listing the discrete choices with their cardinalities would improve reproducibility and clarity.
  4. [§7.2] The paper says 'we randomly sampled 100 styles without replacement for evaluation' but does not report the random seed or whether the same styles were used across all models. Please clarify whether the 100 styles are identical across models and provide the seed for reproducibility.
  5. [Appendix E, Table 6] Table 6 uses arrows (D, ✗, –) with p-values in cells. The notation is not intuitive; a legend would help. Also, the interaction-effect rows are valuable but are not discussed in the main text.
  6. [Throughout] The paper uses the phrase 'rigorous empirical evaluation' (e.g., §7 and §10). Given the limitations of the TableQA and user studies, I recommend a more measured description such as 'initial empirical exploration' or 'case-study-based evaluation.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: POML's claims rest on new implementations, a user study, and fresh accuracy measurements; the TableQA 'optimal style' lookups are descriptive fits, not predictions.

full rationale

The paper's central derivation is self-contained. The TableQA study (Section 7.2) samples 100 styles, measures accuracy on 283 WikiTQ examples, and reports min/max values; the 'optimal style' entries in Table 1 are the best among those samples, i.e., descriptive maxima, not predicted values. PomLink's 'Auto' table-format option then looks up those empirically determined best formats (Section 7.1), which is a fitted input used as a feature, not a prediction of unseen style performance. The other load-bearing evidence for POML's utility is the PomLink implementation, the seven-participant user study (Section 8), and the IDE/SDK toolkit description, none of which reduce by construction to the claims being validated. Self-citations (e.g., [18], [48], [103]) appear as background or as design motivation; the format-sensitivity premise is also supported by independent external citations [73,76,89] and by the paper's own fresh measurements, so these self-citations are not load-bearing. No equation, definition, or fitted parameter is renamed as a prediction. Statistical and validity weaknesses in TableQA (no baseline, extreme min/max without confidence intervals) concern experimental rigor, not circularity, and are left out of this score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that formatting is separable from content, that the evaluated models and tasks are representative, and that the rendering pipeline is faithful. The only fitted quantities are the per-model optimal styles and the hand-constructed style search space. No new physical or conceptual entities are postulated beyond the POML language itself.

free parameters (2)
  • Model-specific optimal styling configurations (Table 1) = e.g., GPT-3.5 Turbo: Overall Syntax=Markdown, Table Syntax=TSV, Instruction Header=Plain-Upper-Colon, Example Caption=He
    These configurations are the best among 100 randomly sampled styles evaluated on 283 WikiTQ samples, selected post hoc for each model. They are descriptive fits, not validated predictions on held-out data.
  • Style search space dimensionality (Figure 10) = 74k combinations from hand-chosen style options
    The set of style dimensions and candidate values (e.g., caption styles, table syntaxes, example body formats) is chosen by the authors rather than derived from theory, and the 100 evaluated styles are a random subset of this space.
assumptions (4)
  • domain assumption LLM prompt formatting sensitivity is a real, general phenomenon that can be measured on WikiTQ.
    Invoked in Section 2.1 and Section 7.2 via cited references and the TableQA experiment. If format sensitivity were negligible or task-specific, the central motivation for the styling system weakens.
  • domain assumption The 8 selected low-cost LLMs and the 283 WikiTQ samples are representative of LLM prompt behavior.
    Used in Section 7.2 to draw conclusions about model-specific optimal styles. The sample is small and the paper acknowledges the models are all low-cost as of February 2025.
  • domain assumption A single POML template combined with stylesheets can faithfully generate all 74k prompt variations without hidden interactions between content and styling.
    Assumed throughout Section 7.2, where the rendering pipeline is trusted to preserve content while changing presentation. Section 6 describes the architecture but provides no independent verification of rendering faithfulness.
  • domain assumption The user study's think-aloud sessions and self-reported metrics capture real workflow benefits.
    Section 8 relies on qualitative feedback from 7 participants, with no control condition and minimal intervention from facilitators.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompt Orchestration Markup Language." pith.science (2026). https://pith.science/paper/QA37IJOR

@misc{pith2026250813948,
  author       = {Pith},
  title        = {Pith review of: Prompt Orchestration Markup Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QA37IJOR}},
  note         = {Machine review of arXiv:2508.13948}
}
read the original abstract

Large Language Models (LLMs) require sophisticated prompting, yet current practices face challenges in structure, data integration, format sensitivity, and tooling. Existing methods lack comprehensive solutions for organizing complex prompts involving diverse data types (documents, tables, images) or managing presentation variations systematically. To address these gaps, we introduce POML (Prompt Orchestration Markup Language). POML employs component-based markup for logical structure (roles, tasks, examples), specialized tags for seamless data integration, and a CSS-like styling system to decouple content from presentation, reducing formatting sensitivity. It includes templating for dynamic prompts and a comprehensive developer toolkit (IDE support, SDKs) to improve version control and collaboration. We validate POML through two case studies demonstrating its impact on complex application integration (PomLink) and accuracy performance (TableQA), as well as a user study assessing its effectiveness in real-world development scenarios.

Figures

Figures reproduced from arXiv: 2508.13948 by the authors.

Figure 2
Figure 2. An example illustrating POML’s structured markup [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of POML data components demonstrating integration of diverse data types (§ 4.2). (a) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Demonstrating POML styling capabilities (§ 4.3). (a) Default rendering of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Example of POML’s templating engine (§ 4.4): using [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 8
Figure 8. Figure 8: Using POML Software Development Kits (SDKs) to integrate into programming workflows. (a) JavaScript/TypeScript SDK example using JSX-like tagged template literals. (b) Python SDK example using a context manager approach. logs and streams the LLM’s response in real-time…
Figure 9
Figure 9. Figure 9: PomLink iOS interface, powered by POML. (1) Main [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the prompt styling search space [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Frequency of POML component usage across user [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: The POML three-pass rendering architecture: (1)->(2) The Parser transforms POML markup into React JSX components, [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 14
Figure 14. Figure 14: Correlation matrix showing relationships be [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 13
Figure 13. Figure 13: Example of POML usage for the TableQA case [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When the Code Autopilot Breaks: Why LLMs Falter in Embedded Machine Learning

    cs.SE 2025-09 conditional novelty 5.0 of 10

    LLM-based sketch generation for embedded ML is fragile, with success rates below 40%, and prompt structure alone can swing outcomes from 15% to 30%.

Reference graph

Works this paper leans on

107 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [1]

    Azure AI. 2024. Evaluate your Generative AI application locally with the Azure AI Evaluation SDK. https://learn.microsoft.com/en-us/azure/ai-foundry/how- to/develop/evaluate-sdk

    2024

  2. [2]

    Cursor AI. 2025. Generate Cursor Project Rule (.mdc). https://cursor.directory/ generate

    2025

  3. [3]

    LangChain AI. 2022. LangChain. https://www.langchain.com/

    2022

  4. [4]

    LangChain AI. 2023. LangSmith. https://www.langchain.com/langsmith

    2023

  5. [5]

    Angular. 2010. Introduction to components and templates. https://v17.angular. io/guide/architecture-components

    2010

  6. [6]

    Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. https:// assets.anthropic.com/m/61e7d27f8c8f5919/original/Claude-3-Model-Card.pdf. Preprint, April 2025, arXiv POML Team

    2024

  7. [7]

    Anthropic. 2024. Use XML tags to structure your prompts. https://docs.anthropic. com/en/docs/build-with-claude/prompt-engineering/use-xml-tags

    2024

  8. [8]

    anysphere. 2024. priompt. https://github.com/anysphere/priompt

    2024

Show all 107 references
  1. [9]

    Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, and Elena L Glassman. 2024. ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–18

    2024

  2. [10]

    AutosseyAI. 2024. prxmpt. https://github.com/AutosseyAI/prxmpt

    2024

  3. [11]

    Stephen H Bach, Victor Sanh, Zheng-Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, et al. 2022. Promptsource: An integrated development environment and reposi- tory for natural language prompts. arXiv preprint ...

    2022 arXiv

  4. [12]

    Tim Berners-Lee. 1991. Re: status. Re: X11 BROWSER for WWW. https://lists. w3.org/Archives/Public/www-talk/1991SepOct/0003.html

    1991

  5. [13]

    Bradybry. 2023. ChatXML: A proposal for a structured LLM prompt method. https://github.com/Bradybry/chatXML

    2023

  6. [14]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    2020 arXiv

  7. [15]

    Fernando Miguel Carvalho, Luis Duarte, and Julien Gouesse. 2020. Text web templates considered harmful. In Web Information Systems and Technologies: 15th International Conference, WEBIST 2019, Vienna, Austria, September 18–20, 2019, Revised Selected Papers 15 . Springer, 69–95

    2020

  8. [16]

    ChatPDF. 2024. ChatPDF. https://www.chatpdf.com/

    2024

  9. [17]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    2021 arXiv

  10. [18]

    Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2024. VisEval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics (2024)

    2024

  11. [19]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)

    2021 arXiv

  12. [20]

    Hai Dang, Lukas Mecke, Florian Lehmann, Sven Goller, and Daniel Buschek

  13. [21]

    Google Deepmind. 2024. Introducing Gemini 2.0: our new AI model for the agentic era. https://blog.google/technology/google-deepmind/google-gemini- ai-update-december-2024/

    2024

  14. [22]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL]

    2024 arXiv

  15. [23]

    Ant Design. 2015. Ant Design. https://ant.design/

    2015

  16. [24]

    Michael Desmond and Michelle Brachman. 2024. Exploring Prompt Engineering Practices in the Enterprise. arXiv preprint arXiv:2403.08950 (2024)

    2024 arXiv

  17. [25]

    Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason E Weston. [n. d.]. Chain-of-Verification Reduces Hallucination in Large Language Models. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  18. [26]

    MDN Web Docs. 2025. CSS: Cascading Style Sheets. https://developer.mozilla. org/en-US/docs/Web/CSS

    2025

  19. [27]

    MDN Web Docs. 2025. HTML: HyperText Markup Language. https://developer. mozilla.org/en-US/docs/Web/HTML

    2025

  20. [28]

    MDN Web Docs. 2025. JavaScript. https://developer.mozilla.org/en-US/docs/ Web/JavaScript

    2025

  21. [29]

    Django documentation. 2024. Templates. https://docs.djangoproject.com/en/5. 1/topics/templates/

    2024

  22. [30]

    edspencer. 2024. mdx-prompt. https://github.com/edspencer/mdx-prompt

    2024

  23. [31]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a Graph: Encoding Graphs for Large Language Models. In The Twelfth International Conference on Learning Representations

    2024

  24. [32]

    fixie ai. 2024. AI.JSX — The AI Application Framework for Javascript. https: //github.com/fixie-ai/ai-jsx/

    2024

  25. [33]

    gensx inc. 2024. gensx. https://github.com/gensx-inc/gensx

    2024

  26. [34]

    Handlebars. 2011. Handlebars. https://handlebarsjs.com/

    2011

  27. [35]

    Jia He, Mukund Rungta, David Koleczek, Arshdeep Sekhon, Franklin X Wang, and Sadid Hasan. 2024. Does Prompt Formatting Have Any Impact on LLM Performance? arXiv preprint arXiv:2411.10541 (2024)

    2024 arXiv

  28. [36]

    Xinyi He, Yihao Liu, Mengyu Zhou, Yeye He, Haoyu Dong, Shi Han, Zejian Yuan, and Dongmei Zhang. 2025. TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models. arXiv:2503.04396 [cs.CL] https://arxiv.org/abs/2503.04396

    2025 arXiv

  29. [37]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations

    2021

  30. [38]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guil- laume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-An...

    2024 arXiv

  31. [39]

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can Language Models Resolve Real-world Github Issues?. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum...

    2024

  32. [40]

    Jinja. 2022. Jinja – Jinja documentation. https://jinja.palletsprojects.com/en/3.1. x/

    2022

  33. [41]

    latitude dev. 2024. latitude-llm. https://github.com/latitude-dev/latitude-llm

    2024

  34. [42]

    Avraham Leff and James T Rayfield. 2001. Web-application development using the model/view/controller design pattern. In Proceedings fifth ieee international enterprise distributed object computing conference . IEEE, 118–127

    2001

  35. [43]

    leforestier. 2016. yattag. https://github.com/leforestier/yattag

    2016

  36. [44]

    Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2024. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems 36 (2024)

    2024

  37. [45]

    Demiao Lin. 2024. Revolutionizing Retrieval-Augmented Generation with En- hanced PDF Structure Recognition. arXiv:2401.12599 [cs.AI] https://arxiv.org/ abs/2401.12599

    2024 arXiv

  38. [46]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. arXiv:2304.08485 [cs.CV] https://arxiv.org/abs/2304.08485

    2023 arXiv

  39. [47]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2023. Agent...

    2023 arXiv

  40. [48]

    Yuanye Liu, Jiahang Xu, Li Lyna Zhang, Qi Chen, Xuan Feng, Yang Chen, Zhongxin Guo, Yuqing Yang, and Peng Cheng. 2025. Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimiza- tion. arXiv:2502.04295 [cs.CL] https://arxiv.org/abs/2502.04295

    2025 arXiv

  41. [49]

    Arvid Lunnemark. 2023. Prompt Design. https://arvid.xyz/posts/prompt- design/

    2023

  42. [50]

    mannaandpoem. 2025. Building OpenManus as a Service. https://openmanus. org/

    2025

  43. [51]

    mattnigh. 2023. ChatGPT3-Free-Prompt-List: A free guide for learning to create ChatGPT3 Prompts. https://github.com/mattnigh/ChatGPT3-Free-Prompt-List

    2023

  44. [52]

    Microsoft. 2016. Language Server Protocol. https://microsoft.github.io/language- server-protocol/

    2016

  45. [53]

    Microsoft. 2023. Guidance. https://github.com/guidance-ai/guidance

    2023

  46. [54]

    Microsoft. 2023. Semantic Kernel. https://github.com/microsoft/semantic- kernel

    2023

  47. [55]

    Microsoft. 2024. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv:2404.14219 [cs.CL] https://arxiv.org/abs/2404. 14219

    2024 arXiv

  48. [56]

    Yasuhiko Minamide. 2005. Static approximation of dynamically generated web pages. In Proceedings of the 14th international conference on World Wide Web . 432–441

    2005

  49. [57]

    Aditi Mishra, Utkarsh Soni, Anjana Arunkumar, Jinbin Huang, Bum Chul Kwon, and Chris Bryan. 2023. Promptaid: Prompt exploration, perturbation, testing and iteration using visual analytics for large language models. arXiv preprint arXiv:2304.01964 (2023)

    2023 arXiv

  50. [58]

    MUI. 2014. Material UI. https://mui.com/

    2014

  51. [59]

    OpenAI. 2020. Playground. https://platform.openai.com/playground

    2020

  52. [60]

    OpenAI. 2023. ChatGPT. https://chatgpt.com/

    2023

  53. [61]

    OpenAI. 2024. GPT-3.5 Turbo. https://platform.openai.com/docs/models/gpt- 3.5-turbo

    2024

  54. [62]

    OpenAI. 2024. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] https://arxiv. org/abs/2303.08774

    2024 arXiv

  55. [63]

    OpenAI. 2024. GPT-4o System Card. https://cdn.openai.com/gpt-4o-system- card.pdf

    2024

  56. [64]

    Panupong Pasupat and Percy Liang. 2015. Compositional Semantic Parsing on Semi-Structured Tables. In Proceedings of the 53rd Annual Meeting of the Associ- ation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: L...

    2015

  57. [65]

    pezzolabs. 2024. pezzo. https://github.com/pezzolabs/pezzo

    2024

  58. [66]

    {Structured} Prompt. 2023. {Structured} Prompt. https://structuredprompt.com/

    2023

  59. [67]

    promptfoo. 2023. promptfoo. https://www.promptfoo.dev/

    2023

  60. [68]

    PromptML. 2023. PromptML (Prompt Markup Language). https://www. promptml.org/

    2023

  61. [69]

    React. 2013. Introducing JSX. https://legacy.reactjs.org/docs/introducing-jsx. html

    2013

  62. [70]

    React. 2013. React. https://react.dev/

    2013

  63. [71]

    React. 2024. Server Components. https://react.dev/reference/rsc/server- components. Prompt Orchestration Markup Language Preprint, April 2025, arXiv

    2024

  64. [72]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927 (2024)

    2024 arXiv

  65. [73]

    Abel Salinas and Fred Morstatter. 2024. The butterfly effect of altering prompts: How small changes and jailbreaks affect large language model performance. arXiv preprint arXiv:2401.03729 (2024)

    2024 arXiv

  66. [75]

    Tobias Schnabel and Jennifer Neville. 2024. Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization. arXiv:2404.02319 [cs.CL] https://arxiv.org/abs/2404.02319

    2024 arXiv

  67. [76]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations

    2024

  68. [77]

    Azure AI Services. 2024. Chat Markup Language ChatML (Preview). https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chat- markup-language

    2024

  69. [78]

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. ALFWorld: Aligning Text and Em- bodied Environments for Interactive Learning. InProceedings of the International Conference on Learning Representations (ICLR) . https:...

    2021 arXiv

  70. [79]

    Hendrik Strobelt, Albert Webson, Victor Sanh, Benjamin Hoover, Johanna Beyer, Hanspeter Pfister, and Alexander M Rush. 2022. Interactive and visual prompt engineering for ad-hoc task adaptation with large language models. IEEE transactions on visualization and computer graphic...

    2022

  71. [80]

    Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. 2024. Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study. arXiv:2305.13062 [cs.CL] https: //arxiv.org/abs/2305.13062

    2024 arXiv

  72. [81]

    Michiaki Tatsubori and Toyotaro Suzumura. 2009. HTML templates that fly: a template engine approach to automated offloading from server to client. In Proceedings of the 18th international conference on World wide web . 951–960

    2009

  73. [82]

    CodeGemma Team. 2024. CodeGemma: Open Code Models Based on Gemma. arXiv:2406.11409 [cs.CL] https://arxiv.org/abs/2406.11409

    2024 arXiv

  74. [83]

    Gemini Team. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530 [cs.CL] https://arxiv.org/abs/ 2403.05530

    2024 arXiv

  75. [84]

    Llama Team. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  76. [85]

    Twig. 2009. Twig - The flexible, fast, and secure template engine for PHP. https://twig.symfony.com/

    2009

  77. [86]

    uithub. 2024. uithub. https://uithub.com

    2024

  78. [87]

    Juho Vepsäläinen, Arto Hellas, and Petri Vuorimaa. 2023. The State of Disap- pearing Frameworks in 2023. In Proceedings of the 19th International Conference on Web Information Systems and Technologies, WEBIST 2023 (International Con- ference on Web Information Systems and Tech...

    2023 doi

  79. [88]

    Juho Vepsäläinen, Arto Hellas, and Petri Vuorimaa. 2023. The Rise of Disappear- ing Frameworks in Web Development . Springer Nature Switzerland, 319–326. doi:10.1007/978-3-031-34444-2_23

    2023 doi

  80. [89]

    Anton Voronov, Lena Wolf, and Max Ryabinin. 2024. Mind your format: Towards consistent evaluation of in-context learning improvements. arXiv preprint arXiv:2401.06766 (2024)

    2024 arXiv

  81. [90]

    VSCode. 2024. Prompt-tsx. https://github.com/microsoft/vscode-prompt-tsx

    2024

  82. [91]

    Vue.js. 2014. Components Basics. https://vuejs.org/guide/essentials/component- basics.html

    2014

  83. [92]

    W3C. 2025. Accessibility Fundamentals Overview. https://www.w3.org/WAI/ fundamentals/

    2025

  84. [93]

    W3C. 2025. Accessibility Principles. https://www.w3.org/WAI/fundamentals/ accessibility-principles/

    2025

  85. [94]

    W3C. 2025. W3C. https://www.w3.org/

    2025

  86. [95]

    Ming Wang, Yuanzhong Liu, Xiaoming Zhang, Songlian Li, Yijie Huang, Chi Zhang, Daling Wang, Shi Feng, and Jigang Li. 2024. LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language. arXiv preprint arXiv:2402.16929 (2024)

    2024 arXiv

  87. [96]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In The Eleventh International Conference on Learning Representations

    2023

  88. [97]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reason- ing in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  89. [98]

    Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022. Promptchainer: Chaining large language model prompts through visual programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts . 1–10

    2022

  90. [99]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489 (2024)

    2024 arXiv

  91. [100]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems 35 (2022), 20744–20757

    2022

  92. [101]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Act- ing in Language Models. In The Eleventh International Conference on Learning Representations

    2023

  93. [102]

    JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang

  94. [103]

    Yuge Zhang, Qiyang Jiang, Xingyu Han, Nan Chen, Yuqing Yang, and Kan Ren. 2024. Benchmarking Data Science Agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (...

    2024 doi

  95. [104]

    Jeffrey Zheng. 2023. Role-Task-Format (RTF) framework for prompting. https: //x.com/thejeffreyzheng/status/1660223969755111427

    2023

  96. [105]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. [n. d.]. Large Language Models are Human-Level Prompt Engineers. In The Eleventh International Conference on Learning Repre- sentations

  97. [106]

    file in files

    Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. 2024. ProSA: Assessing and understanding the prompt sensitivity of LLMs. arXiv preprint arXiv:2410.12405 (2024). Preprint, April 2025, arXiv POML Team A POML vs. Other Prompt Markup Languages The...

    2024 arXiv

  98. [2022]

    arXiv preprint arXiv:2209.01390 (2022)

    How to prompt? Opportunities and challenges of zero-and few-shot learning for human-AI interaction in creative applications of generative models. arXiv preprint arXiv:2209.01390 (2022)

    2022 arXiv

  99. [2023]

    In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

    Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.