From Static to Intelligent: Evolving SaaS Pricing with LLMs

Antonio Ruiz-Cort\'es; Francisco Javier Cavero; Juan C. Alonso

REVIEW 4 major objections 5 minor 19 references

From Static to Intelligent: Evolving SaaS Pricing with LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an LLM-driven Information Extractor can automate the first step of converting static SaaS pricing pages into machine-readable intelligent-pricing models, with validation on 30 commercial SaaS websites reporting mean…

desk verdict Useful extractor with released code; evaluation credits unextracted dynamic content as correct, inflating the reported accuracy. read the letter →

arxiv 2507.12104 v1 pith:7TNIJYJD submitted 2025-07-16 cs.SE cs.AI

Francisco Javier Cavero , Juan C. Alonso , Antonio Ruiz-Cortés This is my paper

classification cs.SEcs.AI

keywords SoftwareasaServiceiPricingLargeLanguageModelinformationextractionSaaSautomationPricing4Saa

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that SaaS pricing pages, which are currently maintained by hand and grow exponentially complex, can be automatically turned into intelligent pricing: dynamic, machine-readable models in YAML. It presents AI4Pricing2Yaml, an LLM-driven information extractor that scrapes a pricing URL and uses a large language model to pull out plans, features, usage limits, and add-ons, then reports validation on 30 commercial SaaS sites. The strongest results are for features (88.2% mean accuracy, 96.4% recall) and plans (100% recall), while add-ons and usage limits are far less reliable (53.5% and 67% mean accuracy). The paper frames the work as the first step toward fully automated pricing transformation.

What carries the argument

The load-bearing mechanism is the Information Extractor, a pipeline that pairs Selenium, which renders JavaScript-generated HTML, with the Gemini 1.5 Flash LLM, whose large context window lets the model see an entire pricing page at once. Prompt engineering directs the model to output the four pricing elements—plans, features, usage limits, and add-ons—in the structure defined by the Pricing4SaaS metamodel, ready to be serialized with the Pricing2Yaml syntax. The extractor is designed for pages that contain a comparison table with all plans and features; pages that hide information behind clicks or lack structured tables are known failure cases.

What would settle it

Take a new sample of 20 SaaS pricing pages known to contain add-ons in structured tables, run AI4Pricing2Yaml on them, and have an independent annotator score every extracted item without peeking at the tool's output. If add-on precision remains around 60% or accuracy near 50% while the gold standard counts only visible items, the claim that the extractor achieves high accuracy across the pipeline would fail for the weakest component.

Watch

Extended reading notes

Core claim

The central claim is that an LLM-based Information Extractor can automate the first stage of transforming a static HTML pricing page into an intelligent pricing expressed with the Pricing4SaaS metamodel, and that the resulting extraction is accurate and complete enough to be feasible. Concretely, the paper reports that on a dataset of 30 commercial SaaS websites (more than 150 price configurations), the system achieves 100% recall for plans, 96.4% recall for features, 77.8% for usage limits, and 81% for add-ons, with precision trailing at 61.4%, 91.1%, 83.8%, and 63% respectively. The authors state that feature extraction performed well, that plans are extracted with perfect recall, and that challenges remain in add-ons and usage limits, as well as in dynamic or complex pages.

Load-bearing premise

The evaluation treats the manually created gold standard for the 30 websites as correct, and counts items that were never extracted (because they sit behind clicks or in unstructured layouts) as True Negatives, which inflates all reported accuracy numbers.

Editorial extensions

If this is right

If the extraction is reliable, DevOps teams could stop hand-maintaining pricing models; changes to a pricing page could be reflected in a machine-readable model automatically.
High recall on plans and features means an automated competitive-analysis tool could monitor dozens of competitors' pricing pages for changes.
The remaining gap lies in add-ons and usage limits; improving those requires either better prompts or a separate validation stage, since precision for add-ons is already low.
Since only 15 of 30 sites yielded features, usage limits, and add-ons, the approach's current scope is effectively limited to comparison-table-based pricing pages; broadening it requires handling dynamic content.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the paper's claim of feasibility is accepted for feature extraction, a natural next step would be to treat the extractor as a monitoring sensor that watches a pricing page over time and emits diffs, turning pricing evolution into a versioned software artifact.
The reported accuracy may be optimistic: counting items hidden behind clicks as True Negatives inflates scores, so an independent annotation that treats unseen data as unknown, rather than correct, would likely lower the numbers, especially for add-ons.
Because the pipeline's success is tied to comparison-table layouts, a testable extension would be an LLM agent that clicks through dynamic content; the paper itself suggests this, and it would directly address the 15 sites where the extractor returned nothing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a real, reproducible engineering contribution with a flawed evaluation and overstated conclusions. The extractor exists, the code and raw outputs are on Zenodo, and running it against 30 live SaaS pages is a legitimate check. Feature extraction picked up most of what was there. That is the part worth keeping.

The soft spots are exactly where the reader's report puts them. In Section 4.1, a True Negative is defined as data not extracted because it was absent from the scraped information, “although it might be accessible through dynamic interactions.” The rubric gives full credit for those. That means the system is rewarded for not clicking buttons. The paper then says only 15 of 30 sites yielded features, usage limits, and add-ons; the other 15 are silently dropped from those metrics. Every headline accuracy/recall figure in Figure 3 inherits that inflation. The abstract claim about 'high accuracy and recall across all steps' does not survive contact with the definition.

The other overreach is scope. Section 3.2 says only the Information Extractor was implemented; the Process Engine and Results Modeler are not built or tested. But the abstract and conclusion talk about automating the full transformation 'across all steps.' That is a mismatch.

The gold standard is the authors' own dataset and schema. That is not fatal, but with the TN rule it makes it hard to tell what should have been in the expected-positive set. There are also no baselines—no comparison with a scraping-only pipeline, a smaller-context LLM, or a cheaper heuristic. So we don't learn much about whether the LLM is doing the work.

None of these are deep conceptual flaws. The scoring definition can be fixed by separating true negatives from unextracted dynamic content, reporting the 15/15 split, and restricting claims to the implemented component. The artifact is worth refereeing because it is concrete and independent results are cheap to check. I'd send it to peer review with expectations of major revision, and I'd want to see the revised evaluation before citing it. For a reading group, it would be a useful case study in how metric definitions create headline results.

Referee Report

4 major / 5 minor

Summary. The paper introduces the notion of 'intelligent pricing' (iPricing) as a machine-readable, dynamically maintainable representation of SaaS pricing, and presents AI4Pricing2Yaml, an LLM-based system intended to transform static HTML pricing pages into this representation. The proposed architecture has three components (Information Extractor, Process Engine, Results Modeler), but the implementation and evaluation cover only the Information Extractor, which uses Selenium for scraping and Gemini 1.5 Flash with prompt engineering to extract plans, features, usage limits, and add-ons. The evaluation is performed on a dataset of 30 commercial SaaS websites taken from the authors' prior work, with metrics of accuracy, precision, and recall computed per element type. The paper reports high recall for plans and features, lower performance for usage limits and add-ons, and discusses challenges with dynamic content and hallucinations. The central claim is that the system automates the transformation of static pricing into intelligent pricing with high accuracy and recall.

Significance. If the claims were fully supported, the paper would offer a useful reusable component for SaaS pricing analysis. The strengths are the concrete implementation, the use of a real-world dataset of 30 commercial websites, the absence of fitted parameters (the approach relies on prompt engineering of a pre-trained LLM), and the availability of supplementary artifacts including raw outputs and code. However, the evaluation design in Section 4.1 is the load-bearing weakness: the True Negative definition credits the system for not retrieving dynamic content, and 15 of 30 sites are excluded from the per-element metrics. As a result, the headline accuracy and recall figures do not substantiate the abstract's claim that the system extracts the desired elements 'across all steps.' The paper is a reasonable proof-of-concept for the extraction step, but the full transformation claim and the quantitative strength of the results require substantial revision.

major comments (4)

[Section 4.1, Table 1, Fig. 3] The definition of True Negatives ('Data not extracted because it was absent from the scraped information, although it might be accessible through dynamic interactions') credits the extractor for failing to retrieve content that exists behind clicks or modals. Because accuracy is computed as (TP+TN)/total, the system is rewarded exactly for the cases it misses. The text further states that only 15 of the 30 SaaS platforms had features, usage limits, and add-ons successfully extracted; the remaining 15 are not counted in the per-element means. The headline metrics (e.g., 88.2% feature accuracy, 53.5% add-on accuracy, and the corresponding recall values in Fig. 3) are therefore computed on a favorable subset and do not support the abstract's claim that the system extracts 'the desired elements across all steps' for the 30-site dataset. Please recompute the results treating unextracted dynamic content as false negatives, including all 30 sites in every per-element metric, and reporting accuracy both with and without TN contributions.
[Section 3.2 vs. Section 6 and Abstract] Section 3.2 explicitly states that only the Information Extractor was fully implemented ('we have fully implemented the first one, the Information Extractor'), while the Process Engine and Results Modeler are part of the architecture but were not implemented or evaluated. Nevertheless, the abstract and Section 6 conclude that AI4Pricing2Yaml automates the transformation of static HTML pricing into iPricing and 'efficiently extracts key components... achieving high accuracy and recall.' This overclaims: the validation covers only the extraction component, not the full pipeline. Please either implement and evaluate the remaining components or explicitly restrict the conclusions to the extraction step.
[Section 4.1, scoring system] The point-based scoring system with half points is not a standard evaluation protocol and makes the reported metrics difficult to interpret. Awarding half a point to both the TP and FP categories for the same item (e.g., a feature extracted but marked as available in too many plans) counts the same evidence in both the numerator and denominator of precision and inflates the total number of extractions. Please provide raw TP/FP/FN/TN counts per site and per element without fractional credits, or define a multi-label loss function that justifies the scoring. The reported means should be reproducible from raw counts.
[Abstract and Table 1] The phrase 'encompassing over 150 intelligent pricings' is not defined or derivable from Table 1. Table 1 reports per-element counts for 30 SaaS sites; no aggregation or definition shows where 150 comes from. Please clarify what counts as an 'intelligent pricing' in this number, and how it relates to the 30 websites and the extracted plans, features, usage limits, and add-ons.

minor comments (5)

[Section 1] The text contains the typo 'through apric-ing'; this should read 'through a pricing' or similar.
[Section 3.2] The context window is given as 'up to 106 tokens'; this should read '10^6 tokens'.
[Section 4.1] The sentence 'A full point is awarded for each correct extraction (TP or TN) and for incorrect extractions (FP or FN)' appears to be a typo: incorrect extractions should receive zero points, not a full point.
[Table 1] Table 1 is very hard to read because the column headers repeat 'TP FP FN TN T A (%) P (%) R (%)' for each of the four element types without visual separation; please reformat the table or split it into separate tables per element type.
[Section 4.6] The proposal of an 'ideal SaaS pricing webpage' is based on the authors' six-year analysis, but the text does not present evidence or criteria for how the ideal structure was derived; this should be stated as a recommendation rather than a validated finding.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: extraction scores come from external websites; self-cited dataset/metamodel are context, not derivation.

full rationale

The claimed derivation chain is empirical rather than definitional. Section 4 applies the Section 3.2 Information Extractor to 30 live SaaS pricing pages and scores the outputs against manually built annotations using standard TP/FP/FN/TN counts. No fitted parameter is later relabeled as a prediction, and no quantity is defined in terms of the outcome it is supposed to establish; accuracy, precision, and recall are computed as standard ratios of observed classifications. The dataset from reference [7] and the Pricing4SaaS metamodel from reference [5] come from the authors' own prior work, so the benchmark is self-referential in provenance, but the ground-truth content is external (the SaaS websites) and the reported counts are checkable from the supplementary raw outputs [1]. The evaluation's TN rule, which credits content absent from the scraped information as correct even when it might be reachable through dynamic interactions, may inflate accuracy, and Section 6's 'high accuracy and recall' overstates the mean add-on accuracy of 53.5% and the fact that only 15 of 30 sites yielded features, usage limits, and add-ons. These are genuine evaluation-validity and overclaiming concerns, but they are not circularity: the extraction outputs are not derived from the benchmark, and no self-citation chain or definitional equivalence forces the reported result. The self-citations are contextual and not load-bearing for the extraction scores.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The system has no fitted numeric parameters. It relies on the authors' own Pricing4SaaS metamodel as the target schema, on the correctness of their manual gold standard, and on a generous True Negative definition. The 'iPricing' concept is a rebranding of the authors' prior metamodel, not a new independent entity.

assumptions (4)

ad hoc to paper The manual annotation of pricing elements for the 30 SaaS websites is correct and complete.
Used as the ground truth for all accuracy, precision, and recall calculations in Section 4.
ad hoc to paper Items not extracted because they are behind dynamic interactions are correctly treated as True Negatives.
Defined in Section 4.1 and used in scoring; this can overstate accuracy.
domain assumption The Pricing4SaaS metamodel is an appropriate target representation for SaaS pricing.
The output schema is taken from the authors' prior work (references [5], [12]), not independently validated.
domain assumption Selenium's rendered HTML completely represents the pricing information needed for extraction.
The LLM only sees what Selenium scrapes; dynamic content that requires clicks is excluded.

invented entities (1)

iPricing (intelligent pricing)
purpose: Labels the concept of a dynamic, machine-readable pricing artifact that can be managed as software.
The concept is a rebranding of the Pricing4SaaS metamodel from the authors' own prior work; it makes no independent falsifiable prediction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Static to Intelligent: Evolving SaaS Pricing with LLMs." pith.science (2026). https://pith.science/paper/7TNIJYJD

@misc{pith2026250712104,
  author       = {Pith},
  title        = {Pith review of: From Static to Intelligent: Evolving SaaS Pricing with LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7TNIJYJD}},
  note         = {Machine review of arXiv:2507.12104}
}

read the original abstract

The SaaS paradigm has revolutionized software distribution by offering flexible pricing options to meet diverse customer needs. However, the rapid expansion of the SaaS market has introduced significant complexity for DevOps teams, who must manually manage and evolve pricing structures, an approach that is both time-consuming and prone to errors. The absence of automated tools for pricing analysis restricts the ability to efficiently evaluate, optimize, and scale these models. This paper proposes leveraging intelligent pricing (iPricing), dynamic, machine-readable pricing models, as a solution to these challenges. Intelligent pricing enables competitive analysis, streamlines operational decision-making, and supports continuous pricing evolution in response to market dynamics, leading to improved efficiency and accuracy. We present an LLM-driven approach that automates the transformation of static HTML pricing into iPricing, significantly improving efficiency and consistency while minimizing human error. Our implementation, AI4Pricing2Yaml, features a basic Information Extractor that uses web scraping and LLMs technologies to extract essential pricing components, plans, features, usage limits, and add-ons, from SaaS websites. Validation against a dataset of 30 distinct commercial SaaS, encompassing over 150 intelligent pricings, demonstrates the system's effectiveness in extracting the desired elements across all steps. However, challenges remain in addressing hallucinations, complex structures, and dynamic content. This work highlights the potential of automating intelligent pricing transformation to streamline SaaS pricing management, offering implications for improved consistency and scalability in an increasingly intricate pricing landscape. Future research will focus on refining extraction capabilities and enhancing the system's adaptability to a wider range of SaaS websites.

Figures

Figures reproduced from arXiv: 2507.12104 by the authors.

**Figure 2.** An overview of the proposed automatic intelligent pricing modeler. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Summary of accuracy, precision, and recall metrics per extracted element. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages

[1]

https://doi.org/10.5281/zenodo.14062533

Cavero, F.J., Alonso, J.C., Ruiz-Cortés, A.: AI4Pricing - Supplementary Material (2024). https://doi.org/10.5281/zenodo.14062533

work page doi:10.5281/zenodo.14062533 2024
[2]

Cavero et al

Figma: Plans & pricings.https://www.figma.com/pricing/, accessed 14 Oct 2024 12 F.J. Cavero et al

work page 2024
[3]

In: Actas de las XIX Jor- nadas de Ciencia e Ingeniería de Servicios (JCIS)

García-Fernández, A., Parejo, J.A., Ruiz-Cortés, A.: Pricing-driven Development and Operation of SaaS: Challenges and Opportunities. In: Actas de las XIX Jor- nadas de Ciencia e Ingeniería de Servicios (JCIS). SISTEDES (2024)

work page 2024
[4]

In: Service- Oriented Computing

García-Fernández, A., Parejo, J.A., Cavero, F.J., Ruiz-Cortés, A.: Racing the Mar- ket: An Industry Support Analysis for Pricing-Driven DevOps in SaaS. In: Service- Oriented Computing. Lecture Notes in Computer Science, vol. 15405, pp. 260–275. Springer Nature Singapore (2025)

work page 2025
[5]

In: Intelligent Information Systems

García-Fernández, A., Parejo, J.A., Ruiz-Cortés, A.: Pricing4SaaS: Towards a Pric- ing Model to Drive the Operation of SaaS. In: Intelligent Information Systems. CAiSE. Lecture Notes in Business Information Processing, vol. 520, pp. 47–54. Springer Nature Switzerland (2024)

work page 2024
[6]

In: Web Engineering

García-Fernández, A., Parejo, J.A., Trinidad, P., Ruiz-Cortés, A.: Towards Pric- ing4SaaS: A Framework for Pricing-Driven Feature Toggling in SaaS. In: Web Engineering. ICWE. pp. 389–392. Springer (2024)

work page 2024
[7]

García-Fernández, A., Parejo, J.A., Cavero, F.J., Ruiz-Cortés, A.: SaaS Analysis - Supplementary Material (2024).https://doi.org/10.5281/zenodo.13857484

work page doi:10.5281/zenodo.13857484 2024
[8]

arXiv (2024).https://doi.org/10.48550/arXiv.2403.05530

Gemini Team: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contex. arXiv (2024).https://doi.org/10.48550/arXiv.2403.05530

arXiv doi:10.48550/arxiv.2403.05530 2024

Show all 19 references

[9]

plans for every developer

GitHub: Pricing. plans for every developer. https://github.com/pricing, ac- cessed 14 Oct 2024

2024
[10]

Google: Generate structured output with the gemini api.https://ai.google.dev/ gemini-api/docs/structured-output, accessed 14 Oct 2024

2024
[11]

Google: Intro to function calling with the Gemini API.https://ai.google.dev/ gemini-api/docs/function-calling, accessed 14 Oct 2024

2024
[12]

https://pricing4saas-docs.vercel.app/docs/2.0

ISA Group: Pricing2Yaml. https://pricing4saas-docs.vercel.app/docs/2.0. 1/category/pricing2yaml, accessed 14 Oct 2024

2024
[13]

ACM Comput

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM Comput. Surv. 55(12) (2023)

2023
[14]

In: 2009 Congress on Services - I

Jiang, Z., Sun, W., Tang, K., Snowdon, J.L., Zhang, X.: A pattern-based design approach for subscription management of software as a service. In: 2009 Congress on Services - I. pp. 678–685 (2009)

2009
[15]

arXiv (2024).https://doi.org/10

Llama Team: The llama 3 herd of models. arXiv (2024).https://doi.org/10. 48550/arXiv.2407.21783

2024 doi arXiv
[16]

arXiv (2024)

OpenAI: Gpt-4 technical report. arXiv (2024). https://doi.org/10.48550/ arXiv.2303.08774

2024 doi arXiv
[17]

In: Advances in Neural Information Processing Systems

Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)

2017
[18]

arXiv (2023).https://doi.org/10.48550/arXiv.2303.18223

Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J.Y., Wen, J.R.: A survey of large language models. arXiv (2023).https://doi.o...

2023 doi arXiv
[19]

Zoom: Plans & pricing for zoom.https://zoom.us/pricing, accessed 14 Oct 2024

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

https://doi.org/10.5281/zenodo.14062533

Cavero, F.J., Alonso, J.C., Ruiz-Cortés, A.: AI4Pricing - Supplementary Material (2024). https://doi.org/10.5281/zenodo.14062533

work page doi:10.5281/zenodo.14062533 2024

[2] [2]

Cavero et al

Figma: Plans & pricings.https://www.figma.com/pricing/, accessed 14 Oct 2024 12 F.J. Cavero et al

work page 2024

[3] [3]

In: Actas de las XIX Jor- nadas de Ciencia e Ingeniería de Servicios (JCIS)

García-Fernández, A., Parejo, J.A., Ruiz-Cortés, A.: Pricing-driven Development and Operation of SaaS: Challenges and Opportunities. In: Actas de las XIX Jor- nadas de Ciencia e Ingeniería de Servicios (JCIS). SISTEDES (2024)

work page 2024

[4] [4]

In: Service- Oriented Computing

García-Fernández, A., Parejo, J.A., Cavero, F.J., Ruiz-Cortés, A.: Racing the Mar- ket: An Industry Support Analysis for Pricing-Driven DevOps in SaaS. In: Service- Oriented Computing. Lecture Notes in Computer Science, vol. 15405, pp. 260–275. Springer Nature Singapore (2025)

work page 2025

[5] [5]

In: Intelligent Information Systems

García-Fernández, A., Parejo, J.A., Ruiz-Cortés, A.: Pricing4SaaS: Towards a Pric- ing Model to Drive the Operation of SaaS. In: Intelligent Information Systems. CAiSE. Lecture Notes in Business Information Processing, vol. 520, pp. 47–54. Springer Nature Switzerland (2024)

work page 2024

[6] [6]

In: Web Engineering

García-Fernández, A., Parejo, J.A., Trinidad, P., Ruiz-Cortés, A.: Towards Pric- ing4SaaS: A Framework for Pricing-Driven Feature Toggling in SaaS. In: Web Engineering. ICWE. pp. 389–392. Springer (2024)

work page 2024

[7] [7]

García-Fernández, A., Parejo, J.A., Cavero, F.J., Ruiz-Cortés, A.: SaaS Analysis - Supplementary Material (2024).https://doi.org/10.5281/zenodo.13857484

work page doi:10.5281/zenodo.13857484 2024

[8] [8]

arXiv (2024).https://doi.org/10.48550/arXiv.2403.05530

Gemini Team: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contex. arXiv (2024).https://doi.org/10.48550/arXiv.2403.05530

arXiv doi:10.48550/arxiv.2403.05530 2024