Pith. sign in

REVIEW 4 major objections 6 minor 44 references

An LLM-enabled semantic-centric framework to consume privacy policies

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs turn privacy policies into formal knowledge graphs at web scale.

desk verdict Useful applied contribution with real resources, but the top-100 Pr2Graph accuracy is asserted, not demonstrated; the 10-policy benchmark cannot carry that weight on its own. read the letter →

arxiv 2509.01716 v1 pith:CTJJ5HD5 submitted 2025-09-01 cs.AI cs.CL

Rui Zhao , Vladyslav Melnychuk , Jun Zhao , Jesse Wright , Nigel Shadbolt This is my paper · ORCID
classification cs.AIcs.CL
keywords privacypoliciesknowledgegraphslargelanguagemodelsDataVocabularyformalpolicyODRLpsDToUfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a seven-step LLM pipeline can automatically read a natural-language privacy policy and output a knowledge graph, called Pr2Graph, in which the policy's data practices are grounded in the standard Data Privacy Vocabulary (DPV). Fine-tuned GPT-4o-family models reach F1 scores near 0.9 overall, comparable to legal expert annotators, at roughly a tenth of the cost. The authors release the Pr2Graph for the top-100 most-visited websites, plus code, annotations, and converted ODRL and psDToU formal policy samples, arguing that this fills the missing step of producing formal policies at scale for compliance checking and web auditing.

What carries the argument

The pipeline decomposes policy analysis into six recognition and classification steps—data entities, purpose entities, parties, actions, DPV classification of data and purpose, and relation extraction—each carried out as a single JSON-schema-constrained LLM query per line-segment. The central object is the DataPractice node in Pr2Graph, which links DPV-grounded data types, purposes, parties, and the original text span into one machine-readable statement about a privacy practice. The critical evaluation mechanism is the enriched annotation dataset: two legal expert annotators added DPV grounding and fine-grained entity and event labels to the Policy-IE corpus, enabling F1 benchmarking of base

What would settle it

Take ten random policies from the released top-100 Pr2Graph, have legal experts annotate them with the same schema, and run the published pipeline over them; if the resulting F1 is substantially below the reported ~0.9 overall score, the claim that the released resource reliably summarizes those websites' privacy practices fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that modern LLMs, with only shallow fine-tuning on a small expert-annotated dataset, can reliably identify data entities, purposes, parties, actions, and their relations in privacy policies, and map the recognized text onto canonical DPV terms. This produces Pr2Graph, a knowledge graph centred on a DataPractice node that ties together what data is used, why, by whom, and with whom it is shared, while retaining the original policy text for auditing. Benchmarking on an enriched Policy-IE dataset of ten expert-annotated policies shows macro F1 around 0.9 for most pipeline steps, with high precision on empty segments, indicating the models rarely invent practices. Th

Load-bearing premise

The accuracy measured on ten expert-annotated privacy policies is assumed to hold for the one hundred unverified website policies in the released Pr2Graph.

Editorial extensions

If this is right

  • Privacy-policy analysis that currently requires expert annotators and costs roughly $10–$20 per policy can be automated for about $2.2 per policy, making large-scale audits feasible.
  • The released top-100 Pr2Graph provides a standardized, auditable snapshot of the privacy practices of popular websites, with original text retained for verification of every extracted practice.
  • Downstream formal policy languages such as ODRL and psDToU can be populated automatically from Pr2Graph, enabling machine-checkable data usage agreements for agentic and Solid-style data sharing.
  • The low amount of fine-tuning data needed (at most 120 training points) suggests the approach is reproducible for other document types or vocabularies without prohibitive annotation costs.
  • Because every practice keeps its source segment, the graph supports both human auditing and automated contradiction or compliance checks, addressing hallucination concerns by making outputs transparent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension, not explored in the paper, is applying the same pipeline to Terms of Service documents, which share the reading-burden problem; the pipeline's generic structure suggests it could transfer with modest annotation effort.
  • The published Pr2Graph for the top-100 websites was produced without manual verification of the outputs; the reported accuracy rests on ten annotated policies, so a quick sampling audit of the released graph would clarify how the benchmark transfers in practice.
  • The pipeline's per-line segmentation and reliance on commercial LLM APIs means accuracy and cost will drift as models change; a local-model variant would be needed for reproducible longitudinal monitoring of privacy-policy changes.
  • The resource's value increases if re-run over time: comparing Pr2Graph snapshots could reveal when a website silently changes its privacy practices, a direction the paper mentions but does not implement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an LLM-based NLP pipeline, pp-analyzer, that automatically converts natural-language privacy policies into a knowledge graph, Pr2Graph, grounded in the Data Privacy Vocabulary (DPV). The pipeline performs entity, purpose, party, action and relation recognition, followed by DPV-based classification, and the resulting graph is designed to support formal policy representations such as ODRL and psDToU. To evaluate the pipeline, the authors enrich the Policy-IE dataset by adding expert annotations for 10 policies, benchmark several GPT-4-family and reasoning models, and release a Pr2Graph constructed from the top-100 most visited websites. The central claims are that state-of-the-art LLMs with shallow fine-tuning achieve performance comparable to human annotators and that the released top-100 Pr2Graph accurately summarizes the privacy practices of those websites.

Significance. If the claims are substantiated, the paper addresses a genuine gap: the lack of scalable, semantic, machine-readable formal policies derived from natural-language privacy policies. The public release of code, expert annotations, and the top-100 Pr2Graph is a valuable contribution to the privacy-policy and semantic-web communities. The demonstration of downstream conversion to ODRL and psDToU is useful and helps motivate the KG-centric design. However, the current evidence base is too thin to support the headline claims, because the central numeric table is missing and the transfer from the 10-policy benchmark to the 100-policy resource is not validated.

major comments (4)
  1. [§5.3, Table 1] The central numeric evidence is missing from the manuscript. The text refers to 'Table 1' and makes quantitative claims such as 'most tasks have f1-score of about 0.9 or higher' and that f1-non-empty values are around 0.5–0.7, but the table contents do not appear in the provided submission. Without the per-model, per-task F1 values (including standard F1, f1-non-empty, f1-empty, and relaxed matching scores), the paper's core performance claim cannot be audited. This is load-bearing and must be fixed by including the full table with all reported metrics, and preferably with the number of segments per task and confidence intervals.
  2. [§4.2 and §5.3] The transfer of the 10-policy benchmark results to the released top-100 Pr2Graph is unsupported. Section 4.2 states that the top-100 graph was produced using 'the best-performing models' selected on the 10 annotated policies, but no manual verification, spot-check, or independent evaluation on a sample from the top-100 set is reported. The benchmark policies were selected alphabetically from Policy-IE, while the top-100 policies come from Tranco and the Princeton-Leuven dataset; these sets likely differ in language, structure, jurisdiction, and policy length. Because the released Pr2Graph is a central contribution, the authors should either (a) provide evidence that the benchmark performance transfers (e.g., a manually verified random sample of practices from the top-100 graph), or (b) explicitly reframe the resource as an unverified pipeline output and soften the accuracy claims accordi
  3. [§5.1 and §5.3] The evaluation protocol raises overfitting concerns. The text says 'We iteratively refined the query details and fine-tuning data size selection' and reports that prompts reached version 4, but no held-out test set or cross-validation is described. Fine-tuning data sizes were selected heuristically on the same 10 policies used for evaluation. With only 10 policies and iterative model/prompt selection on that same data, the reported F1 scores are likely optimistically biased. The paper should describe the exact train/validation/test split (or use k-fold cross-validation), report per-fold variance, and avoid claiming 'capability' from a single small-sample run with no confidence intervals.
  4. [§5.3] The claim that model performance is 'reasonably comparable' to human annotators is not supported by the evidence cited. The authors compare model F1 on automated tasks with inter-annotator agreement measured as accuracy percentages (55%, 72%, 85%) from a different annotation phase and task. These are not commensurate metrics: F1 is a different measure from raw accuracy, and the human agreement values are not computed on the same reconciled labels used to score the models. To support the comparability claim, the authors should compute annotator F1 on the same task and labels (e.g., by treating one annotator as the reference and the other as the prediction), or explicitly restrict the claim to 'models achieve high empty-segment detection and moderate F1 on non-empty segments.'
minor comments (6)
  1. [§5.2] The precision and recall formulas appear to have swapped denominators: precision should be tp/(tp+fp) and recall tp/(tp+fn). Please correct.
  2. [§3.1] Typo: 'advertizement' should be 'advertisement'.
  3. [Abstract] Typo: 'The mist of data privacy practices' likely should be 'The midst of data privacy practices' or similar.
  4. [§6] Typo: 'forseeable' should be 'foreseeable'.
  5. [§5.3] The table is labeled 'T able 1' with spacing; the label should be 'Table 1'. Also ensure the table body is rendered in the final version.
  6. [§5.2] The definition of f1-empty is ambiguous: it says 'macro f1 over data that should be predicted empty.' Does this treat correct empty predictions as true positives, or is it a negative-class F1? Clarify the computation and how empty segments contribute to precision/recall.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; pipeline is benchmarked against externally created human annotations, and the only self-citation (psDToU) is a downstream demonstration, not load-bearing.

full rationale

The paper's central derivation is an LLM-based pipeline that extracts entities, purposes, parties, actions, and relations from privacy policies and assembles them into a Pr2Graph. The evaluation compares these outputs to human-created annotations on an enriched Policy-IE dataset. The ground-truth labels were produced by legal experts independently of the model's outputs, so the benchmark is a standard supervised evaluation rather than a self-definitional or fitted-input-called-prediction circularity. The only self-citation is [42] (psDToU), which is cited as background and as a downstream conversion target; it does not justify the pipeline's accuracy or constrain the benchmark outcome, so it is not load-bearing. The released top-100 Pr2Graph is generated by the same pipeline without manual verification, and the paper acknowledges that only two event types are covered due to limited ground-truth data (Section 6); this is an external-validity and domain-transfer limitation, not a circularity. The statement in Section 5.1 that the authors 'iteratively refined the query details and fine-tuning data size selection' on the same annotated dataset could introduce optimistic bias in the reported F1 scores, but this is a model-selection/overfitting concern rather than a reduction of the prediction to its input by construction. Given the one minor, non-load-bearing self-citation, the circularity score is 2.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The paper's central contribution is an integration of LLM extraction, DPV grounding, and a new KG schema. The evaluation depends on a small, self-created annotation dataset and several hand-tuned settings (prompt version, fine-tuning data sizes, matching threshold), which is typical of applied NLP but limits the strength of the performance claim.

free parameters (5)
  • Relaxed matching threshold (LCS ratio) = 0.9
    Set in Section 5.2 to count partially overlapping text spans as true positives; chosen by hand and directly affects F1.
  • Segmentation unit = line
    Section 3.1 says segmentation by line is empirically chosen to balance cost and accuracy; affects which spans are seen together.
  • Fine-tuning data selection sizes = 10-30-2-6, 20-20-4-4, 40-80-10-20
    Section 5.1 describes iteratively and heuristically determined sizes; reported scores come from the best such choice without a separate held-out test set.
  • Prompt version = 4th version
    Section 6 notes the data entity extraction prompt is the 4th version; prompt tuning is part of the method performance.
  • Top-100 model choice = best-performing models from Table 1 (gpt-4o family)
    Section 4.2 uses best-performing benchmark models to build the resource; no manual verification of the 100 outputs.
assumptions (5)
  • domain assumption DPV (Data Privacy Vocabulary) provides a sufficiently expressive and correct grounding for privacy practices in natural-language policies.
    Section 3.1 classifies data and purpose entities into DPV leaf nodes; if DPV concepts do not align with policy content, the graph misrepresents the policy.
  • domain assumption The Policy-IE derived annotation schema plus the authors' extension is a valid ground truth for privacy practice extraction.
    Section 4.1 uses legal-expert annotations on 10 policies to define evaluation targets; all F1 scores depend on this schema.
  • domain assumption LLMs can reliably perform the NER and relation extraction subtasks when prompted with the given JSON schema and system messages.
    The whole pipeline in Section 3.1 assumes gpt-4o-family models return parseable and semantically correct extractions; the paper tests this but the numeric evidence is incomplete.
  • domain assumption Privacy policies from the Princeton-Leuven dataset for the Tranco top-100 are representative and correctly retrieved.
    Section 4.2 builds the public resource from those sources; stale or mismatched policies would make the resource inaccurate.
  • domain assumption Human legal-expert annotation is a reliable reference, and reconciliation resolves disagreements into a correct ground truth.
    Section 4.1 states the final dataset is formed after reconciliation; the ground truth assumes reconciliation produced correct labels.
invented entities (1)
  • Pr2Graph schema (urn:pp-analyze:core#DataPractice and related classes)
    purpose: Knowledge-graph representation of privacy practices linking data type, purpose, parties, and original text segment (Section 3.2).
    This ontology is introduced by the paper; no external benchmark or falsifiable prediction checks whether the schema faithfully encodes policies, so its validity is argued by design rather than independent evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An LLM-enabled semantic-centric framework to consume privacy policies." pith.science (2026). https://pith.science/paper/CTJJ5HD5

@misc{pith2026250901716,
  author       = {Pith},
  title        = {Pith review of: An LLM-enabled semantic-centric framework to consume privacy policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CTJJ5HD5}},
  note         = {Machine review of arXiv:2509.01716}
}
abstract

In modern times, people have numerous online accounts, but they rarely read the Terms of Service or Privacy Policy of those sites, despite claiming otherwise, due to the practical difficulty in comprehending them. The mist of data privacy practices forms a major barrier for user-centred Web approaches, and for data sharing and reusing in an agentic world. Existing research proposed methods for using formal languages and reasoning for verifying the compliance of a specified policy, as a potential cure for ignoring privacy policies. However, a critical gap remains in the creation or acquisition of such formal policies at scale. We present a semantic-centric approach for using state-of-the-art large language models (LLM), to automatically identify key information about privacy practices from privacy policies, and construct $\mathit{Pr}^2\mathit{Graph}$, knowledge graph with grounding from Data Privacy Vocabulary (DPV) for privacy practices, to support downstream tasks. Along with the pipeline, the $\mathit{Pr}^2\mathit{Graph}$ for the top-100 popular websites is also released as a public resource, by using the pipeline for analysis. We also demonstrate how the $\mathit{Pr}^2\mathit{Graph}$ can be used to support downstream tasks by constructing formal policy representations such as Open Digital Right Language (ODRL) or perennial semantic Data Terms of Use (psDToU). To evaluate the technology capability, we enriched the Policy-IE dataset by employing legal experts to create custom annotations. We benchmarked the performance of different large language models for our pipeline and verified their capabilities. Overall, they shed light on the possibility of large-scale analysis of online services' privacy practices, as a promising direction to audit the Web and the Internet. We release all datasets and source code as public resources to facilitate reuse and improvement.

Figures

Figures reproduced from arXiv: 2509.01716 by the authors.

Figure 1
Figure 1. Their main tasks are: – Data entity recognition: identify the text spans from the PP that resemble data entities (e.g. medical data); – Purpose entity recognition: identify the text spans from the PP that resemble purpose entities (e.g. advertizement purpose); – Party recognition: identify the text spans from the PP that describes a party, and classify it as party type (e.g. first-party and third-party); – Action re… view at source ↗
Figure 1
Figure 1. NLP pipeline, and downstream task – Relation recognition: identify the relations between the previously recognized entities – how the previously identified types of information relate to each other, especially what entities belong to what data practice, and serve what role (e.g. the party X is a data provider for an action Y). Each step is performed as one query of the LLM3 , to a segment of the privacy policy, to i… view at source ↗
Figure 2
Figure 2. Overall classes and relations in Pr2Graph [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Sample annotation produced by annotators The annotation took three phases, where we used the first phase (2 policies) to refine the guideline instructions for corner cases and allow the annotators to familiarize themselves with the procedure; the second phase (3 polici…
Figure 4
Figure 4. Figure 4: Statistics of entities and events in the annotation 4.2 Pr2Graph for top-100 websites We used the NLP pipeline with the best-performing models to process the pri￾vacy policies of top-100 most visited websites, accordingly to Tranco list [22]4 , and using privacy polici…
Figure 5
Figure 5. Figure 5: Top-10 classes for data and purpose, in Pr2Graph of top-100 websites Different downstream tasks can be performed over Pr2Graph. For example, one may construct actionable formal policies using a policy model, such as ODRL [1] or app policy in the psDToU language [42] […
Figure 6
Figure 6. Figure 6: Conversion to Pr2Graph, and further conversion to formal policy representation in ODRL and psDToU, using data class as data identifier for demonstration [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: General structure of the system message of our LLM prompts [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 30 canonical work pages

  1. [1]

    https://www.w3.org/TR/odrl-model/ (2018)

    ODRL Information Model 2.2. https://www.w3.org/TR/odrl-model/ (2018)

    work page 2018

  2. [2]

    Ahmad, W., Chi, J., Le, T., Norton, T., Tian, Y., Chang, K.W.: Intent Classi- fication and Slot Filling for Privacy Policies. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Processing (Volume 1: Long Pa- pers). pp. 4402–4417. Association for Comp...

    doi:10.18653/v1/2021.acl-long.340 2021

  3. [3]

    In: Cohn, T., He, Y., Liu, Y

    Ahmad, W., Chi, J., Tian, Y., Chang, K.W.: PolicyQA: A Reading Comprehension Dataset for Privacy Policies. In: Cohn, T., He, Y., Liu, Y. (eds.) Findings of the AssociationforComputationalLinguistics:EMNLP2020.pp.743–749.Association for Computational Linguistics, Online (Nov 2020).https://doi.org/10.18653/ v1/2020.findings-emnlp.66 9 For the human annotato...

    work page 2020

  4. [4]

    In: Proceedings of the Web Conference 2021

    Amos, R., Acar, G., Lucherini, E., Kshirsagar, M., Narayanan, A., Mayer, J.: Pri- vacy Policies over Time: Curation and Analysis of a Million-Document Dataset. In: Proceedings of the Web Conference 2021. pp. 2165–2176. Association for Com- puting Machinery, New York, NY, USA (Jun 2021).https://doi.org/10.1145/ 3442381.3450048

    arXiv 2021

  5. [5]

    In: 28th USENIX Security Symposium (USENIX Security 19)

    Andow, B., Mahmud, S.Y., Wang, W., Whitaker, J., Enck, W., Reaves, B., Singh, K., Xie, T.: {PolicyLint}: Investigating Internal Privacy Policy Contradictions on Google Play. In: 28th USENIX Security Symposium (USENIX Security 19). pp. 585–602 (2019)

    work page 2019

  6. [6]

    In: 29th USENIX Security Symposium (USENIX Security 20)

    Andow,B.,Mahmud,S.Y.,Whitaker,J.,Enck,W.,Reaves,B.,Singh,K.,Egelman, S.: Actions Speak Louder than Words: {Entity-Sensitive} Privacy Policy and Data Flow Analysis with {PoliCheck}. In: 29th USENIX Security Symposium (USENIX Security 20). pp. 985–1002 (2020)

    work page 2020

  7. [7]

    Re- quirements Engineering 19(3), 281–307 (Sep 2014)

    Breaux, T.D., Hibshi, H., Rao, A.: Eddy, a formal language for specifying and analyzing data flow specifications for conflicting privacy requirements. Re- quirements Engineering 19(3), 281–307 (Sep 2014). https://doi.org/10.1007/ s00766-013-0190-7

    work page 2014

  8. [8]

    Proceedings on Privacy Enhancing Technolo- gies 2021(2), 88–110 (Apr 2021).https://doi.org/10.2478/popets-2021-0019

    Bui, D., Shin, K.G., Choi, J.M., Shin, J.: Automated Extraction and Presentation of Data Practices in Privacy Policies. Proceedings on Privacy Enhancing Technolo- gies 2021(2), 88–110 (Apr 2021).https://doi.org/10.2478/popets-2021-0019

    work page doi:10.2478/popets-2021-0019 2021

Show all 44 references
  1. [9]

    Council of European Union: Regulation - 2016/679 - EN - gdpr - EUR-Lex (2016)

    2016

  2. [10]

    Council of European Union: Regulation - 2022/2065 - EN - DSA - EUR-Lex (2022)

    2022

  3. [11]

    In: 32nd USENIX Security Sym- posium (USENIX Security 23)

    Cui, H., Trimananda, R., Markopoulou, A., Jordan, S.: {PoliGraph}: Automated Privacy Policy Analysis using Knowledge Graphs. In: 32nd USENIX Security Sym- posium (USENIX Security 23). pp. 1037–1054 (2023)

    2023

  4. [12]

    European Data Protection Law Review 5(3), 352–366 (2019).https://doi.org/10.21552/edpl/2019/3/9

    Efroni, Z., Metzger, J., Mischau, L., Schirmbeck, M.: Privacy Icons: A Risk-Based Approach to Visualisation of Data Processing. European Data Protection Law Review 5(3), 352–366 (2019).https://doi.org/10.21552/edpl/2019/3/9

    2019 doi

  5. [13]

    Emami-Naeini, P., Agarwal, Y., Faith Cranor, L., Hibshi, H.: Ask the Experts: What Should Be on an IoT Privacy and Security Label? In: 2020 IEEE Symposium on Security and Privacy (SP). pp. 447–464 (May 2020). https://doi.org/10. 1109/SP40000.2020.00043

    2020

  6. [14]

    Sensors 23(3), 1477 (Jan 2023)

    Fallatah, K.U., Barhamgi, M., Perera, C.: Personal Data Stores (PDS): A Review. Sensors 23(3), 1477 (Jan 2023). https://doi.org/10.3390/s23031477

    2023 doi

  7. [15]

    SSRN Electronic Journal (2023).https://doi.org/10

    Guha, N., Nyarko, J., Ho, D.E., Ré, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., Zambrano, D., Talisman, D., Hoque, E., Surani, F., Fagan, F., Sarfaty, G., Dickinson, G.M., Porat, H., Hegland, J., Wu, J., Nudell, J., Niklaus, J., Nay, ...

    2023

  8. [16]

    In: 27th USENIX Security Symposium (USENIX Security 18)

    Harkous, H., Fawaz, K., Lebret, R., Schaub, F., Shin, K.G., Aberer, K.: Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning. In: 27th USENIX Security Symposium (USENIX Security 18). pp. 531–548 (2018)

    2018

  9. [17]

    In: International Con- ference on Learning Representations (Oct 2020) 16 R

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., Steinhardt, J.: Measuring Massive Multitask Language Understanding. In: International Con- ference on Learning Representations (Oct 2020) 16 R. Zhao et al

    2020

  10. [18]

    ACM Trans

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., Liu, T.: A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans. Inf. Syst. 43(2), 42:1–42:55 (Jan 2025).https://doi.or...

    2025 doi

  11. [19]

    Pandit, H., Esteves, B., P

    J. Pandit, H., Esteves, B., P. Krog, G., Ryan, P., Golpayegani, D., Flake, J.: Data Privacy Vocabulary (DPV) – Version 2.0. In: Demartini, G., Hose, K., Acosta, M., Palmonari, M., Cheng, G., Skaf-Molli, H., Ferranti, N., Hernández, D., Hogan, A. (eds.) The Semantic Web – ISWC ...

    2024 doi

  12. [20]

    de Jong, J.: Josdejong/jsonrepair (Oct 2024)

    2024

  13. [21]

    nutritionlabel

    Kelley,P.G.,Bresee,J.,Cranor,L.F.,Reeder,R.W.:A"nutritionlabel"forprivacy. In: Proceedings of the 5th Symposium on Usable Privacy and Security. pp. 1–12. SOUPS’09,AssociationforComputingMachinery,NewYork,NY,USA(Jul2009). https://doi.org/10.1145/1572532.1572538

  14. [22]

    In: Proceedings 2019 Network and Distributed System Security Symposium

    Le Pochat, V., Van Goethem, T., Tajalizadehkhoob, S., Korczynski, M., Joosen, W.: Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipula- tion. In: Proceedings 2019 Network and Distributed System Security Symposium. Internet Society, San Diego, CA (2019).https...

    2019 doi

  15. [23]

    IEEE Transactions on Knowledge and Data Engineering34(1), 50– 70 (Jan 2022).https://doi.org/10.1109/TKDE.2020.2981314

    Li, J., Sun, A., Han, J., Li, C.: A Survey on Deep Learning for Named Entity Recognition. IEEE Transactions on Knowledge and Data Engineering34(1), 50– 70 (Jan 2022).https://doi.org/10.1109/TKDE.2020.2981314

    2022

  16. [24]

    ACM Comput

    Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput. Surv. 55(9), 195:1–195:35 (Jan 2023). https://doi. org/10.1145/3560815

    2023 doi

  17. [25]

    Information, Communication & Society23(1), 128–147 (Jan 2020).https://doi.org/10.1080/ 1369118X.2018.1486870

    Obar, J.A., Oeldorf-Hirsch, A.: The biggest lie on the Internet: Ignoring the privacy policies and terms of service policies of social networking services. Information, Communication & Society23(1), 128–147 (Jan 2020).https://doi.org/10.1080/ 1369118X.2018.1486870

    2020

  18. [26]

    IEEE Transactions on Knowledge and Data Engineering 36(7), 3580–3599 (Jul 2024)

    Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., Wu, X.: Unifying Large Language Models and Knowledge Graphs: A Roadmap. IEEE Transactions on Knowledge and Data Engineering 36(7), 3580–3599 (Jul 2024). https://doi.org/10.1109/ TKDE.2024.3352100

    2024

  19. [27]

    Ravichander, A., Black, A.W., Wilson, S., Norton, T., Sadeh, N.: Question An- swering for Privacy Policies: Combining Computational and Legal Perspectives. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Lan- guage Processing and the 9th International Jo...

    2019 doi

  20. [28]

    Computing (Aug 2024).https: //doi.org/10.1007/s00607-024-01331-9

    Rodriguez, D., Yang, I., Del Alamo, J.M., Sadeh, N.: Large language models: A new approach for privacy policy analysis at scale. Computing (Aug 2024).https: //doi.org/10.1007/s00607-024-01331-9

    2024 doi

  21. [29]

    Sadeh, N., Acquisti, A., Breaux, T.D., Cranor, L.F., McDonald, A.M., Reiden- berg, J.R., Smith, N.A., Liu, F., Russell, N.C., Schaub, F., Wilson, S.: The Usable Privacy Policy Project:. Tech. rep. (Dec 2013)

    2013

  22. [30]

    Sahoo, P., Singh, A.K., Saha, S., Jain, V., Mondal, S., Chadha, A.: A System- atic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (Mar 2025).https://doi.org/10.48550/arXiv.2402.07927 An LLM-enabled semantic-centric framework to consume priva...

    2025 doi arXiv

  23. [31]

    MIT CSAIL & Qatar Computing Research Institute, Tech

    Sambra,A.,Mansour,E.,Hawke,S.,Zereba,M.,Greco,N.,Ghanem,A.,Zagidulin, D., Aboulnaga, A., Berners-Lee, T.: Solid: A Platform for Decentralized Social Applications Based on Linked Data. MIT CSAIL & Qatar Computing Research Institute, Tech. Rep. (2016)

    2016

  24. [32]

    In: Gorodetsky, V., Popyack, L., Skormin, V

    Sandhu, R., Park, J.: Usage Control: A Vision for Next Generation Access Control. In: Gorodetsky, V., Popyack, L., Skormin, V. (eds.) Computer Network Security. pp. 17–31. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg (2003). https://doi.org/10.1007/978-3-540...

    2003 doi

  25. [33]

    Frontiers in Artificial Intelligence 6, 1279794 (Nov 2023).https://doi.org/10.3389/frai.2023.1279794

    Savelka, J., Ashley, K.D.: The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal texts. Frontiers in Artificial Intelligence 6, 1279794 (Nov 2023).https://doi.org/10.3389/frai.2023.1279794

    2023

  26. [34]

    Zhao et al

    Srivastava, A., Rastogi, A., Rao, A., Shoeb, A.A.M., Abid, A., Fisch, A., Brown, A.R., Santoro, A., Gupta, A., Garriga-Alonso, A., Kluska, A., Lewkowycz, A., Agarwal, A., Power, A., Ray, A., Warstadt, A., Kocurek, A.W., Safaya, A., Tazarv, A., Xiang, A., Parrish, A., Nie, A., ...

    2023

  27. [35]

    In: Proceedings of the Demon- strations Session at EACL 2012

    Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., Tsujii, J.: Brat: A Web-based Tool for NLP-Assisted Text Annotation. In: Proceedings of the Demon- strations Session at EACL 2012. Association for Computational Linguistics, Avi- gnon, France (Apr 2012)

    2012

  28. [36]

    Tang, C., Liu, Z., Ma, C., Wu, Z., Li, Y., Liu, W., Zhu, D., Li, Q., Li, X., Liu, T., Fan, L.: PolicyGPT: Automated Analysis of Privacy Policies with Large Language Models (Sep 2023).https://doi.org/10.48550/arXiv.2309.10238

    2023 doi arXiv

  29. [37]

    https://www.w3.org/TR/rdf11- concepts/ (Feb 2014)

    W3C: RDF 1.1 Concepts and Abstract Syntax. https://www.w3.org/TR/rdf11- concepts/ (Feb 2014)

    2014

  30. [38]

    https://www.w3.org/TR/owl2-overview/ (Dec 2012)

    W3C OWL Working Group: OWL 2 Web Ontology Language Document Overview (Second Edition). https://www.w3.org/TR/owl2-overview/ (Dec 2012)

    2012

  31. [39]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Wadhwa, S., Amir, S., Wallace, B.: Revisiting Relation Extraction in the era of Large Language Models. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 15566– 15589. Association for Computational Linguisti...

    2023 doi

  32. [40]

    In: Proceedings of the 33rd International Conference on Neural Information Processing Systems

    Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.R.: SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems....

    2019

  33. [41]

    In: Erk, K., Smith, N.A

    Wilson, S., Schaub, F., Dara, A.A., Liu, F., Cherivirala, S., Giovanni Leon, P., Schaarup Andersen, M., Zimmeck, S., Sathyendra, K.M., Russell, N.C., Norton, T.B., Hovy, E., Reidenberg, J., Sadeh, N.: The Creation and Analysis of a Web- site Privacy Policy Corpus. In: Erk, K.,...

    2016

  34. [42]

    In: Proceedings of The ACM Web Conference 2024

    Zhao, R., Zhao, J.: Perennial Semantic Data Terms of Use for Decentralized Web. In: Proceedings of The ACM Web Conference 2024. pp. 2238–2249. ACM, Singa- pore (May 2024).https://doi.org/10.1145/3589334.3645631

    2024

  35. [43]

    ACM Comput

    Zhao, X., Deng, Y., Yang, M., Wang, L., Zhang, R., Cheng, H., Lam, W., Shen, Y., Xu, R.: A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers. ACM Comput. Surv.56(11), 293:1–293:39 (Jul 2024).https: //doi.org/10.1145/3674501

    2024 doi

  36. [44]

    Zimmeck, S., Story, P., Smullen, D., Ravichander, A., Wang, Z., Reidenberg, J., Russell, N.C., Sadeh, N.: MAPS: Scaling Privacy Compliance Analysis to a Million Apps. Proceedings on Privacy Enhancing Technologies (2019).https://doi.org/ 10.2478/popets-2019-0037 A LLM prompting...

    2019 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.