{"id":"390a950c-bfd9-40a7-8a7f-505beeb77226","arxiv_id":"2501.01131","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"PriBOM is a UI-widget-indexed privacy bill of materials for creating and maintaining app privacy notices, supported by positive but self-reported survey feedback.","lead":"This paper introduces PriBOM, a widget-level privacy information inventory that development teams can use to coordinate and generate mobile app privacy notices. It reports a 150-participant survey showing strong perceived usefulness, though the approach has not yet been tested in real development settings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that PriBOM enables accurate privacy notice generation is unsupported: the pre-fill pipeline is never validated against ground truth, and the survey measures only perceived usefulness.","rationale":"The reader's weakest assumption correctly identifies that survey agreement from 150 mostly junior, small-team participants may not predict real-world utility in large, sophisticated teams. My concern is adjacent but more fundamental: even for the surveyed population, the paper provides no evidence that the pre-fill is accurate or that PriBOM improves notice accuracy. The survey shows only that participants found a hand-picked example intuitive and plausible; it does not demonstrate that the actual pipeline output is correct. This is a distinct load-bearing gap because the central claim includes 'accurate generation,' which requires both a correct inventory and a demonstrated effect on notice quality. The absence of any objective evaluation of Section 4's pre-fill is an internal validity threat, whereas the reader's concern is an external validity threat. Both support a conditional verdict, and the reader already assigned CONDITIONAL; therefore my read does not change the verdict. The paper's contribution as a design proposal with preliminary perception data is still valuable, but the strong accuracy claim must be downgraded or explicitly deferred pending validation. A precision/recall benchmark of the pre-fill, plus a small controlled user study against ground-truth labels, would settle whether the concern lands.","tokens_in":22774,"tokens_out":4036,"duration_ms":41419,"concrete_test":"Run the Section 4.1 pre-fill pipeline on a benchmark set of 20-30 open-source Android apps whose widget-level privacy practices have been manually verified (e.g., from existing privacy-label or permission ground-truth datasets). For each widget, compute precision and recall of the extracted permissions, data types, TPL names/versions, and matched privacy-policy/label disclosures against the ground truth. If any core field (e.g., data type) has precision or recall below 0.85, the pre-fill cannot support the 'accurate generation' claim. Additionally, run a small controlled experiment: have developers create a Google Play Data Safety label for a test app both with and without PriBOM, and compare their output to a verified ground-truth label; this directly tests whether PriBOM improves notice accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and introduction, is that PriBOM facilitates 'transparent, collaborative, and accurate generation of privacy notices.' The only evidence offered is a Likert-scale survey of 150 Prolific participants (Section 5) who rated statements about a single hand-picked example row (Table 2). The survey measures agreement, not whether the pre-fill output is correct or whether using PriBOM improves actual notice accuracy. Section 4.1's static-analysis pipeline (JADX, GATOR, AndroGuard, LibScan, coarse permission-to-data-type mapping) is never evaluated: there is no ground-truth comparison, no precision/recall computation, no baseline against existing generators, and no user study using the actual pre-fill output. If the pre-fill produces false positives or misses privacy-relevant flows (especially since the UI-widget granularity deliberately excludes non-UI components, as acknowledged in Section 3.1), the generated notices could be incomplete or misleading, directly undermining the 'accurate' part of the claim. The authors themselves note in Section 6 that pre-fill quality depends on static-analysis performance and that 'real practicability needs to be further verified,' but the headline claim is not correspondingly qualified. The load-bearing concern is therefore the missing objective validation of the artifact's correctness and effectiveness, not merely the survey's external validity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PriBOM (Privacy Bills of Materials), a UI-widget-indexed privacy information inventory intended to support transparent, collaborative, and accurate generation of privacy notices in mobile app development. The authors derive three developer-facing privacy challenges from a small literature review, design PriBOM with four sections (UI Widget Identifier, Codebase and Permission, Third-Party Library, and Privacy Notice Disclosure), and present a pre-fill pipeline combining static analysis and privacy notice analysis. They evaluate PriBOM through a 150-participant Prolific survey measuring perceived usefulness, finding positive agreement on intuitiveness, traceability, and communication. The paper claims that PriBOM enables accurate privacy notice generation, but the evaluation only supports perceived usefulness, and the authors themselves note that real practicability requires further verification.","tokens_in":23068,"tokens_out":3675,"duration_ms":34881,"significance":"If PriBOM's central claims were fully validated, the concept could provide a genuinely useful bridge between technical and non-technical roles in privacy documentation and maintenance, addressing a real gap in existing privacy notice tools. The paper's strengths include a clear and well-motivated artifact design, a modular pre-fill pipeline built from established tools (JADX, GATOR, AndroGuard, LibScan), a relatively large survey (150 participants), and public availability of the implementation and questionnaire. However, the headline claim of 'accurate' notice generation is not validated: the pre-fill is never checked against ground truth, and the survey measures only self-reported agreement. The sample also skews toward junior developers in small teams, which does not match the paper's motivating scenario of large, crowded development teams. The significance is therefore conditional on reframing the claims to match the evidence or adding objective validation.","major_comments":[{"comment":"The central claim that PriBOM enables 'accurate' generation of privacy notices (Abstract and Section 1) is not supported by the evidence in the paper. The pre-fill pipeline (Section 4.1 static analysis and Section 4.2 privacy notice analysis) is never evaluated against ground-truth privacy practices; no precision/recall, no comparison with existing generators, and no user study using the actual pre-fill output are reported. The example in Table 2 is a single hand-picked row without verification of correctness. To support the accuracy claim, the authors should add an objective evaluation of pre-fill correctness on a benchmark of apps with known privacy practices, or substantially qualify the claim to perceived usefulness only.","section":"Abstract; Section 4.1"},{"comment":"The participant sample does not match the paper's motivating scenario. The authors motivate PriBOM for 'sophisticated mobile apps with complex features and in crowded development teams' (Section 1), but Table 3 shows that 39% of participants are junior developers and 59% work in teams of fewer than 10 people. The survey therefore provides limited evidence about the target population. The authors should either recruit participants from large, multi-role teams (e.g., with legal and design roles) or explicitly narrow the claimed scope of the findings.","section":"Section 5.2; Table 3"},{"comment":"The survey results are reported as mean Likert scores and agreement percentages without inferential statistics or a baseline comparison. Statements such as 'significantly higher' in Section 5.4.2 (e.g., legal team vs. UI designers on S10) are not backed by significance tests, and the absence of a control/baseline means the positive responses may reflect acquiescence bias or the participants' desire to please the researchers. The authors should report appropriate statistical tests and discuss the absence of a baseline, or explicitly frame the analysis as descriptive.","section":"Section 5.4; Table 4"},{"comment":"The Limitations section candidly acknowledges that pre-fill quality depends on static analysis performance and that 'real practicability needs to be further verified,' but the abstract and conclusion repeat the unqualified claim that PriBOM supports 'accurate' notice generation. The main text should carry these qualifications forward, for example by replacing 'accurate' in the headline with 'transparent and collaborative' and stating that accuracy of the pre-fill is future work.","section":"Section 6; Conclusion"},{"comment":"The UI-widget granularity deliberately excludes non-UI components, and the authors acknowledge that this 'may lead to gaps in capturing practices that circumvent standard permission protocols.' Since privacy notices must be comprehensive to be accurate, this design trade-off directly threatens the 'accurate' part of the central claim. The paper should either evaluate the extent of such gaps (e.g., by analyzing how many privacy-relevant data flows are not reachable from UI widgets) or explicitly state that PriBOM targets a subset of privacy practices.","section":"Section 3.1"}],"minor_comments":[{"comment":"In the contributions list, 'comprehsnsively' should be 'comprehensively'.","section":"Section 1"},{"comment":"The text states that 'senior developers agree more than junior developers' on S26, but then reports scores for senior developers (4.05) and UI designers (3.78); the comparison should be against junior developers, not UI designers, or the text should be corrected.","section":"Section 5.4.2"},{"comment":"Reference [8] is titled 'Iubenda' but the URL points to termly.io; this appears to be a citation error and should be corrected to the appropriate service.","section":"References"},{"comment":"The phrase 'access the usefulness' should be 'assess the usefulness'.","section":"Section 6"},{"comment":"The abbreviations 'Ave.' and 'Distr.' in Table 4's header are not defined in the caption; they should be spelled out or defined.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of PoPETs and could become a useful contribution if the authors rework the claims to match the evidence. The main issue is not the novelty of PriBOM as a concept but the gap between the 'accurate generation' claim and the perceived-usefulness survey. I would encourage the editors to request a revision that either validates the pre-fill objectively (even on a small benchmark) or qualifies the central claim accordingly. The paper's treatment of existing tools and its qualitative analysis of participant feedback are solid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely useful idea -- a BOM-style, UI-widget-indexed privacy inventory for mobile dev teams -- and the paper is clearly written and honest about its limits. But the central claim that PriBOM enables \"accurate\" privacy notice generation is not supported by the evaluation. The 150-person survey measures perception, not accuracy; the pre-fill pipeline is never checked against ground truth, and no baseline or precision/recall is reported. That is a real gap, but it is fixable rather than fatal.\n\nWhat is genuinely new: PriBOM is the first privacy inventory indexed by UI widget, designed as a collaboration tool across developer roles. The systematic summary of privacy notice tools in the formative study is a useful reference, and the pre-fill pipeline built from JADX, GATOR, AndroGuard, LibScan, and permission-data mapping is sensible and modular. The survey itself is reasonably designed: 26 Likert statements, pilot-tested, with participants spanning multiple roles, and the authors report agreement rates and role-based differences. The decision to link widgets to code, permissions, third-party libraries, and notice disclosures is a practical step forward for privacy engineering.\n\nThe soft spots are proportionate to the claims. The stress-test note is on target: the word \"accurate\" appears in the abstract and introduction, but the only evidence is perceived usefulness. Participants rated a single hand-picked example row, not the actual tool output. The pre-fill's correctness is acknowledged as dependent on static analysis performance (Section 6), but the headline is not correspondingly qualified. The sample skews junior developers and small teams (59% under 10 people), and there is no statistical significance testing. These are standard limitations for a first evaluation, so I would not call them disqualifying, but they do mean the paper should either validate the pre-fill against ground truth or visibly soften the accuracy language.\n\nWho is this for? Researchers and practitioners in privacy engineering, usable privacy, and DevOps. It is a legitimate subfield contribution, not a breakthrough. The concept deserves referee time, especially since the implementation and questionnaire are released.\n\nRecommendation: send to peer review. It needs major or minor revision depending on the venue's bar -- at minimum, the accuracy claim must be reconciled with the perceived-usefulness evidence, and a ground-truth evaluation of the pre-fill (even on a small app set) would substantially strengthen it. I would not desk-reject this.","headline":"Useful new privacy-inventory concept, but the 'accurate' claim outruns the evidence -- the survey only measures perceived usefulness, and the pre-fill is never validated against ground truth.","tokens_in":23571,"tokens_out":1949,"would_cite":true,"duration_ms":21549,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PriBOM, a UI-widget-indexed privacy information inventory, enables transparent, collaborative, and more accurate privacy notice generation in mobile app development, according to a 150-participant survey.","keywords":["privacy bills of materials","privacy notice generation","mobile app privacy","UI widget indexing","privacy transparency","Android static analysis","privacy labels","DevOps privacy collaboration"],"falsifier":"Run the pre-fill on a set of Android apps known to access sensitive data through reflection or native code and check whether PriBOM's inventory omits the corresponding permissions and privacy-notice disclosures; any such omission would falsify the claim that the inventory supports accurate notice generation for sophisticated apps.","tokens_in":22628,"feed_emoji":"📋","tokens_out":9055,"duration_ms":76865,"temperature":0.7,"pith_summary":"Privacy notices for mobile apps are often inaccurate because developers lack privacy expertise and the writing of policies and labels falls mainly on legal teams who cannot see how the code actually handles data. This paper proposes PriBOM, a structured privacy information inventory indexed by the app's UI widgets, so that what each screen element collects, which permissions and third-party libraries back it, and what the privacy policy and label say about it are recorded together. The authors argue that this turns privacy notice generation from an individual, post-hoc task into a transparent, collaborative part of development, and they support the idea with a survey in which 83.33% of 150 participants agreed PriBOM improves privacy-related communication. A pre-fill pipeline using static analysis and privacy-notice analysis shows how the inventory could be populated automatically. The paper's contribution is the concept and its perceived usefulness, not yet proof that adopting PriBOM improves notice accuracy in real large teams.","feed_headline":"A UI-widget index keeps app privacy notices in sync","feed_subtitle":"PriBOM links each screen element to its permissions, code and disclosures, giving teams one shared privacy record.","key_machinery":"The load-bearing object is the PriBOM table: a privacy information inventory whose rows are UI widgets and whose columns are organized into four sections: UI Widget Identifier, Codebase and Permission, Third-Party Library, and Privacy Notice Disclosure. The UI widget is the pivot because it is the visible element users interact with, the component that triggers data-handling callbacks, and therefore a vocabulary that front-end developers, back-end developers, UI designers, and legal teams can share. The pre-fill is a modular static-analysis pipeline that disassembles the APK, extracts widgets and their callback methods, builds call graphs for those callbacks, extracts reachable Android permissions, maps permissions to data types, detects third-party libraries, and matches segmented privacy-policy sentences and privacy-label declarations back to those data types. This machinery operationalizes the paper's two named benefits: tracing from a reported UI problem back to code and disclosures, and tracking from a code change forward to every notice entry that needs updating.","core_discovery":"The paper's central claim is that a UI-widget-indexed privacy inventory, called PriBOM, lets mobile development teams generate and keep privacy notices aligned with actual app behavior. Each widget anchors four sections of information: its identifier and source, the events and Android permissions in the codebase behind it, the third-party libraries it touches, and the corresponding statements in the privacy policy and privacy label disclosures. The authors provide a pre-fill that extracts widgets, callbacks, and permissions from an APK through static analysis, maps permissions to data types, detects third-party libraries, and segments existing privacy policies and labels to fill the disclosure fields. In a survey of 150 participants spanning developers, UI designers, project managers, and legal team members, they report positive perceived usefulness, with 83.33% agreement that PriBOM enhances privacy-related communication, 85.3% for design intuitiveness, 72% for traceability, and 78.76% for information relevance. They conclude that PriBOM is a systematic solution for privacy support in mobile app DevOps, while noting that real-world practicability still needs verification.","pith_inferences":["The same widget-level index could be extended into an automated notice-diffing tool: when a code change alters a widget's call graph, the corresponding policy and label entries could be flagged for revision within the same commit; the paper motivates this need but does not build it.","Because the pre-fill relies on static analysis, a runtime-monitoring companion could catch data practices that are invisible statically, such as reflection, native code, or permission circumvention; the paper explicitly sets these aside as a trade-off in favor of ease of use.","Adoption may hinge less on the design than on maintenance cost: participants already flagged learning curves, team habits, and the effort to keep the inventory current, so the decisive test is whether teams sustain PriBOM after the initial pre-fill.","The widget-as-pivot idea generalizes to any interactive software with a UI layer, such as web apps, desktop software, or IoT dashboards, although the paper only demonstrates the concept on Android."],"forward_implications":["When a user reports a privacy issue on a particular screen element, the team can trace from that widget back to the handler, permissions, and code paths involved.","When a code change alters data practices, PriBOM lets the team track forward to every privacy policy and label entry that must be updated.","Legal and non-technical roles gain a common widget-level vocabulary for privacy discussions, reducing reliance on a few privacy specialists who carry the work alone.","The pre-fill pipeline can populate much of the inventory automatically from the APK and existing privacy notices, lowering the burden of initial setup.","PriBOM is designed as a modifiable format and a modular pipeline, so teams can customize fields or swap in stronger analysis modules and adapt it beyond Android to iOS and other software contexts."],"supporting_citations":[{"why":"Supplies the bill-of-materials concept (SBOM) that PriBOM adapts to privacy information.","marker":"[5]"},{"why":"Provides the static reference analysis for GUI objects used to extract widgets and callback methods.","marker":"[72]"},{"why":"Provides the static control-flow analysis of user-driven callbacks that maps events to handlers.","marker":"[86]"},{"why":"Provides third-party library detection with version identification used in the TPL section.","marker":"[80]"},{"why":"Supplies the privacy-policy segmentation technique that fills the Privacy Notice Disclosure section.","marker":"[66]"},{"why":"Documents developers' difficulty creating accurate privacy labels, the core problem PriBOM targets.","marker":"[57]"},{"why":"Represents the IDE-plugin approach to notice generation that PriBOM positions itself against for collaborative teams.","marker":"[54]"},{"why":"Provides the grouping of Android permissions into data types that maps widgets to collected data.","marker":"[60]"}],"fun_headline_variants":["Widget-level privacy map syncs app code with disclosures","New inventory links UI widgets to permissions and policies","PriBOM anchors privacy info to each app screen element","One shared record tracks app privacy from code to notice","Privacy tool helps teams keep app notices accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Likert-scale agreement from 150 survey participants, many of them junior developers in teams of under ten people, predicts whether PriBOM will actually improve privacy-notice accuracy and be adopted in large, complex development teams.","fun_headline_variants_meta":{"raw":{"variants":["Widget-level privacy map syncs app code with disclosures","New inventory links UI widgets to permissions and policies","PriBOM anchors privacy info to each app screen element","One shared record tracks app privacy from code to notice","Privacy tool helps teams keep app notices accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00026,"raw_usage":{"total_tokens":1585,"prompt_tokens":935,"completion_tokens":650,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":576}},"tokens_in":551,"tokens_out":650,"duration_ms":6770,"temperature":1.0,"reasoning_tokens":576,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:33:45.295467+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pre-fill on a set of Android apps known to access sensitive data through reflection or native code and check whether PriBOM's inventory omits the corresponding permissions and privacy-notice disclosures; any such omission would falsify the claim that the inventory supports accurate notice generation for sophisticated apps.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the static reference analysis for GUI objects used to extract widgets and callback methods."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the static control-flow analysis of user-driven callbacks that maps events to handlers."},{"cited_title":"2023.{LibScan}: Towards More Precise{Third-Party} Library Identification for Android Applications","cited_arxiv_id":null,"evidence_quote":"Provides third-party library detection with version identification used in the TPL section."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the privacy-policy segmentation technique that fills the Privacy Notice Disclosure section."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents developers' difficulty creating accurate privacy labels, the core problem PriBOM targets."},{"cited_title":"Matcha: An IDE Plugin for Creating Accurate Privacy Nutrition Labels","cited_arxiv_id":"2402.03582","evidence_quote":"Represents the IDE-plugin approach to notice generation that PriBOM positions itself against for collaborative teams."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the grouping of Android permissions into data types that maps widgets to collected data."}],"review_version":1}