{"id":"0f3e03a0-d41c-4dcb-ad47-92ae24b23f07","arxiv_id":"2505.22778","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The paper proposes and benchmarks two transparency mechanisms for open ML models: Sigstore-based model signing and a zero-knowledge set for proving (non-)inclusion of data points in a training set.","lead":"This paper argues that open ML model hubs face serious supply-chain risks, from malware-laden models to poisoned training data, and proposes two practical fixes: signing models with Sigstore and committing to training data using cryptographic sets. If adopted, these tools could let users verify a model's authenticity and check whether specific data was used in training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The dataset-commitment scheme proves membership only relative to a trainer-chosen commitment, not relative to the data actually used for training; Section 6.4 concedes this, so the advertised data-provenance benefit overreaches the construction.","rationale":"The reader's weakest assumption is exactly the gap I consider most load-bearing: the zero-knowledge set gives soundness and privacy for a trainer-chosen set, but nothing in the construction binds that set to the data used in training. Because the paper explicitly acknowledges this in Section 6.4 and in the Impact Statement, the concern is one of overclaimed scope rather than hidden error; it does not require rejecting the paper. The model-signing contribution is separate and appears technically sound, and the performance measurements are useful. The reader's CONDITIONAL verdict is appropriate: the authors should either reframe the dataset-commitment claims as verifiable membership against a trainer-selected set, or add an attestation layer (proofs of training, TEEs, or regulated witnessing) that ties the commitment to the actual training data. I also note that Figure 3 contains an independent implementation-breaking inconsistency in the accumulator verification branches, which further supports treating this half of the paper as conditional rather than fully validated.","tokens_in":16038,"tokens_out":9640,"duration_ms":108285,"concrete_test":"Run an end-to-end adversarial demonstration using Parakeet or a faithful implementation of Figure 3: commit to a public dataset P, train a small model on a disjoint dataset T, then produce an inclusion proof for an element of P and a non-inclusion proof for an element of T. If a verifier accepts both proofs and cannot tell from the cryptographic outputs that the commitment does not match the actual training data, the claimed training-data transparency is not delivered by the commitment protocol alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For the central claim that the scheme lets model trainers 'commit to the data they use to train their models' and later prove (non-)inclusion, the load-bearing requirement is that the committed set is the true training set. The zero-knowledge set is sound only against the commitment that the trainer outputs; the algorithms in Figure 3 establish no relationship between that commitment and any actual training run. A malicious or negligent trainer can commit to an arbitrary set, and every inclusion/non-inclusion proof will still verify, so a verifier learns only about the self-declared set, not about the model's training data. Section 6.4 states this directly: 'the proofs provided against this commitment become meaningful only when we can be sure the commitment was computed correctly,' and it defers attestation to future work. The abstract's claim that the work enables publishers to 'prove properties about the datasets they use' is therefore not supported by the cryptographic construction alone. The protocol is a useful building block for verifiable set membership, but as presented it does not supply verifiable data provenance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the open ML model ecosystem faces significant supply-chain risks, including model tampering, malicious uploads, and unverifiable training-data claims. It proposes two interventions: first, signing model artifacts with Sigstore and storing signing metadata in the Rekor transparency log, with an implementation and benchmarks showing hashing-dominated costs (e.g., 800 s for a 1 TB model); second, adapting a zero-knowledge set construction (inspired by SEEMless and Parakeet) to let a model trainer commit to a list of data elements and later prove (non-)inclusion of specific points, with benchmarks showing commit times of 96 s for 100 M entries. The paper explicitly acknowledges in Section 6.4 that these proofs are meaningful only if the commitment was computed over the actual training data, which it leaves to future work.","tokens_in":16243,"tokens_out":3930,"duration_ms":42856,"significance":"The model-signing contribution is practical and clearly presented; the open-source library, integration with Sigstore, and measurements of large-model hashing costs are useful engineering results. The zero-knowledge set is a sensible adaptation of known key-transparency techniques, and the benchmark data are informative. The paper's central data-provenance claim, however, overreaches the construction: without a mechanism tying the commitment to the real training run, (non-)inclusion proofs only attest to a trainer-chosen set. The work is therefore best read as a building block plus a deployment-oriented first step, not as a complete solution to verifiable dataset provenance.","major_comments":[{"comment":"The conditional in ZKS.Verify is reversed. When resp=0 (non-inclusion), the algorithm calls Acc.VerIncl, and when resp=1 (inclusion) it calls Acc.VerNonIncl, whereas ZKS.Query returns resp=1 with Acc.ProveIncl and resp=0 with Acc.ProveNonIncl. As written, an honest proof will always fail verification (or a dishonest one may pass). This should be corrected to use Acc.VerIncl when resp=1 and Acc.VerNonIncl when resp=0.","section":"Figure 3, ZKS.Verify"},{"comment":"There is an index mismatch. Section 6.2 states that the accumulator index for an entry D is H(VRF.Eval(sk,D)), and ZKS.Commit computes di = H(VRF.Eval(sk,Di)). However, ZKS.Query sets d = VRF.Eval(sk,D) and uses d directly as the accumulator position, while ZKS.Verify passes that same d to Acc.VerIncl/VerNonIncl. The proof πVRF is over d, but it is not linked to the hashed value used to place entries in the tree. The pseudocode should separate the VRF output v from the derived index d = H(v), and ZKS.Verify should recompute d from the verified v before checking the accumulator proof.","section":"Figure 3, ZKS.Query and ZKS.Verify"},{"comment":"The abstract claims the work enables publishers to 'prove properties about the datasets they use,' but the construction only proves (non-)inclusion with respect to a commitment that the trainer chooses. Section 6.4 concedes this: 'the proofs provided against this commitment become meaningful only when we can be sure the commitment was computed correctly.' This is not a technical flaw in the zero-knowledge set itself, but the presentation should temper the data-provenance claim or explicitly frame the ZKS as a primitive that must be combined with a separate attestation mechanism (e.g., proofs of training, TEEs) before it delivers the advertised transparency benefit.","section":"Section 6.4 / Abstract"},{"comment":"The claim that the simplification of the SEEMless construction 'achiev[es] the same soundness and privacy guarantees' is not accompanied by a proof or a precise statement of the security model, even though the paper relies on external results for soundness. Since the construction modifies leaf storage and ordering compared to a standard accumulator, the privacy claim should either be formally argued or explicitly attributed to the SEEMless proof with a careful reduction.","section":"Section 6.2"}],"minor_comments":[{"comment":"There is a typo: 'ranging from 1 Bto 1 TB' lacks a space; also, 'the time required to hash a file of a size ranging from 1 B to 1 TB' could be clearer.","section":"Section 5.4"},{"comment":"In the sentence 'for entry D we compute its new index j as H(VRF.Eval(sk, Di))', the subscript 'i' appears in a context where the element is D, not Di; this is confusing given that Di denotes the i-th training element.","section":"Section 6.2"},{"comment":"The phrase 'a non-private proof of training data to an entity who is trusted to see the data but not collude with the model trainer' is a sentence fragment; please rephrase for clarity.","section":"Section 6.4"},{"comment":"The reference for the AI Act is incomplete (no authors or publication venue); if it is a legislative document, a stable URL and date would be helpful.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a systems-oriented position paper with a useful threat survey and a concrete Sigstore integration. The main technical issue is the incorrect pseudocode in Figure 3, which is fixable but currently invalidates the described ZKS protocol as written. The broader data-provenance framing should be aligned with the acknowledged limitation in Section 6.4. With these revisions, the paper could be a solid ICML-style contribution on ML supply-chain transparency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take. The model-signing half of this paper is solid and worth a real read. It adapts Sigstore to ML model hubs, ships an open-source library, reports honest benchmarks (hashing a 1TB model in 800 seconds, signing overhead dominated by hashing), and is already being piloted with Kaggle. That's a concrete, deployable intervention against a documented problem (malware, namesquatting, model tampering on Hugging Face). The threat survey in Sections 3–4 is also well done and will be a useful citation.\n\nThe dataset-commitment half is where I have concerns, and the stress-test lands. The ZKS construction is a simplified static-set version of SEEMless/Parakeet, which is fine as an adaptation, but the paper does not provide a proof of the claimed soundness/privacy equivalence; it just asserts it. More importantly, the protocol only proves membership in the trainer-chosen commitment. It does not tie that commitment to the data actually used for training. Section 6.4 says this explicitly: 'the proofs provided against this commitment become meaningful only when we can be sure the commitment was computed correctly.' That is a load-bearing gap, and the abstract's phrase 'prove properties about the datasets they use' overstates what the construction alone delivers.\n\nThere is also a concrete bug the reader caught: in Figure 3, ZKS.Verify calls Acc.VerIncl when resp=0 and Acc.VerNonIncl when resp=1, which is backwards. ZKS.Query returns resp=1 for inclusion and resp=0 for non-inclusion. As written, the verifier would reject valid inclusion proofs and accept invalid non-inclusion ones. This is fixable, but it needs correcting before anyone relies on the code.\n\nNone of this is fatal to the paper's value. The model-signing work stands on its own. The ZKS part is honestly presented as preliminary and the authors flag the attestation problem as future work; the issue is more that the framing in the abstract and introduction doesn't match the caveat. That, plus the pseudocode bug, means the dataset half needs revision.\n\nWho is this for? People working on ML supply chain security, model registry design, and data provenance tooling. It deserves a serious referee: the model signing contribution is practical and timely, and the ZKS part, once fixed and reframed, is a usable building block. I'd accept it for peer review with the expectation that the authors address the verification bug and soften the provenance claims.","headline":"Useful model-signing work; dataset-commitment part has a backwards verification branch and a conceded gap between commitment and actual training data.","tokens_in":16786,"tokens_out":2085,"would_cite":true,"duration_ms":19626,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that Sigstore-based model signing and a zero-knowledge set over training data give open ML a practical supply-chain fix.","keywords":["ML supply chain security","model signing","Sigstore","Rekor transparency log","zero-knowledge sets","training data provenance","Merkle tree commitments","verifiable random functions"],"falsifier":"Build a 1 TB model file on a comparable server, run the released hashing library, and compare the 800-second figure and the roughly 19 percent overhead against saving the file; a materially larger overhead would falsify the practicality claim. For the commitment scheme, a 100-million-entry registry that takes substantially longer than 96 seconds to commit on ordinary hardware—or proofs that take more than the reported tens of microseconds—would falsify the scaling claim.","tokens_in":15851,"feed_emoji":"🔐","tokens_out":8246,"duration_ms":91588,"temperature":0.7,"pith_summary":"The paper's central claim is that the open ecosystem for downloadable machine-learning models carries real, already-exploited supply-chain risks—malware-laden or tampered models, poisoned training data, and impersonated publishers—and that two cryptographic mechanisms can make model publication verifiable at acceptable cost. The first is Sigstore-based signing: publishers sign model hashes with short-lived certificates tied to their identity, and the signing metadata is recorded in a public transparency log so tampering or differentiated distribution can be detected. The second is a zero-knowledge set over the training data: a trainer commits to the data once and can later prove, without revealing the rest of the set, that a specific data point was or was not included. The paper reports concrete costs—hashing a 1 TB model takes about 800 seconds, committing a 100-million-entry registry takes 96 seconds—as evidence that these steps are practical enough for hubs to adopt. A sympathetic reader would take away that transparency for open models is achievable today with standard cryptographic building blocks, provided the trainer's commitment is made honestly.","feed_headline":"Two crypto fixes could secure open AI model downloads","feed_subtitle":"Signing adds minutes per upload; trainers can later prove a data point was or wasn't in training.","key_machinery":"The load-bearing object for integrity is the Sigstore bundle: an ephemeral public key, a certificate binding that key to an OIDC identity, and a signature plus a Merkle inclusion proof in the public Rekor transparency log, turning 'this is the publisher's intended model' into evidence any user can check and any outsider can audit. The load-bearing object for provenance is a zero-knowledge set built from a verifiable random function and a Patricia Merkle tree: the VRF pseudo-randomly permutes entries so the tree's structure reveals no ordering, and leaf values are hiding commitments, so membership and non-membership proofs release nothing beyond the queried point.","core_discovery":"The paper's discovery is that two existing, standardized cryptographic tools can be reassembled to cover the two weakest points of the open-model supply chain. For integrity, it adapts Sigstore's workflow to models: a publisher authenticates to an identity provider, obtains a short-lived certificate binding an ephemeral key to that identity, signs the hash of the model, and submits the signature, certificate, and hash to the Rekor transparency log, returning a Merkle inclusion proof to the consumer. For data provenance, it constructs a zero-knowledge set: entries (individual training points or whole datasets) are placed at pseudorandom positions determined by a verifiable random function, a Patricia Merkle tree is built over those positions, and leaf values are stored as hiding commitments, so a prover can prove membership or non-membership of a queried point while learning nothing else. The paper measures both mechanisms on realistic scales and reports that hashing dominates model signing—800 seconds for a 1 TB file against 4,151 seconds to save it—and that commitments to 100 million entries take 96 seconds, with individual proofs around tens of microseconds. It is explicit that these proofs are only as meaningful as the commitment itself: nothing in the construction ties the committed set to the data actually used in training.","pith_inferences":["If hubs adopt signing as the default upload path, the transparency log becomes a natural place to also record dataset commitments, letting users cross-check which model version was trained against which committed data registry without building new infrastructure.","The paper's commitment benchmarks stop at 100 million in-memory entries; a natural test is to run the same construction over a multi-billion-document web-scale corpus with persistent storage, where the paper only extrapolates.","The same VRF-plus-Merkle machinery could eventually back queries in agentic systems—proving which tool, document, or retrieved context drove a given action—though the paper does not pursue that direction.","A useful next experiment is to measure how often real consumers query non-inclusion proofs; if demand is rare, the limiting factor for transparency will be the honesty of the initial commitment, not proof cost."],"forward_implications":["Model users can verify that a downloaded file is the exact version the publisher signed, and a publisher who watches the transparency log can detect tampered or targeted copies.","Model hubs can integrate signing into their upload API with negligible extra code, making verification available by default without changing how models are trained or stored.","A trainer can publish one compact commitment at release time and later prove, without revealing the dataset, whether a specific document or image was part of training.","The measured overhead is small relative to model production: hashing a 1 TB model takes about 800 seconds, and committing to 100 million training entries takes 96 seconds.","Non-inclusion proofs make it possible to answer copyright and regulatory queries about whether particular data was used, even when the full dataset is private."],"supporting_citations":[{"why":"Supplies the Sigstore protocol and Rekor transparency log that the model-signing design builds on.","marker":"(Newman et al., 2022)"},{"why":"Provides the SEEMless zero-knowledge set construction and its soundness proof, which the training-data commitment adapts.","marker":"(Chase et al., 2019)"},{"why":"Offers the Parakeet verifiable registry code used for the commitment benchmarks.","marker":"(Malvai et al., 2023)"},{"why":"Documents the deployed key-transparency data structure that motivates the commitment design.","marker":"(Lawlor & Lewi, 2023)"},{"why":"Demonstrates practical web-scale training-data poisoning, a core motivation for verifiable data commitments.","marker":"(Carlini et al., 2024)"},{"why":"Surveys broken data provenance in AI and frames why verifiable training-data claims are needed.","marker":"(Longpre et al., 2024)"},{"why":"Documents low PGP signature adoption in package registries and motivates replacing key management with Sigstore's short-lived certificates.","marker":"(Schorlemmer et al., 2024)"}],"fun_headline_variants":["Crypto duo could lock down open AI model supply chain","Two crypto tools may fix open model integrity and provenance","Sigstore-style signing plus ZK sets could secure model downloads","Sign your models: two crypto tricks tame open ML supply chains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole data-provenance mechanism rests on the trainer forming the commitment over data actually used in training; a trainer who commits to a different set makes every membership proof vacuous, and the paper leaves verifying that step to future work.","fun_headline_variants_meta":{"raw":{"variants":["Crypto duo could lock down open AI model supply chain","Two crypto tools may fix open model integrity and provenance","Sigstore-style signing plus ZK sets could secure model downloads","Sign your models: two crypto tricks tame open ML supply chains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000846,"raw_usage":{"total_tokens":3677,"prompt_tokens":932,"completion_tokens":2745,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":2677}},"tokens_in":548,"tokens_out":2745,"duration_ms":22692,"temperature":1.0,"reasoning_tokens":2677,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:00:17.157147+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a 1 TB model file on a comparable server, run the released hashing library, and compare the 800-second figure and the roughly 19 percent overhead against saving the file; a materially larger overhead would falsify the practicality claim. For the commitment scheme, a 100-million-entry registry that takes substantially longer than 96 seconds to commit on ordinary hardware—or proofs that take more than the reported tens of microseconds—would falsify the scaling claim.","supporting_citations":[{"cited_title":"SEEMless : Secure end-to-end encrypted messaging with less trust","cited_arxiv_id":null,"evidence_quote":"Provides the SEEMless zero-knowledge set construction and its soundness proof, which the training-data commitment adapts."},{"cited_title":"Parakeet: Practical key transparency for end-to-end encrypted messaging","cited_arxiv_id":null,"evidence_quote":"Offers the Parakeet verifiable registry code used for the commitment benchmarks."},{"cited_title":"and Lewi, K","cited_arxiv_id":null,"evidence_quote":"Documents the deployed key-transparency data structure that motivates the commitment design."},{"cited_title":"A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., and Tramèr, F","cited_arxiv_id":null,"evidence_quote":"Demonstrates practical web-scale training-data poisoning, a core motivation for verifiable data commitments."},{"cited_title":"Data authenticity, consent, & provenance for AI are all broken: what will it take to fix them? In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024","cited_arxiv_id":null,"evidence_quote":"Surveys broken data provenance in AI and frames why verifiable training-data claims are needed."},{"cited_title":"R., Kalu, K","cited_arxiv_id":null,"evidence_quote":"Documents low PGP signature adoption in package registries and motivates replacing key management with Sigstore's short-lived certificates."}],"review_version":1}