{"id":"221fa66b-6055-4dd3-9497-849b4bf6591e","arxiv_id":"2512.09762","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Structured data can be diffed and merged across schema changes by recording changes as operations and transferring them between histories.","lead":"This paper introduces Operational Differencing, which records changes to structured data as operations, enabling fine-grained diffing and merging even across schema changes. It could give database and spreadsheet users a more intuitive way to version and merge data without a central repository.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim depends on projection/retraction rules that are admitted incomplete and whose roundtrip law is only informally tested; diff optimization uses the roundtrip law, so a counterexample would break the main contribution.","rationale":"We carefully read the paper and agree with the reader's identification of the weakest assumption. The strongest claim—Contribution 2—is that operation-based differencing gives fine-grained diffing and merging across structural transformations. For this to hold, the Project/Retract rules must be complete and must preserve operation intention, at least in the roundtrip sense. The paper explicitly admits the implementation is incomplete (§3.2 Note), calls the rules 'axioms' with 'no objective answer' to their correctness (§3.2 Remark), and does not even present the retraction rules. The roundtrip law is asserted to be tested but no test results are given, and the diff optimization algorithm in §3.4 depends on it for order-independence. Since a single counterexample would make the diff result order-dependent and the 'optimal diff' claim false, this is the most load-bearing gap. The query operationalization and no-repo model are downstream of the same transfer machinery, so they inherit the same risk. The evaluation is honest but limited: four of eight challenge problems solved, one solution explicitly questioned, and the implementation incomplete. We considered whether the lack of formal verification alone is disqualifying; it is not, because the paper is upfront about its exploratory nature. But the combination of admitted incompleteness, unproven roundtrip law, and reliance of the core algorithm on that law justifies a conditional verdict. We therefore recommend no change to the reader's verdict; the paper should be accepted (or revised) subject to the concrete test passing. The authors deserve credit for the playable demo and explicit discussion of limitations, which make the technical gap clear and testable.","tokens_in":22226,"tokens_out":4253,"duration_ms":41147,"concrete_test":"Implement (or fully specify in a declarative DSL) the missing projection/retraction rules for all operations in Figures 2 and 9 plus deletePresent/deleteAbsent from Figure 10. Then run property-based testing of the roundtrip law: for random states and random pairs of operations (or small operation sequences), check that transferring an operation from branch A to B and then back yields a noop or the original operation, as required by §3.2. Specifically, generate 10,000 random histories of length ≤5 over the full operation set and assert the roundtrip property; also test that diff optimization (§3.4) yields the same result regardless of branch-interleaving order. If any counterexample appears, the paper's central claims about high-fidelity diffing/merging and query rewriting are unsupported. If the rule set cannot be completed without violating the law, the approach as presented is not a v","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution—fine-grained diffing and merging across schema changes—rests entirely on the Project and Retract functions (Figures 4–6). The authors are transparent that the rule set is not known to be complete: §3.2 Note says 'the implementation is incomplete' because procedural if/switch rules have become 'unwieldy'; §3.2 Remark states there is 'no objective answer' to whether the rules preserve intention, calling them 'axioms'; and retraction rules are not even presented ('we skip over them in the interest of brevity'). More concretely, the diff optimization algorithm in §3.4 relies on the roundtrip law: its order-independence is justified by 'because of the roundtrip law.' But the roundtrip law is only 'test[ed]' informally, not proven, and no test results are reported. If a single pair of operations violates the roundtrip law—e.g., a projection followed by a retraction that does not reduce to noop—then the optimized diff can depend on the order of transfers, undermining the 'optimal' diff claim and the transfer semantics on which the TODO, normalization, and query-rewriting examples all depend. The database layer compounds this: Split/Join are said to be 'composed from a set of primitive operations' with 'subtle tradeoffs' (§4 Note), but those primitives and their projection rules are unspecified, so it is unclear whether the demonstrated normalization example generalizes. The evaluation (§6) is a self-assessment against four challenge problems, with the authors themselves disqualifying half the problems and noting one solution 'arguably doesn't work' (§4.1 Note). Without a complete, verified rule set, the central claim is a strong conjecture rather than a demonstrated result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Baseline, an operation-based platform for data version control and schema evolution. It defines a rich data model in which record fields and list elements carry permanent IDs, and it introduces a large set of value and type operations, including refactorings such as ListOf and database operations such as Split and Join. The central technical proposal is Operational Differencing, built from two primitive functions, Project and Retract, that map operations across intervening changes. The paper claims this enables fine-grained diffing and merging across structural transformations, a repo-less version control model, database normalization and bidirectional transfer, operationalized queries in the form of future timelines, and query rewriting that falls out 'for free' from transfer. The evaluation is a self-assessment against four of eight challenge problems from prior work, with several caveats acknowledged. The paper is written as an exploratory systems paper, using small examples and diagrams rather than formal semantics or proofs.","tokens_in":22662,"tokens_out":4211,"duration_ms":47618,"significance":"If the proposed mechanism works as claimed, it would be a genuinely useful unification of version control and schema evolution for structured data, with potential implications for database tooling and live programming systems. The paper has notable strengths: it presents concrete, runnable examples; it is unusually candid about limitations and open problems; and it situates the work in a broad related-work landscape with an accompanying playable demo. However, the central mechanism is currently specified only through selected projection rules and examples; the retraction rules, the roundtrip law, and the primitives underlying Split/Join are either omitted or left informal. As a result, the paper's load-bearing claims about high-fidelity transfer and query rewriting cannot yet be independently verified or reproduced. The significance is therefore conditional on completing and validating the rule system.","major_comments":[{"comment":"The Retract function is central to the paper: Transfer (§3.3), diff optimization (§3.4), selective undo (Appendix B), and query rewriting (§5.2) all depend on it. Yet the retraction rules are not presented at all—the paper says 'we skip over them in the interest of brevity.' Without a specification of these rules, the paper's central claims cannot be reproduced, checked, or independently implemented. Please include the retraction rules, at least for the operations used in the examples, or provide a supplementary technical report/artifact that fully specifies them.","section":"§3.2, Figure 6"},{"comment":"The claim that diff optimization is order-independent is justified by the roundtrip law: 'The order in which the two branches are interleaved does not matter because of the roundtrip law.' The roundtrip law is introduced in §3.2 as something the authors 'test' informally, but no proof or test results are reported. If any pair of operations violates the roundtrip law, the optimized diff can depend on the iteration order, undermining the 'maximal shared state/minimal branches' claim. Please either prove the roundtrip law for the complete rule set or provide exhaustive property-based tests over the operation set and report the results.","section":"§3.4 Note"},{"comment":"The database layer is built on Split and Join, which are said to be 'composed from a set of primitive operations' with 'subtle tradeoffs on transfer semantics.' Those primitive operations and their projection/retraction rules are not specified. The normalization example in §4 therefore does not demonstrate that the demonstrated behavior generalizes beyond the specific hand-crafted sequence. Please specify the primitives and their transfer behavior, at least for nested tables and set-valued links, or clearly mark the database example as a prototype scenario rather than a solved general feature.","section":"§4 Note"},{"comment":"The deduplication example contains an admitted transfer anomaly: if an address change is made to e1 in the old schema, 'our current implementation drops the change' because e3 overwrites e1. This directly contradicts the paper's claim of high-fidelity transfer and occurs in the flagship schema-evolution example. The note says a set-like datatype would help, but that datatype is not specified. At minimum, this case should be presented as an open problem rather than a solved instance of challenge problem #2/#6, and the evaluation should not count it as a full solution without describing how the proposed fix changes the semantics.","section":"§4.1 Note"},{"comment":"The claim that query rewriting falls out 'for free' from operational differencing is only as strong as the transfer machinery on which it depends. Since retraction rules are incomplete and the roundtrip law is unproven, the rewriting result is conditional. Moreover, the operationalized query language covers only selects and joins, as the paper acknowledges. Please either provide the missing semantics for the operations used in query rewriting or explicitly state in the contributions that the rewriting claim is a consequence of the (incomplete) transfer semantics and is demonstrated only on the given fragment.","section":"§5.2"}],"minor_comments":[{"comment":"Typos: 'informaton' in the abstract, and 'iin' in the first paragraph of the introduction. These should be corrected.","section":"Abstract and §1"},{"comment":"Inconsistent spellings: 'Customer Relationship department' appears as 'Cutomer Relationship department' in §5; also 'the scheme evolution operations' in §4.1 should be 'the schema evolution operations.'","section":"§4.1 and §5"},{"comment":"The text in §5 refers to 'removePresent/removeAbsent' operations, but Figure 10 defines 'deletePresent' and 'deleteAbsent.' Please align the terminology.","section":"Figure 10 and §5"},{"comment":"The sentences 'Using an open source onboarding library was a mistake' and 'our early results are mixed' are presented without data or context. If user feedback is being reported, provide a short summary; otherwise, rephrase as subjective lessons learned rather than evaluation results.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"This is a compelling vision paper with a working demo, but the manuscript currently overclaims relative to its formal support. The major revision should focus on making the projection/retraction rules complete and checkable, proving or thoroughly testing the roundtrip law, and clearly marking which evaluation items are fully solved versus prototyped. If the authors are unwilling to include the full rule set, the paper could be reframed as a problem statement/vision with the central claims appropriately hedged; in its present form, the technical content is not sufficient for the claims made."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: Operational Differencing is a genuinely interesting new combination — Project/Retract as a generalized OT, repo-less versioning by copying histories, and operationalized queries as speculative future timelines — but the paper's foundational mechanism is not yet demonstrated. The projection/retraction rules are admitted incomplete, and the roundtrip law that the diff optimization relies on is assumed rather than proven.\n\nWhat's actually new and worth credit: Project/Retract generalize OT's inclusion/exclusion transformations in a meaningful way — returning paired operations, allowing retraction to fail on dependencies, and dropping TP1/TP2 in favor of human-in-the-loop conflict resolution. The copy-based, repo-less version control model is a real simplification, and treating queries as timelines that can be transferred and rewritten is a provocative idea that could open a new design space. The paper is also refreshingly honest. The authors flag the incomplete implementation, the unwieldy procedural rules, the axiomatic status of intention preservation, the \"arguably doesn't work\" dedup example, and the disclaimed challenge problems. That honesty is a plus.\n\nThe soft spots are real and load-bearing. Transfer, diff optimization, and query rewriting all depend on the projection and retraction rules being complete and satisfying the roundtrip law. The roundtrip law is only informally tested, and no test results are reported. Retraction rules are not presented at all — \"skipped in the interest of brevity\" — and the database Split/Join primitives are unspecified. So the central claim of fine-grained diffing and merging across schema changes is a strong conjecture, not a demonstrated result. If a pair of operations violates the roundtrip law, the optimal diff and transfer semantics break. That's exactly the kind of thing a proper referee should demand be pinned down.\n\nThat said, the paper doesn't overclaim. It calls query operationalization a conjecture, and the limitations are in the text. I think the stress-test concern is fair but proportionate: it's a reason to demand more evidence, not a reason to dismiss the ideas. The paper is a research program with worked examples, not a finished system.\n\nWho should read it: anyone working on schema evolution, OT/CRDTs, bidirectional transformations, or data version control. It deserves a serious referee. I'd send it to peer review with the expectation of major revision: complete the rule set, formalize or at least property-test the roundtrip law, and specify the database primitives.","headline":"Operational Differencing is a genuinely new combination and the paper is honestly written, but the central mechanism rests on incomplete rules and an unproven roundtrip law — a strong conjecture, not a demonstrated result.","tokens_in":23074,"tokens_out":2117,"would_cite":true,"duration_ms":20559,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Recording edits as operations lets you diff, merge, and rewrite queries even when the data's schema has changed.","keywords":["operational differencing","schema evolution","data version control","projection and retraction","query rewriting","data migration","programming by demonstration","rich data operations"],"falsifier":"Enumerate every pair of supported operations and run the roundtrip test: retract an operation through another operation, then project it back. If any pair yields a different operation than the original (other than a no-op) or fails where the paper claims success, the rule set is incomplete; a concrete known case is transferring an address change from a pre-deduplication state into a deduplicated customers table, where the current implementation drops the change rather than splitting the row.","tokens_in":22136,"feed_emoji":"🔄","tokens_out":4943,"duration_ms":48590,"temperature":0.7,"pith_summary":"The paper introduces Operational Differencing, a way to manage richly structured data by recording high-level operations—including schema refactorings—and then 'diffing' two histories by projecting operations across each other so that the same edit can be carried through structural transformations. The central claim is that because changes are tracked as operations with intent, rather than as before/after states, one can compute fine-grained differences and merge or cherry-pick changes even after a schema change has moved, renamed, or reshaped the data. The authors build a repo-free version control model where branching is just copying, with append-only histories, and show that database schema migration operations like splitting a table work as first-class operations. They further conjecture that queries can be expressed as sequences of operations—'operationalized queries'—and demonstrate that query rewriting after schema change then falls out of the same projection machinery. A sympathetic reader would care because this promises to unify editing, collaboration, and schema evolution under one mechanism, which today are handled by separate tools.","feed_headline":"Schema changes no longer break diffs and merges","feed_subtitle":"Recording edits as high-level operations keeps them meaningful even when the data's shape changes.","key_machinery":"The central mechanism is the pair of functions Project and Retract defined on operations in a rich data model of typed nested lists and records with permanent IDs, tombstones, and type operations that migrate values (renames, inserts, moves, ListOf, Split, Join, and others). Project converts a pre-operation and a base operation into a post-operation that preserves the pre-operation's intent after the base has happened; Retract does the reverse, and its failure marks a dependency between operations. Transfer uses Retract to pull an operation back to a common ancestor state, then Project to push it forward through the other branch. This same machinery, applied to formula-like operation timelin","core_discovery":"The paper's central discovery is that two primitive functions on operations—Projection and Retraction—let one shift an operation across another operation while preserving its intention, and that this suffices to reconstruct diffing, merging, and query rewriting across schema changes. Projection maps a 'pre' operation through a 'base' operation into a 'post' operation that does the same thing in the new state, even when the base operation has moved or reshaped the data; Retraction goes backward, with failure signaling that the later operation depends on the earlier one. On top of these, the paper shows that every document can carry an append-only history, diffs can be synthesized by optimizin","pith_inferences":["A natural next step would be to define the projection and retraction rules in a declarative DSL with property-based roundtrip testing; the first counterexample found would pinpoint the boundary of operations whose intent can be preserved.","The tombstone-via-insert-projection idea suggests a way to handle deletions in operation logs without persistent tombstones, which might simplify future implementations of operation-based systems.","The deduplication anomaly noted in the paper implies that the choice of base operations, not just the projection rules, determines transfer semantics; a set-valued datatype with equivalence classes would be a direct test of whether the approach can be made intuitive.","If the operationalized-query conjecture holds for a larger fragment, it could open an unexplored design space where query languages are defined by direct manipulation rather than by textual expressions."],"forward_implications":["Schema evolution becomes reversible and transferable: a database can be migrated by transferring type operations instead of writing ad hoc SQL scripts.","Data version control works across refactorings: a change made before a value was wrapped in a list or split into a separate table can be applied after those transforms.","No repository is needed for small-scale use; branches are copies with histories, and diffs are synthesized on demand.","Queries can be built by demonstration (sculpting a copy) and embedded as formulas that are automatically rewritten when the schema changes.","Operationalized queries give a bidirectional, updatable-view-like behavior with the known limitations of view updates."],"fun_headline_variants":["Shifting operations across schema changes preserves diff fidelity","Diffs and merges that adapt when data's shape changes","Projection and retraction enable schema-proof version control","Operation-based versioning: diffs survive structural changes","How to diff data even after its schema evolves"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything rests on the assumption that the projection and retraction rules form a complete and intuitive axiom set for 'preserving intention' across all operations; the paper itself notes this is not objectively checkable and that the current implementation of the rules is incomplete.","fun_headline_variants_meta":{"raw":{"variants":["Shifting operations across schema changes preserves diff fidelity","Diffs and merges that adapt when data's shape changes","Projection and retraction enable schema-proof version control","Operation-based versioning: diffs survive structural changes","How to diff data even after its schema evolves"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000693,"raw_usage":{"total_tokens":2993,"prompt_tokens":784,"completion_tokens":2209,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":2133}},"tokens_in":528,"tokens_out":2209,"duration_ms":16232,"temperature":1.0,"reasoning_tokens":2133,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T17:20:42.319544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Enumerate every pair of supported operations and run the roundtrip test: retract an operation through another operation, then project it back. If any pair yields a different operation than the original (other than a no-op) or fails where the paper claims success, the rule set is incomplete; a concrete known case is transferring an address change from a pre-deduplication state into a deduplicated customers table, where the current implementation drops the change rather than splitting the row.","supporting_citations":[],"review_version":1}