{"id":"eeaa86d5-1d15-45bd-aea9-cd4dee647a41","arxiv_id":"2507.12955","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Rubin Observatory's data movement relies on Rucio and FTS for transfers, plus three custom tools that tie Rucio to the Data Butler registry.","lead":"The Vera C. Rubin Observatory will move petabytes of telescope images among three continents every year. This paper describes the software pipeline that ships and registers those images at each facility.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central auto-ingestion claim rests on an unverified path-preservation convention between Rucio LFNs and Butler datastore roots; a single live end-to-end transfer would settle it.","rationale":"The reader's weakest assumption was the unvalidated 7-second transfer latency target. That is a performance target and is peripheral to the central architectural claim: whether replicated files are automatically registered into the destination Butler repository. The more load-bearing condition is the path-preservation and metadata-sufficiency convention that makes auto-ingestion possible. The paper asserts this convention in a single sentence and provides no evidence that it holds across the heterogeneous storage systems at the three facilities. This is not an accusation of error; it is an unverified implementation detail on which the entire synchronization model depends. Because the relevant components are open source and a single controlled transfer can test the convention, the concern is concrete and easily settled. I would not reject the paper, but I would condition acceptance on a small end-to-end demonstration or on a precise specification of the LFN and RSE-prefix conventions. The 7-second latency concern can remain a caveat, but it is not the single weakest point of the central claim.","tokens_in":6024,"tokens_out":10111,"duration_ms":132355,"concrete_test":"Set up a destination RSE with a deliberately different protocol prefix from the source, register one representative raw exposure and one derived dataset with rucio_register, transfer them via FTS, and run ingestd. Then query the destination Butler for both files and verify that their registry locations match the actual physical paths. If paths are not preserved or the metadata is insufficient for ingestion, the central claim fails; if the Butler query returns the files correctly, the paper's path-preservation convention is confirmed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The architecture's core claim is that a replicated file lands at the exact path the destination Butler expects and is then auto-registered by ingestd. The only stated mechanism for this is the assertion in Section 3 that all RSEs use Rucio's identity logical-to-physical filename mapping, which is said to ensure paths are preserved relative to the Butler datastore. This inference is not automatic across heterogeneous storage: an RSE protocol prefix mismatch, or a difference between the source and destination Butler datastore roots, changes the final physical pathname. The identity mapping only fixes how an LFN is converted to a PFN; it does not by itself align the storage prefixes at XrootD, dCache, and S3 endpoints. The paper does not validate that rucio_register emits LFNs equal to Butler-relative paths, nor that every facility's RSE root is exactly the Butler datastore root. Similarly, the 'minimal set' of JSON metadata is asserted to suffice for Butler ingestion, but no example, schema, or test is shown. If any facility has a different prefix or a file type whose metadata is incomplete, replicated files land outside the Butler datastore, ingestion fails or registers incorrect locations, and pipelines cannot access the data. The 7-second latency target identified by the reader is a performance requirement; even if it were unmet, the architecture could still be correctly implemented. The path-preservation and metadata convention is a correctness condition for every replicated file, so it is more load-bearing.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes the data movement architecture for the Vera C. Rubin Observatory, covering three use cases: raw data transfer from the summit in Chile to the US archive at SLAC, replication from the archive to the France and UK data facilities for processing, and distribution of annual data releases to data access centers. The proposed model combines Rucio and FTS for transfer management with three custom tools: rucio_register (selection and registration of Butler files into Rucio datasets), HermesK (Kafka-based notification routing of successful transfers), and ingestd (automatic ingestion of replicated files into destination Butler repositories). The paper argues that this design keeps each facility's Butler repository synchronized without manual intervention, relying on Rucio's identity logical-to-physical filename mapping to preserve paths relative to the Butler datastore.","tokens_in":6264,"tokens_out":5003,"duration_ms":57198,"significance":"If the described architecture works as claimed, it is a valuable operational blueprint for a multi-petabyte, multi-continental scientific data system, and it offers a concrete integration pattern between Rucio and a domain-specific storage abstraction (Data Butler). The paper's strengths are its clear use-case decomposition, explicit tool choices with public code repositories, and the identification of design conventions (e.g., separate input and product RSEs, cost-motivated avoidance of registering intermediate files in Rucio). However, the central automatic-ingestion claim rests on a path-preservation invariant and a minimal metadata schema that are asserted but not demonstrated; no end-to-end transfer, schema, or test is reported. The paper is also entirely free of performance measurements, which is acceptable for an architecture description only if the targets are explicitly labeled as targets.","major_comments":[{"comment":"The automatic ingestion claim depends on the invariant that a replicated file's physical path at the destination is exactly the path expected by the destination Butler datastore. The identity logical-to-physical filename mapping preserves relative pathnames only if every RSE's root prefix coincides with the corresponding Butler datastore root, or if rucio_register emits LFNs that are already Butler-relative. The paper does not state this configuration invariant, nor does it provide evidence from a live transfer. Because Section 3.2 says ingestd registers files into the local Butler repository, a path mismatch would place files outside the repository and break pipeline access. Please state the invariant explicitly and provide at least one end-to-end transfer test, or clearly label this as an unvalidated design assumption.","section":"Section 3, 'All RSEs are configured...'"},{"comment":"The JSON metadata attached by rucio_register is described as a 'minimal set of information' sufficient for Butler ingestion, but no schema, example, or coverage test is given. Without this, a reader cannot assess whether all file types (FITS, Parquet, JSON/YAML, and the Zip aggregates mentioned in Section 3.1) are handled correctly. Please include the metadata schema (or a link to it) and describe how it is validated for the distinct file categories.","section":"Section 3.1, 'rucio_register' paragraph"},{"comment":"The seven-second end-to-end latency target for transferring four gigabytes of compressed exposure data is load-bearing for the 60-second prompt-alert requirement, but it is presented as a target with no measurements, network characterization, or reference. If this is a design target, say so explicitly and cite any related network validation or requirements document; if it has been tested, report the test conditions. This is a performance claim rather than a correctness claim, but it should be labeled to avoid overstatement.","section":"Section 2.1, latency target"}],"minor_comments":[{"comment":"There is a typo: 'webDA V protocol' should read 'WebDAV protocol'.","section":"Section 3, first paragraph"},{"comment":"The phrase 'triggering o ff known synchronization points' appears to contain a spacing error ('off').","section":"Section 3.2, last paragraph"},{"comment":"The sentence contains a duplicated article: 'part of the the published release'.","section":"Section 2, first paragraph"},{"comment":"The 'approximately ten times' ratio for intermediate data relative to final products is given without a citation or derivation; please add a reference or label it as an operational estimate.","section":"Section 2.2, paragraph on replication"},{"comment":"The paper mentions idempotent batch ingestion and stateless daemons, but does not discuss failure handling (e.g., retries for failed transfers, handling of partial or duplicate notifications). A brief statement on how consistency is maintained under failures would strengthen the operational description.","section":"Section 3.2, paragraph on ingestd"},{"comment":"The flow from processing facilities back to the archive is described at a high level, but it is not stated whether the same rucio_register/subscription mechanism is used symmetrically for final data products; please clarify.","section":"Section 2.2, back-flow paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper is a useful and internally consistent systems description, and the author team is well positioned to address the missing validation details. The main risk is that the auto-ingestion claim is presented as a working property while the path-preservation invariant is only asserted; this should be fixed before publication. The lack of any performance measurement is less concerning if the seven-second number is explicitly framed as a target. The manuscript may be better suited to a technical/proceedings venue than a broad journal, but that is an editorial call."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a straightforward architecture description of how Rubin will move and register data between its processing centers, built from Rucio, FTS, and the Butler plus three small custom pieces. The useful new material is the integration pattern—rucio_register, HermesK, ingestd, and the Zip bundling of small Butler files—shown with enough code pointers that someone else could reproduce the setup. It is honest about being a system description: no performance claims beyond a stated 7-second target for a 4 GB exposure, and no failure-mode analysis. For an observatory about to take data, that is a reasonable scope.\n\nThe soft spots are real but modest. The paper asserts that identity LFN-to-PFN mapping automatically preserves Butler-relative paths across RSEs with different storage systems (XrootD, dCache, S3). That is not actually automatic: the identity mapping only controls the LFN-to-PFN transformation; the RSE root prefix still has to line up with the Butler datastore root. The paper does not show any end-to-end test or validation that a replicated file lands at the exact path ingestd expects. The stress-test note is right that a single live transfer would settle it. Similarly, the 'minimal set' of JSON metadata is asserted to be sufficient for Butler ingestion, but no schema or example is shown. These are gaps in verification, not necessarily errors in the design; the configuration could well be correct. And the paper is not pretending to offer measured performance, so the latency target should be read as a requirement, not a result.\n\nThe description is internally consistent, the citations are appropriate (including the team's own technical notes, which is fine here), and the code is actually posted. No circular argument is hiding in the text. This is a perfectly citable reference for the Rubin data movement stack.\n\nVerdict: I would accept it for peer review at a conference like ADASS or SPIE, and would be happy to see it published after a short revision that addresses the path-preservation verification and shows at least one real ingestion event (or explicitly says it is in progress). It is not deep research, but it is exactly the kind of engineering documentation that should be peer-reviewed rather than left as internal notes.","headline":"A clear, honest systems description of Rubin's data movement stack; the load-bearing path-preservation claim needs one end-to-end check before publication.","tokens_in":6932,"tokens_out":2429,"would_cite":true,"duration_ms":25894,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that Rubin's replicated data can be made self-registering at every facility by chaining Rucio and FTS transfers into Butler ingestion via three custom daemons, guaranteeing that pipelines see transferred data immediately.","keywords":["Vera C. Rubin Observatory","Legacy Survey of Space and Time","data movement","Rucio","File Transfer Service","Data Butler","data replication","time-domain alerts"],"falsifier":"Measure the wall-clock time from a completed exposure readout at the summit to its ingestion into the US archive Butler repository via the production transfer path: a consistent per-exposure time above about seven seconds, or an alert issued more than sixty seconds after readout, would falsify the prompt-processing model; likewise, any file Rucio marks as transferred but that is missing from the destination Butler registry would show the synchronization chain is broken.","tokens_in":5841,"feed_emoji":"🔭","tokens_out":10480,"duration_ms":108396,"temperature":0.7,"pith_summary":"Every night the Vera C. Rubin Observatory's 3.2-gigapixel camera will record roughly five petabytes of new images per year, and those images must flow from the Chilean summit to processing centers on three continents, with alerts about transients due within sixty seconds of readout. This paper presents the operational design that keeps that multi-site dataset coherent: the generic transfer pair Rucio and FTS moves files between storage elements, while three custom tools—rucio_register, HermesK, and ingestd—ensure that what arrives at a facility is immediately known to that facility's Butler repository, the storage-access layer the science pipelines actually query. The reason to care is that the entire LSST science program, from nightly alert generation to annual data releases, depends on replicated data becoming available to pipelines the moment a transfer completes, not merely sitting on a disk somewhere.","feed_headline":"New Rubin data lands pre-registered at every processing site","feed_subtitle":"A thin chain of three daemons makes every completed transfer instantly visible to the science pipelines.","key_machinery":"The load-bearing mechanism is the registration-to-ingestion chain: rucio_register → Rucio Datasets and subscriptions → FTS transfer → HermesK Kafka topic → ingestd → Butler ingestion, with identity filename mapping preserving the paths the Butler expects at the destination. The named objects are the three custom components: rucio_register, which registers selected Butler files into Rucio with Rubin metadata; HermesK, which reroutes Rucio transfer-completion notices through Kafka topics named per RSE; and ingestd, a stateless daemon that performs idempotent batch ingestion of newly arrived files into the local Butler. The chain also solves a small-file problem: JSON and YAML metadata files from a processing run are zipped into single containers so Rucio and FTS move one large object instead of tens of thousands of tiny ones.","core_discovery":"The central claim is that a closed integration loop makes inter-facility replication self-registering. rucio_register selects already-written files from a source Butler repository, attaches a compact JSON record of Rubin-specific metadata, and groups them into Rucio Datasets; preconfigured subscriptions translate dataset registrations into transfer rules that Rucio hands to FTS. Each facility exposes Rucio Storage Elements configured with identity logical-to-physical filename mapping, so pathnames relative to the Butler datastore are preserved verbatim. When FTS reports a completed transfer, HermesK, a modification of Rucio's Hermes daemon, publishes a message to a Kafka topic named after the destination storage element, and the receiving facility's ingestd daemon consumes that message and ingests the new file into the local Butler repository. If the model behaves as designed, every replicated file is automatically registered at its destination and immediately visible to the processing pipelines.","pith_inferences":["Editorial inference: the register-transfer-notify-ingest pattern is generic; any multi-site experiment whose local catalog must reflect replicated files could reuse this design, because the daemons are stateless and ingestion is batched and idempotent.","Editorial inference: the seven-second transfer target is a design budget, not a measured result, so an instrumented end-to-end test of exposure-to-ingestion latency on the production links would be the natural validation of the 60-second alert commitment.","Editorial inference: the paper does not describe monitoring for divergence between Rucio's record of completed transfers and the destination Butler registry; a lag alert on the HermesK-to-ingestd path would close the one loop where silent inconsistency could enter.","Editorial inference: once the annual Data Release is registered as Rucio Datasets, the same subscription machinery could drive the tiered distribution to independent data access centers, reducing each release campaign to dataset-to-RSE rules."],"forward_implications":["A successfully replicated file is automatically ingested into the destination Butler repository, so science pipelines see newly arrived data without any manual registration step.","Rucio tracks only files that are replicated between facilities; intermediates, which can be ten times more numerous than final products, stay in local Butler registries and keep the global catalog small.","Naming Kafka topics after the destination storage element means each facility receives only notifications for its own endpoints, letting ingestd scale to multiple consumers as the file count grows.","Grouping files into Rucio Datasets with spatially meaningful names lets subscriptions associate sky regions with storage elements and provides a deterministic dataset-closure signal.","Maintaining separate input and products storage elements lets the project prioritize raw-data transfers over regenerable data products and protects raw data against modification."],"supporting_citations":[{"why":"States the 60-second alert budget from which the seven-second transfer latency target follows.","marker":"[2]"},{"why":"Describes the multi-facility distributed image processing infrastructure that this data movement model serves.","marker":"[5]"},{"why":"Defines the Data Butler repository abstraction whose per-facility registries must be updated when files arrive.","marker":"[7]"},{"why":"Supplies the dataset, subscription, and RSE machinery that triggers file transfers towards the destination facilities.","marker":"[9]"},{"why":"Executes the actual file transfers between storage elements on behalf of Rucio.","marker":"[10]"},{"why":"Is the data movement protocol exposed by the US and UK facility storage elements.","marker":"[11]"},{"why":"Is the storage system and protocol used by the France facility's storage elements.","marker":"[12]"},{"why":"Provides the file-count and size distribution from a data preview run that motivates batching small ancillary files into Zip containers.","marker":"[16]"}],"fun_headline_variants":["Rubin's data pipeline auto-registers every replicated file","Self-registering data flow for Rubin Observatory","Rubin data moves with automatic registration","Closed loop makes Rubin replication self-registering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design assumes the end-to-end latency target of seven seconds for four gigabytes of compressed data per exposure is actually achievable over the international network links, but the paper states this as a target without presenting measurements or network validation.","fun_headline_variants_meta":{"raw":{"variants":["Rubin's data pipeline auto-registers every replicated file","Self-registering data flow for Rubin Observatory","Rubin data moves with automatic registration","Closed loop makes Rubin replication self-registering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000453,"raw_usage":{"total_tokens":2260,"prompt_tokens":907,"completion_tokens":1353,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":523,"completion_tokens_details":{"reasoning_tokens":1294}},"tokens_in":523,"tokens_out":1353,"duration_ms":10955,"temperature":1.0,"reasoning_tokens":1294,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:33:14.500841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the wall-clock time from a completed exposure readout at the summit to its ingestion into the US archive Butler repository via the production transfer path: a consistent per-exposure time above about seven seconds, or an alert issued more than sixty seconds after readout, would falsify the prompt-processing model; likewise, any file Rucio marks as transferred but that is missing from the destination Butler registry would show the synchronization chain is broken.","supporting_citations":[{"cited_title":"Juri ´c, T","cited_arxiv_id":null,"evidence_quote":"States the 60-second alert budget from which the seven-second transfer latency target follows."},{"cited_title":"The Vera C. Rubin Observatory Data Butler and Pipeline Execution System","cited_arxiv_id":"2206.14941","evidence_quote":"Defines the Data Butler repository abstraction whose per-facility registries must be updated when files arrive."},{"cited_title":"Barisits, T","cited_arxiv_id":null,"evidence_quote":"Supplies the dataset, subscription, and RSE machinery that triggers file transfers towards the destination facilities."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Executes the actual file transfers between storage elements on behalf of Rucio."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the data movement protocol exposed by the US and UK facility storage elements."}],"review_version":1}