{"id":"a334466a-4eb8-40ee-a92d-3ffd8f214ed1","arxiv_id":"2507.16109","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Under network delay and partition faults, cloud-edge Kubernetes deployments show tighter response-time distributions than cloud-only deployments, while cloud deployments stay more stable under bandwidth throttling and packet loss.","lead":"This paper builds an automated framework that injects faults into Kubernetes clusters and measures how cloud and cloud-edge deployments respond, running nearly 12,000 failure experiments. It claims cloud-edge setups stay more stable under network delay while cloud setups cope better with bandwidth limits, which could guide where to place critical services.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 80%/47% headline figures never appear in the results, and the only normalization described (§4.4.1, Eq. 1) uses a 25%-fault baseline, so the claimed advantages may be artifacts of that reference point rather than intrinsic resilience.","rationale":"The reader's verdict is REJECT, and my stress-test does not change that. The engineering contribution—an orchestrated fault-injection framework with public code and a large claimed dataset—is real, but the central quantitative claim (the 80%/47% deployment guidance) is not auditable from the manuscript. The results section describes qualitative patterns and shows z-score distributions, but the exact percentages appear only in the abstract and conclusion. The normalization framework in §4.4.1 is the only quantitative bridge between raw measurements and the headline figures, and its choice of a 25%-fault baseline is a genuine correctness risk: it defines 'normal' as already-perturbed behavior, and the edge environment's inherent latency jitter can inflate its baseline standard deviation, making injected delay look comparatively small. This is a specific, testable mechanism by which the reported edge stability advantage could be an artifact. The reader identified the same weakness, and my analysis agrees. I also note the abstract's claim that Gremlin and ChaosBlade are integrated, whereas §3 says the implementation focuses on Chaos Mesh with 'extensibility provisions' for others; this is an overstatement but secondary to the load-bearing normalization issue. The correct remedy is not merely adding text but providing the missing derivation and rerunning the analysis with an undisturbed baseline; until then, REJECT is appropriate.","tokens_in":18129,"tokens_out":5942,"duration_ms":59299,"concrete_test":"Request from the authors the exact derivation of the 80% and 47% figures (no such derivation appears in §4.4 or §4.5), and then recompute them from the raw per-request logs using an undisturbed baseline: set μ and σ from 0% fault-intensity measurements, re-apply Eq. (1), and recalculate the z-score variance comparison for the delay/partition and bandwidth scenarios. Alternatively, rerun a representative subset (e.g., 4-node Sock-Shop delay and bandwidth scenarios) with 0% injection as the baseline. If the ordering or the magnitude of the 80%/47% claims changes materially, the 25%-fault baseline is a load-bearing artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the abstract/conclusion assertion that cloud-edge deployments show 80% superior response stability under delay/partition while cloud shows 47% better resilience under bandwidth limits. For that claim to hold, the percentages must be derived from a metric whose reference point is meaningful. Two facts undermine this. First, the results section (§4.4, Figs. 5–7) never states or derives the 80% or 47% figures; §4.5 only gives qualitative 'favors cloud/edge' statements. Second, the only quantitative normalization defined is the z-score in Eq. (1), whose baseline μ and σ are taken 'using 25% fault intensity measurements' (§4.4.1). Thus z = 0 corresponds to a perturbed cluster, not an undisturbed one. Because the edge environment already has 200 ms latency with ±10% jitter (§4.1), its σ at 25% intensity can be much larger than cloud's, so an equal added delay yields smaller z for edge. The reported pattern—edge within ±1σ while cloud reaches 3–6σ—could therefore be an artifact of normalizing by a noisier edge baseline. The 47% figure is also unexplained. Without the raw derivation or an undisturbed baseline, the headline percentages are unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a framework for automated fault injection and load testing of Kubernetes clusters in pure-cloud and cloud-edge configurations. The framework orchestrates Chaos Mesh fault injection with Locust workloads and is used to run 11,965 experiments across two cluster sizes, monolithic and microservices applications, and several fault types at four intensity levels, collecting response-time and error data. Based on z-score normalized response times, the authors report that under network delay and partition faults the cloud-edge deployment shows greater response stability, while under bandwidth limitation and network loss the cloud deployment is more stable. The paper claims quantitative results: 80% superior response stability for cloud-edge and 47% better resilience for cloud under bandwidth limits, and it positions a dataset of roughly 30 GB and 57 million requests as a first-of-its-kind contribution. The framework code is said to be publicly available on GitHub, while the dataset is currently in a private repository.","tokens_in":18380,"tokens_out":8130,"duration_ms":85526,"significance":"If the quantitative results were properly derived, this would be a useful empirical benchmark for deployment decisions in cloud-edge Kubernetes. The main strengths are the scale of the fault-injection study (nearly 12,000 scenarios), the direct comparison of monolithic versus microservices architectures under identical faults, the open-source framework code, and the use of both absolute and normalized metrics. However, the paper's central numerical claims are not substantiated in the results section, and the chosen normalization baseline is questionable; these issues must be resolved before the findings can be accepted as quantitative guidance.","major_comments":[{"comment":"The central quantitative claims—80% superior response stability under network delay/partition and 47% better resilience under bandwidth limitations—appear only in the Abstract and the Conclusion. The results section presents qualitative statements (e.g., Table VII's 'preferred deployment' column) and z-score ranges (e.g., §4.4.4: cloud medians reaching 3.0σ and whiskers extending to 6σ, edge within ±1σ), but no formula or calculation shows how the 80% and 47% figures are derived. Please provide the exact computation for both percentages (metric, aggregation, baseline, and fault intensities used), or remove the percentages from the Abstract and Conclusion if they cannot be substantiated.","section":"Abstract; §4.4; §5"},{"comment":"The z-score baseline is not a no-fault baseline. The text states that 'Baselines are established using 25% fault intensity measurements,' so z=0 refers to a cluster already under a nonzero fault load. Because the edge environment is configured with 200 ms base latency, ±10% jitter, and 10% packet loss (§4.1, Table III), its response-time variance at 25% fault intensity is plausibly larger than the cloud's. Normalizing by this noisier edge baseline compresses edge z-scores, so the reported contrast (edge always within ±1σ while cloud reaches 3–6σ) could arise from the baseline choice rather than from intrinsic deployment resilience. Please use measurements from an undisturbed cluster as μ and σ, or present the analysis in absolute terms (e.g., raw response-time increase) alongside the normalized values.","section":"§4.4.1, Eq. (1)"},{"comment":"The resilience comparisons are based on qualitative descriptions of z-score distributions (e.g., 'peak z-scores reaching 1.5σ versus cloud's 0.5σ' in §4.4.3), but no sample sizes, confidence intervals, or significance tests are reported. The term 'z-variance' in Table VII is never defined, and the figures do not show error bars. Without this information, the reader cannot determine whether the reported cloud-versus-edge differences are distinguishable from run-to-run noise, particularly since the results aggregate across applications, cluster sizes, thread counts, and timeout settings. Please add per-condition summary statistics and a statistical test or effect-size measure.","section":"§4.4.3–§4.4.5"},{"comment":"The claimed framework capabilities exceed what is implemented and evaluated. Table I marks 'Our Work' as supporting control-plane fault injection, and §2.3 says the framework 'uniquely combines control-plane fault injection with integrated workload generation'; however, §3.1's Injection Controller describes only Chaos Mesh integration, with 'extensibility provisions' for other tools, and no control-plane fault experiment (e.g., etcd or API-server disruption) appears in §4. The Abstract similarly lists Gremlin and ChaosBlade as combined platforms, while §4.2 states 'Our fault injection leverages Chaos Mesh.' These claims should be brought in line with the actual implementation, or supplemented by experiments using the advertised tools.","section":"§2.3, Table I, §3.1"}],"minor_comments":[{"comment":"The four experiment counts in Table VI sum to 11,964, not 11,965 as stated throughout the paper; please reconcile this discrepancy.","section":"§4.3, Table VI"},{"comment":"The Author Contributions section says 'The author independently conducted all aspects of this study,' but the title lists three authors; please clarify the contribution statement for all authors.","section":"Author Contributions"},{"comment":"The footnote defines failure rate as 'number of failed experiments ÷ total experiment' (with what appears to be a missing word 'count'), which measures the proportion of experiments that failed rather than a request-level failure rate; please clarify the definition and keep it consistent with the dataset description in §4.3.","section":"Table VI, footnote"},{"comment":"The dataset is described as a key contribution, but §4.3 states that it is hosted in a private repository and Appendix B says it is not included and must be requested by email; please provide a concrete data-availability plan, such as an anonymous access link or an archival repository, for review and eventual publication.","section":"§4.3, Appendix B"},{"comment":"The z-score distribution plots would be easier to interpret with sample sizes and clear per-intensity labels; currently the reader cannot infer how many experiments underlie each distribution.","section":"Figures 5–7"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like a thorough technical report, and its strongest asset is the scale of the experiment. The headline quantitative findings need to be either properly derived from the data or removed, and the z-score baseline should be reconsidered. There is also an unusual mismatch in the author contributions statement (singular author vs three listed authors) and several self-overstated capability claims; these should be checked during revision. I do not see evidence of deliberate misconduct, but the private dataset and the exact scope of the released framework should be verified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you care about chaos engineering for Kubernetes in edge setups. The genuinely new thing is a working orchestration framework that runs fault injection (Chaos Mesh) plus load generation (Locust) across cloud and cloud-edge clusters, and a large experiment campaign: roughly 12k fault-injection scenarios on 4- and 8-node clusters, with monolithic and microservice apps. The code is on GitHub. That is real engineering and worth building on.\n\nThe qualitative finding is plausible: under bandwidth throttling and packet loss, cloud looks more stable; under added latency and partition, edge looks more stable in z-score terms. That inversion is the paper's interesting claim, and it is exactly where the soft spots are.\n\nFirst, the abstract numbers—80% superior response stability for edge, 47% better resilience for cloud—appear nowhere in the results section. No derivation, no formula, no figure. They show up only in the abstract and conclusion. That is a load-bearing gap because those are the numbers a practitioner would take away.\n\nSecond, the z-score normalization in Eq. (1) uses the 25% fault intensity condition as baseline μ and σ, not an undisturbed cluster. The edge environment already has 200 ms latency with jitter, so its σ at 25% intensity is likely larger than cloud's. Normalizing by that larger σ compresses edge z-scores, which can manufacture exactly the \"edge stays within ±1σ\" pattern. The paper dismisses this choice in one sentence, but it is the crux of the inversion. With an undisturbed baseline, the result might flip.\n\nThird, the abstract says the framework \"combines multiple fault injection platforms\" including Gremlin and ChaosBlade; Section 3 admits only Chaos Mesh is actually integrated. That is an overstatement. And the \"first comprehensive dataset\" is in a private repo—contact author for access. Not reproducible yet. Minor: Table VI defines failure rate as failed experiments divided by total experiments, which looks like a typo for failed requests.\n\nThe paper is not junk. The framework is modular, the experiment design is systematic, and the limitations section is honest about simulated edge conditions. But the headline percentages are unsupported, and the normalization baseline is a legitimate alternative explanation for the main finding. This needs major revision, not acceptance as is.\n\nRecommendation: send it to peer review. A good referee will ask for the derivation of 80%/47%, an undisturbed baseline or a strong justification for the 25% choice, and dataset release. If those are fixable, it becomes a useful empirical contribution. I would not cite it in its current form, but I would bring it to reading group to discuss the normalization choices.","headline":"Useful engineering, unsupported headline numbers: the 80%/47% claims need a derivation and a fairer baseline before the main finding can be trusted.","tokens_in":18882,"tokens_out":3228,"would_cite":false,"duration_ms":33134,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that cloud-edge Kubernetes deployments are the more stable choice when network delay or partition faults dominate, while pure cloud deployments are more resilient under bandwidth throttling and network loss, and it backs…","keywords":["Kubernetes","cloud-edge computing","resilience evaluation","fault injection","chaos engineering","microservices","z-score normalization","performance analysis"],"falsifier":"Re-run the z-score normalization with an undisturbed cluster (no injected fault) as the baseline instead of the 25% intensity measurements. If the reported 80% stability advantage for cloud-edge under delay and partition, or the 47% cloud advantage under bandwidth, shrinks below statistical significance or reverses sign, the central claim is an artifact of the normalization baseline. A complementary check is to repeat the network-loss experiments while logging successful and failed requests separately, since the paper notes survivorship bias at 100% loss.","tokens_in":17912,"feed_emoji":"⚡","tokens_out":5815,"duration_ms":59002,"temperature":0.7,"pith_summary":"The paper sets out to turn deployment advice for Kubernetes in cloud-edge settings from folklore into measurement: it builds an orchestration framework that injects pod, node, and network faults under realistic load, and runs it across cloud-only and cloud-edge clusters running both a monolithic and a microservices application. The central empirical claim is that the better deployment depends on the dominant failure mode. Under injected network delay and partition faults, cloud-edge deployments keep response times far more stable, reported as 80% superior response stability; under bandwidth throttling and network loss, cloud deployments show 47% better resilience. If true, this gives operators a quantitative rule: prefer the edge when latency disruptions dominate, prefer the cloud when throughput constraints dominate. The paper also releases what it calls the first comprehensive resilience dataset for hybrid cloud-edge Kubernetes, spanning 11,965 fault-injection scenarios.","feed_headline":"Cloud-edge beats cloud on delay faults, loses on bandwidth","feed_subtitle":"11,965 fault-injection experiments map where each Kubernetes deployment strategy holds up best.","key_machinery":"The load-bearing mechanism is the measurement and normalization pipeline, not any single fault. The framework orchestrates fault injection through Chaos Mesh custom resources, workload generation through Locust, health checks, and recovery through a five-phase workflow, then compares deployments using dual metrics: absolute average response time and a z-score of response time, $z = (x-\\mu)/\\sigma$, where the baseline mean $\\mu$ and standard deviation $\\sigma$ are taken from measurements at 25% fault intensity. The z-score is what converts raw latency numbers into a claimed stability advantage, and the paper's resilience conclusions rest on the variance of these normalized distributions across cloud versus cloud-edge deployments.","core_discovery":"The authors claim to have found a resilience inversion between cloud and cloud-edge Kubernetes deployments that absolute response-time averages hide. Measured with z-score normalization, edge deployments remain within about 1 standard deviation of their baseline under network delay and partition faults while cloud deployments stretch to 3-6 standard deviations, which they report as 80% superior response stability for latency-sensitive faults. Under bandwidth limitation and partial network loss, the pattern reverses: cloud z-scores stay near 0-0.6 standard deviations while edge medians exceed 1 standard deviation, a 47% resilience advantage for the cloud under throughput-constrained faults. The paper argues this inversion follows from architecture: edge services use shorter local paths and local caches, so delay and partition hit them less, while cloud networks have higher aggregate bandwidth and redundant paths, so throughput throttling hurts them less.","pith_inferences":["Because the baseline used for z-score normalization is itself a perturbed state (25% fault intensity), the 80% and 47% figures should be read as relative stability under already-degraded conditions, not as absolute resilience margins; a reader comparing SLOs should re-derive the comparison with a clean baseline.","The same dual-metric methodology could be applied to service meshes or multi-cloud topologies to see whether the resilience inversion is specific to Kubernetes or a general property of distributed placement.","The simulated edge conditions (200 ms latency, 10% loss) may understate real edge heterogeneity; testing on physical edge devices could reveal whether the patterns hold when compute and storage constraints interact with network faults.","Separating failed requests from successful ones in the 100% network-loss regime would remove the survivorship bias noted in the paper and yield a cleaner resilience metric."],"forward_implications":["Under network delay or partition fault regimes, operators should prefer cloud-edge placement when predictable response times matter more than peak speed.","Under bandwidth throttling or partial network loss, cloud deployment should be preferred when stable throughput is the priority.","Microservices pay for modularity with higher tail latency and failure rates; the dataset quantifies this gap and suggests longer client timeouts for microservices.","Scaling the cluster from 4 to 8 nodes reduces mean and P95 response times and lowers experiment failure rates in both deployment modes.","The public dataset can serve as a baseline for future Kubernetes resilience studies without rerunning 11,965 fault-injection scenarios."],"supporting_citations":[{"why":"Supplies the z-score normalization reference and the control-plane fault-injection motivation that the framework builds on.","marker":"[8]"},{"why":"Provides the data-normalization method used to compute resilience z-scores and compare environments with different baselines.","marker":"[55]"},{"why":"Chaos Mesh is the fault-injection platform integrated into the framework's Injection Controller.","marker":"[38]"},{"why":"Gremlin's systematic resilience testing of microservices motivates the fault-injection approach and tool comparison.","marker":"[23]"},{"why":"ORCAS contributes the idea of architecture-aware automatic fault-injection experiment generation.","marker":"[31]"},{"why":"Sock Shop is the 13-service microservices benchmark deployed under identical fault scenarios.","marker":"[54]"},{"why":"Documents that Kubernetes liveness and readiness probes miss performance-degradation faults, justifying application-level latency and error metrics.","marker":"[27]"},{"why":"Explains cascading failure propagation across microservices, the failure mode the experiments are designed to expose.","marker":"[7]"},{"why":"Provides the prior reliability comparison of microservices versus monolithic architectures that this study extends to cloud-edge settings.","marker":"[9]"}],"fun_headline_variants":["Cloud-edge wins delay, cloud wins bandwidth in Kubernetes","K8s resilience flips: edge for latency, cloud for bandwidth","Fault injection shows edge beats cloud on delays, loses on bandwidth","11,965 faults reveal: edge for delay, cloud for bandwidth","Resilience inversion in Kubernetes: edge vs cloud"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline percentages assume that the 25% fault-intensity measurements used to set the z-score baseline represent normal operation; if that perturbed baseline is not a fair reference, the reported stability advantages could be artifacts of normalization rather than intrinsic deployment properties.","fun_headline_variants_meta":{"raw":{"variants":["Cloud-edge wins delay, cloud wins bandwidth in Kubernetes","K8s resilience flips: edge for latency, cloud for bandwidth","Fault injection shows edge beats cloud on delays, loses on bandwidth","11,965 faults reveal: edge for delay, cloud for bandwidth","Resilience inversion in Kubernetes: edge vs cloud"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1333,"prompt_tokens":911,"completion_tokens":422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":336}},"tokens_in":527,"tokens_out":422,"duration_ms":4418,"temperature":1.0,"reasoning_tokens":336,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:17:29.722900+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the z-score normalization with an undisturbed cluster (no injected fault) as the baseline instead of the 25% intensity measurements. If the reported 80% stability advantage for cloud-edge under delay and partition, or the 47% cloud advantage under bandwidth, shrinks below statistical significance or reverses sign, the central claim is an artifact of the normalization baseline. A complementary check is to repeat the network-loss experiments while logging successful and failed requests separately, since the paper notes survivorship bias at 100% loss.","supporting_citations":[{"cited_title":"Mutiny! how does kubernetes fail, and what can we do about it?","cited_arxiv_id":null,"evidence_quote":"Supplies the z-score normalization reference and the control-plane fault-injection motivation that the framework builds on."},{"cited_title":"Investigating the impact of data normalization methods on predicting electricity consumption in a building using different artificial neural network models,","cited_arxiv_id":null,"evidence_quote":"Provides the data-normalization method used to compute resilience z-scores and compare environments with different baselines."},{"cited_title":"Chaos mesh – open source chaos engineering platform for kubernetes,","cited_arxiv_id":null,"evidence_quote":"Chaos Mesh is the fault-injection platform integrated into the framework's Injection Controller."},{"cited_title":"Gremlin: Systematic resilience testing of microservices,","cited_arxiv_id":null,"evidence_quote":"Gremlin's systematic resilience testing of microservices motivates the fault-injection approach and tool comparison."},{"cited_title":"Orcas: Efficient resilience benchmarking of microservice architectures,","cited_arxiv_id":null,"evidence_quote":"ORCAS contributes the idea of architecture-aware automatic fault-injection experiment generation."},{"cited_title":"A review of resilience testing in microservices architectures: Implementing chaos engineering for fault tolerance and system reliability,","cited_arxiv_id":null,"evidence_quote":"Sock Shop is the 13-service microservices benchmark deployed under identical fault scenarios."},{"cited_title":"A study on the aging and fault tolerance of microservices in kubernetes,","cited_arxiv_id":null,"evidence_quote":"Documents that Kubernetes liveness and readiness probes miss performance-degradation faults, justifying application-level latency and error metrics."},{"cited_title":"Impacttracer: Root cause local- ization in microservices based on fault propagation modeling,","cited_arxiv_id":null,"evidence_quote":"Explains cascading failure propagation across microservices, the failure mode the experiments are designed to expose."},{"cited_title":"Reliability evaluation of microservices and monolithic architectures,","cited_arxiv_id":null,"evidence_quote":"Provides the prior reliability comparison of microservices versus monolithic architectures that this study extends to cloud-edge settings."}],"review_version":1}