{"id":"ca47f9e5-066d-4d51-aabc-35f736a68235","arxiv_id":"2501.10513","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ConfigBot uses Bayesian optimization over Linux cgroups and ROS message adaptors to automatically find resource configurations that meet developer-specified robot performance targets.","lead":"ConfigBot is a system that automatically adjusts how a robot's limited computing power is divided among its apps by tuning operating-system limits and by throttling data streams inside the robot software. In tests on three robot platforms, it kept core navigation services stable while using fewer CPUs than the default setup and still running extra apps.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline result validates only that ConfigBot can hit a developer-chosen publish-rate metric; it never validates that hitting that metric produces stable, collision-free behavior, so the claimed end-to-end stability is unsupported.","rationale":"The reader identified this same assumption as weakest; I agree. It is load-bearing because the optimization problem in §VI-A hard-codes Ji(c) ≥ Jo_i as the only notion of correctness, and all reported 'satisfaction rates' measure only these Ji values. A wrong proxy is not merely a measurement issue: ConfigBot would actively optimize toward it, and the selected configuration could be unsafe while scoring 100% on the metric. This is especially plausible in the Spot case because command-velocity publish frequency does not encode data freshness or trajectory quality, and the paper's own §IV-C shows how resource limits can cause internal contention and degraded outputs without necessarily stopping publishes. The absence of end-to-end safety validation, combined with single-run results and no artifacts, makes the conditional rating appropriate. The proposed safety-event check is a single experiment that would settle whether the metric proxy is valid; if it passes, the central claim is substantially strengthened.","tokens_in":11202,"tokens_out":6943,"duration_ms":79585,"concrete_test":"Run the §VIII-A Spot experiment with the Table IV ConfigBot config and the Default config (same CPU budget) over, e.g., 20 repeated navigation trials, logging both command-velocity frequency and an independently assessed safety score (collisions, falls, deviation from commanded path). If ConfigBot's config shows ~99% Ji satisfaction but its safety-event rate is not significantly lower than Default's, the chosen Ji is not a sufficient proxy and the stability claim is unsupported; if safety events track Ji satisfaction across trials, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the §VI-A assumption that each developer-defined Ji and target Jo_i are accurate proxies for correct robot behavior. In the headline Spot experiment (§VIII-A, Fig. 1), the only core constraint is command-velocity publish frequency ≥35 Hz. ConfigBot optimizes cgroup limits and adaptor thresholds against this Ji, and Table IV reports only satisfaction of Ji. Nothing measures whether the 35 Hz commands are computed from fresh LiDAR/odometry or produce collision-free, stable motion. Because adaptors selectively drop messages (§V) and cgroups throttle processing, a configuration can sustain 35 Hz publishes while starving perception or computing on stale state; such failures would not register as constraint violations. Table III's failure modes (stumbling, collisions, pauses) are attributed to low update rates, but no experiment shows that restoring the update rate actually removes those failure modes. Tables IV–VII therefore establish that ConfigBot finds configs that satisfy Ji, not that ConfigBot maintains the end-to-end stability claimed in the abstract and Table III.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ConfigBot, a runtime resource-allocation framework for ROS-based service robots. ConfigBot treats core robot services as constrained optimization targets and non-core apps as objectives, then uses Bayesian optimization to tune per-node Linux cgroup CPU limits and ROS-layer adaptor thresholds that throttle message rates on selected subscription edges. The system monitors for new processes and nodes and triggers relearning when context changes. Experiments on three robots (Spot, Jackal, Cobot) and six stacks compare ConfigBot against default OS configurations, three random configurations, and a cgroups-only ablation, reporting core constraint satisfaction rates and non-core app performance. Headline results include meeting a 35 Hz command-velocity target on Spot with about 1 CPU where the default configuration fails with 4 CPUs, and improved non-core performance of 49-193% over default.","tokens_in":11367,"tokens_out":5377,"duration_ms":55633,"significance":"If the empirical results hold, ConfigBot would be a practical contribution to robot resource management: it combines OS-level cgroup control with a transparent, application-agnostic ROS-layer adaptor abstraction, and it evaluates on multiple real robots and stacks with several baseline comparators, including random configurations and an ablation. These are genuine strengths. However, the central evidence currently consists of single-run point estimates, and the success metric is the same developer-specified objective that ConfigBot optimizes; no behavioral validation links satisfaction rates to collision-free or stable operation. The claimed end-to-end stability and real-world effectiveness therefore require additional evidence before the paper's central claims are fully supported.","major_comments":[{"comment":"All headline quantities in Tables IV-VII are single-run percentages or single non-core performance values, with no standard deviations, confidence intervals, or numbers of repeated trials. Because Bayesian optimization is stochastic and robot experiments are noisy, differences such as ConfigBot 100% vs. Random-1 99.3% in Table V, or ConfigBot 100% vs. ConfigBot-cg 100% in the Basic and Intermed rows, may be within noise. The claims of 'near-perfect satisfaction' and '49-193% better' non-core performance are not yet supported by the reported data. Please report repeated runs with distributions, and use an appropriate significance test or effect-size metric, or explain why single runs are representative.","section":"§VIII-A, §VIII-B, Tables IV-VII"},{"comment":"The abstract and Table III claim that ConfigBot maintains system stability, listing failure modes such as stumbling, collisions, and pauses and attributing them to low update rates. However, Tables IV-VII measure only whether the developer-specified metric Ji (e.g., a 35 Hz command-velocity publish frequency) is satisfied and what the non-core performance value is; no experiment records collisions, stumbles, pauses, or manipulation errors under ConfigBot versus default configurations. Since adaptors drop messages and cgroups throttle computation, satisfying the publish-rate constraint does not by itself imply fresh perception inputs or stable control. The conclusion that ConfigBot 'maintains system stability' is therefore unsupported; please add direct behavioral outcome measurements or explicitly limit the claims to meeting developer-specified performance constraints.","section":"§VI-A, §VIII-A, Table III"},{"comment":"The optimization problem and the evaluation both rest on the assertion that developer-provided metrics and targets are 'reasonable' and 'often easy to define,' but no support or validation is given. This assumption is load-bearing because ConfigBot optimizes only Ji; if a 35 Hz publish rate can coexist with stale odometry or starved perception, the satisfaction rates in Tables IV-VII do not imply effective robot behavior. Please validate the proxy for at least the headline Spot/Basic case (e.g., by showing that command velocities are computed from fresh LiDAR and odometry under the tuned configuration), or reframe the contribution as meeting developer specifications rather than achieving end-to-end stability.","section":"§VI-A"},{"comment":"The dynamic-adaptation contribution is not quantified. Section VIII-D reports that the eBPF monitor detects new nodes in under 2 seconds, but the time to converge to a new configuration (number of Bayesian optimization iterations, the 5-second profiling rounds, total downtime, and whether constraints were violated during retraining) is not reported. Without this information, the claim that ConfigBot 'quickly reapplies' configurations and handles dynamic changes is not fully supported. Please report the timing and overhead of initial learning and relearning, including the configuration-library hit path.","section":"§VI-B, §VIII-D"}],"minor_comments":[{"comment":"The abstract contains a typo: 'ConifgBot' should be 'ConfigBot'.","section":"Abstract"},{"comment":"The non-core performance values in parentheses (e.g., '2.12', '1.45') lack units and a definition; please state what 'obj performance' measures (frames per second, messages per second, or a composite score) in the text or table caption.","section":"Table V and §VIII-B"},{"comment":"The entry '45.31 ( 1.52)' appears to be missing a percent sign or decimal point; it should likely read '45.3% (1.52)'.","section":"Table VI"},{"comment":"The Bayesian optimization implementation is underspecified: the surrogate model, acquisition function, and hyperparameters are not given, which makes reproduction difficult. Please add these details or cite a specific implementation.","section":"§VI-B"},{"comment":"The adaptor symbol in Figure 3 appears as a blank placeholder '( )' in the rendered text; please ensure the symbol is visible in the final version.","section":"Fig. 3"},{"comment":"The sentence 'between 49% (for Intermed) to 193% (for Terrain)' is ambiguous about the baseline and whether the percentages are relative improvements; please clarify with a formula or explicit comparison.","section":"§VIII-B"}],"recommendation":"major_revision","confidential_remarks":"For the editor: this is a systems paper with a solid idea and a broad evaluation, but the empirical support is currently fragile because the reported numbers are single runs and the success metric coincides with the optimized objective. I do not think rejection is warranted, but the revision needs repeatability data and at least one behavioral validation on a headline experiment. Also, the paper does not mention code release; for a systems contribution, making the adaptor implementation and evaluation scripts available would strengthen reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a real systems paper with a genuinely useful new mechanism, but the headline stability claims go further than the data supports. It deserves peer review, not acceptance as is.\n\nWhat's new and good: the per-subscriber ROS adaptor is a real contribution. Intercepting message flows on individual subscription edges and throttling them independently is a lever ROS doesn't expose, and the ~150-line ros_comm modification makes it plausible for others to adopt. The paper also asks a good question—can automated config search replace manual cgroup tuning and static over-provisioning—and answers it with a sensible Bayesian optimization loop over cgroup limits and adaptor thresholds. The evaluation across three robots and six stacks, with default, random, and cgroups-only baselines, is genuinely useful. The cross-evaluation in Table VI, showing no globally optimal config, is a nice honest result that supports their context-dependent relearning argument.\n\nSoft spots: the strongest claims are about end-to-end stability (stumbling, collisions, pauses), but the experiments only measure developer-specified message-rate constraints. On Spot the core constraint is 35 Hz command-velocity publishes; nothing checks whether those commands are computed from fresh perception or produce collision-free motion. The adaptor's entire purpose is dropping messages, so a config could sustain 35 Hz while starving perception, and the metric wouldn't care. Table III attributes failures to low update rates, but no experiment restores update rate and then measures those failure modes directly. That gap is real, and the abstract's \"maintaining system stability\" goes beyond what the measurements show.\n\nMethodologically, the headline percentages are single-run, without error bars or significance tests. Table IV's \"Default (4x CPUs)\" only reaches 84.94% satisfaction, which does support the over-provisioning argument, but the 49–193% non-core improvement ranges need variance. There's also an oddity in Table V: ConfigBot-cg shows 100% satisfaction on Terrain while ConfigBot shows 96%—the text explains the non-core tradeoff, but not the satisfaction drop. No code or data is provided, which hurts reproducibility. The VI-A assumption that developer-defined metrics capture correct robot behavior is asserted rather than argued; the paper says metrics are \"often easy to define,\" and the whole evaluation inherits that assumption.\n\nWho this is for: systems researchers working on resource management for ROS robots. The adaptor idea alone is worth citing. The mechanism is sound, the evaluation is suggestive but not airtight, and the stability claims need a direct behavioral test. I'd send it to review: the adaptor contribution is enough to merit expert scrutiny, and the authors can reasonably be asked to add repeated trials and, ideally, a behavioral check (actual collision counts or stabilization success) before publication.","headline":"A genuine per-subscriber adaptor abstraction with a useful broad evaluation, but the end-to-end stability claims outrun the metric-only evidence; worth refereeing, with a request for direct behavioral validation.","tokens_in":11931,"tokens_out":2057,"would_cite":true,"duration_ms":20529,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ConfigBot claims a Bayesian-optimization loop over OS cgroup limits and ROS data-flow throttles can automatically find robot configurations that meet developer-specified performance requirements, letting a navigation stack run on one CPU…","keywords":["ConfigBot","adaptive resource allocation","Bayesian optimization","cgroups","ROS adaptors","performance specification","service robots","runtime profiling"],"falsifier":"Run the Basic navigation stack on Spot under ConfigBot's tuned ~1-CPU configuration through an obstacle course while logging both command-velocity frequency and physical stability (IMU-based stumble or collision events); if the robot stumbles during intervals where the 35 Hz constraint is continuously satisfied, the link between the optimized metric and robot stability is broken.","tokens_in":10988,"feed_emoji":"🤖","tokens_out":4362,"duration_ms":34957,"temperature":0.7,"pith_summary":"ConfigBot is a system that automatically retunes a robot's compute resources so that apps meet developer-defined performance targets without manual intervention. The paper argues that default Linux scheduling is a poor fit for robot stacks, and that tuning cgroup CPU limits plus a new 'adaptor' layer that throttles ROS message flows can restore performance. ConfigBot frames resource allocation as a constrained black-box optimization: core services are hard constraints, non-core apps are objectives, and Bayesian optimization searches the configuration space. On a Boston Dynamics Spot running a basic navigation stack, ConfigBot met the 35 Hz command-velocity spec with about one CPU, where the default OS failed to meet it even with four CPUs. The paper reports similar success across other stacks and robots (Jackal, Cobot), supporting the view that automated configuration tuning is a promising direction for robot deployments.","feed_headline":"ConfigBot hits robot speed specs with 4x fewer CPUs","feed_subtitle":"Automated tuning of CPU limits and ROS data-flow throttles beat the default OS on Spot, Jackal, and Cobot.","key_machinery":"The load-bearing pieces are (1) cgroup cpu.max controls that cap CPU usage per ROS node, (2) adaptors, a ~150-line modification to ROS's communication library that inserts a per-subscriber message filter to throttle data flows without code changes, and (3) Bayesian optimization over the combined cgroup/adaptor configuration space, using a developer spec that separates core services (constraints) from non-core apps (objectives). Runtime monitoring via eBPF, rosmaster polling, and a constraint monitor detects context shifts and triggers re-optimization or reuse of cached configs.","core_discovery":"The central claim is that robot resource allocation can be treated as an online, context-aware configuration search rather than a static provisioning problem. ConfigBot shows that a combination of OS-level cgroup limits, a thin per-subscription message-throttling layer ('adaptors') in ROS, and Bayesian optimization can satisfy developer-specified core-service constraints while maximizing non-core app performance. On Spot's Basic stack, the learned configuration achieved 99.42% constraint satisfaction with roughly one CPU, while the default OS configuration scored 0% and even a default config with 4 CPUs only reached 84.94%. The paper further shows that the optimal configuration is context-specific: a good config for one navigation stack loses satisfaction when applied to another, motivating continuous monitoring and relearning.","pith_inferences":["A promising extension is to apply ConfigBot's search to GPU, memory, and network budgets, where the same contention pathologies likely appear.","The paper's future-work sketch of RL-driven proactive tuning could be combined with ConfigBot's reactive loop: a predictive model could pre-empt context shifts before the constraint monitor fires.","Because adaptors work at the ROS pub/sub layer, the same mechanism could be applied to ROS2 and other middleware with similar subscription APIs, potentially widening the impact."],"forward_implications":["If ConfigBot's approach holds, robot developers can replace manual trial-and-error tuning with a specification-driven search that keeps core apps stable under resource pressure.","The combination of cgroup limits and data-flow throttling lets non-core apps degrade gracefully under contention, avoiding the thread-oversubscription collapse seen with pure CPU throttling.","Context-specific configs mean robots need continuous monitoring and relearning; a static 'one good config' does not transfer across workloads or environments.","The framework's success on three robot platforms with six stacks suggests OS- and ROS-level automated tuning generalizes beyond the evaluated hardware."],"supporting_citations":[{"why":"Supplies the Bayesian optimization method that ConfigBot adapts from cloud configuration search to robot configuration search.","marker":"[5]"},{"why":"Provides the context-conditioned configuration reuse design used for ConfigBot's configuration library.","marker":"[3]"},{"why":"The terrain-aware navigation stack used as the Terrain workload on Spot.","marker":"[11]"},{"why":"The object detection app used as the non-core workload in the Spot benchmarks.","marker":"[24]"},{"why":"The Phoenix navigation stack used to test ConfigBot on Jackal.","marker":"[26]"},{"why":"The Robofleet web dashboard used as a non-core telemetry workload in evaluations.","marker":"[12]"}],"fun_headline_variants":["ConfigBot meets robot specs with one CPU, not four","Auto-tuning robot configs: one CPU beats four","Context-aware config search saves robot CPUs","Bayesian tuning hits 99% robot specs with one CPU","Robot stability with fewer CPUs? ConfigBot does it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The developer-supplied performance metric and its target value faithfully capture what it means for the robot to behave correctly, so that meeting the numeric spec (e.g., 35 Hz command velocity) guarantees the robot is actually stable.","fun_headline_variants_meta":{"raw":{"variants":["ConfigBot meets robot specs with one CPU, not four","Auto-tuning robot configs: one CPU beats four","Context-aware config search saves robot CPUs","Bayesian tuning hits 99% robot specs with one CPU","Robot stability with fewer CPUs? ConfigBot does it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00049,"raw_usage":{"total_tokens":2367,"prompt_tokens":859,"completion_tokens":1508,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":1432}},"tokens_in":475,"tokens_out":1508,"duration_ms":11344,"temperature":1.0,"reasoning_tokens":1432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:10:00.297353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Basic navigation stack on Spot under ConfigBot's tuned ~1-CPU configuration through an obstacle course while logging both command-velocity frequency and physical stability (IMU-based stumble or collision events); if the robot stumbles during intervals where the 35 Hz constraint is continuously satisfied, the link between the optimized metric and robot stability is broken.","supporting_citations":[{"cited_title":"{CherryPick}: Adaptively unearthing the best cloud configu- rations for big data analytics,","cited_arxiv_id":null,"evidence_quote":"Supplies the Bayesian optimization method that ConfigBot adapts from cloud configuration search to robot configuration search."},{"cited_title":"Self-supervised terrain representation learning from unconstrained robot experience,","cited_arxiv_id":null,"evidence_quote":"The terrain-aware navigation stack used as the Terrain workload on Spot."},{"cited_title":"Phoenix stack army research lab,","cited_arxiv_id":null,"evidence_quote":"The Phoenix navigation stack used to test ConfigBot on Jackal."},{"cited_title":"Robofleet: Open source communication and management for fleets of autonomous robots,","cited_arxiv_id":null,"evidence_quote":"The Robofleet web dashboard used as a non-core telemetry workload in evaluations."}],"review_version":1}