{"id":"c674c939-bafd-4df6-af99-d5cdd018aab9","arxiv_id":"2412.13437","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"This survey proposes a layered framework (execution, resource, model, agent, application) for deploying foundation-model-powered agent services across edge-cloud environments, and reviews optimization techniques at each layer.","lead":"This paper is a survey of techniques for deploying AI agents powered by foundation models across edge and cloud devices, and it organizes these techniques into a layered framework. A generalist reader might use it as a map of the current engineering landscape for running large models and agents efficiently.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The survey's sections describe generic FM inference and separate agent frameworks, so the claimed unified 'FM-powered agent services deployment' scope is not actually supported by the cited literature.","rationale":"The reader correctly identified the implicit organizing assumption that the surveyed technologies compose into a coherent deployment stack. In good faith, the survey has real strengths: it is clearly structured, covers a broad range of system and model optimization techniques, and offers a useful vocabulary for discussing edge-cloud FM serving. However, the central novelty claim depends on the existence of a literature specifically about deploying FM-powered agent services. The manuscript does not demonstrate that the works in Sections II-IV are agent-specific, and Section V does not show how its agent-level techniques are deployed with the lower-layer optimizations. The paper's own lesson statements acknowledge that agent-layer elasticity and heterogeneous edge-cloud FM serving are open problems, which supports the interpretation that the unified framework is aspirational rather than descriptive. This is not a fatal flaw but it is a scope mismatch in the central claim. The survey could be accepted if the authors either moderate the claim to 'a survey of techniques relevant to deploying FM-powered agents, with a proposed unified framework' or provide evidence that the reviewed systems actually address agent-service deployment. A conditional acceptance with that requested revision is therefore the most honest verdict. No ad hominem is intended; the issue is about the relationship between the paper's title, its central claim, and the content of the cited literature.","tokens_in":43590,"tokens_out":5479,"duration_ms":57255,"concrete_test":"Audit the 20 most prominent works cited in Sections II-IV (for example, vLLM/PagedAttention, PowerInfer, EdgeMoE, Splitwise, SARATHI) and the 20 cited in Section V. For each, determine from the abstract or system description whether it (a) targets an agent-service workload, (b) runs across heterogeneous edge-cloud devices, or (c) both. If no paper satisfies both (a) and (b), then the unifying 'FM-powered agent services deployment' literature does not yet exist, and the claim should be narrowed to a proposed taxonomy rather than a comprehensive survey of an established field.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the paper is the first comprehensive survey of deploying real-time FM-powered agent services in heterogeneous devices. The load-bearing premise is that such a coherent literature exists and that the reviewed papers are about agent-service deployment. The manuscript does not establish this: Sections II-IV cover hardware accelerators, resource allocation, parallelism, compression, and token reduction for generic FM/LLM inference; none of the cited systems in these sections is evaluated on or designed for agent workloads such as tool calling, planning loops, multi-agent coordination, or memory retrieval. Section V reviews agent frameworks but from an algorithmic and application perspective, with almost no deployment or system contribution. The paper itself concedes in Section VII-A3 that there is a 'significant gap in elasticity at the agent layer' and in Section VII-A1 that heterogeneous edge-cloud FM serving is 'under-explored.' Thus the framework in Figure 1 is a proposal for future integration rather than a scheme that organizes an existing body of work on agent-service deployment. The firstness claim is therefore at risk: the survey may be a survey of two adjacent literatures, FM inference systems and AI agents, stitched together under a new label, not a comprehensive survey of a pre-existing research area.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This survey proposes a five-layer unified framework for deploying foundation model (FM) powered agent services on heterogeneous edge-cloud devices. The layers are: execution optimization (hardware-specific computation, memory and communication optimizations), resource allocation and parallelism (cloud/edge resource scaling and various parallelism strategies), model-layer optimizations (model compression, quantization, distillation, token reduction), agent-layer components (multi-agent frameworks, planning, memory, tool use), and application-layer concerns (batching and representative applications). The paper reviews a large number of recent works in each area, provides several summary tables, and concludes with lessons learned and future research directions. Its central claim is that it is the first comprehensive survey of the deployment of real-time FM-powered agent services in heterogeneous devices.","tokens_in":1557,"tokens_out":2153,"duration_ms":63466,"significance":"The paper has clear value as a broad compilation of recent work on efficient FM inference and on AI-agent architectures. The taxonomy is readable and the tables (e.g., Table II on integrated frameworks, Table XIII on batching) provide useful entry points for readers. The paper is also honest in Section VII about current gaps, which is a strength. However, the significance of the paper as claimed depends entirely on whether the surveyed literature actually constitutes a coherent area of 'FM-powered agent service deployment.' The evidence in the manuscript does not support this: the systems in Sections II-IV are generic FM/LLM inference systems, and the agent work in Section V is algorithmic rather than deployment-oriented. The proposed framework is a plausible future integration agenda, but the claim of a first comprehensive survey of an existing research area is overstated. If the scope were reframed accordingly, the survey would still be a useful reference.","major_comments":[{"comment":"The paper's central claim that it is 'the first comprehensive survey to review and discuss the deployment of real-time FM-powered agent services in heterogeneous devices' (Introduction) is not supported by the material surveyed. None of the systems cited in Sections II-IV (e.g., FlashAttention, vLLM, PowerInfer, SpotServe, the resource-allocation works in Table III) is designed for or evaluated on agent workloads such as tool calling, planning loops, multi-agent coordination, or memory retrieval. Conversely, the agent frameworks in Section V (AgentVerse, Toolformer, DEPS, etc.) are described from an algorithmic/application perspective with no system-level deployment contributions. The paper itself concedes in Section VII-A3 that there is a 'significant gap in elasticity at the agent layer' and in Section VII-A1 that heterogeneous edge-cloud FM serving is 'under-explored.' Thus, on the evidence provided, the framework in Figure 1 is a proposal for future integration rather than a taxonomy of an existing body of work on agent-service deployment. This missing evidence is load-bearing because the firstness claim is the paper's principal contribution.","section":"I (Introduction, firstness claim) and Sections II-V"},{"comment":"The connection between the surveyed infrastructure and agent services is never established. For example, Table II lists llama.cpp, MLC-LLM, FastChat, and similar frameworks, but the discussion does not explain how these frameworks support agent-specific requirements such as maintaining multi-turn tool-call state, sharing KV cache across planning iterations, or dynamically deciding when to offload subtasks to different devices. Likewise, the resource-allocation methods in Table III target generic DNN/LLM inference and do not model agent-specific request graphs, inter-agent communication, or memory-retrieval latency. The survey would need at least one worked example or a dedicated analysis showing how the layers compose for an agent service; without this, the unified framework is only a juxtaposition of two adjacent literatures.","section":"II-D and III"},{"comment":"Section VII-B3 lists 'Specific serving system for agents' as a future direction, acknowledging that current serving systems are designed for FM inference rather than for agent services. This is an honest statement, but it directly contradicts the introductory claim that the paper surveys the deployment of FM-powered agent services. The conclusion in Section VII-C repeats that the framework 'showcases the latest advancements' in this area, which is not supported by the content. The authors should either reframe the paper as a survey of building blocks plus a research agenda, or substantially expand the survey to include systems (if any exist) that actually address agent-service deployment end to end.","section":"VII-B3 and VII-C"}],"minor_comments":[{"comment":"There are many typographical errors and inconsistencies, including stray letters in the author affiliations (e.g., 'Y . Fan', 'V'), inconsistent use of backslashes in Table V, and a duplicated 'Section V' label in Figure 2 (the application layer should be Section VI).","section":"Throughout"},{"comment":"The statistic on ChatGPT users is cited to a non-academic blog-style source (Nerdynav). A more authoritative source, such as a company report or a peer-reviewed citation, would be preferable for a survey.","section":"I"},{"comment":"The sentence beginning 'This rethinking method helps...' is grammatically incomplete, and the reference to Hatalis is cited without a first author name or paper title. Please ensure all citations are complete and the prose is polished.","section":"V-C"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is valid and lands. The paper is essentially a survey of two separate literatures, FM/LLM inference systems and AI-agent algorithms, joined by an aspirational framework. The firstness claim needs to be either dropped or carefully scoped, and the paper should be repositioned as a survey of building blocks with a research agenda. This is fixable within the manuscript's scope, so I do not recommend rejection. I would also encourage the editor to have the authors verify that no other recent survey already covers agent serving systems specifically, as that would affect the novelty framing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful survey of two adjacent literatures—LLM inference systems and AI agents—but the packaging as a survey of \"FM-powered agent services deployment\" overstates how much the cited work is actually about agent workloads. The paper itself admits the integration is under-explored, so the five-layer framework in Figure 1 is a proposal for future integration, not a map of an existing field.\n\nWhat the paper does well: the coverage is broad and mostly accurate. The tables on resource allocation, parallelism, quantization, pruning, token reduction, and batching are genuinely useful for newcomers. I checked a sample of the summaries against the cited papers; they are consistent. The lessons-learned section is honest—it flags that heterogeneous edge-cloud FM serving is under-explored and that agent-layer elasticity is missing. That candor is a real strength.\n\nMain soft spot: the central claim. Sections II–IV review generic FM/LLM inference techniques—FlashAttention, vLLM, quantization, token pruning—none of which is evaluated on agent workloads like tool calling, planning loops, or multi-agent coordination. Section V reviews agent frameworks from an algorithmic perspective with almost nothing on serving systems. The few exceptions (Parrot, SGLang, INFERCEPT) don't carry the weight of the framing, so the stress-test note is right: this is really a survey of \"LLM serving + agents\" as two related topics, not a survey of a pre-existing \"agent-service deployment\" literature. The paper would be more accurate if it said it identifies a gap and surveys the building blocks. Also, there is no systematic search methodology or inclusion criteria, which is a limitation for a survey though not fatal. Some sections are thin—the agent-memory part is more about agent design than deployment.\n\nWho this is for: newcomers who want a bird's-eye view, and researchers looking for references on either LLM inference optimization or agent frameworks. It is not the final word on agent-service deployment, and to its credit the paper mostly says so.\n\nRecommendation: send it to peer review with a request to revise the framing. It deserves referee time because it is comprehensive and honest, and a revision that either softens the firstness claim or clearly separates \"existing LLM serving systems\" from \"agent frameworks\" before positioning integration as future work would make it a solid contribution. Accept with revisions, not a desk reject.","headline":"A useful survey of LLM serving and agent frameworks whose framing as a survey of 'agent-service deployment' overstates how much the cited work is actually about agents; the paper's own lessons section concedes the integration is open.","tokens_in":44336,"tokens_out":2596,"would_cite":true,"duration_ms":25590,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey argues that delivering real-time foundation-model agent services at scale requires a unified deployment stack linking hardware execution, resource management, model compression, agent components, and applications.","keywords":["foundation models","AI agents","edge-cloud computing","model serving systems","model compression","token reduction","resource allocation","parallelism"],"falsifier":"A concrete check would be to search the literature for a prior survey that already covers real-time FM-powered agent deployment across heterogeneous edge-cloud devices under a unified framework; if one exists, the paper's first-comprehensive claim fails. A second check would be an end-to-end experiment combining representative techniques from all five layers (say token reduction, pipeline parallelism, and agent tool calling) to see whether their benefits add up or interfere; the paper reports no such experiment.","tokens_in":43389,"feed_emoji":"🤖","tokens_out":5137,"duration_ms":46063,"temperature":0.7,"pith_summary":"The paper sets out to establish that deploying foundation-model (FM) powered agent services in heterogeneous edge-cloud environments is one problem, not a pile of separate ones, and that a unified five-layer framework is the right way to organize both research and practice. The layers run from low-level execution optimization (computation, memory, communication), through resource allocation and parallelism, to model compression and token reduction, then to agent components and applications. The paper reviews existing work inside each layer and argues that jointly optimizing computational and communication resources across these layers is what makes real-time, high-QoS agent services achievable. A sympathetic reader would take the contribution to be a coherent map of the field that identifies where techniques fit and which gaps matter most.","feed_headline":"AI agent serving needs five coordinated layers","feed_subtitle":"New survey ties execution, resource, model, agent, and application optimizations into one framework for real-time edge-cloud AI services.","key_machinery":"The central object is the five-layer framework in the paper's Figure 1: an Execution layer, a Resource layer, a Model layer, an Agent layer, and an Application layer. It functions as both a taxonomy and a compositional claim: low-level inference optimizations, resource-allocation and parallelism strategies, model compression and token reduction, agent capabilities, and batching are treated as mutually dependent design choices within one serving stack. The framework carries the argument by showing where each surveyed technique sits and by exposing the missing elasticity at the agent layer as the binding constraint on real-time agent services.","core_discovery":"On its own terms, the paper's central claim is that it provides the first comprehensive survey of real-time FM-powered agent service deployment across heterogeneous devices, and that this deployment is best understood through a unified framework of five stacked layers: execution, resource, model, agent, and application. Each lower layer supplies capabilities to the one above it: execution-layer optimizations make inference feasible on diverse hardware; resource-layer parallelism and scaling make the system elastic; model-layer compression and token reduction make large models lightweight enough for edge-cloud use; agent-layer components (multi-agent frameworks, planning, memory, tool use) turn the model into a service; application-layer batching and applications deliver the user-facing QoS. The paper's discovery is organizational rather than empirical: it claims these bodies of work belong to a single design space and that their integration, not any single technique, is the open research agenda.","pith_inferences":["Editorial inference: the layered framework implies a concrete design recipe — start from agent-level QoS requirements and derive lower-layer optimization targets — which the paper describes but does not itself validate end-to-end.","Editorial inference: the identified agent-layer elasticity gap could be addressed by a scheduler that dynamically selects planning depth, tool-use rate, and collaboration topology under latency constraints; testing such a scheduler would be a natural next step beyond the survey.","Editorial inference: cross-layer interactions may create non-compositional effects not quantified in the survey; for example, token reduction changes KV-cache size and attention patterns, which in turn shifts the optimal parallelism and batching strategy.","Editorial inference: if the framework is accepted, a useful benchmark would be an open testbed that measures the same agent workload across different layer configurations, making the survey's taxonomy directly actionable."],"forward_implications":["If the framework is right, a serving system for FM agents should be designed with cross-layer budgets: a latency or accuracy target at the application layer should be traceable down to choices in execution, resource, and model layers.","The survey's own lesson about agent-layer elasticity implies that future serving systems will need adaptive agents that decide when to call APIs, retrieve knowledge, or collaborate with other agents based on current load and task complexity.","Edge-cloud deployment of large FMs becomes viable only when parallelism, model compression, and communication optimization are co-designed, since no single hardware class can host the full model.","Multi-modal and mixture-of-experts models will require new serving-system mechanisms, because their activated modules and resource demands vary with the input.","Batching and scheduling must become heterogeneity-aware, grouping requests by length, service characteristics, and per-request adapters rather than assuming a uniform model."],"supporting_citations":[{"why":"Defines foundation models and their opportunities and risks, setting the object the whole survey is about.","marker":"[1]"},{"why":"Surveys LLM-based agents and supplies the agent components (planning, memory, tool use) that the framework's agent layer organizes.","marker":"[2]"},{"why":"Reviews hardware accelerators for large language models, providing the execution-layer material on FPGAs, ASICs, and other devices.","marker":"[4]"},{"why":"Summarizes scheduling techniques in computing and network convergence, grounding the resource-allocation and communication parts.","marker":"[5]"},{"why":"Surveys efficient generative LLM serving from algorithms to systems, the system-layer baseline the paper extends toward agent services.","marker":"[6]"},{"why":"Reviews edge-cloud deployment for AIGC, the closest prior survey the paper distinguishes its unified framework from.","marker":"[7]"}],"fun_headline_variants":["Five layers unify AI agent deployment","AI agent services demand five-layer coordination","Survey maps five layers for agent service deployment","One framework for real-time AI agent services"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the techniques surveyed at different layers can meaningfully be integrated into one coherent deployment stack for agent services, and that the paper's selection of topics is representative enough to support its conclusions about open problems; the paper does not implement or demonstrate that the layers compose in practice.","fun_headline_variants_meta":{"raw":{"variants":["Five layers unify AI agent deployment","AI agent services demand five-layer coordination","Survey maps five layers for agent service deployment","One framework for real-time AI agent services"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1372,"prompt_tokens":895,"completion_tokens":477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":424}},"tokens_in":511,"tokens_out":477,"duration_ms":5221,"temperature":1.0,"reasoning_tokens":424,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:06:05.499464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to search the literature for a prior survey that already covers real-time FM-powered agent deployment across heterogeneous edge-cloud devices under a unified framework; if one exists, the paper's first-comprehensive claim fails. A second check would be an end-to-end experiment combining representative techniques from all five layers (say token reduction, pipeline parallelism, and agent tool calling) to see whether their benefits add up or interfere; the paper reports no such experiment.","supporting_citations":[],"review_version":1}