{"id":"1fcaa66f-54ed-443c-8de0-fcc229e533ec","arxiv_id":"2501.00881","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A practitioner-oriented survey proposing a 'Cognitive Skills' module as a standardized building block for vertical AI agents.","lead":"This paper is a guide to building vertical AI agents, describing how large language models combine memory, reasoning, tools, and a proposed 'cognitive skills' module to tackle industry-specific tasks. It is written for practitioners who want a map of agent architectures, from simple task-specific agents to multi-agent and human-in-the-loop systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Cognitive Skills module's central benefit claim is asserted, not demonstrated: Section 3.3.3 does not establish that fine-tuned LLMs with tool access are insufficient, and the module's boundary with the Tools module is never operationally defined.","rationale":"The reader's CONDITIONAL verdict is appropriate: the paper works as a survey and taxonomy, but the central conceptual contribution, the Cognitive Skills module, is under-evidenced. I found the same weakest assumption as the reader: the necessity of the module rests on the unstated assertion that fine-tuned LLMs with tools are insufficient for precision-critical domain tasks. I additionally flag an internal tension: Section 2.1.1 credits fine-tuning with delivering exceptional precision, while Section 3.3.3 dismisses fine-tuned models as lacking precision. The paper also does not define how a cognitive skill is invoked differently from a tool, so the proposed new building block is not crisply distinct from the existing Tools module. None of this makes the paper worthless; it makes the benefit claim a design opinion rather than a demonstrated result. No verdict change is warranted, but the conditional acceptance should explicitly require either empirical support for the cognitive-skills advantage or a repositioning of that contribution as an architectural suggestion.","tokens_in":10520,"tokens_out":3476,"duration_ms":31966,"concrete_test":"Run a controlled comparison on one of the paper's own examples, e.g., contract risk assessment: (A) a ReAct-style agent using a fine-tuned LLM plus RAG/API tools; (B) the same agent with a separately deployed Cognitive Skills risk classifier. Use a fixed test set of 200 contracts, measure precision/recall and false-positive rate; if (A) matches (B) within a pre-registered tolerance, the module's claimed necessity is not supported. Also specify the invocation interface for a Cognitive Skill; if it is identical to a tool call, the proposed boundary collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise for the Cognitive Skills module is in Section 3.3.3: 'Fine-tuned models often lack the precision and specialization required for complex, domain-specific tasks.' The paper gives no citation, benchmark, or ablation for this claim, and it sits in tension with Section 2.1.1, where vertical agents are said to achieve 'exceptional precision' precisely by using LLMs 'fine-tuned for specialized knowledge and workflows.' If general fine-tuning plus tool access (RAG, APIs, classifiers) already delivers the required precision, the 'new module' is redundant. The paper also never specifies an operational distinction between a Cognitive Skill and a Tool: the examples listed (OCR, image classifiers, guardrail classifiers, risk-assessment models) are all callable models, and the Tools section already includes knowledge retrieval, APIs, and legacy-system interfaces. Since the central claim is that this module 'fills the gap' and is a new core building block, the burden is to show a task where a fine-tuned LLM with tool access fails while adding a purpose-built model succeeds, and to define how the Reasoning Engine invokes the skill differently from a tool. Neither is provided. The taxonomy may still be useful as a design vocabulary, but the benefit claim is currently an unsupported design opinion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This position paper proposes a standardization vocabulary for Vertical AI agents by identifying four core building blocks of an LLM agent (Memory, Reasoning Engine, Cognitive Skills, Tools) and introduces the Cognitive Skills module as a new core component: a hub of purpose-built, domain-specific inference models that are claimed to bridge general LLM reasoning and specialized tasks. The paper then offers a taxonomy of agentic systems (task-specific, multi-agent, human-augmented), describes several design patterns including a RAG Agent Router and a RAG Orchestrated Multi-Agent System, and surveys industry and academic efforts. The central technical claim is that the Cognitive Skills module is necessary and novel because fine-tuned LLMs and tools alone are insufficient for precision-critical, domain-specific tasks.","tokens_in":10711,"tokens_out":1931,"duration_ms":17697,"significance":"If the proposed standardization and the Cognitive Skills module were rigorously supported, this could be a useful design vocabulary for practitioners building vertical agents, especially in regulated industries. The paper's strengths are its clear rhetorical organization, its concrete examples (e.g., guardrail classifiers, OCR, risk-assessment models) that illustrate what the intended module would contain, and its effort to situate the proposal within current industry frameworks (AutoGen, Semantic Kernel, Vertex AI Agent Builder). However, the manuscript contains no empirical evaluation, no formal specification, and no operationalization of the key boundary between Cognitive Skills and Tools; the significance is therefore that of an unsupported design opinion rather than a validated architecture.","major_comments":[{"comment":"The load-bearing premise that 'Fine-tuned models often lack the precision and specialization required for complex, domain-specific tasks' is asserted without citation, benchmark, or ablation, and it directly contradicts §2.1.1, which states that vertical agents achieve 'exceptional precision' precisely by using LLMs 'fine-tuned for specialized knowledge and workflows.' If a fine-tuned LLM with tool access already delivers the required precision, the Cognitive Skills module is redundant; if it does not, the paper must provide evidence and concrete failure cases. Without such support, the central contribution is an unverified claim.","section":"§3.3.3 vs. §2.1.1"},{"comment":"The manuscript never defines an operational distinction between a Cognitive Skill and a Tool. The examples of Cognitive Skills (OCR, image classifiers, guardrail classifiers, risk-assessment models) are all callable models, while the Tools module already includes knowledge retrieval, APIs, and legacy-system interfaces. The paper should specify how the Reasoning Engine invokes a Cognitive Skill differently from a Tool (e.g., different control flow, different discovery mechanism, different training/serving pipeline) and provide at least one task where a fine-tuned LLM with tool access fails but adding a purpose-built model succeeds. Without this, the claim that Cognitive Skills 'fills the gap' is not falsifiable.","section":"§3.3.3 vs. §3.3.4"},{"comment":"The taxonomy of agentic systems into task-specific agents, multi-agent systems, and human-augmented agents is presented as exhaustive and primary, but no classification criterion is given and no argument is provided that these three categories cover the design space. The categories are not mutually exclusive: a task-specific agent can be a component of a multi-agent system, and human-in-the-loop can augment either a task-specific or a multi-agent system. The paper should state the intended dimension of classification (e.g., number of agents, degree of human involvement) and address overlaps, or explicitly present the taxonomy as one of several possible views rather than as an exhaustive typology.","section":"§4.3"},{"comment":"The paper promises 'a level of standardization' and describes core modules, but it never specifies the interfaces, data flows, or control flows between Memory, Reasoning Engine, Cognitive Skills, and Tools. Consequently, the proposed Cognitive Skills module cannot be implemented, tested, or compared against alternative designs from the manuscript alone. At minimum, the paper should provide a formal-ish definition of the module boundary, the inputs/outputs of a Cognitive Skill, and a protocol for the Reasoning Engine to select among skills; without this, the standardization claim is not actionable.","section":"§3.3 and §4.2"}],"minor_comments":[{"comment":"The text contains typographical inconsistencies: 'A WS' should be 'AWS' (also in §4.2.1), and 'Vertical Ai agent solutions' should be 'Vertical AI agent solutions.'","section":"§2.1.3"},{"comment":"The phrase 'as aversatile, purpose-driven system' is a typo; it should read 'as a versatile, purpose-driven system.'","section":"§3.3.3"},{"comment":"Reference [9] (OpenAI DevDay) and [10] (AWS SageMaker) lack URLs and access dates, and reference [12] (MedAide) lacks a venue or arXiv identifier; please complete the bibliographic entries.","section":"References"},{"comment":"Figures 1, 2, and 4 are referenced in the text but not displayed in the manuscript; please include the figures or clearly indicate that they are omitted.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a competent, readable practitioner survey whose only new idea is naming a fourth building block, the Cognitive Skills module. The rest is a sensible review of familiar patterns — ReAct, RAG, multi-agent orchestration, HITL — with concrete examples that are genuinely useful for someone building vertical agents. The prose is clean and the taxonomy (task-specific, multi-agent, human-augmented; memory, reasoning, cognitive skills, tools) is plausible as a shared vocabulary. The RAG router and orchestrated multi-agent walk-throughs are well done and would help a team starting out.\n\nThe soft spots are real but not fatal if the paper is positioned correctly. The central claim — that Cognitive Skills is a new core building block that 'fills the gap' and 'ensures' precision and reliability — is asserted, not demonstrated. Section 3.3.3 says fine-tuned models 'often lack the precision and specialization required for complex, domain-specific tasks,' but Section 2.1.1 credits vertical agents with 'exceptional precision' precisely because their reasoning engines are fine-tuned. That is a direct tension, and the paper never resolves it. There is no citation, benchmark, or ablation. The boundary with the Tools module is also fuzzy: OCR, classifiers, and guardrails are callable models, just like APIs and retrieval systems. If a fine-tuned LLM with tool access already does the job, the new module is a relabeling, not a contribution. The conclusion also overreaches with 'paradigm shift' and 'revolutionize industries.'\n\nThe stress-test note is right on all of this. I'd add that the paper's own internal logic is coherent enough — the module makes sense as a design opinion, not as an established finding. The authors should either provide a concrete example where a fine-tuned LLM plus tools fails and the skill hub succeeds, or explicitly frame the module as a recommended pattern rather than a validated requirement.\n\nWho is this for? Practitioners and teams looking for a common vocabulary to talk about vertical agent architectures. It is not for researchers expecting empirical results. I would send it to a serious referee if the venue accepts position or survey papers, because the taxonomy is useful and the Cognitive Skills idea is worth a second opinion — but I would expect the referee to demand a major reframing and honest disclaimers. If the venue demands original results, desk reject with an invitation to resubmit as a design guide.","headline":"A clearly written practitioner taxonomy with one genuinely new named module, but the module's core benefit is asserted, not demonstrated; useful as a design guide, thin as a research contribution.","tokens_in":11257,"tokens_out":1494,"would_cite":false,"duration_ms":15782,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A proposed 'Cognitive Skills' module aims to make vertical AI agents precise and reliable by bridging general LLM reasoning and domain-specific inference.","keywords":["vertical AI agents","LLM agents","cognitive skills","agentic systems","multi-agent systems","human-in-the-loop","retrieval-augmented generation","design patterns"],"falsifier":"A concrete test would be to take a vertical domain task, say contract risk assessment or medical image screening, and compare an agent built from a fine-tuned LLM plus tools against the same agent augmented with a Cognitive Skills module containing a purpose-built model for that task, measuring task-specific accuracy on a fixed benchmark. If the fine-tuned-plus-tools agent already performs as well, the module's premise fails.","tokens_in":10295,"feed_emoji":"🤖","tokens_out":4619,"duration_ms":37986,"temperature":0.7,"pith_summary":"This paper argues that vertical AI agents—agents built for a specific industry—should include a fourth core module, which it calls 'Cognitive Skills': a hub of purpose-built inference models tailored to domain tasks. The claim is that general-purpose or fine-tuned LLMs alone, even with tools, lack the precision for specialized tasks such as risk assessment, compliance monitoring, OCR, and guardrails. By adding this module, agents become both adaptable and precise, bridging the LLM's general reasoning with domain-specific inference. The paper also offers a taxonomy of agentic systems—task-specific, multi-agent, and human-augmented—with architectural patterns for each. A sympathetic reader would take this as a proposal for standardizing vertical AI agent design, not an experimentally proven result.","feed_headline":"A new Cognitive Skills module sharpens vertical AI agents","feed_subtitle":"The paper argues general LLMs plus tools miss domain precision; a purpose-built model hub fixes it.","key_machinery":"The central object is the Cognitive Skills module: a collection of purpose-built inference models that sit alongside the LLM's reasoning engine and external tools. It is described as a 'bridge' between general reasoning and domain-specific inference. In the paper's architecture, the Reasoning Engine (the LLM) orchestrates inputs from Memory, Cognitive Skills, and Tools; the Cognitive Skills module supplies high-precision outputs for specialized tasks, while Tools handle knowledge retrieval and environment interaction. The paper also uses a three-category taxonomy and specific patterns (RAG Agent Router, orchestrated multi-agent, HITL) as design patterns to organize the space.","core_discovery":"The central discovery is a design claim: the authors propose that the core building blocks of an LLM agent should be expanded from memory, reasoning, and tools to also include a Cognitive Skills module. This module acts as a model hub containing purpose-built models specialized for tasks that general LLMs and even fine-tuned LLMs struggle with, such as credit underwriting risk assessment, vulnerability detection, compliance monitoring, OCR, image classification, speech processing, and responsible-AI guardrails. The paper states that fine-tuned models often lack the precision and specialization required for complex, domain-specific tasks, and that the Cognitive Skills module bridges the gap between the LLM's general reasoning abilities and domain-specific inference tasks. It further advances a three-category taxonomy of agentic systems (task-specific, multi-agent, human-augmented) and illustrates implementation patterns including a RAG Agent Router, a RAG Orchestrated Multi-Agent System, and a Human-in-the-Loop pattern.","pith_inferences":["The paper does not compare the Cognitive Skills module against a strong baseline; a natural extension is to test whether the module outperforms fine-tuning alone on a suite of vertical benchmarks.","The distinction between Cognitive Skills and Tools is conceptual rather than operational—both are external model calls—so a stricter boundary (e.g., inference-only vs. retrieval/action) would make the proposal easier to evaluate.","The taxonomy excludes self-improving or learning agents; a future revision might add a fourth category to account for agents that update their own skills from feedback."],"forward_implications":["If the Cognitive Skills module is adopted, vertical AI agents will standardize on four core modules—Memory, Reasoning Engine, Cognitive Skills, Tools—making architectures more comparable across vendors.","Specialized inference models like risk classifiers and guardrails become first-class components rather than ad-hoc add-ons, which could improve reliability and auditability of agent outputs.","The three-category taxonomy provides a common language for describing agentic systems, which could accelerate adoption and reduce design ambiguity.","The RAG Agent Router and orchestrated multi-agent patterns offer concrete starting points for organizations building domain-specific retrieval systems."],"supporting_citations":[{"why":"Supplies the ReAct pattern of interleaving reasoning and action used to define task-specific agents.","marker":"[13]"},{"why":"Provides the retrieval-augmented generation survey that grounds the RAG patterns in the paper's tool and router designs.","marker":"[6]"},{"why":"Describes a generalist multi-agent system that the paper contrasts with vertical specialization, supporting the argument for domain-specific design.","marker":"[5]"},{"why":"Presents a cost-efficient router/search agent pattern that motivates the RAG Agent Router example.","marker":"[11]"},{"why":"Offers a collaborative medical multi-agent system used as an example of the multi-agent category.","marker":"[12]"},{"why":"Demonstrates a hierarchical multi-agent framework with knowledge graphs for medical diagnosis, another example of the multi-agent category.","marker":"[14]"}],"fun_headline_variants":["Cognitive Skills module: missing link for vertical AI agents","Vertical AI agents gain a Cognitive Skills model hub","New Cognitive Skills module sharpens domain-specific AI agents","Why LLM agents need a Cognitive Skills module for vertical AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the claim, made without cited evidence, that fine-tuned LLMs even with tools cannot deliver the precision needed for complex domain-specific tasks; if a fine-tuned model with tool access already achieves that precision, the Cognitive Skills module is redundant.","fun_headline_variants_meta":{"raw":{"variants":["Cognitive Skills module: missing link for vertical AI agents","Vertical AI agents gain a Cognitive Skills model hub","New Cognitive Skills module sharpens domain-specific AI agents","Why LLM agents need a Cognitive Skills module for vertical AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000434,"raw_usage":{"total_tokens":2179,"prompt_tokens":881,"completion_tokens":1298,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":1234}},"tokens_in":497,"tokens_out":1298,"duration_ms":9192,"temperature":1.0,"reasoning_tokens":1234,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:39:45.105112+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test would be to take a vertical domain task, say contract risk assessment or medical image screening, and compare an agent built from a fine-tuned LLM plus tools against the same agent augmented with a Cognitive Skills module containing a purpose-built model for that task, measuring task-specific accuracy on a fixed benchmark. If the fine-tuned-plus-tools agent already performs as well, the module's premise fails.","supporting_citations":[{"cited_title":"Medaide: Towards an omni medical aide via specialized llm-based multi-agent col- laboration, 2024","cited_arxiv_id":null,"evidence_quote":"Offers a collaborative medical multi-agent system used as an example of the multi-agent category."},{"cited_title":"Kg4diagnosis: A hier- archical multi-agent llm framework with knowledge graph enhancement for medical diagnosis, 2024","cited_arxiv_id":null,"evidence_quote":"Demonstrates a hierarchical multi-agent framework with knowledge graphs for medical diagnosis, another example of the multi-agent category."}],"review_version":1}