{"id":"dc4a911c-d51f-4fea-8578-7f1a800cf714","arxiv_id":"2412.07786","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A multi-agent LLM workflow automatically defines small database views as a semantic layer, validated only by structural statistics on two datasets.","lead":"Three AI agents talk to each other to explore a database and produce small SQL views that act as a simpler version of the schema. The authors show early structural results on two commercial datasets, but not yet proof that the views make analysis easier.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Width reduction is built into the view-generation mechanism, so Section 3's median-width comparison cannot by itself validate 'refined schema'; no task-based or user-level evidence supports the interpretability claim.","rationale":"I read the paper in good faith. The contribution is genuinely interesting: a multi-agent LLM pipeline that produces executable views, with code released and the Verifier agent executing views in a real engine, which at least ensures syntactic/execution correctness. The case studies are honest about limitations (e.g., CMS coverage 18.87%). The problem is not internal inconsistency; it is that the central claim of a semantic layer that is 'easy-to-interpret' and 'immediately insightful' has no validation. The only quantitative evidence for refinement is column-width reduction, but width reduction is tautologically guaranteed by decomposing wide tables into views. The paper's own Appendix A.1 shows that almost all original relationships (99.5%) are discarded in one case, which could be fine only if the retained views carry semantic value, and that is exactly what is unmeasured. The reader's weakest assumption ('small view width equates to semantic interpretability') matches the load-bearing gap I find, so I agree with the reader. A user study or task-based accuracy comparison is the decisive check; without it, CONDITIONAL is the right verdict: the mechanism may work, but the usefulness claim is not yet supported. My stress test does not move the verdict, hence UNCHANGED.","tokens_in":8245,"tokens_out":4032,"duration_ms":38416,"concrete_test":"Run a between-subjects task on the Braze case study: recruit 20 SQL-proficient participants, randomly assign 10 to the original 61-table schema and 10 to the semantic layer (views plus entity/relationship mapping). Give both groups the same 5 analytical questions (e.g., 'find the campaign with the highest conversion rate among users who opened an email') and ask them to write SQL or identify the needed tables/columns. Score by exact result-set correctness against the demo database, recording time and self-rated confidence. If the semantic-layer group does not significantly outperform the original-schema group on correctness or time, the claim that narrow views provide immediate semantic insight is unsupported and the paper should be revised to a purely structural claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 3 is that generated views 'serve as a refined version of the original schema with tables effectively decomposed into smaller components,' and the evidence is the width distribution (median 3 vs 28). But the refinement mechanism in Section 2.1 defines views as projections/filters of original tables or query subexpressions, so a narrower output is a direct consequence of the method, not a discovered semantic property. A random decomposition of the 61 wide tables into 3-column fragments would produce the same median width without yielding 'easy-to-interpret' semantics. The paper also states (Section 3) that a new user 'can immediately gain insight' and that entities/relationships are mapped to views, but this rests on LLM-generated labels and self-assessment by the same multi-agent process; there is no human study, no query-accuracy benchmark, and no Text-to-SQL comparison. In Appendix A.1, the CMS case preserves only 0.5% of original column co-occurrences; the authors call this expected, but it highlights that the method discards most relational structure, and the only offered justification is the semantic claim that was never measured. The absence of any baseline or repeated runs compounds the issue: 1146 views from one run on one dataset, with no variance or comparison, cannot distinguish a meaningful semantic layer from an arbitrary projection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes building a semantic layer over complex databases as a set of small, reusable SQL views, discovered by a multi-agent LLM simulation. Three agents (Analyst, Critic, Verifier) iteratively define, refine, and validate views, with the aim of distilling the original schema into entities and relationships that are easier for users to understand. The authors report a proof-of-concept study on a public Braze demo database (61 tables, 1146 views, median view width 3 vs. median table width 28) and an additional case study on CMS data feeds (113 tables, 632 views), plus an entity-relationship mapping derived from the views. The central claim is that the generated views constitute a refined, semantically meaningful version of the original schema that gives users immediate insight into the database.","tokens_in":8499,"tokens_out":3374,"duration_ms":32739,"significance":"If validated, the multi-agent view-discovery framework would be a practical contribution to automated semantic-layer construction, addressing a real pain point in database exploration and text-to-SQL systems. The approach is concrete: views are executable SQL objects, the Verifier checks them by executing them in a database engine, and the code is publicly available. The paper also demonstrates that the method can scale to wide, realistic schemas. However, the current evidence is structural statistics only; the semantic-usefulness claim is asserted rather than measured. The lack of any baseline, user study, task-based accuracy metric, or repeated-run variance means the central value proposition—that narrower views are easier to interpret and that the ER model aids comprehension—remains unsupported. The contribution is therefore a promising system description rather than a validated method.","major_comments":[{"comment":"The median-width comparison (3 vs. 28) is presented as validating that the views are a refined version of the schema, but that conclusion does not follow. The view-generation mechanism in Section 2.1 defines views as projections, filters, or query subexpressions of the original tables, so producing narrower outputs is a direct consequence of the mechanism rather than a discovered semantic property. A random decomposition into 3-column fragments would achieve the same median width without any claim of interpretability. To support the refinement claim, the authors need a task-based or human evaluation (e.g., measuring whether users or text-to-SQL systems answer questions more accurately using the semantic layer) or at least a comparison against a non-semantic baseline such as random column groupings.","section":"Section 3, Fig. 3"},{"comment":"The claim that a new user 'can immediately gain insight' and that each entity/relationship is 'mapped to database views, validated for correctness and usefulness' relies on labels and validation performed by the same multi-agent framework that generated the views. The Verifier checks only that views execute successfully, not that they are semantically meaningful. There is no user study, no external benchmark, and no comparison of query results with and without the semantic layer. This self-referential validation is a circularity risk: the framework's own assessment of 'usefulness' cannot substitute for an independent measure of whether the views actually improve exploration or query formulation.","section":"Section 3, final paragraph"},{"comment":"In the CMS case study, only 0.5% of original column co-occurrences are preserved in the views, yet the authors call this 'expected' and use it to argue that the refinement discards only non-semantic relations. This is a load-bearing assumption: if the discarded relations include many that are semantically important, the 0.5% preservation rate could indicate that the decomposition is largely arbitrary. The paper provides no evidence that the preserved relations are more semantically meaningful than the discarded ones, nor any downstream task that would demonstrate the semantic loss is acceptable. A concrete test would be to check whether typical analytic queries over the original schema can still be answered (or answered more simply) using the views.","section":"Appendix A.1, Table 2"},{"comment":"The experimental evaluation consists of two datasets, each with a single run of the multi-agent simulation, and no baselines, no variance, and no error analysis. The paper acknowledges the results are preliminary, but the abstract and introduction make general claims about 'our approach' paving the way for LLM-powered database exploration. Since the central usefulness claim rests entirely on the reported structural statistics, at least one additional supporting experiment is needed: for example, repeated runs to show stability, a comparison against a non-agentic or random view-generation baseline, or a small user study measuring comprehension of the schema with and without the semantic layer. Without such evidence, the results cannot distinguish a meaningful semantic layer from an arbitrary set of projections.","section":"Section 3 and Appendix A.1"}],"minor_comments":[{"comment":"The SQL for query Q' uses the alias 's' in the WHERE and GROUP BY clauses ('o.staff_id = s.staff_id', 'GROUP BY s.staff_id') even though the table is aliased as 'i' in the FROM clause; the query would not execute as written. Please fix to use 'i.staff_id' consistently.","section":"Figure 1"},{"comment":"Several origin table names contain an erroneous space, e.g., 'USERS_BEHA VIORS_UPGRADEDAPP' and 'USERS_BEHA VIORS_UNINSTALL' in Table 1; these should be 'USERS_BEHAVIORS_...'.","section":"Table 1 and Appendix A.1"},{"comment":"The caption says 'We ignore the top 1% views in terms of width' but does not state whether this exclusion affects the reported median view width and relation counts; please clarify the exact filtering rule and its effect on the statistics.","section":"Figure 3 caption"},{"comment":"The phrase 'abstracting away too detailed information' is unclear; please rephrase to explain which information is abstracted away and why it is considered too detailed.","section":"Appendix A.1"},{"comment":"The sentence 'thus validating that the generated views serve as a refined version' overstates what a width distribution can show; consider replacing 'validating' with 'suggesting' or 'illustrating' to match the preliminary nature of the study.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The paper describes a plausible and appealing system, but the evaluation does not yet establish the central semantic-usefulness claim. I recommend major revision with a specific request for either (a) a task-based or user-level evaluation showing that the semantic layer improves comprehension or query accuracy, or (b) a substantive reframing of the paper as a system demonstration with claims limited to structural properties. The two-dataset, single-run evaluation is too thin to support the general claims in the abstract. The authors appear to be aware of the preliminary nature, but the manuscript's current framing goes beyond what the evidence supports. The code availability and executed-view verification are strengths that should be highlighted in the revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new combination—multi-agent LLM collaboration applied to automatic view discovery for semantic layers—and it's written clearly and honestly. But the evidence for the main claim is mostly structural and self-referential, so the paper is a proof-of-concept, not a validation.\n\nWhat's good: the refinement idea is concrete. The Analyst proposes queries, the Critic suggests decompositions, and the Verifier actually executes the resulting views in a database engine. That means the artifacts are real SQL objects that can be checked independently, and the authors ship code. The running example and the Appendix chat transcript make the mechanism transparent. The paper also calls itself preliminary, which is the right level of modesty.\n\nThe soft spot is load-bearing: Section 3 uses median view width (3 vs. 28) as evidence that the views are a 'refined version of the original schema.' But the refinement mechanism defines views as projections, filters, and grouped subexpressions of the base tables. Narrower output is a direct consequence of the method, not a discovered semantic property. A random partitioning of wide tables into 3-column fragments would give the same median width without being easy to interpret. The CMS appendix makes this worse: only 0.5% of original column co-occurrences survive. The paper calls that expected, but it also means most relational structure is discarded, and the only justification is the semantic claim that is never measured.\n\nThe usefulness claim is also self-referential. Entities and relationships are 'validated for correctness and usefulness via our multi-agent schema refinement framework'—the same framework that created the views. There is no user study, no task-based accuracy metric, no text-to-SQL comparison, no baseline, and no repeated runs. So the central claim that a new user can 'immediately gain insight' is unsupported.\n\nI agree with the reader's conditional verdict and with the stress-test: the width comparison cannot validate semantic refinement. But I don't think this is a fatal flaw in the idea—only in the current evidence. The mechanism is plausible, and the paper would be much stronger with an external evaluation: for example, comparing text-to-SQL accuracy with and without the semantic layer, or a small human study measuring how quickly analysts can answer questions using the derived views, plus a random-projection baseline.\n\nBottom line: this deserves a serious referee slot at a workshop because it introduces a useful idea and the implementation is real. For a full conference or journal, it would need that task-based evaluation. I'd read it, keep it on my radar, but not cite it as evidence yet. I'd encourage the authors to run the missing experiments—the approach may genuinely work, but the paper hasn't shown it.","headline":"A promising workshop paper on multi-agent LLM view discovery whose central interpretability claim rests on a statistic that is largely built into the mechanism.","tokens_in":8987,"tokens_out":2656,"would_cite":false,"duration_ms":28541,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a semantic layer made of small, reusable database views and shows that a multi-agent LLM simulation can generate such views, decomposing a 61-table schema into 1,146 narrow views.","keywords":["semantic layer","database views","schema refinement","multi-agent LLM","view discovery","entity-relationship model","data exploration","text-to-SQL"],"falsifier":"Run a controlled comparison in which human analysts or a text-to-SQL system answer a fixed set of questions against either the original schema or the generated semantic layer. If query correctness, completion time, or confidence does not improve with the views, the claim that the views form an easier-to-interpret refined schema would be falsified.","tokens_in":8066,"feed_emoji":"🗃️","tokens_out":8262,"duration_ms":68286,"temperature":0.7,"pith_summary":"Large enterprise databases are hard to explore: tables are wide, column names are cryptic, and understanding what the data means takes tedious work. This paper argues that a semantic layer built from small, reusable database views can stand between the schema and the user, distilling entities and relationships into manageable pieces. To build that layer without a human expert, it runs a multi-agent LLM simulation in which an Analyst, a Critic, and a Verifier iteratively decompose complex queries into views and validate them by executing them in the database. On a 61-table commercial database the process produced 1,146 views with median width 3, down from a median table width of 28, and on a second, extremely wide public dataset it produced 632 views. The paper is a preliminary demonstration: the claims are about feasibility and structural effects, not about measured gains in user performance.","feed_headline":"LLM agents carve wide tables into 1,146 narrow views","feed_subtitle":"A multi-agent simulation distills messy schemas into small reusable views for faster data exploration","key_machinery":"The engine is a schema refinement mechanism implemented as a multi-agent conversation. A view is a virtual table defined by a stored SQL query. The Analyst proposes analytics tasks, writes SQL, and defines intermediate views; the Critic reviews the views for refinement quality and suggests alternatives; and the Verifier executes the views in a database engine to confirm correctness. To keep the process tractable, sessions run on connected components of the primary-key/foreign-key schema graph, focus each session with a graph retrieval algorithm, and pass chat summaries between sessions so later work reuses existing views and avoids repeating earlier tasks. The output is the set of validated views that acts as the semantic layer, and a post-processing step clusters the views into an entity-relationship model.","core_discovery":"The central claim is that schema refinement can be automated: given only the schema, a data sample, and minimal seed instructions, a multi-agent LLM simulation discovers a set of views that serve as a refined version of the original schema, with tables effectively decomposed into smaller components. The authors demonstrate this on a commercial customer-engagement database with 61 tables and 1,770 columns, where the simulation produced 1,146 views with median width 3 compared with the original median table width of 28; the views cover 80.79% of the original columns, preserve 54.84% of the original pairwise column relationships, and add 7,229 new relationships through joins. They also report a second case study on a large public healthcare dataset where 632 views were generated from 113 tables, several of which had hundreds of columns. The authors further claim that grouping the views and extracting an entity-relationship model gives users immediate insight into the semantic content of the database, with each entity or relationship mapped to validated SQL views.","pith_inferences":["A testable extension the paper leaves open is whether narrowness itself drives interpretability; a user study comparing fact-finding and query-writing on the original schema versus the semantic layer would turn the structural claim into a behavioral one.","The second case study's low column coverage (18.87%) suggests that on extremely wide schemas the process deliberately discards most columns; checking whether real queries ever need the discarded columns would show whether low coverage is a weakness or a useful abstraction.","The discovered views could be compared with the output of standard normalization based on functional dependencies; where the LLM-driven decomposition agrees with syntactic normalization, the semantic value is confirmed, and where it diverges, the difference would reveal what external knowledge contributes.","Because the views are validated by execution inside the database, the same pipeline could be re-run when a schema evolves, providing a low-cost way to keep a semantic layer up to date."],"forward_implications":["Users can bootstrap understanding of an unfamiliar database by reading the entity-relationship model instead of exploring dozens of wide tables.","Analytics queries become shorter and more reusable because they can be rewritten against the discovered views using standard SQL.","The semantic layer surfaces hidden connections, adding 7,229 pairwise column relationships that no original table captured.","Because each session handles only connected schema components and keeps memory between sessions, the approach scales to schemas larger than a single LLM context window.","The mapping from entities and relationships to validated views gives a direct path from a conceptual question to executable SQL, lowering the barrier to querying without deep schema knowledge."],"supporting_citations":[{"why":"supplies the multi-agent conversation framework used to run the Analyst, Critic, and Verifier simulation.","marker":"[24]"},{"why":"powers the agents with the large language model used in the experiments.","marker":"[1]"},{"why":"provides the graph retrieval algorithm that focuses each session on relevant schema subgraphs.","marker":"[8]"},{"why":"documents schema complexity as a key cause of text-to-SQL failure that the semantic layer targets.","marker":"[5]"},{"why":"defines database views as stored queries, the building blocks of the proposed semantic layer.","marker":"[20]"}],"fun_headline_variants":["LLM agents shrink 61 tables into 1,146 narrow views","Multi-agent LLMs distill messy schemas into 1,146 views","LLM agents auto-refine schemas into 1,146 query-ready views","Agentic schema refinement: LLMs produce 1,146 views"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a view with fewer columns is more interpretable and useful, since the paper uses median view width as its main evidence of refinement and does not include a user study or task-based metric connecting narrow views to better comprehension or query accuracy.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents shrink 61 tables into 1,146 narrow views","Multi-agent LLMs distill messy schemas into 1,146 views","LLM agents auto-refine schemas into 1,146 query-ready views","Agentic schema refinement: LLMs produce 1,146 views"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001265,"raw_usage":{"total_tokens":5113,"prompt_tokens":815,"completion_tokens":4298,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":431,"completion_tokens_details":{"reasoning_tokens":4218}},"tokens_in":431,"tokens_out":4298,"duration_ms":26752,"temperature":1.0,"reasoning_tokens":4218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:45:21.688956+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled comparison in which human analysts or a text-to-SQL system answer a fixed set of questions against either the original schema or the generated semantic layer. If query correctness, completion time, or confidence does not improve with the views, the claim that the views form an easier-to-interpret refined schema would be falsified.","supporting_citations":[{"cited_title":"NL2SQL is a solved problem","cited_arxiv_id":null,"evidence_quote":"documents schema complexity as a key cause of text-to-SQL failure that the semantic layer targets."},{"cited_title":"Database Management Systems","cited_arxiv_id":null,"evidence_quote":"defines database views as stored queries, the building blocks of the proposed semantic layer."}],"review_version":1}