{"id":"01373d54-7b23-4411-898b-48c5d89b8612","arxiv_id":"2507.21964","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A zero-shot activity recognition system that classifies smart-home sensor windows by comparing sentence embeddings of data summaries with embeddings of activity descriptions, without prompting an LLM.","lead":"This paper describes a zero-shot human activity recognition method for smart homes that replaces LLM prompting with sentence-embedding similarity: sensor data is summarized as text, activity labels are described as text, and the closest description label is predicted. It might matter because it offers a privacy-preserving, offline alternative to LLM-based HAR with comparable accuracy on six datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's central claim of 'comparable performance to three LLM-based SOTA baselines' is unsupported: only two baselines are shown, on at most two datasets, with no numerical results or statistics.","rationale":"I read the paper as a case study arguing that a zero-shot HAR system can avoid LLM prompting while retaining comparable accuracy to LLM-based methods. The method itself is coherent: it converts sensor windows into textual summaries, converts activity labels into textual descriptions, and classifies by cosine similarity in a pre-trained sentence-embedding space. The ablation study in Table 1 supports the need for both modules and the choice of encoder, and the few-shot extension is a reasonable nearest-example mechanism. The most load-bearing assumption for the paper's stated impact, however, is not the transferability of the templates (which the authors partly acknowledge as a limitation) but the claim of 'comparable performance' to SOTA baselines. That claim is presently unfalsifiable because Figure 2 gives no numbers, no error bars, and no statistical comparison, and because a third promised baseline never appears. The reader's verdict already flagged the baseline discrepancy and lack of code/data, so my concern does not move the verdict; it sharpens the condition. I partially agree with the reader's weakest_assumption: template transferability is a genuine risk on new homes, and the Cairo/Kyoto7 results illustrate it, but the missing quantitative SOTA comparison is the first gate that must be passed before transferability can even be evaluated. A conditional acceptance requiring the numerical comparison, code/data release, and a statement on per-dataset design choices remains the right outcome.","tokens_in":8872,"tokens_out":7166,"duration_ms":83632,"concrete_test":"Obtain from the authors, or reconstruct by running the released code of [5] and [10] on the same test splits, the exact per-dataset accuracy and weighted/macro F1 for every claimed baseline, including the missing third baseline mentioned in Section 1, and compute pairwise differences with bootstrap confidence intervals. If the proposed method is not within five absolute points of each baseline on the datasets where that baseline was originally evaluated, or if the third baseline cannot be identified, the 'comparable performance' claim fails. This single check would settle whether the central assertion is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in Section 1, is that the proposed embedding-only method achieves comparable performance to three LLM-based SOTA baselines. The only evidence for this is Figure 2 in Section 4.2, which qualitatively compares against two methods, LLM + Iterative Evolution [10] and ADL-LLM [5], and appears to do so on one dataset each (ARAS and MARBLE, respectively). No numerical accuracy or F1 values for the baselines are reported anywhere, Table 1 contains only the proposed method and ablations, and there are no error bars or significance tests. Moreover, the abstract and introduction promise three baselines, but only two appear in the evaluation. As a result, the paper's impact claim—that offline, privacy-preserving HAR can match LLM prompting—rests entirely on an unquantified visual comparison. This is not a claim about internal inconsistency in the method, but about the evidence supporting the headline assertion; without the missing numbers, a reader cannot tell whether 'comparable' means within one point or within twenty points. The authors' own discussion acknowledges that performance degrades on semantically similar classes and that summary generation is limited, which makes a precise, quantitative baseline comparison especially important.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a zero-shot human activity recognition (HAR) method for smart homes that replaces LLM prompting with language modeling: sensor data are converted into a hand-crafted textual summary, activity labels are converted into hand-crafted textual descriptions, and a pre-trained sentence encoder (all-distilroberta-v1) is used to embed both; classification is performed by cosine similarity to the nearest class description. The method is evaluated in a zero-shot setting on six public datasets (Aruba, Milan, Cairo, Kyoto7, MARBLE, ARAS), reporting accuracies above 60% on four of them, a few-shot extension, and ablations over the summary module, the activity-description module, the sentence encoder, and the similarity metric. The central claim is that this embedding-based approach achieves performance comparable to LLM-based SOTA baselines while avoiding external LLM service dependence and privacy risks.","tokens_in":9097,"tokens_out":2761,"duration_ms":36632,"significance":"If the central claim is supported, the contribution is practically significant: a zero-shot HAR pipeline that runs offline with a small pre-trained sentence encoder and no labeled data would address real privacy and reliability concerns of LLM-prompting approaches. The paper's ablations demonstrate that both the summary-generation and activity-description components are necessary for the reported accuracy, which is a useful design finding, and the few-shot extension is a sensible human-in-the-loop direction. However, the evidence presented is not yet rigorous enough to support the headline claim of comparability to SOTA LLM baselines, because the baseline comparison is qualitative only, and no code or textual artifacts are released for reproduction. The method also performs poorly on the semantically difficult Cairo and Kyoto7 datasets, and the authors correctly acknowledge that limitation in Section 4.5.","major_comments":[{"comment":"The central claim of 'comparable performance to three LLM-based SOTA baselines' is not supported by the evidence shown. Figure 2 compares only two baselines—LLM + Iterative Evolution on ARAS and ADL-LLM on MARBLE—and only visually, with no numerical accuracy or F1 values for those baselines, no error bars, and no significance tests. The text in Section 1 promises three baselines but only two appear in the evaluation. Because 'comparable' cannot be quantified from the figure, the reader cannot tell whether the difference is one point or twenty points. Please provide numerical results for all three claimed baselines on all datasets on which they are evaluated, with variance estimates, and correct the baseline count or explicitly list the three baselines.","section":"Section 4.2, Figure 2, and Section 1"},{"comment":"The method is not reproducible as described because it relies on manually engineered text artifacts that are not shipped: the exact summary skeleton templates, the per-dataset special heuristic rules of Section 3.1.5, and the complete set of activity descriptions of Section 3.2 are not provided in full for all six datasets. Since these textual definitions are the entire predictive machinery, a reader cannot re-implement the method without reverse-engineering them from examples. Moreover, the contribution of the special rules is not isolated in the ablation: Table 1 removes either the whole summary module or the whole activity-description module, but never removes only the special rules. Please release the templates and descriptions (e.g., as supplementary material) and add an ablation that toggles the special rules.","section":"Sections 3.1, 3.2, and 4.2 / Table 1"},{"comment":"The evaluation lacks a comparison against trivial baselines, which is necessary to interpret the reported accuracies and confusion matrices. On Cairo the proposed method obtains 0.46 accuracy and on Kyoto7 0.50, and Figure 3 shows confusion between semantically similar classes; without reporting the majority-class baseline or a random-guess baseline (adjusted for class imbalance), it is unclear whether the method is actually extracting signal on these datasets or merely exploiting label priors. Please report per-dataset class distributions, majority-class accuracy, and the macro-F1 values for such baselines, alongside the confusion matrices already shown.","section":"Section 4.2, Table 1, and Figure 3"}],"minor_comments":[{"comment":"The header 'w/osummary' appears to be a typo; it should read 'w/o summary' for consistency with the other ablation rows.","section":"Table 1"},{"comment":"The sentence beginning 'on the MARBLE dataset' should start with a capital 'On'.","section":"Section 4.2, paragraph 1"},{"comment":"The sentence encoder all-distilroberta-v1 is mentioned without a citation; please add a reference for the model so that the exact encoder version is identifiable.","section":"Section 3.3"},{"comment":"The figures are likely difficult to read in print because the panels are small and no numeric labels are given on the bars; please use larger panels, add value labels, and describe in the captions what is being compared and on which dataset each comparison is performed.","section":"Figure 2 and Figure 3"},{"comment":"The description of special rules is underspecified: please state how many rules were used per dataset and which activities they applied to, because these rules are a form of dataset-specific engineering that affects the claimed data-independent nature of the method.","section":"Section 3.1.5"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is sound and the empirical direction is promising, but the headline claim of SOTA-comparable performance is currently supported only by a visual comparison. The missing numerical baseline results, the unreleased text templates, and the absent trivial baselines are all fixable within the manuscript's scope, so I recommend major revision rather than rejection. I would also encourage the editor to ask for the supplementary artifacts to be made available at revision time, since the method is essentially defined by those hand-crafted texts."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this paper has a real idea—replace LLM prompting with sentence-embedding similarity between hand-crafted data summaries and activity descriptions—and the six-dataset case study is useful. But the central claim that it matches LLM-based SOTA is not supported by the evidence shown. Two baselines appear, no numbers are reported, there are no error bars or significance tests, and Figure 2 is a qualitative visual comparison on one dataset per baseline. The abstract and introduction promise three baselines; only two appear.\n\nWhat is actually new: Fritsch et al. explored embedding-space zero-shot without activity descriptions; this paper adds activity descriptions and runs a six-dataset study. That is a legitimate incremental contribution. The method is genuinely zero-shot in the sense that no sensor data is used for training, and it runs offline on a small encoder, which addresses real privacy and reliability concerns of LLM prompting. The ablation shows both components matter, and the few-shot extension is a sensible add-on. The discussion is honest about failures on semantically similar classes (Cairo, Kyoto7) and about the manual nature of summary generation.\n\nSoft spots: the evaluation is thin exactly where the headline claim is strongest. Without baseline numbers, 'comparable performance' is unverifiable. The summaries and descriptions are hand-crafted per dataset, including special heuristic rules (e.g., MARBLE phone sensors). That makes the method less automatic than the zero-shot framing suggests, and this should be stated more prominently. No code or data is released, so the numbers in Table 1 are not reproducible. The few-shot results show variance and some degradation with more samples; the authors acknowledge this, which is fair.\n\nWho this is for: researchers working on zero-shot HAR or privacy-preserving smart home analytics will get value from the case study and the clear articulation of the design space. It deserves a serious referee, but only with a request for major revision: quantitative SOTA comparison, code/data, and a clearer separation of automated vs. hand-engineered components.\n\nI would not cite this version in my own work yet, but the idea is worth watching.","headline":"A plausible zero-shot HAR alternative to LLM prompting, but the headline SOTA comparison is unquantified and the paper needs code/data before the claim holds.","tokens_in":9625,"tokens_out":2670,"would_cite":false,"duration_ms":28946,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Smart-home activity recognition can work without sending data to an LLM.","keywords":["human activity recognition","zero-shot classification","smart homes","sentence embeddings","language modeling","sensor data","activity descriptions","LLM-free recognition"],"falsifier":"A concrete test would be to run the method on a new smart home dataset without modifying the summary skeleton or descriptions; if the accuracy drops to the level of the Cairo and Kyoto7 results for activities whose descriptions are not well defined semantically, the transferability assumption fails. A direct comparison would be to ask different human annotators to write activity descriptions for the same dataset and show that the performance varies widely with the wording of the descriptions.","tokens_in":8632,"feed_emoji":"🏠","tokens_out":2821,"duration_ms":26540,"temperature":0.7,"pith_summary":"The paper claims that zero-shot human activity recognition (HAR) in smart homes can be done without large language models and without any training data, by converting sensor windows and activity labels into natural-language text and comparing their sentence embeddings. If true, this would remove the privacy and reliability problems of existing 'prompt-the-LLM' systems and make HAR deployable offline and locally in new homes. The authors demonstrate the approach on six public datasets, reporting accuracies above 60% on four of them and comparable results to two LLM-based baselines, while showing that both hand-crafted summary generation and activity descriptions are necessary.","feed_headline":"Smart-home activity recognition without prompting an LLM","feed_subtitle":"Text summaries and sentence embeddings identify activities in new homes with no training data needed.","key_machinery":"The central mechanism is embedding-space arithmetic between two hand-crafted text modules: a Summary Generation module that converts a sensor window into a fixed-skeleton sentence covering time, duration, top-k locations, and top-k sensors with contextual tags, and an Activity Descriptor module that produces one sentence per activity class from floorplan and metadata. A pre-trained sentence encoder, all-distilroberta-v1, maps both to vectors, and cosine similarity selects the class whose description sits closest to the datapoint summary embedding.","core_discovery":"The paper argues that the essential ingredient for zero-shot HAR is generating the two textual descriptions at the right granularity: a datapoint summary that captures the time of occurrence, duration, top locations, and top fired sensors with their object context, and a one-sentence activity description that captures the duration, location, and sensor signatures expected for that activity. Once these texts are embedded with a pre-trained sentence encoder (all-distilroberta-v1), the correct activity label is simply the class description whose embedding has the highest cosine similarity to the datapoint summary embedding. The authors support this by an ablation that shows a drastic drop in accuracy if either component is removed, and by a comparison in which their zero-shot performance is comparable to that of LLM-based SOTA baselines.","pith_inferences":["A natural next step the paper leaves implicit is automating the summary and description generation, e.g., from floorplans and sensor metadata, since the current hand-crafted templates are the main barrier to scaling the approach to arbitrary smart homes.","The paper's own results on Cairo and Kyoto7 suggest that the method's ceiling is set by how precisely an activity can be verbally distinguished from its semantic neighbors, which connects to the deeper question of whether linguistic descriptions alone can separate activities that differ mainly in fine-grained sensor signatures.","Because the method exposes a clean embedding geometry, a testable extension would be a fully automatic pipeline that uses the same sentence encoder to generate data summaries from raw event streams and to propose activity descriptions, removing the need for human-written templates.","The observed sensitivity to randomly chosen few-shot samples points toward combining this embedding approach with an active-learning selector that picks maximally informative sensor windows, which would turn the few-shot extension into a more reliable human-in-the-loop tool."],"forward_implications":["If the approach holds, smart-home HAR can be deployed in new homes without collecting any labeled data and without any network connection to a language model service.","The same pipeline extends to few-shot recognition simply by adding the embeddings of a few labeled sensor windows to the set of candidate class descriptions, which the paper reports improves weighted F1 by about 13.5% on average across five of six datasets.","The comparison with two LLM-based baselines suggests that the commonsense knowledge needed for this task can be supplied by hand-written summaries and activity descriptions rather than by prompting an LLM.","The ablation results imply that the accuracy of such systems depends more on the descriptive fidelity of the generated text than on the choice of similarity metric."],"supporting_citations":[{"why":"LLM + Iterative Evolution baseline that the authors compare their zero-shot approach against on the MARBLE and ARAS datasets.","marker":"[10]"},{"why":"ADL-LLM baseline, an LLM-based zero-shot HAR method, used as a SOTA comparison on the MARBLE and ARAS datasets.","marker":"[5]"},{"why":"TDOST, the prior layout-agnostic use of textual sensor descriptions that the authors build on and whose test setup they reuse for the CASAS datasets.","marker":"[14]"},{"why":"Source for the observation that the same LLM prompt produces degrading HAR performance across LLM versions, motivating the no-LLM design.","marker":"[13]"},{"why":"The related zero-shot approach that also explores replacing LLM inference with embeddings, but without activity descriptions.","marker":"[9]"},{"why":"Source of the ARAS multi-resident dataset used in the evaluation.","marker":"[1]"},{"why":"Source of the MARBLE dataset used in the evaluation.","marker":"[2]"},{"why":"Source of the CASAS smart home datasets (Aruba, Milan, Cairo, Kyoto7) used in the evaluation.","marker":"[7]"}],"fun_headline_variants":["Zero-shot smart-home HAR without LLM prompts","Embeddings replace LLM prompts for activity recognition","Language modeling for zero-shot activity recognition","Skip the prompt: language embeddings do zero-shot HAR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that the hand-written sentence templates and activity descriptions, derived from floorplans and dataset metadata, transfer to any new smart home and capture enough of an activity to separate it from every other class in the embedding space.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot smart-home HAR without LLM prompts","Embeddings replace LLM prompts for activity recognition","Language modeling for zero-shot activity recognition","Skip the prompt: language embeddings do zero-shot HAR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1431,"prompt_tokens":876,"completion_tokens":555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":497}},"tokens_in":492,"tokens_out":555,"duration_ms":6595,"temperature":1.0,"reasoning_tokens":497,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:09:23.632619+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test would be to run the method on a new smart home dataset without modifying the summary skeleton or descriptions; if the accuracy drops to the level of the Cairo and Kyoto7 results for activities whose descriptions are not well defined semantically, the transferability assumption fails. A direct comparison would be to ask different human annotators to write activity descriptions for the same dataset and show that the performance varies widely with the wording of the descriptions.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LLM + Iterative Evolution baseline that the authors compare their zero-shot approach against on the MARBLE and ARAS datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ADL-LLM baseline, an LLM-based zero-shot HAR method, used as a SOTA comparison on the MARBLE and ARAS datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TDOST, the prior layout-agnostic use of textual sensor descriptions that the authors build on and whose test setup they reuse for the CASAS datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source for the observation that the same LLM prompt produces degrading HAR performance across LLM versions, motivating the no-LLM design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The related zero-shot approach that also explores replacing LLM inference with embeddings, but without activity descriptions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the ARAS multi-resident dataset used in the evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the MARBLE dataset used in the evaluation."}],"review_version":1}