{"id":"e45d3ce1-56e4-46b2-9dfc-bbdbfb63df9a","arxiv_id":"2506.17685","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SeqDG uses a masked sequence reconstruction loss and cross-domain sequence mixing to improve egocentric action recognition in unseen kitchens, reporting a 2.4-point gain on EPIC-KITCHENS-100 Action accuracy.","lead":"This paper introduces SeqDG, a method that improves egocentric action recognition across unseen environments by training on sequences of actions and reconstructing masked actions using text and video cues. It reports small accuracy gains on EPIC-KITCHENS-100 and EGTEA, suggesting that sequence context helps domain generalization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hyperparameters and design choices appear to be selected on the EPIC-KITCHENS-100 target-domain validation split, so the 'unseen environment' claim is not tested as stated; the reported +2.4 Action gain may be optimistic.","rationale":"The paper has genuine strengths: the method is clearly motivated, ablated across components, and compared against strong sequence-based baselines such as MTCN; the no-text variant suggests that sequence information alone contributes. The reader's weakest assumption about cross-domain sequence consistency is a real limitation, but it is partly mitigated by SeqMix and by the intra-domain results, and it is not the point where the reported evidence is least secure. The more load-bearing issue is target-domain validation leakage: the paper reports results on the target validation split of EPIC-KITCHENS-100 and does not state that all design choices were fixed using source-only validation. Since the headline improvement is small in absolute terms and many ablated choices produce differences of similar magnitude, the reported +2.4 Action gain could be an artifact of tuning on the target validation set. The concrete test above would settle this. In addition, there are minor reporting inconsistencies — the abstract's +0.6% on EGTEA does not match Table 3's +1.5% over MTCN, and the 'relative' improvement in the abstract is actually an absolute percentage-point gain — but these are secondary to the validation protocol. Therefore the reader's CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":14402,"tokens_out":6007,"duration_ms":61187,"concrete_test":"Repeat the EPIC-KITCHENS-100 cross-domain experiment using only source-domain validation for all model-selection decisions: hold out a subset of source kitchens (or a random split within the source split) to pick lambda_rV/lambda_rT, window size W, language model, and backbone; then evaluate the selected model on the target validation split across 3 seeds. If the Action Top-1 gain over Source Only is no longer near +2.4 and falls within seed noise, the claim that SeqDG improves recognition in unseen environments is not supported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing concern is not the sequence-consistency intuition itself, but that the paper's central empirical claim — cross-domain improvement in 'unseen environments' — is established under model selection that appears to use the target domain. All cross-domain results in Table 1 are reported on the EPIC-KITCHENS-100 target validation set, and the ablations and hyperparameter analyses in Tables 4, 6, 7, 8 and Figure 5 (lambda_rV/lambda_rT, sequence length, language model, backbone) are reported without stating that a source-only validation split was used for selection. If lambda values, window size, LM, and backbone were chosen by maximizing target-validation accuracy, the +2.4 Action Top-1 improvement over Source Only is an optimistic selection artifact, and the method's ability to generalize to truly unseen domains is unverified. This is consequential because many design choices yield differences comparable to the headline gain: for example, Table 8 shows RGB-only Action accuracy of 12.1, 12.8, and 12.6 for sequence lengths 3, 5, and 7, a 0.7-point spread around the chosen setting. The paper should specify the model-selection protocol and re-run with source-only validation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes SeqDG, a domain generalization method for egocentric action recognition. The key idea is that action sequences are more consistent across visual domains than individual action appearances. SeqDG uses two components: SeqRec, a visual-text sequence reconstruction objective that masks the central action and reconstructs it from surrounding actions using cross-attention between visual and textual features, and SeqMix, a data augmentation that replaces an action in a sequence with another action from a different source domain but with the same label. The training loss combines a classification loss on the central action with two reconstruction losses. The method is evaluated on EPIC-KITCHENS-100 in a cross-domain setting and on EGTEA in an intra-domain setting, reporting improvements over source-only baselines and prior methods, including a +2.4 Action Top-1 gain on EPIC-KITCHENS-100 and 74.0 Top-1 accuracy on EGTEA.","tokens_in":14598,"tokens_out":7385,"duration_ms":75887,"significance":"The idea of leveraging action-sequence consistency across visual domains is timely and intuitively plausible, especially for scripted activities such as cooking. The paper is thorough in its experimental coverage: ablations of each component, multiple modalities (RGB, Flow, Audio), several backbones (TSN, I3D, TSM, TBN), sequence lengths, language models, and loss weights, and the authors release code and data. If the empirical claims survive unbiased model selection, SeqDG would be a solid contribution to egocentric domain generalization. At present, however, the central empirical claims are weakened by apparent target-validation-based hyperparameter selection and by the absence of uncertainty estimates, and the reported gains over the nearest competitors are small.","major_comments":[{"comment":"All cross-domain results are reported on the EPIC-KITCHENS-100 target validation set, as stated in the caption of Table 1, and the ablation and hyperparameter analyses in Tables 4, 6, 7, 8 and Figure 5 do not indicate that a source-only validation split was used for model selection. Because the values of λ_rV, λ_rT, sequence length W, SeqMix probability p, language model, and backbone are all selected using these target-validation numbers, the reported +2.4 Action Top-1 improvement over Source Only is not an unbiased estimate for truly unseen environments. This is load-bearing: in Table 8, varying W from 3 to 7 changes RGB-only Action accuracy by 0.7 points around the chosen W=5, which is the same order as the headline gain. Please specify the model-selection protocol and either re-run the experiments with source-only validation for all hyperparameter choices or report the full grid of results on a source-only validation split.","section":"§4.4, Tables 1, 4, 6, 7, 8, Fig. 5"},{"comment":"No variance or significance is reported. In Table 1, SeqDG's Action Top-1 accuracy (20.6) differs from EgoZAR (20.3) and RNA (20.0) by at most 0.6 points, and in Table 3 the EGTEA lead over MTCN is 1.5 points. Without standard deviations across seeds or significance tests, these gaps are within the range of noise typical of this benchmark, so the claims of 'state-of-the-art' performance and '+2.4%' improvement are not yet supported. Please report means and standard deviations over at least three runs and, for the main SOTA comparisons, a paired significance test or confidence intervals. In addition, the Source Only baselines differ across method blocks in Table 1 (e.g., Action accuracies of 19.2, 19.0, 19.6, 18.2, 19.4), so comparing relative improvements across methods is not a controlled comparison.","section":"§4.4, Tables 1 and 3"},{"comment":"The textual narrations used for L_rT are described in §4.3 as 'simple verb and noun combinations', which are effectively close to the action labels themselves. If this is the case, the reconstruction objective L_rT in Eq. (11) may be little more than masked-label prediction rather than a domain-agnostic semantic prior, which weakens the claim that text is 'less tied to the domain' (§3). This does not invalidate the method, but it affects the interpretation of SeqRec and of the SeqDG (No text) ablation. Please clarify what the textual branch encodes, compare against a variant that uses only label embeddings for the text branch, or evaluate on a benchmark with genuinely free-form narrations.","section":"§3.2, §4.3"}],"minor_comments":[{"comment":"The abstract states '+2.4% relative average improvement', but Table 1 shows an absolute increase of 2.4 percentage points (18.2 to 20.6 Action Top-1), which is a relative improvement of about 13.2%. Please use the correct terminology.","section":"Abstract"},{"comment":"Equation (5) gives Z_V ∈ R^{(W+2)×D} and the text says 'append a learnable classification token', while §4.3 mentions separate classification tokens for Verb and Noun. Please clarify how many tokens are appended and align the dimension notation.","section":"Eq. (5), §4.3"},{"comment":"The caption of Figure 2 says the counts are for 'sub-sequences of any smaller (or equal) length that repeat across domains'; this makes the reported numbers difficult to interpret. Please state whether the counts are unique repeated sequences or total occurrences, and whether shorter sequences are counted within longer ones.","section":"Fig. 2"},{"comment":"Section 3.5 states that textual annotations are not used at inference time, but Tables 2 and 3 report results for SeqDG† and MTCN† that use a language model at test time to filter predictions. Please clarify what the LM filtering uses and whether this contradicts the earlier statement.","section":"§3.5, Tables 2 and 3"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the target-validation-based hyperparameter selection. If the authors can demonstrate that the results hold under source-only model selection and provide variance estimates, the contribution is likely publishable. The stress-test concern raised in the review is valid and should be the primary request for revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SeqDG is worth a look if you work on egocentric action recognition or domain generalization. The new piece is the combination of SeqRec, a masked sequence-reconstruction objective with cross-attention between visual and textual decoders, and SeqMix, which mixes same-label actions across source domains. Text is used only in training, and the no-text variant still beats the source-only baseline, so the sequence intuition carries real weight. The ablations are more thorough than average in this line of work: multiple backbones, modality sets, sequence lengths, and loss weights, with directionally consistent improvements. That is honestly earned credit. The soft spots are in the empirical claims. The paper never states which validation split was used for model selection. The published tables and figures look like target-domain validation analyses; if lambda values, sequence length, language model, and backbone were all chosen on the EPIC-KITCHENS-100 target split, then the cross-domain '+2.4' is optimistic and the 'unseen environments' framing is not tested as stated. That is the load-bearing issue. Second, no variance or significance is reported anywhere, and on these metrics a 1-2 point gap is often within run-to-run noise. Third, the abstract says '+0.6% over SOTA on EGTEA,' but Table 3 shows SeqDG at 74.0 vs MTCN at 72.5 - a 1.5-point gap. That inconsistency will be noticed by any careful reader. Fourth, code is promised but not actually shipped; for an engineering-heavy method, that hurts reproducibility. Minor point: the text reconstruction loss uses narrations that are essentially verb+noun labels, so part of the 'sequence' signal is label statistics rather than free text, which is fine but should be acknowledged. The related work and citations are fair; the self-citations are to baselines, and the positioning against MTCN and EgoZAR is clear. None of the soft spots kills the idea. The architectural combination is new in this setting, and the ablations are honest enough that the method will probably survive a re-run. But the paper would be much stronger with a source-only model-selection curve, multiple seeds, and a corrected abstract. For peer review, I would send this to referees rather than desk-reject. As a personal verdict, I would not bet on the reported gains until code or a revision clarifies the selection protocol.","headline":"A credible sequence-based DG method with genuinely thorough ablations, but the headline +2.4 gain is not trustworthy until the model-selection protocol is disclosed.","tokens_in":749,"tokens_out":1195,"would_cite":false,"duration_ms":36729,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Action sequences that repeat across kitchens let egocentric action recognition generalize to unseen environments.","keywords":["domain generalization","egocentric action recognition","action sequences","masked sequence reconstruction","cross-modal learning","SeqDG","EPIC-KITCHENS-100","EGTEA"],"falsifier":"Run SeqDG and its source-only baseline on a benchmark whose action orderings differ by domain (or on a subset of EPIC-KITCHENS-100 selected for low sequence repetition), and check whether the +2.4% improvement disappears or reverses; alternatively, compute the correlation between per-domain sequence repetition counts and SeqDG's gain and look for a near-zero slope.","tokens_in":14171,"feed_emoji":"🍳","tokens_out":5986,"duration_ms":54606,"temperature":0.7,"pith_summary":"The paper argues that egocentric action recognition models fail in new environments because they latch onto visual cues specific to one kitchen, and that the fix is to exploit the fact that goal-directed action sequences—take eggs, crack them, fry them—repeat across kitchens even when appearances differ. To test this, the authors build SeqDG, which trains a transformer on windows of actions, masks the central action, and reconstructs it from surrounding visual and textual features, while a SeqMix augmentation swaps in same-label actions from other domains. On EPIC-KITCHENS-100 this yields a +2.4% relative average improvement in cross-domain action accuracy over its source-only baseline, and on EGTEA it reaches 74.0% Top-1 accuracy, surpassing previous methods. Text is used only during training, so at inference the model needs only video.","feed_headline":"Repeated action sequences boost cross-domain egocentric recognition","feed_subtitle":"SeqDG turns consistent user intent across kitchens into a training signal, gaining +2.4% on unseen EPIC-KITCHENS-100 domains.","key_machinery":"The central object is the action sequence $S_i = \\{a_{i-W/2}, \\ldots, a_i, \\ldots, a_{i+W/2}\\}$, an ordered window of $W$ actions centered on the target, each represented by frozen visual and textual features. The mechanism is masked cross-modal reconstruction, called SeqRec, combined with a mixing augmentation, SeqMix. In SeqRec, the central action's features are zeroed and two transformer decoders reconstruct them, with the visual decoder cross-attending to unmasked textual features and the textual decoder cross-attending to unmasked visual features; the classification head uses a CLS token. SeqMix replaces an action with a same-label action from another source domain with probability 0.5. The full objective is $\\mathcal{L} = \\mathcal{L}_C + \\lambda_{rV}\\mathcal{L}_{rV} + \\lambda_{rT}\\mathcal{L}_{rT}$, jointly training classification, visual reconstruction, and textual reconstruction. This machinery is what forces the encoder to rely on relational, domain-agnostic structure instead of per-domain appearance.","core_discovery":"Human actions do not occur in isolation: the order in which people perform goal-directed steps, such as opening the fridge, picking up milk, opening the lid, and pouring milk, stays consistent across kitchens and users even when the visual appearance shifts. SeqDG turns this regularity into a training objective: features from $W$ surrounding actions are encoded through a transformer, the central action's visual and textual features are masked, and two decoders reconstruct the masked features by cross-attending to the other, unmasked modality. This reconstruction pushes the encoder to capture temporal and cross-modal dependencies rather than domain-specific appearance, and SeqMix further widens the training distribution by replacing the target action with a same-label action from a different source domain. The result is a model that generalizes to unseen kitchens and temporal shifts without ever seeing target data, and that outperforms both source-only baselines and existing domain-adaptation methods.","pith_inferences":["If sequence regularity is the active ingredient, cross-domain gains should shrink on datasets with weakly ordered or random activities; a direct test would correlate per-domain sequence repetition counts with SeqDG's improvement over its baseline.","SeqMix's constraint that swapped actions share a label may leak label information across domains; an ablation swapping same-verb but different-noun actions could isolate how much of the gain comes from the mixing rule rather than cross-modal reconstruction.","The same recipe could transfer to procedure-heavy first-person tasks such as surgery or assembly, provided the domain's sequence repetition rate is checked first.","The choice of text encoder matters little (CLIP, MiniLM, BERT give similar results), which suggests the reconstruction objective is doing the work rather than the specific language model."],"forward_implications":["Sequence context alone, without text, already improves cross-domain accuracy by +1.3%, so the temporal ordering of actions is itself a generalizable cue.","SeqDG outperforms UDA methods that use unlabeled target data during training, suggesting target-free domain generalization can be competitive with adaptation.","Gains are larger for Noun and Action metrics than for Verb, consistent with the idea that manipulated objects persist across consecutive actions more reliably than verbs.","The approach improves results across TSN, I3D, TSM, and TBN backbones, so the sequence-reconstruction objective is not tied to one visual encoder.","Because text is absent at inference, the method can be deployed with video-only inputs, and test-time language-model filtering adds little."],"supporting_citations":[{"why":"Supplies the EPIC-KITCHENS-100 dataset and the repeated-sequence statistics (5882 repeated verb sequences) that motivate the method.","marker":"[10]"},{"why":"Defines the UDA benchmark with source/target kitchen splits and the evaluation protocol used for cross-domain comparisons.","marker":"[39]"},{"why":"MTCN is the closest sequence-based baseline; SeqDG compares against it in both intra-domain and cross-domain settings.","marker":"[9]"},{"why":"CIR is a source-only DG baseline whose cross-instance reconstruction SeqDG contrasts with.","marker":"[4]"},{"why":"EgoZAR is the strongest DG competitor on Action accuracy and the paper compares SeqDG's relative improvement against it.","marker":"[5]"},{"why":"RNA provides the norm-alignment DG baseline and the largest relative improvement (+1.8%) that SeqDG claims to exceed (+2.4%).","marker":"[31]"},{"why":"Masked autoencoders provide the reconstruction-objective precedent that SeqRec adapts from pixel-level to action-sequence-level.","marker":"[32]"},{"why":"BERT encodes the free-form narrations used as textual features in the SeqRec reconstruction loss.","marker":"[43]"}],"fun_headline_variants":["SeqDG: Action sequences bridge domains for egocentric recognition","Action order is the key to cross-domain egocentric recognition","SeqDG: Reconstructing action sequences for domain generalization","Consistent action steps across environments boost egocentric recognition","Unseen domains? Action sequences ground egocentric models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the order in which people perform actions is largely the same across environments; if a deployment setting has no repeated action sequences, the reconstruction objective will learn source-specific statistics rather than general temporal structure.","fun_headline_variants_meta":{"raw":{"variants":["SeqDG: Action sequences bridge domains for egocentric recognition","Action order is the key to cross-domain egocentric recognition","SeqDG: Reconstructing action sequences for domain generalization","Consistent action steps across environments boost egocentric recognition","Unseen domains? Action sequences ground egocentric models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001373,"raw_usage":{"total_tokens":5574,"prompt_tokens":962,"completion_tokens":4612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":4533}},"tokens_in":578,"tokens_out":4612,"duration_ms":36906,"temperature":1.0,"reasoning_tokens":4533,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:03:12.821095+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SeqDG and its source-only baseline on a benchmark whose action orderings differ by domain (or on a subset of EPIC-KITCHENS-100 selected for low sequence repetition), and check whether the +2.4% improvement disappears or reverses; alternatively, compute the correlation between per-domain sequence repetition counts and SeqDG's gain and look for a near-zero slope.","supporting_citations":[{"cited_title":"Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100,","cited_arxiv_id":null,"evidence_quote":"Supplies the EPIC-KITCHENS-100 dataset and the repeated-sequence statistics (5882 repeated verb sequences) that motivate the method."},{"cited_title":"Scaling egocentric vision: The epic-kitchens dataset,","cited_arxiv_id":null,"evidence_quote":"Defines the UDA benchmark with source/target kitchen splits and the evaluation protocol used for cross-domain comparisons."},{"cited_title":"With a little help from my temporal context: Multimodal egocentric action recognition,","cited_arxiv_id":null,"evidence_quote":"MTCN is the closest sequence-based baseline; SeqDG compares against it in both intra-domain and cross-domain settings."},{"cited_title":"What can a cook in italy teach a mechanic in india? action recognition generalisation over scenarios and locations,","cited_arxiv_id":null,"evidence_quote":"CIR is a source-only DG baseline whose cross-instance reconstruction SeqDG contrasts with."},{"cited_title":"Egocentric zone-aware ac- tion recognition across environments,","cited_arxiv_id":null,"evidence_quote":"EgoZAR is the strongest DG competitor on Action accuracy and the paper compares SeqDG's relative improvement against it."},{"cited_title":"Relative norm alignment for tackling domain shift in deep multi-modal classification,","cited_arxiv_id":null,"evidence_quote":"RNA provides the norm-alignment DG baseline and the largest relative improvement (+1.8%) that SeqDG claims to exceed (+2.4%)."}],"review_version":2}