{"id":"fae66657-059e-4fb8-abed-82f236055571","arxiv_id":"2411.19041","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"TAMT pre-trains on a source video dataset and fine-tunes small temporal-aware adapters plus a global moment-pooling head on the target, reporting new state-of-the-art cross-domain few-shot action recognition results.","lead":"This paper proposes a training recipe for cross-domain few-shot action recognition: pre-train a video model on a large source dataset, then tune only a small set of temporal adapters on the target domain. The authors report large accuracy gains over existing CDFSAR methods on several video benchmarks, with lower training cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 13–31% SOTA margin is not isolated: Tab. 2 shows SSL+SL pre-training alone lifts the FFT baseline from 54.67 to 64.30, while TAMT adds only ~2.5 points over that matched baseline, so the headline gain is largely a pre-training-recipe effect.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the headline 13–31% advantage is not isolated from the SSL+SL pre-training recipe. The paper is an internally consistent empirical study with a plausible technical contribution; Tab. 2 is itself the control that exposes the confound, and the paper should be credited for reporting it. The efficiency numbers (19 vs 88 GPU days) are relevant and support the decoupled-paradigm claim, as does the linked but not independently executed code. However, the strongest advertised claim, that TAMT outperforms counterparts by 13–31% and achieves new SOTA, conflates the contribution of the proposed HTTN modules with the much larger effect of switching to SSL+SL source pre-training. Because the central claim would still be true if HTTN were the main source of the gains, but Tab. 2 makes that attribution unlikely, the finding warrants a conditional verdict rather than acceptance: the authors should either provide matched pre-training baselines or restate the claim to separate recipe gains from adapter/moment-tuning gains. No suspicion of bad faith is warranted; this is an experimental-isolation gap, and the proposed concrete test would settle it.","tokens_in":18945,"tokens_out":8201,"duration_ms":77916,"concrete_test":"Retrain the strongest baseline, CDFSL-V, on top of the same SSL→SL VideoMAE-S checkpoint and the same 112×112, 10,000-episode protocol used for TAMT, for K-400→{HMDB, SSV2, Diving, UCF, RareAct}, and compare with Tab. 1. If the matched CDFSL-V average is close to the Tab. 2 FFT level (roughly 64% on the three shared targets) rather than 47.24%, the claimed 13–31% advantage is mostly a pre-training-recipe effect. If CDFSL-V code is unavailable, a cheaper decisive proxy is to extend the Tab. 2 internal control: run the FFT baseline under SSL+SL on the two missing targets (HMDB, RareAct); if its five-target average approaches the TAMT average within ~3 pp, the adapters contribute only a small residual and the headline attribution should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that TAMT outperforms CDFSAR counterparts by 13–31 percentage points because its decoupled paradigm plus HTTN (TAA/GTMT) unlocks the potential of frozen pre-trained models. For that claim to be supported, the margin over baselines should survive when comparators are given the same pre-training recipe, backbone, and resolution. The paper does not demonstrate this. In Tab. 2, with K-400 as source and ViT-S at 112×112, replacing SSL-only pre-training with SSL+SL raises the FFT baseline average on SSV2/Diving/UCF from 54.67 to 64.30; TAMT under SSL+SL reaches 66.76, only +2.46 over that matched FFT baseline. Yet the headline Table 1 contrasts TAMT with CDFSL-V (47.24 average on K-400) and other methods that do not use the SSL+SL recipe. CDFSL-V's average on the same three targets is only about 44.4, so the pre-training recipe alone accounts for roughly the entire 13–31 pp gap, leaving only a small residual for the adapter and moment-tuning modules. SEEN and DMSD are additionally marked at 224 resolution with ResNet-18, so they are not matched in resolution or backbone. No baseline is retrained under the SSL+SL recipe, so the attribution of the headline gains to HTTN is unresolved.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TAMT, a decoupled pre-training/fine-tuning approach for cross-domain few-shot action recognition (CDFSAR). It freezes a VideoMAE backbone and introduces Temporal-Aware Adapters (TAA) into the last transformer blocks plus a Global Temporal-aware Moment Tuning (GTMT) module with Efficient Long-Short Temporal Covariance (ELSTC) for video representation. Experiments across five source and five target datasets report 5-way 5-shot accuracy gains of 13% to 31% over prior CDFSAR methods, along with reduced training memory and a claim of 19 GPU days versus 88 GPU days for CDFSL-V when training on K-400 for five targets.","tokens_in":19345,"tokens_out":6693,"duration_ms":64751,"significance":"If the reported margins hold under controlled comparison, TAMT would offer a valuable parameter-efficient baseline for CDFSAR, and the decoupled paradigm would reduce re-training cost when a single source is paired with multiple targets. The paper also provides ablations showing that TAA and GTMT each contribute under a fixed SSL+SL recipe. However, the headline state-of-the-art comparison does not isolate the proposed modules from the stronger SSL+SL pre-training recipe, nor does it match prior methods in backbone and resolution; the central claim therefore currently rests on an attribution that is not fully supported by the experiments as presented.","major_comments":[{"comment":"The headline 13%–31% comparison is not controlled for the pre-training recipe. Table 2 shows that for the K-400 source with ViT-S, moving from SSL-only to SSL+SL pre-training raises the FFT baseline average on SSV2/Diving/UCF from 55.01 to 64.30, and TAMT from 52.80 to 66.76; TAMT is only +2.46 over FFT under the same SSL+SL recipe. Since prior methods in Table 1 (e.g., CDFSL-V average 47.24 on K-400 across five targets) are not trained with the same SSL+SL VideoMAE recipe, the pre-training recipe alone can account for a substantial portion of the reported gains. The authors should retrain at least the main baselines under the same recipe, backbone, and resolution, or explicitly restrict the claim to beating the published numbers under the original protocols.","section":"§4.2, Table 1 and Table 2"},{"comment":"The comparison with SEEN and DMSD is confounded by resolution and backbone: SEEN and DMSD are marked as running at 224×224 resolution with ResNet-18, while the unstarred TAMT runs at 112×112 with ViT-S. Because input resolution and backbone have a large effect on few-shot video accuracy, the reported outperformance over SEEN and DMSD cannot be attributed to the proposed modules from this table alone. Please provide matched-protocol comparisons (same backbone and resolution for all methods, or both configurations for TAMT) or clearly state that the comparison is cross-protocol.","section":"§4.2, Table 1"},{"comment":"The ablation margins used to justify specific design choices are small: ELSTC over TCov gives +0.75 average accuracy, GTMT over ELSTC gives +0.82, and L=2 versus L=3 differs by −0.01, yet no confidence intervals or multi-seed statistics are reported despite averaging 10,000 episodes. These differences are load-bearing for the claims that ELSTC is preferable to TCov and that adding the first-order moment is beneficial; without variance estimates, the reader cannot assess whether these differences are meaningful. Please add confidence intervals and/or results over multiple seeds.","section":"§4.3, Table 4(a) and Table S5"}],"minor_comments":[{"comment":"The characteristic function expansion motivates the use of moments, but the final representation in Eq. (10) is a linear combination H(M2) ⊕ M1 that does not use the coefficients α_p from Eq. (4). Consider presenting Eq. (4) as a heuristic motivation and explicitly stating that the aggregation is a learned linear combination rather than an evaluation of the characteristic function.","section":"§3.3, Eq. (4) and Eq. (10)"},{"comment":"Reference [33] lists the venue as 'Advances in Neural Information Processing Systems (ICCV)'; the correct venue for ST-Adapter is NeurIPS.","section":"References, [33]"},{"comment":"The phrase 'the timestamps t and t′ always have a temporal gap Δ, ranging from 0 to (T−G) with an interval of G' is confusing, because grouping along the temporal dimension changes the indexing of frames within each group. Please define t and t′ clearly with respect to the grouped feature X_ge.","section":"§3.3, ELSTC description"},{"comment":"The asterisk on TAMT* in some blocks is used for the 224×224 version, but in the K-100 block the same marker appears on SEEN*† and DMSD*† as well; the caption should state explicitly which entries are at 224×224 versus 112×112 and which use ResNet-18.","section":"Table 1 caption and Fig. 1b"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the proposed module-level design is reasonable, but the empirical support for the main 'state-of-the-art' claim is not yet isolated. The key missing piece is a set of controlled experiments in which prior CDFSAR methods are retrained with the same SSL+SL VideoMAE recipe, same backbone, and same resolution. Without those, the claimed 13%–31% margin is likely to be questioned by reviewers in this area. I recommend major revision rather than rejection because the deficiency appears fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TAMT is a reasonable parameter-efficient tuning recipe for cross-domain few-shot action recognition. The decoupled pre-train-then-finetune idea is not deep, but it is the right thing to try, and the efficiency argument (one pre-training for many targets versus retraining per target pair) is real. The adapter design is a temporal extension of SSF-style scale-and-shift, and the moment pooling is a grouped covariance variant. The combination is new for CDFSAR and the paper reports consistent small gains over matched tuning baselines.\n\nWhat's good: Tab. 2 and Tab. 4 give the reader what is needed to see the actual contribution. Under the same SSL+SL pre-training, TAMT beats full fine-tuning by about 2.5 points average (66.76 vs 64.30), and beats the best adapter counterpart (ST-Adapter) by about 2.6 points. The efficiency numbers (0.1x memory, 0.7x time vs FFT) are attractive. Code and data splits are promised; I did not run it.\n\nWhere it is soft: the headline claim that TAMT outperforms recently proposed counterparts by 13–31% is not supported as an attribution to the method. Those comparisons pair TAMT's SSL+SL pre-trained VideoMAE backbone at 112x112 against prior methods trained with SSL-only or at 224x224 with ResNet-18. The paper's own Tab. 2 shows that switching pre-training from SSL to SSL+SL lifts the FFT baseline by about 10 points (54.67 to 64.30), which is the same order as most of the claimed headline gains. So the 13–31% story is mostly a recipe effect, not the adapter. The stress-test note is right about this. This is a fixable reporting problem: retrain CDFSL-V or SEEN under the same recipe, or at least clearly separate 'recipe gain' from 'tuning gain.' The absence of error bars on 10k episodes is a minor omission. The novelty of the 'decoupled paradigm' is overstated; it is standard transfer learning, and the paper should not claim 'first attempt' without acknowledging image-domain decoupled few-shot transfer work more carefully.\n\nBottom line: the method's real contribution is a cheap temporal PEFT module that gives a few points over FFT and other adapters in a matched setting. That is a modest but useful result for the CDFSAR subfield. The paper needs a rewrite around attribution before acceptance, but it deserves a serious referee.\n\nRecommendation: send to peer review; my own verdict would be major revision.","headline":"TAMT is a sensible parameter-efficient tuning recipe for CDFSAR, but the headline 13–31% margin is mostly a pre-training and resolution effect; the method's own matched-setting gains are a modest 2–3 points.","tokens_in":19860,"tokens_out":1928,"would_cite":false,"duration_ms":20223,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that cross-domain few-shot action recognition can be solved by pre-training once on the source and fine-tuning only a handful of temporal-aware parameters on the target, beating joint-training approaches by 13–31% at a…","keywords":["cross-domain few-shot action recognition","temporal adapter","moment pooling","parameter-efficient fine-tuning","VideoMAE","self-supervised pre-training","transfer learning","prototypical network"],"falsifier":"Train CDFSL-V, SEEN, or DMSD under exactly the same protocol as TAMT—same VideoMAE SSL+SL pre-trained checkpoint at 112×112 resolution, same number of fine-tuning epochs, same prototypical evaluation—and compare. If the margin collapses, the claimed advantage is not caused by TAA and GTMT but by the pre-training recipe or resolution.","tokens_in":18765,"feed_emoji":"🎬","tokens_out":9837,"duration_ms":79535,"temperature":0.7,"pith_summary":"Cross-domain few-shot action recognition normally trains source and target data together, so every new target means retraining the whole model. This paper tries to establish that a decoupled pipeline works better: pre-train the video model once on the source data, freeze it, and adapt to the target with a small set of temporal-aware modules. The proposed TAMT baseline reports accuracy gains of 13–31% over joint-training state-of-the-art methods across five source and five target benchmarks, while using far less training compute. The central claim is that a frozen spatio-temporal backbone plus lightweight local adapters and moment-based global pooling is enough to close the cross-domain gap.","feed_headline":"Pre-train once, then tune tiny adapters: 13–31% CDFSAR gain","feed_subtitle":"Decoupled training plus tiny temporal adapters outperforms joint training at a fraction of the compute.","key_machinery":"The load-bearing object is HTTN, composed of Temporal-Aware Adapters (TAA) and Global Temporal-aware Moment Tuning (GTMT). TAA computes a per-frame temporal scale $\\gamma$ and bias $\\beta$ from the pooled features of a frozen transformer block, using a shared two-layer temporal convolution with a bottleneck ratio of 4 and kernel size 3; these scale and shift parameters recalibrate intermediate features with only 2.8 million trainable parameters. GTMT produces the video representation from two moments: the global-average-pooled first moment and a second moment computed by the Efficient Long-Short Temporal Covariance layer (ELSTC), which splits the frame sequence into groups, computes long-short temporal covariance matrices inside each group, and aggregates them with convolutions. The two moments are combined and matched by Euclidean distance in a prototypical network. This object carries the argument because it is the only component tuned on the target data; the backbone stays frozen.","core_discovery":"The central discovery claimed is that a VideoMAE backbone pre-trained on source data with self-supervised reconstruction followed by supervised classification can be frozen and transferred to an unseen target domain using only about 2.8 million trainable parameters. The transfer is done by the Hierarchical Temporal Tuning Network (HTTN): local Temporal-Aware Adapters rescale and shift frame-level features inside the last transformer blocks, while Global Temporal-aware Moment Tuning pools the resulting features into a representation built from first-order and second-order temporal covariance statistics. With a prototypical nearest-neighbour classifier, this beats full fine-tuning and outperforms prior joint-training CDFSAR methods by 13–31% on the reported 5-way 5-shot benchmarks, for example 68.37% average accuracy with K-400 as the source at 112×112 resolution.","pith_inferences":["Because the paper's own ablation shows SSL+SL pre-training lifts full fine-tuning from 54.67 to 64.30 average, a matched-recipe comparison with earlier baselines would isolate how much of the headline margin comes from the HTTN modules rather than from the pre-training recipe.","Because TAA and GTMT are plug-in modules on a frozen encoder, they could be reused for other video transfer tasks such as temporal action localization or video retrieval with little modification.","The ELSTC grouping trick suggests that the temporal-covariance representation could be sharpened further with learnable temporal grouping, a direction the paper does not explore.","The decoupled paradigm turns cross-domain few-shot action recognition into a parameter-efficient transfer problem, so the gains should compound when the frozen source model is a larger video foundation model."],"forward_implications":["One source pre-training serves many target datasets: with K-400 as source, TAMT is reported to use 19 GPU days across five targets versus CDFSL-V's 88.","Under the same SSL+SL pre-training, TAMT beats full fine-tuning by about 1–3% on average while using roughly one tenth of the GPU memory and one tenth of the trainable parameters.","The gains extend to other shot settings: with K-100 as source, TAMT reports average improvements over CDFSL-V of 24.08% at 1-shot and 34.13% at 20-shot.","After tuning on UCF or HMDB, the model transfers to four unseen target datasets without further tuning, beating CDFSL-V by 13.83% and 15.26% on average.","The same HTTN tuning also works for standard few-shot action recognition with a ViT-B backbone, matching or exceeding full fine-tuning of CLIP-based methods."],"supporting_citations":[{"why":"Defines the CDFSAR benchmark, the non-overlapping source–target protocol, and the joint-training baseline that TAMT must beat.","marker":"[39]"},{"why":"SEEN is the seminal joint-training CDFSAR method that TAMT compares against.","marker":"[50]"},{"why":"DMSD is the most recent joint-training CDFSAR baseline that TAMT outperforms.","marker":"[18]"},{"why":"VideoMAE supplies the frozen backbone and self-supervised pre-training used by TAMT.","marker":"[44]"},{"why":"Prototypical networks provide the metric-based few-shot inference pipeline.","marker":"[41]"},{"why":"Adapter modules are the parameter-efficient tuning precedent and a direct comparison baseline for TAA.","marker":"[21]"},{"why":"Scaling and shifting features is the direct baseline for TAA's scale–bias recalibration.","marker":"[26]"},{"why":"Moment probing supplies the theoretical basis for using first- and second-order feature moments as representations.","marker":"[11]"},{"why":"ST-Adapter is the temporal adapter baseline that TAMT compares with.","marker":"[33]"}],"fun_headline_variants":["Freeze backbone, add tiny adapters: 13–31% CDFSAR boost","TAMT: Decoupled tuning with temporal adapters beats joint training","One pre-train, many targets: TAMT's tiny adapters win big","Cross-domain few-shot action: TAMT gains 13–31% with 2.8M params","Frozen source model + tiny temporal adapters = SOTA CDFSAR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 13–31% advantage over earlier methods is caused by the temporal-aware tuning modules rather than by the stronger two-stage pre-training recipe (self-supervised then supervised) and different input resolution that those earlier methods did not receive.","fun_headline_variants_meta":{"raw":{"variants":["Freeze backbone, add tiny adapters: 13–31% CDFSAR boost","TAMT: Decoupled tuning with temporal adapters beats joint training","One pre-train, many targets: TAMT's tiny adapters win big","Cross-domain few-shot action: TAMT gains 13–31% with 2.8M params","Frozen source model + tiny temporal adapters = SOTA CDFSAR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000307,"raw_usage":{"total_tokens":1805,"prompt_tokens":1043,"completion_tokens":762,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":650}},"tokens_in":659,"tokens_out":762,"duration_ms":6785,"temperature":1.0,"reasoning_tokens":650,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:35:52.186590+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CDFSL-V, SEEN, or DMSD under exactly the same protocol as TAMT—same VideoMAE SSL+SL pre-trained checkpoint at 112×112 resolution, same number of fine-tuning epochs, same prototypical evaluation—and compare. If the margin collapses, the claimed advantage is not caused by TAA and GTMT but by the pre-training recipe or resolution.","supporting_citations":[{"cited_title":"CDFSL-V: Cross-domain few-shot learning for videos","cited_arxiv_id":null,"evidence_quote":"Defines the CDFSAR benchmark, the non-overlapping source–target protocol, and the joint-training baseline that TAMT must beat."},{"cited_title":"Cross-domain few-shot ac- tion recognition with unlabeled videos","cited_arxiv_id":null,"evidence_quote":"SEEN is the seminal joint-training CDFSAR method that TAMT compares against."},{"cited_title":"DMSD-CDFSAR: Distillation from mixed-source domain for cross-domain few-shot action recognition","cited_arxiv_id":null,"evidence_quote":"DMSD is the most recent joint-training CDFSAR baseline that TAMT outperforms."},{"cited_title":"Video- MAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training","cited_arxiv_id":null,"evidence_quote":"VideoMAE supplies the frozen backbone and self-supervised pre-training used by TAMT."},{"cited_title":"Prototypical networks for few-shot learning","cited_arxiv_id":null,"evidence_quote":"Prototypical networks provide the metric-based few-shot inference pipeline."},{"cited_title":"Parameter-efficient transfer learning for NLP","cited_arxiv_id":null,"evidence_quote":"Adapter modules are the parameter-efficient tuning precedent and a direct comparison baseline for TAA."},{"cited_title":"Scaling & shifting your features: A new baseline for efficient model tuning","cited_arxiv_id":null,"evidence_quote":"Scaling and shifting features is the direct baseline for TAA's scale–bias recalibration."},{"cited_title":"Tuning pre-trained model via moment probing","cited_arxiv_id":null,"evidence_quote":"Moment probing supplies the theoretical basis for using first- and second-order feature moments as representations."},{"cited_title":"ST-Adapter: Parameter-efficient image-to-video transfer learning","cited_arxiv_id":null,"evidence_quote":"ST-Adapter is the temporal adapter baseline that TAMT compares with."}],"review_version":1}