{"id":"d91e5a50-9f52-47b4-b0c1-7e7de1a799ab","arxiv_id":"2412.06674","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A 5M-parameter backbone with shared-weight spanning window attention sets new accuracy records across classification, detection, and generation benchmarks.","lead":"EMOv2 proposes a lightweight vision backbone built from a single improved attention block that mixes local and distant image information with shared weights, reaching 79.4% ImageNet top-1 with 5.1M parameters. For mobile AI, this maps the current performance ceiling for a 5M-parameter model across classification, detection, segmentation, video, and generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'frontier' claim rests on cross-paper baselines with divergent training recipes; the paper's own Sec. 4.1/4.2 disclaimer makes baseline comparability the load-bearing risk.","rationale":"The paper has real independent support: the controlled EMOv2-vs-EMOv1 ablation is internally consistent and reproduced across classification/detection/segmentation; the attention-mode ablation (Tab. 17a) shows spanning attention adds +1.0 over the scaled no-attention baseline at equal parameter count; and Tab. 6 shows EMOv2 is not overfit to a single recipe. These facts make the core mechanism credible. The weak point is not the mechanism but the 'frontier' claim, which is quantified against heterogeneous published baselines. The paper itself flags the lack of a unified experimental standard (Sec. 4.1, 4.2, Tab. A1), so this is not an external objection but an internal admission of missing support. The reader's CONDITIONAL verdict is therefore appropriate: accept the architecture and its controlled gains as a solid contribution, but require a unified re-benchmark before treating EMOv2-5M as the established accuracy frontier for 5M-parameter backbones. I found no internal inconsistency in the FLOP/parameter accounting or the spanning-attention equivalence that would justify a stronger objection.","tokens_in":34084,"tokens_out":19966,"duration_ms":199165,"concrete_test":"Controlled re-benchmark: using the released code, train EMOv2-5M and the closest 5M-scale comparators (EdgeNeXt-S 5.6M, EfficientFormerV2-S1 6.1M, MobileViTv2-1.0 4.9M, tiny-MOAT-1 5.1M) from scratch on ImageNet-1K for 300 epochs at 224x224 with the exact EMOv2 recipe from Sec. 4.1/Tab. A1 (AdamW, lr 6e-3, bs 2048, warmup 20, label smoothing 0.1, RandAugment 9/0.5/1, stochastic depth 0.1; no MixUp, CutMix, EMA, or multi-scale). Record Top-1. Then fine-tune the same checkpoints into RetinaNet with the official MMDetection config (AdamW, 1x schedule) and compare mAP. If EMOv2-5M stays at least +0.5 Top-1 above every close-scale baseline and its RetinaNet margin over EdgeViT-XXS holds at +2.0 mAP, the frontier claim stands; otherwise it needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"EMOv2-5M's headline 'state-of-the-art / frontier' status is established by comparing its 79.4 Top-1 (Tab. 7) and its downstream numbers (Tabs. 8-10) against results taken from other papers. Sec. 4.1 explicitly warns that 'Different SoTA methods use various training recipes that could lead to potentially unfair comparisons,' and Sec. 4.2 warns that 'different methods lack a unified experimental standard.' Tab. A1 documents the gap: MobileNetV4 uses NAS+KD and 500 epochs, GhostNetV3 uses LAMB+KD+re-parameterization, EdgeNeXt uses multi-scale training and position embeddings, whereas EMOv2 uses none of these. The paper only tests EMOv2 under other recipes (Tab. 6: 78.5-79.1); it never re-trains the baselines under a common protocol. Consequently the reported margins over EATFormer-Tiny (+0.9), EdgeViT-XXS (+2.8 RetinaNet), and others could shrink or invert under a unified setting. The controlled EMOv2-vs-EMOv1 comparisons (+1.0 classification, +1.7 SSDLite, +2.6 RetinaNet) are solid and support the mechanism, but the broader '5M frontier' assertion is under-supported by cross-paper numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EMOv2, a family of lightweight vision backbones built from a single one-residual block called i2RMB. The block combines an inverted-residual structure with expanded-window multi-head self-attention (EW-MHSA) and depthwise convolution, and introduces SEW-MHSA, a parameter-shared dual-window attention mechanism that models both nearby and distant tokens in parallel. The authors claim new state-of-the-art results at the 1M/2M/5M parameter scale, including 79.4% ImageNet-1K top-1 accuracy with 5.1M parameters and 1.03G FLOPs, and consistent improvements over EMOv1 in object detection, semantic segmentation, video classification, and diffusion-based image generation. The controlled EMOv2-versus-EMOv1 comparisons support the block-level improvements; the broader \"frontier\" claim relies on cross-paper comparisons whose training recipes differ substantially.","tokens_in":34382,"tokens_out":7702,"duration_ms":79935,"significance":"The paper has a clean architectural message and unusually broad evaluation: the i2RMB block is simple, the ablations in Tables 5 and 17 are systematic, the code is promised, and the controlled EMOv2-versus-EMOv1 results are internally consistent. If the empirical claims hold, the spanning-attention mechanism is a useful contribution for lightweight vision models. However, the headline state-of-the-art status is established by comparing against published baselines that use different training recipes; Table A1 documents large recipe differences (e.g., MobileNetV4 uses NAS and KD, GhostNetV3 uses LAMB and re-parameterization, EdgeNeXt uses multi-scale training, while EMOv2 uses none of these). The controlled comparisons cannot by themselves establish the \"5M frontier\" claim, and the unproved FLOP-saving equivalence in Section 3.2.2 leaves part of the efficiency story unsupported. The strengths and weaknesses together point to a revise-and-resubmit rather than acceptance in current form.","major_comments":[{"comment":"The paper explicitly acknowledges in Sec. 4.1 that \"Different SoTA methods use various training recipes that could lead to potentially unfair comparisons\" and in Sec. 4.2 that \"different methods lack a unified experimental standard.\" These caveats apply directly to the headline comparisons in Tabs. 7-11 and Tab. 19: e.g., MobileNetV4 uses NAS and KD with 500 epochs, GhostNetV3 uses LAMB, KD, and re-parameterization, and EdgeNeXt uses multi-scale training and position embeddings, whereas EMOv2 is trained with a weaker recipe. The paper reports EMOv2 under other recipes (Tab. 6) but does not retrain the published baselines under a common protocol, so margins such as +0.9 over EATFormer-Tiny (Tab. 7) and +2.8 mAP over EdgeViT-XXS (Tab. 9) could shrink or invert under a unified setting. The controlled EMOv2-vs-EMOv1 numbers (+1.0 classification, +1.7 SSDLite, +2.6 RetinaNet) support the mechanism, but the \"frontier\" claim needs either common-protocol retraining of key baselines or a carefully narrowed claim.","section":"Sec. 4.1, Sec. 4.2, Tab. A1"},{"comment":"The paragraph introducing \"pre-attention\" states that when the number of groups in MLPe equals the number of heads in EW-MHSA, the multiplication order can be exchanged, and that matrix multiplication before MLPe therefore reduces FLOPs. This proposition is asserted without proof or citation. It is load-bearing because the FLOP counts in Tabs. 2 and 7 (e.g., 1035M for EMOv2-5M) and the claimed efficiency of EW-MHSA rely on this pre-attention implementation. Please provide a short derivation or a precise citation; if the equivalence is only approximate, the reported complexity numbers should be revised.","section":"Sec. 3.2.2, \"Efficient equivalent implementation\""},{"comment":"The default hyperparameters are stated inconsistently across the paper. Section 4.1 reports a batch size of 2,048 and Tab. A1 lists batch size 2,048 with drop path rate 0.1, while Tab. 17c shows the best results at batch size 1,024 and drop path rate 0.05 and suggests batch size 1,024 as the default. This ambiguity affects reproducibility of the main 79.4 result and should be resolved by specifying one canonical configuration that is used for all reported comparison tables.","section":"Sec. 4.1 vs Sec. 4.3, Tab. 17c"}],"minor_comments":[{"comment":"The footnote uses the same symbol '*' for both \"Neural Architecture Search\" and \"stronger training strategy displayed in Tab. 17(e)\"; use distinct symbols or letters for the two definitions.","section":"Tab. 7 footnotes"},{"comment":"Resolutions appear as \"2242\", \"2562\", and \"5122\", which should be rendered as 224², 256², and 512²; as printed they are easy to misread as literal dimensions.","section":"Throughout Tab. 7 and related tables"},{"comment":"The terms \"pre-attention\" (Sec. 3.2.2) and \"post-attention\" (Sec. 3.3.1) are confusing because both refer to the placement of the channel-expansion MLP relative to the attention-map multiplication rather than to attention operating before or after convolution; consider renaming them to \"expand-before-attention\" and \"expand-after-attention\".","section":"Sec. 3.3.1, \"Non-linearity for post-attention\""},{"comment":"The sentences \"Tao et al. [53] introduces additional learnable tokens\" and \"Chen et al. [53] design a parallel structure\" both cite reference [53], but [53] is LightViT by Huang et al.; the second sentence appears to refer to MobileFormer (reference [33]) and should be re-cited.","section":"Sec. 2, Related Work"},{"comment":"The UNet-based segmentation model is referred to as U-EMO in the text and as \"U-EMOv2-5M\" in the table header, but the backbone column of the last row reads \"EMOv2-5M\"; please use a consistent name.","section":"Sec. 4.2, Tab. 12"}],"recommendation":"major_revision","confidential_remarks":"The core architecture and the controlled EMOv2-vs-EMOv1 experiments are probably publishable, and the paper is a reasonable TPAMI extension of the authors' ICCV'23 EMO work. The main risk is the state-of-the-art claim, which rests on cross-paper baselines with substantially different training recipes; the authors' own Tab. A1 makes this risk explicit. I would urge the editor to require either a common-recipe comparison for at least the closest competitors (e.g., EdgeNeXt, EATFormer, MobileNetV4) or a clear reframing of the contribution as a mechanism study rather than a claim of frontier status. The unproved pre-attention equivalence should also be fixed before publication. If these points are addressed, I would expect the paper to be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, incremental extension of the authors' own EMOv1, with one genuinely new mechanism and unusually thorough ablations. The headline \"5M frontier\" is plausible but not fully proven, because the comparisons against other methods rest on numbers from papers using different training recipes. The internal controlled comparisons are the strong part.\n\nWhat's new: SEW-MHSA reuses the neighbor window attention weights for a second, stride-sampled \"remote\" window partition, so the model gets distant interactions with no extra parameters and modest FLOPs. That is a real idea, not present in EMOv1 or the other cited work. The Meta Mobile Block abstraction (one-residual expand/operator/shrink) is a reasonable way to unify IRB, MHSA, and FFN; it is a framing, but the i2RMB built from it performs well. The paper earns credit for breadth: classification, SSDLite, RetinaNet, Mask R-CNN, segmentation, video, and DiT-style generation, plus depth/width/kernel/attention ablations.\n\nThe controlled comparisons are the evidence that holds: EMOv2-5M beats EMOv1-5M by +1.0 under the same recipe, and downstream gaps of +1.7/+2.6 on SSDLite/RetinaNet are meaningful because both use the same detection framework and the same backbone lineage. That supports the mechanism.\n\nThe soft spots are real but not fatal. First, the \"frontier\" claim against EATFormer, EdgeViT, MobileNetV4, GhostNetV3, and others uses numbers lifted from external papers with divergent recipes; Sec. 4.1 and Sec. 4.2 say exactly that. The paper tests EMOv2 under other recipes but never retrains the baselines under one protocol. Tab. A1 shows how different those recipes are, so margins over those baselines could shrink. Second, the pre-attention FLOP-saving equivalence in Sec. 3.2.2 is stated as a proposition but no proof is given; it is checkable, but a referee should ask. Third, code is promised, not shipped in the preprint, so the numbers cannot yet be independently reproduced. Some recent mobile backbones are also missing from the RetinaNet/Mask R-CNN comparison tables, which weakens those tables but not the internal story.\n\nWho it's for: people building lightweight backbones for mobile or dense prediction, and anyone tracking the accuracy-per-parameter frontier. It deserves a serious referee. The mechanism is novel enough, the ablations are extensive, and the main claims are backed by controlled experiments. I'd send it to review, and ask for proof of the equivalence, a clearer statement of which margins are cross-recipe, and ideally code plus one unified-recipe baseline comparison.","headline":"A solid, incremental extension of EMOv1 with a genuinely new parameter-shared spanning attention mechanism; the internal controlled comparisons hold up, but the '5M frontier' claim rests partly on cross-paper baselines with different training recipes.","tokens_in":34974,"tokens_out":2023,"would_cite":true,"duration_ms":20131,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EMOv2-5M, built from one shared-weight spanning-attention block, reaches 79.4% top-1 on ImageNet-1K with 5.1M parameters.","keywords":["lightweight vision backbone","inverted residual block","spanning window attention","Meta Mobile Block","ImageNet classification","dense prediction","efficient attention","mobile vision"],"falsifier":"Retrain similarly sized baselines such as EdgeNeXt-S, MobileViT-S, and EMOv1-5M under EMOv2's exact training recipe, then compare ImageNet top-1 and COCO RetinaNet mAP; if the reported margins shrink to noise or reverse, the claimed frontier position would not hold.","tokens_in":33887,"feed_emoji":"⚡","tokens_out":8716,"duration_ms":81580,"temperature":0.7,"pith_summary":"This paper proposes EMOv2, a family of lightweight vision backbones whose 5M-parameter variant reaches 79.4% top-1 accuracy on ImageNet-1K with 5.1M parameters and 1.0G FLOPs. The authors argue that a single one-residual block, combining depthwise convolution with a shared-weight spanning window attention, defines the current accuracy frontier at the 5M scale. A sympathetic reader would care because mobile and edge applications operate under strict parameter and latency budgets, and the paper offers a uniform building block that transfers to detection, segmentation, video, and image generation.","feed_headline":"5M-parameter EMOv2 hits 79.4% ImageNet top-1","feed_subtitle":"Shared-weight spanning attention lets a tiny backbone see far, beating larger models on detection and generation.","key_machinery":"The load-bearing object is the spanning attention mechanism (SEW-MHSA), a parameter-shared dual-window attention that partitions the same Q, K, V into adjacent local windows and into remote windows sampled with stride $[H/h, W/w]$, then fuses the two attention maps. Because both partitions reuse the same Q, K, V projections and attention weights, it adds no parameters and only a small FLOP overhead; this is what lets a 5M model see distant interactions in high-resolution inputs. The surrounding i2RMB block wraps this attention with a large-kernel depthwise convolution, a post-attention nonlinearity, and a single residual connection.","core_discovery":"The central claim is that the Inverted Residual Block of MobileNetV2 and the MHSA/FFN modules of Transformers share a common one-residual meta-structure, which the paper abstracts as the Meta Mobile Block with expansion ratio $\\lambda$ and efficient operator $F$. Instantiating $F$ as a cascade of expanded-window multi-head self-attention and depthwise convolution gives the iRMB; adding a parameter-shared second window partition that samples distant tokens at stride $[H/h, W/w]$ produces the improved i2RMB with spanning attention. Using only i2RMB blocks, EMOv2-5M reaches 79.4% top-1 at 5.1M parameters and 1.0G FLOPs, surpassing EMOv1-5M by +1.0, and 41.5 mAP with RetinaNet, +2.6 over EMOv1. The paper also reports 82.9% top-1 at 512 resolution with knowledge distillation and 1000 epochs, and scales the design to 20M and 50M variants.","pith_inferences":["A testable extension the paper does not run: train EMOv2-5M and EMOv1-5M at increasing resolutions from 224 to 512 and measure the accuracy gap, which should widen if spanning attention's distant modeling is the cause.","The dual-window partition is a general token-grouping trick, so it could be inserted into other window-attention or state-space vision models without changing the attention computation itself.","Because the Meta Mobile Block abstracts IRB, MHSA, and FFN into one residual structure, a future design could choose the efficient operator F per stage or per hardware target while keeping the same surrounding block."],"forward_implications":["At the 5M scale, EMOv2-5M outperforms published CNN-, Transformer-, and RNN-based lightweight models on ImageNet-1K, including MobileNetV3-L-1.25, EdgeNeXt-S, EfficientFormerV2-S1, and Vim-Ti.","The same ImageNet-pretrained backbone raises dense-prediction results, reaching 29.6 mAP with SSDLite, 41.5 mAP with RetinaNet, 42.3 box AP with Mask R-CNN, and 39.8 mIoU with DeepLabv3 on ADE20K.","Replacing the Transformer block in DiT with i2RMB cuts FID from 68.4 to 46.3 at the S scale and from 19.5 to 9.6 at the XL scale while using fewer parameters.","Extending i2RMB to the temporal dimension gives V-EMOv2-5M 65.2% top-1 on Kinetics-400 with 5.9M parameters, beating UniFormer-XXS's 63.2% with 9.8M parameters.","With a stronger training recipe of 512 resolution, knowledge distillation, and 1000 epochs, EMOv2-5M reaches 82.9% top-1, indicating the architecture retains headroom."],"supporting_citations":[{"why":"Supplies the inverted residual block that the paper generalizes into the Meta Mobile Block.","marker":"[9]"},{"why":"Defines the EMOv1 baseline and iRMB, the direct predecessor and comparison point for EMOv2.","marker":"[13]"},{"why":"Provides MobileViT, a long-range attention baseline that underperforms at higher resolutions and motivates spanning attention.","marker":"[17]"},{"why":"Introduces the window attention scheme that SEW-MHSA extends by adding a distant-window partition.","marker":"[21]"},{"why":"Supplies the MHSA and FFN modules whose meta-structure is unified with the inverted residual block.","marker":"[35]"},{"why":"Defines MetaFormer, the two-residual architecture that the one-residual Meta Mobile Block is contrasted against.","marker":"[52]"},{"why":"Gives EdgeNeXt, a strong hybrid lightweight baseline used for classification and throughput comparisons.","marker":"[2]"},{"why":"Defines DiT, the diffusion transformer whose blocks are replaced by i2RMB for image generation results.","marker":"[67]"},{"why":"Provides DeiT-Tiny, the columnar transformer used to test iRMB/i2RMB replacement in toy experiments.","marker":"[43]"},{"why":"Provides PVT-Tiny, the pyramidal transformer used to test iRMB/i2RMB replacement in toy experiments.","marker":"[19]"}],"fun_headline_variants":["EMOv2-5M hits 79.4% ImageNet top-1 at 1.0G FLOPs","EMOv2-5M: 79.4% top-1, 41.5 mAP at 5M parameters","Spanning attention pushes 5M-param EMOv2 to 82.9% with KD","A single meta-block unifies CNN and Transformer for mobile vision","EMOv2-5M: 5M params, 1.0G FLOPs, 79.4% ImageNet accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The published results of competing lightweight models, each trained under its own recipe, are directly comparable to the authors' numbers even though the training setups differ.","fun_headline_variants_meta":{"raw":{"variants":["EMOv2-5M hits 79.4% ImageNet top-1 at 1.0G FLOPs","EMOv2-5M: 79.4% top-1, 41.5 mAP at 5M parameters","Spanning attention pushes 5M-param EMOv2 to 82.9% with KD","A single meta-block unifies CNN and Transformer for mobile vision","EMOv2-5M: 5M params, 1.0G FLOPs, 79.4% ImageNet accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001266,"raw_usage":{"total_tokens":5277,"prompt_tokens":1136,"completion_tokens":4141,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":752,"completion_tokens_details":{"reasoning_tokens":4000}},"tokens_in":752,"tokens_out":4141,"duration_ms":30229,"temperature":1.0,"reasoning_tokens":4000,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:25:37.659537+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain similarly sized baselines such as EdgeNeXt-S, MobileViT-S, and EMOv1-5M under EMOv2's exact training recipe, then compare ImageNet top-1 and COCO RetinaNet mAP; if the reported margins shrink to noise or reverse, the claimed frontier position would not hold.","supporting_citations":[{"cited_title":"Metaformer is actually what you need for vision,","cited_arxiv_id":null,"evidence_quote":"Defines MetaFormer, the two-residual architecture that the one-residual Meta Mobile Block is contrasted against."},{"cited_title":"Scalable diffusion models with transformers,","cited_arxiv_id":null,"evidence_quote":"Defines DiT, the diffusion transformer whose blocks are replaced by i2RMB for image generation results."}],"review_version":1}