{"id":"1f23e43a-d999-4299-8e01-59be3534e694","arxiv_id":"2607.23070","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DishSeg24k is a 24k-image dish-level food segmentation benchmark, and the FEAST model reports +3.21 mIoU over prior methods, mostly from its mixture-of-experts decoder.","lead":"A new dataset of 24,096 real dining photos with 112,281 dish outlines and 278 dish types aims to make food segmentation harder and more realistic; a new model built on Mask2Former with expert mixtures reports state-of-the-art gains, though most gains come from the expert module, not the reinforcement-learning part.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +3.21% mIoU gain may be confounded by the undisclosed pretraining of the Mask2Former reference policy π_ref; if π_ref is pretrained on DishSeg24k, the Table 2 baseline comparison is unfair.","rationale":"The reader's weakest assumption is annotation consistency; I agree that no inter-annotator agreement is reported, but the annotation rule is a dataset-wide property that would affect all methods, leaving rankings partially meaningful. The reference policy provenance is a direct confound in the method comparison: the paper's own ablation shows that the RL components contribute only 0.18 mIoU over the MoE-only variant, while the MoE variant contributes 3.03 mIoU. If the MoE variant was initialised from a DishSeg24k-pretrained Mask2Former, the baseline is not comparable. The manuscript is ambiguous enough (π_ref 'pretrained', 'frozen') that this must be resolved by code inspection. I therefore recommend keeping the CONDITIONAL verdict but adding as primary condition the disclosure of π_ref's pretraining data and initialization, and a controlled re-run of the baseline with the same initialization.","tokens_in":14267,"tokens_out":7340,"duration_ms":77735,"concrete_test":"Train the Table 2 Mask2Former baseline using FEAST's exact decoder initialization (i.e., the same π_ref checkpoint) and the same 80k-iteration protocol. If this 'baseline' reaches ≈53.43 mIoU (or improves beyond statistical significance), the +3.21% gain is due to pretraining, not the proposed MoE/RL design. If no improvement appears, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Eq. 6 (Sec. 4.4) anchors the FEAST policy to a 'frozen pretrained Mask2Former decoder π_ref,' and Sec. 5.1 repeats that π_ref is 'the frozen pretrained Mask2Former decoder.' The paper never states what this checkpoint was pretrained on. If π_ref was pretrained on DishSeg24k (or even COCO) and FEAST's decoder is initialized from it, then FEAST receives an additional pretraining stage that the Table 2 Mask2Former baseline does not receive; Sec. 5.1 says only 'All competing methods use official code with the same backbone and training protocol,' implying that baseline was trained from scratch on DishSeg24k. The ablation (Table 3) attributes +3.03 mIoU to the MoE decoder alone, but if the MoE row is initialized from such a checkpoint, that gain is largely initialization, not architecture. The RL components add only +0.18 mIoU over MoE-only (Table 3 D vs B), so the headline +3.21% rests on this unverified confound. This is more directly fatal to the SOTA claim than annotation noise, which would affect all methods roughly symmetrically; the reference-policy asymmetry can fully explain the gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DishSeg24k, a large-scale dish-level food segmentation benchmark with 24,096 images, 112,281 instances, and 278 categories, plus a proposed method FEAST that formulates query-based decoding as an MDP with an RL-guided Mixture-of-Experts decoder. The authors report that FEAST outperforms Mask2Former by +3.21% mIoU on DishSeg24k with a ResNet-50 backbone and generalizes to FoodSeg103 (+1.47% mIoU). The central claim is that the stochastic expert-decoding mechanism is responsible for these gains, and that the new benchmark captures dense overlap, fine-grained similarity, and long-tail distributions.","tokens_in":14602,"tokens_out":2785,"duration_ms":27102,"significance":"If the benchmark is of high annotation quality and the method's gains are real, DishSeg24k would be a valuable resource for food segmentation research, and the RL-guided MoE decoder could be a useful architectural alternative. The paper includes a large-scale dataset with detailed statistics, extensive comparisons, code/model release planned, and a fairly complete ablation study. However, the significance is tempered by two unresolved issues: the marginal contribution of the RL components themselves (Table 4: R5 vs R0 is only +0.18% mIoU) and the undisclosed pretraining of the reference policy, which could confound the headline SOTA comparison.","major_comments":[{"comment":"The reference policy π_ref is described only as 'the frozen pretrained Mask2Former decoder,' but the pretraining data is never stated. If π_ref was pretrained on DishSeg24k (or another food segmentation dataset), then FEAST receives a pretraining advantage that the Mask2Former baseline in Table 2 does not receive, since the baselines are 'trained from scratch' under the stated protocol. This would directly confound the +3.21% mIoU headline gain and the +3.03% MoE ablation gain in Table 3 (B vs A). Please disclose the pretraining data for π_ref and rerun the Mask2Former baseline initialized from the same checkpoint, or otherwise demonstrate that the comparison is fair.","section":"Sec. 5.1, Eq. (6)"},{"comment":"The task reward in Eq. (4) is computed from the same ground-truth masks and the same final-layer bipartite matching used by the segmentation loss L_seg, making it closer to per-layer deep supervision than to an independent RL signal. The claimed central mechanism—RL-guided stochastic decoding—is weakly supported: Table 4 shows that each individual RL component (R1, R2, R3, R4) degrades performance below the MoE-only reference, and the full RL stack (R5) adds only +0.18% mIoU over MoE-only (R0). The paper should either provide direct evidence of the RL mechanism's effect (e.g., reward curves, query trajectory diversity, or a comparison against a non-RL stochastic MoE baseline) or temper the claim that RL decoding is the driving contribution.","section":"Sec. 4.4, Eq. (4), Table 4"},{"comment":"The annotation rule that 'overlapping ingredients are assigned unified masks based on dominant dish identity' presumes that a dominant-dish label is well-defined for every pixel in dense overlap. No inter-annotator agreement or label-noise study is reported for this critical rule. If the rule is applied inconsistently, the small metric differences between methods (including the +0.18% mIoU from the RL components) could be within annotation noise. Please report a label-quality study on a subset (e.g., pixel-level agreement on overlapping regions) to justify the benchmark's use for fine-grained method comparison.","section":"Sec. 3.2, Annotation Workflow"},{"comment":"Table 4's mDice and mAcc columns appear to be swapped relative to Table 3 (Row B: 53.25 mIoU, 60.48 mDice, 60.68 mAcc, whereas Table 4 R0 shows 60.68 mDice and 60.48 mAcc). Please verify the reported numbers; if the columns are swapped, the ablation conclusions about mDice/mAcc need to be corrected.","section":"Sec. 5.4, Table 4"}],"minor_comments":[{"comment":"FoodSeg103 is described as having 104 categories in Sec. 5.5 but 103 in the abstract and elsewhere; please standardize the count.","section":"Abstract / Sec. 5.1"},{"comment":"Typo: 'Doupled rewards' should be 'Decoupled rewards'.","section":"Fig. 4 caption"},{"comment":"CCNet appears twice with different reference numbers ([15] and [37]); the original CCNet is [15], while [37] is the FoodSeg103 benchmark paper. Please disambiguate.","section":"Table 5"},{"comment":"The 'UECFoodPixCompomplete' entry has a typo; should be 'UECFoodPixComplete'.","section":"Table 1"},{"comment":"The MDP is defined as a tuple (S,A,P,R,γ), but no transition probability distribution is specified for stochastic policies; since the transition is deterministic, this should be stated explicitly.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful benchmark contribution and a plausible architecture, but the central SOTA claim hinges on the undefined reference-policy pretraining. The good news is that this is fixable—disclose the checkpoint and rerun baselines under matched initialization. The RL mechanism is also more cosmetic than load-bearing in the current ablation; the authors should either provide cleaner evidence or adjust the framing. I do not see a reason to reject, but the revision must address the confound and the annotation-quality question before the claims can be trusted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nRead this one for the dataset, not for the RL machinery. DishSeg24k — 24k images, 112k instance masks, 278 dish classes, 4.66 instances per image — is a genuine step up in scale and density for dish-level segmentation. The re-annotation of ZSFood plus new images under a unified taxonomy is valuable, assuming the masks are clean and the overlap rule doesn't inject too much label noise. If the data actually ships, it becomes a standard benchmark for food computing.\n\nWhat the paper does well: the ablations are unusually honest. Table 3 shows the MoE decoder gives +3.03 mIoU over the baseline, while stochastic queries and the RL objective add +0.15 combined. Table 4 is even clearer: every RL component alone hurts, and only the full stack gives +0.18 over the MoE-only reference. The abstract and introduction oversell the RL story relative to those numbers, but the data is presented in a way you can check.\n\nThe larger problem is the reference policy. The paper calls it a \"frozen pretrained Mask2Former decoder\" but never says where that checkpoint came from. If it was pretrained on COCO or — worse — on DishSeg24k itself, then FEAST's decoder starts with weights the Table 2 Mask2Former baseline does not get, and the headline +3.21% is partly initialization, not architecture. The Table 3 ablation attributing +3.03 to the MoE decoder has the same issue if that row uses the same pretrained init. This is not necessarily deliberate cheating; it's incomplete reporting, and it is a load-bearing unknown. The annotation-noise concern from the reader is legitimate but secondary, since it affects methods roughly symmetrically.\n\nAlso missing: error bars or significance tests, inter-annotator agreement numbers, and a clear statement of which experiments use the pretrained decoder. All fixable, but they need to be fixed before the SOTA claim is credible.\n\nWho is this for? People building food segmentation benchmarks and anyone working on MoE decoders. The dataset is the contribution. The method is worth a look but not a revelation. I would not take the +3.21% at face value until the pretraining question is answered.\n\nRecommendation: send it to peer review. A serious referee can force the pretraining disclosure and get the dataset released in a verifiable form. That's where the value lies.","headline":"The dish dataset is a real contribution; the RL-decoder story is mostly MoE, and the headline +3.21% needs an honest accounting of the pretrained reference policy before it can be trusted.","tokens_in":15119,"tokens_out":3559,"would_cite":true,"duration_ms":39682,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FEAST, a reinforcement-learning-guided segmentation decoder, sets a new state of the art on a novel 24k-image dish benchmark, beating Mask2Former by 3.21 mIoU.","keywords":["food segmentation","benchmark","instance segmentation","long-tail distribution","mixture-of-experts","Markov decision process","reinforcement learning","DishSeg24k"],"falsifier":"Take a random subset of DishSeg24k test images (e.g., 500) and have a second set of annotators re-label them under the same protocol; compute pixel-level agreement. If agreement is significantly lower in overlapping regions than in non-overlapping regions, or if the mIoU ranking of methods changes when evaluated on a high-agreement subset, then the reported differences between methods may be dominated by annotation noise rather than segmentation quality.","tokens_in":14124,"feed_emoji":"🍲","tokens_out":3680,"duration_ms":37037,"temperature":0.7,"pith_summary":"This paper introduces DishSeg24k, a large-scale benchmark for dish-level food segmentation with 24,096 real-world dining images, 112,281 annotated instances across 278 fine-grained categories, built around dense inter-dish overlap, fine-grained class similarity, and a long-tail distribution. The authors argue that existing benchmarks oversimplify real dining scenes and that query-based segmentation decoders are too deterministic. They propose FEAST, which models query refinement as a Markov Decision Process where each decoder layer applies a continuous 'semantic shift' sampled from a policy, routed through a mixture of shared and specialized experts, and trained with two decoupled critics (task and structure rewards). On DishSeg24k, FEAST with ResNet-50 reaches 53.43% mIoU, outperforming the strongest baseline Mask2Former by +3.21% mIoU, and it also improves FoodSeg103 by +1.47% mIoU. If the reported numbers are accurate, FEAST is the best-performing food segmentation method on both benchmarks, and DishSeg24k becomes the most demanding public testbed for dish-level segmentation.","feed_headline":"RL decoder beats Mask2Former by 3.2 mIoU on dish segmentation","feed_subtitle":"A new 24k-image benchmark with 112k annotations and stochastic expert decoding sets the state of the art on real canteen scenes.","key_machinery":"The key mechanism is the stochastic MoE decoder: object queries are updated via Gaussian-sampled semantic shifts in a residual transition, routed through one shared expert plus E_r routed experts with a stop-gradient router, and trained with dual critics (task and structure) via policy gradient. A KL penalty anchors the learned policy to a frozen pretrained Mask2Former decoder to prevent catastrophic deviation, while the structure reward (routing entropy minus a load-balancing KL) prevents expert collapse under the long-tail distribution.","core_discovery":"The central claim is that modeling query-based mask decoding as a sequential stochastic decision process—rather than deterministic layer-by-layer refinement—improves segmentation in densely overlapping, long-tailed food scenes. FEAST treats each decoder layer as an action that shifts object queries in embedding space (q_{k+1}=q_k+a_k), with the action sampled from a Gaussian distribution produced by an MoE policy network. Two independent reward signals guide the training: a task reward measuring incremental mask and classification improvement, and a structure reward encouraging balanced expert routing. The authors show that the MoE decoder alone gives the largest gain (+3.03% mIoU over the M","pith_inferences":["The benchmark's annotation rule for overlapping ingredients ('unified masks based on dominant dish identity') presumes a single correct label for every pixel; without inter-annotator agreement statistics, the reported mIoU differences between methods could partly reflect label noise rather than pure segmentation ability.","At inference, FEAST sets the action to its mean (a_k = mu_k), making the decoder deterministic; the stochastic exploration benefits are realized only during training. An extension would test whether sampling multiple actions at test time and ensembling the resulting masks improves boundary accuracy in overlap regions.","A direct diagnostic experiment would be to re-annotate a random subset of DishSeg24k test images by a second set of annotators and compute pixel-level agreement; if agreement is low in overlapping regions, the method ranking may shift when evaluated on a high-agreement subset.","The dataset's additionally mentioned ingredient-level annotations could enable a transfer-learning test: pretraining on dish-level masks and then evaluating on ingredient-level benchmarks (or vice versa) would quantify how well the dish-level representation supports finer-grained understanding."],"forward_implications":["If the reported gains are accurate, FEAST establishes a new state of the art for food segmentation, beating Mask2Former by +3.21 mIoU on DishSeg24k and +1.47 mIoU on FoodSeg103.","DishSeg24k becomes the largest dish-level benchmark to date (24,096 images, 112,281 instances, 278 categories, 4.66 instances per image), offering a more demanding testbed for food segmentation and downstream tasks like volume estimation and dietary assessment.","The RL-guided MoE decoding framework is general: it can be applied to other segmentation tasks that exhibit dense overlap and long-tail distributions, not just food.","The decoupled dual-reward design—separating mask-quality improvement from routing-balance—provides a recipe for stabilizing MoE training under imbalanced data and may generalize to other long-tail recognition problems."],"fun_headline_variants":["AI dish segmentation jumps 3.2 mIoU with RL expert routing","New benchmark: 24k dish photos, 112k masks, RL beats Mask2Former","Stochastic decoder lifts food segmentation by 3.2 mIoU","FEAST: RL-guided MoE wins on DishSeg24k, +3.2% mIoU","DishSeg24k: 278 food categories, RL experts score SOTA"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"All method comparisons on DishSeg24k inherit the annotation rule that overlapping ingredients are assigned to the dominant dish, which presumes a single well-defined label for every pixel in dense overlap; no inter-annotator agreement is reported to verify that this rule is applied consistently.","fun_headline_variants_meta":{"raw":{"variants":["AI dish segmentation jumps 3.2 mIoU with RL expert routing","New benchmark: 24k dish photos, 112k masks, RL beats Mask2Former","Stochastic decoder lifts food segmentation by 3.2 mIoU","FEAST: RL-guided MoE wins on DishSeg24k, +3.2% mIoU","DishSeg24k: 278 food categories, RL experts score SOTA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000657,"raw_usage":{"total_tokens":2885,"prompt_tokens":826,"completion_tokens":2059,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":1956}},"tokens_in":570,"tokens_out":2059,"duration_ms":12909,"temperature":1.0,"reasoning_tokens":1956,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T03:40:55.595730+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random subset of DishSeg24k test images (e.g., 500) and have a second set of annotators re-label them under the same protocol; compute pixel-level agreement. If agreement is significantly lower in overlapping regions than in non-overlapping regions, or if the mIoU ranking of methods changes when evaluated on a high-agreement subset, then the reported differences between methods may be dominated by annotation noise rather than segmentation quality.","supporting_citations":[],"review_version":1}