{"id":"528ededd-c05e-45b0-81d5-55b403289ffe","arxiv_id":"2608.08596","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"VideoNIG tests whether multimodal models can turn tour videos into executable navigation instructions, and a two-stage curriculum with preference optimization improves their spatial reasoning.","lead":"This paper introduces VideoNIG, a benchmark where models must write step-by-step navigation instructions from an ego-centric tour video, a starting view, and a goal text or image. A curriculum-based training method improves instruction quality and lets a separate navigation agent follow the generated instructions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MPO is trained on the same distractor taxonomy used in Choice Evaluation, so the headline diagnostic gains may reflect training/evaluation overlap rather than improved spatial reasoning.","rationale":"The reader's weakest assumption (route coverage in Sec. 3.2) is a real limitation for generalization, but it is an explicit design choice of the benchmark. The more load-bearing risk is that the paper's headline diagnostic evidence is contaminated: MPO preference pairs and the Choice Evaluation distractors are drawn from the same five-category taxonomy, and GRPO explicitly optimizes the same Rouge-L/SPICE metrics reported as results. That does not make the paper worthless. The navigation-execution results in Tab. 3 show MPO improves SR on Gold Route (R2R 56.1 vs 40.3; RxR 37.5 vs 29.2) and are not directly optimized by the same rewards, so the executability claim has partial independent support. The paper also has no released code, data, or error bars, which further limits verification of any quantitative claim. The correct disposition is therefore conditional: the empirical central claim should be accepted only if the Choice-Evaluation gains survive decontamination via a held-out distractor-type test, and ideally with released artifacts. This does not move the reader's conditionality, but it shifts the binding condition from route coverage to evaluation validity.","tokens_in":27334,"tokens_out":6720,"duration_ms":74187,"concrete_test":"Hold out one distractor type from MPO training: train MPO with only four of the five negative types (e.g., deletion, addition, change, swap) and then evaluate on the held-out direction/entity-modification subset of Choice Evaluation. If MPO's accuracy on the held-out type falls to the SFT baseline while its accuracy on the trained types stays high, the Choice-Evaluation gains are a training/evaluation overlap artifact rather than improved spatial reasoning.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's key evidence that the curriculum improves spatial reasoning is Choice Evaluation, but the MPO training and this evaluation share the same negative-instruction taxonomy. Section 5.1 states that MPO 'employs preference-based learning using negative samples constructed via our evaluation protocol (see Fig. 4), where one negative sample is randomly selected per instance.' Appendix C.2 confirms that the five negative types are exactly the five Multiple-choice distractor types in Fig. 4 (direction, entity, deletion, addition, change, swap), generated by qwen3-vl-plus. Section 3.4 then evaluates Multiple-choice and Orthogonal-choice with the same five types (Orthogonal-choice isolates direction/entity modifications). The MPO model is therefore optimized to reject the same kinds of distractors on which it is scored. Its large Choice-Evaluation advantage over SFT (Tab. 2: MPO-lora averages around 60.6/62.0 vs 49.8/53.1) may reflect memorized heuristics for these distractor categories rather than general video-grounded spatial reasoning. This matters because Sec. 5.2 uses Choice Evaluation as the diagnostic that 'better reflects practical navigation performance' and as the main evidence that stronger MLLMs struggle while the proposed method helps. The GRPO text-similarity gains are also partly circular (Eq. 2 rewards f_SPICE and f_Rouge-L, the same metrics in Tab. 2), though the authors do acknowledge this. Navigation execution (Tab. 3) provides independent support that MPO instructions are more executable, but it does not validate the diagnostic metrics themselves.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces VideoNIG, a task in which a model must generate step-by-step navigation instructions from an ego-centric tour video, an initial observation, and a textual or visual goal, without using maps or graphs. The authors instantiate this task in a Habitat simulator benchmark built from R2R-CE and RxR-CE, with four video types (Gold Route, Tyro Tour, Curiosity Tour, Explorer Tour) of increasing complexity, together with multimodal start-goal descriptions. They propose a two-stage Curriculum Learning framework consisting of Action Warmup (action-view alignment) followed by Complexity Progression (training on progressively harder tours), and they train Qwen3-VL-8B with SFT, GRPO, and MPO. Evaluation uses Rouge-L/SPICE text similarity, Multiple-choice and Orthogonal-choice spatial-consistency tests, and downstream navigation execution with InternVLA-N1. The central empirical claims are that existing MLLMs struggle on VideoNIG and that the proposed curriculum training improves instruction quality across the diagnostic metrics.","tokens_in":27656,"tokens_out":6569,"duration_ms":64891,"significance":"If the results hold, VideoNIG would be a useful controlled benchmark for video-grounded spatial reasoning in navigation, and the curriculum training pipeline would be a reasonable reference method. Strengths include the large-scale constructed benchmark (about 60K tour videos), the graded difficulty design, the combination of three evaluation perspectives, and the explicit acknowledgment of simulator-only settings and of the route-coverage assumption. The navigation execution results provide at least partially independent evidence that instructions produced by the MPO-trained model are more executable, especially on Gold Route. However, the two text-based evidence channels are partially coupled to the training objectives (GRPO rewards the same text metrics; MPO negatives share the same distractor taxonomy as the choice tests), so the magnitude of the claimed improvements over baselines is not yet established in a fully unbiased way.","major_comments":[{"comment":"The MPO training and the Choice Evaluation share the same five distractor types: direction, entity, deletion, addition, change, and swap. Appendix C.2 states that MPO's rejected instructions are generated by qwen3-vl-plus using these five types, and Sec. 3.4 defines Multiple-choice and Orthogonal-choice with exactly the same categories. Tab. 2 then shows MPO-lora improving average Choice-Evaluation accuracy to roughly 60.6/62.0 from SFT's 49.8/53.1. Because Sec. 5.2 explicitly relies on Choice Evaluation as the diagnostic that 'better reflects practical navigation performance' and as the main evidence that the curriculum helps, these gains may reflect overfitting to the evaluation taxonomy rather than general video-grounded spatial reasoning. Please add a held-out evaluation with distractor types not used in MPO training (or generated by a different procedure) and report per-type accuracies.","section":"Sec. 5.1, App. C.2 vs Sec. 3.4"},{"comment":"The GRPO reward in Eq. (2) includes fSPICE and fRouge-L, and Tab. 2 reports SPICE and Rouge-L as the text-similarity evidence of improvement. Since the optimization directly maximizes these metrics, the higher GRPO text-similarity scores (e.g., average Rouge-L 0.254 vs SFT 0.233) are at least partly by construction. This is acknowledged in Sec. 5.2, but the paper still summarizes the results as 'significantly improves instruction quality across complementary diagnostic metrics' (Abstract). Please either present text-similarity scores as training objectives rather than independent evaluation evidence, or ablate the similarity terms from the reward and show that improvements persist.","section":"Eq. (2), Tab. 2"},{"comment":"The navigation execution results that are cleanly independent of the training/evaluation overlaps support MPO mainly on Gold Route (e.g., R2R SR 56.1 vs 40.3 for System2 baseline), while improvements on Tyro, Curiosity, and Explorer tours are small (e.g., 33.2 vs 31.0 on Tyro, 21.2 vs 21.0 on Curiosity, and 20.4 vs 18.7 on Explorer). The paper does not report variance estimates or significance tests, and it is unclear how many episodes underlie each number. Please provide standard errors or confidence intervals and state the number of evaluation episodes.","section":"Tab. 3"}],"minor_comments":[{"comment":"The text says 'We render four types of videos' but the reader sees three tour types plus the Gold Route; consider clarifying 'four video types, including the Gold Route and three tour variants'.","section":"Sec. 3.2"},{"comment":"Table 2 is extremely wide and difficult to parse because text-similarity and choice-evaluation numbers are interleaved; splitting it into separate tables for text metrics and choice metrics (or reordering columns) would improve readability.","section":"Tab. 2"},{"comment":"Proprietary models are evaluated on only 100 randomly sampled instances per configuration; the choice-evaluation results for these models in Tab. 7 are therefore noisy. Please describe the sampling procedure and ideally report confidence intervals or a larger sample.","section":"App. D.1"},{"comment":"The term 'circuitousness' is nonstandard; consider using 'detour ratio' or defining it more explicitly as the route-to-Euclidean distance ratio.","section":"Sec. 3.3"},{"comment":"The claim that 'the GRPO-trained model achieves higher Rouge-L and SPICE scores' is stated relative to SFT, but on some rows MPO has higher Rouge-L (e.g., Gold Route R2R image goal); the sentence should be reworded to avoid ambiguity about which comparison is meant.","section":"Sec. 5.2"}],"recommendation":"major_revision","confidential_remarks":"The training/evaluation overlap on the choice-based diagnostics is the most serious issue and needs to be addressed with a held-out distractor set. The paper is otherwise competent and the benchmark may be useful to the community, but the current evidence for the central claim of improved spatial reasoning is not fully convincing. The route-coverage assumption is stated transparently and can be treated as a scope restriction rather than a defect."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take on 2608.08596: the task definition is genuinely new and the benchmark construction is solid work; however, the headline choice-based gains for the MPO method are inflated by training on the same distractor taxonomy used for evaluation, and the GRPO text-similarity gains are partly circular by design (the reward includes the metrics). The underlying empirical pattern — MLLMs struggle with this task and curriculum training helps — survives those caveats, but the reported effect sizes should be read with caution.\n\nWhat's actually new: VideoNIG is the first formulation I've seen that generates step-by-step navigation instructions from ego-centric tour videos plus a goal, with no maps or graphs. That distinguishes it from GoViG (start-goal only) and Mobility VLA (topological graphs). The benchmark itself is careful: four graded video types, nDTW filtering of non-optimal routes, outlier detection on instruction/trajectory alignment, and non-discriminative start/goal filtering. The three-pronged evaluation (text similarity, choice tests, downstream execution) is the right instinct, and the ablation showing Action Warmup helps is useful.\n\nWhere it's soft. Eq. (2) rewards SPICE and Rouge-L, and Tab. 2 reports those same metrics as evidence; the authors openly say this, so it's not hidden, but it means the GRPO text gains are not independent confirmation. More serious: the MPO negative samples are constructed from exactly the five distractor types used in Choice Evaluation (Sec. C.2 and Fig. 4). The model is literally trained to reject those categories and then scored on those categories. Orthogonal-choice isolates direction/entity, and navigation execution (Tab. 3) gives some independent signal — MPO does improve Success Rate on Gold Route — so the executability claim has legs. But on Tyro/Curiosity/Explorer tours the execution gains are small or negligible, so 'curriculum helps' is really only strongly supported on the easiest video type. The route-coverage assumption (tours always cover the optimal path) is stated but load-bearing; this benchmark tests subpath selection within a route-covering video, not planning under incomplete priors. Finally, no released code or data and no error bars, which limits verification.\n\nWho it's for: VLN and MLLM researchers working on instruction generation or video spatial reasoning. It deserves a serious referee — the benchmark will be useful even if the method results are weaker than claimed. I'd push the authors to release the artifacts and to add a held-out distractor taxonomy for Choice Evaluation, or at least show that MPO gains persist on distractor types it never saw.\n\nRecommendation: send to peer review, conditional on code/data release and a de-overlapped evaluation.","headline":"VideoNIG is a genuinely new benchmark task with a careful construction pipeline, but the headline MPO gains on Choice Evaluation are partly by construction because training and evaluation share the same distractor taxonomy.","tokens_in":28221,"tokens_out":2873,"would_cite":true,"duration_ms":30650,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper attempts to show that ego-centric tour videos, an initial observation, and a goal description are enough to generate executable navigation instructions, without maps or graphs.","keywords":["navigation instruction generation","vision-and-language navigation","multimodal large language models","curriculum learning","ego-centric tour video","spatial reasoning","benchmark","Habitat simulator"],"falsifier":"Take an Explorer Tour sample, remove the frames that cover one middle corridor of the Gold Route, and ask the trained model to produce instructions for that corridor; if the model still names objects in the unseen segment, it is relying on memorized layout priors rather than video grounding, and if it fails, the benchmark's route-coverage assumption is load-bearing for its results.","tokens_in":27135,"feed_emoji":"🧭","tokens_out":3580,"duration_ms":40102,"temperature":0.7,"pith_summary":"The paper introduces VideoNIG, a goal-oriented navigation instruction generation task where a model must produce step-by-step instructions from an ego-centric tour video, a single initial observation, and a textual or image goal, with no map or graph available. It instantiates the task in a controlled Habitat-based benchmark with roughly 60K tour videos and 37K multimodal prompts, and evaluates outputs through text similarity, distractor-based choice tests, and downstream navigation execution by a VLN agent. The central claim is that this video-grounded formulation is viable but challenging, and that current multimodal LLMs struggle with it, particularly when the tour does not simply follow the optimal route. The paper further argues that a two-stage curriculum, first teaching action-view alignment and then increasing trajectory complexity, substantially improves instruction quality and executability.","feed_headline":"Tour videos alone can generate navigation instructions to any goal","feed_subtitle":"A two-stage curriculum lets an 8B vision-language model stitch the right subpath from a noisy ego-centric tour and execute it.","key_machinery":"The central mechanism is the two-stage Curriculum Learning framework. The first stage, Action Warmup, uses paired tasks, action-from-view and view-from-action, to align the model's spatial representations with discrete ego-centric motions before it encounters long narratives. The second stage, Complexity Progression, orders training data from Gold Route and Tyro Tour videos through Curiosity Tour to the more deviating Explorer Tour videos, so the model gradually handles longer and more exploratory trajectories. The benchmark's tour construction, which guarantees each tour covers the Gold Route's viewpoints while adding start/goal extensions, look-arounds, and neighborhood visits, is what makes the task well-posed.","core_discovery":"VideoNIG defines a task in which the environmental prior is a continuous ego-centric video rather than a graph, map, or landmark set, and it shows that a multimodal LLM can be trained to plan the optimal subpath inside that video and verbalize it as navigation instructions. The benchmark covers three tour complexities, from simple endpoint extensions to tours with substantial detours, and the paper's experiments show that performance degrades predictably as tour complexity rises. The proposed two-stage curriculum, Action Warmup followed by Complexity Progression, is demonstrated to improve spatial grounding on choice-based tests and to raise navigation success when the generated instructions are executed by a VLN agent; for example, on R2R Gold Route the MPO-trained model raises Success Rate from 40.3 to 56.1 in the System2 setting. The paper also claims that choice-based spatial consistency metrics correlate more strongly with downstream navigation success than traditional text similarity metrics.","pith_inferences":["If the central claim holds, a practical device could pre-record a single walk-through of an indoor space and later answer arbitrary goal queries with turn-by-turn instructions, without ever building a semantic map.","The choice-based metrics may transfer to other instruction-generation domains where an output is judged by whether a downstream policy can execute it, not by surface similarity to a reference.","The route-coverage assumption is the key boundary: extending VideoNIG to naturally collected tour videos that may miss parts of the optimal path would likely expose hallucination and would be a direct stress test of the task's grounding claim.","One testable extension would be to replace the tour-type curriculum with a continuous difficulty measure, such as circuitousness or detour ratio, which might yield smoother learning and better generalization than the discrete Gold/Tyro/Curiosity/Explorer ordering."],"forward_implications":["If the task formulation is correct, goal-oriented navigation instruction generation can be decoupled from explicit environmental representations, since richly recorded videos can serve as the spatial prior.","Tour-video complexity, especially route deviation and exploratory behavior, is a scaling bottleneck for current MLLMs, so progress on VideoNIG should track with better long-horizon video grounding.","Choice-based spatial consistency tests, such as the proposed Multiple-choice and Orthogonal-choice evaluations, are better predictors of instruction executability than Rouge-L and SPICE, making them useful diagnostic tools for future instruction-generation research.","Training that first reinforces local action-view alignment and then increases trajectory complexity yields stable gains across text similarity, choice accuracy, and downstream VLN success, suggesting that curriculum order matters for spatial reasoning tasks."],"supporting_citations":[{"why":"R2R-CE and RxR-CE provide the human-annotated routes and instructions that the VideoNIG benchmark renders into tour videos.","marker":"[4,29,30]"},{"why":"Room-Across-Room supplies the multilingual instruction data and route statistics that define R2R and RxR difficulty distributions.","marker":"[30]"},{"why":"GoViG is the endpoint-only goal-conditioned NIG baseline that VideoNIG extends by inserting tour-video priors.","marker":"[63]"},{"why":"Mobility VLA is the closest prior work using tour videos and user queries, but it relies on topological graphs, which VideoNIG explicitly avoids.","marker":"[14]"},{"why":"VSI-Bench supplies the general video spatial-intelligence benchmark used to test whether VideoNIG performance correlates with broader visual-spatial reasoning.","marker":"[64]"},{"why":"NavBench inspires the choice-based evaluation design that measures spatial consistency through multiple-choice and orthogonal-choice distractors.","marker":"[41]"},{"why":"GRPO is the reinforcement-learning optimizer used in the Action Warmup stage and the Complexity Progression stage.","marker":"[45]"},{"why":"MPO is the preference-optimization method that constructs negative samples from the choice-evaluation distractors and improves discriminative quality.","marker":"[54]"},{"why":"SPICE and Rouge-L are the text similarity metrics used to measure surface-level agreement with human-annotated instructions.","marker":"[3,33]"},{"why":"InternVLA-N1 and its DualVLN configuration serve as the downstream VLN agent that executes the generated instructions in the navigation execution evaluation.","marker":"[48,60]"}],"fun_headline_variants":["Tour video priors enable goal-oriented navigation instructions","Two-stage curriculum turns ego-centric videos into navigation prompts","VideoNIG: Plan subpaths in tour videos to reach any goal","From noisy tour videos to executable navigation instructions","8B vision-language model masters goal-oriented navigation from video"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every tour video is constructed so that it visually covers the full optimal route's viewpoints, which means the correct instruction can in principle be read out of the video; if a tour missed a necessary segment of the optimal path, the task could not be solved from the video alone.","fun_headline_variants_meta":{"raw":{"variants":["Tour video priors enable goal-oriented navigation instructions","Two-stage curriculum turns ego-centric videos into navigation prompts","VideoNIG: Plan subpaths in tour videos to reach any goal","From noisy tour videos to executable navigation instructions","8B vision-language model masters goal-oriented navigation from video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000255,"raw_usage":{"total_tokens":1599,"prompt_tokens":999,"completion_tokens":600,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":522}},"tokens_in":615,"tokens_out":600,"duration_ms":6331,"temperature":1.0,"reasoning_tokens":522,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:29:59.481815+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an Explorer Tour sample, remove the frames that cover one middle corridor of the Gold Route, and ask the trained model to produce instructions for that corridor; if the model still names objects in the unseen segment, it is relying on memorized layout priors rather than video grounding, and if it fails, the benchmark's route-coverage assumption is load-bearing for its results.","supporting_citations":[{"cited_title":"In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference","cited_arxiv_id":null,"evidence_quote":"VSI-Bench supplies the general video spatial-intelligence benchmark used to test whether VideoNIG performance correlates with broader visual-spatial reasoning."}],"review_version":1}