{"id":"901d421a-b56c-4c78-b2a5-d4f24030fec1","arxiv_id":"2608.01827","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multimodal search agent fine-tuned on event-graph-synthesized trajectories with intermediate visual dependencies improves long-horizon visual search across ten benchmarks.","lead":"This paper trains an AI search agent that can find new images during a long search, inspect them, and use what it sees to decide what to search next. It shows that supervised fine-tuning on synthetic questions with mid-search visual clues produces large gains over the same model without such training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed vision-in-the-loop skill is never measured on the trained policy; Fig. 3 uses Doubao-2.0-Pro rollouts on synthetic data, so transfer and contamination remain open.","rationale":"The reader's weakest assumption—that guaranteed-by-construction visual necessity on synthetic data transfers to natural benchmarks—is the same load-bearing point I identify. My reading sharpens it: the paper's only behavioral evidence (Fig. 3) is generated by a teacher model on the training distribution, not by the fine-tuned policy on evaluation benchmarks, so the mechanism of vision-in-the-loop is never directly observed. The self-authored BrowseComp-V3 overlap concern further undercuts the strongest individual gain. These are not internal inconsistencies, but they are unverified external-validity assumptions. The paper's ablations, especially the cumulative +7K VIL data gain over a 20K multi-source baseline, are real evidence for the value of the synthetic data, which is why I would not reject or mark unverdictable. CONDITIONAL remains the appropriate verdict: the central claim is plausible and partly supported, but independent confirmation requires the model-level counterfactual test above and an overlap audit.","tokens_in":20525,"tokens_out":5727,"duration_ms":70440,"concrete_test":"On 200 sampled queries from BrowseComp-V3 and VisBrowse-Bench, run the released DeepVoyager-VL-30B-A3B checkpoint twice: (i) normal environment, and (ii) an environment where FETCHIMAGE and CROPIMAGE return only the URL/caption (no pixel observations, no image tokens). Keep all other tools, judge, and turn budget identical. If success rates differ by <5 points or the model issues no fewer subsequent IMAGESEARCH/TEXTSEARCH calls, Eqs. 2–3 are not actually operative at inference and the central claim fails; if the drop is large and later actions change, the transfer concern is substantially answered.","verdict_should_be":"UNCHANGED","load_bearing_attack":"DeepVoyager-VL's central claim is that SFT on structure-before-language trajectories instills a general vision-in-the-loop search skill. The decisive evidence would be trajectories of the fine-tuned policy on held-out benchmarks showing that a visual observation causes a later retrieval action (Eq. 2) and is necessary for the answer (Eq. 3). Section 4.3's 'Trajectory Analysis' does not provide this: Figure 3 reports tool-call composition and horizon for Doubao-2.0-Pro rollouts on EventVoyage-VL training questions and three public datasets, not for the trained DeepVoyager-VL model on evaluation benchmarks. The benchmark results (Table 1) are accuracy-only, so a model that imitates longer, more image-tool-heavy trajectories after generic SFT could produce these numbers without any vision-in-the-loop dependency. This is compounded by the absence of any overlap analysis between the Wikipedia/news event-graph training source and the self-authored BrowseComp-V3 benchmark: the +23.7 gain there could reflect distributional similarity rather than a transferred skill. The counterfactual condition (Eq. 3) is enforced only during synthesis (Sec. 3.2), never checked on the ten evaluation distributions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DeepVoyager-VL, a long-horizon multimodal deep-search agent trained by supervised fine-tuning on synthetic 'vision-in-the-loop' trajectories. The authors construct EventVoyage-VL, a structure-before-language pipeline built on a multimodal event graph, in which intermediate visual evidence is made necessary for later retrieval actions (Eqs. 2–3, 7–8). They then define an agent framework that separates image discovery from active visual acquisition and report large same-backbone gains on ten multimodal search benchmarks, with ablations isolating the contribution of the 7K VIL trajectories and of four framework components. The central claim is that SFT alone on synthetic trajectories with intermediate visual dependencies can induce long-horizon, vision-in-the-loop search behavior, without reinforcement learning.","tokens_in":20920,"tokens_out":5073,"duration_ms":59571,"significance":"If the central claim holds, the paper makes a substantial contribution: it would show that supervised trajectory fine-tuning on structure-before-language synthetic data is a viable alternative to RL for training multimodal deep-search agents. The same-backbone cumulative data ablation (Table 3) is a genuine strength, as is the explicit synthesis-time enforcement of counterfactual visual necessity (Eq. 3). The framework ablation (Table 4) and the careful separation of difficulty estimation from teacher-trajectory generation are also well designed. The main weakness is that the paper never demonstrates the claimed vision-in-the-loop behavior for the trained policy; the trajectory analysis uses rollouts of a different model, and the counterfactual property is never verified on evaluation benchmarks. The self-authored BrowseComp-V3 benchmark and the absence of released artifacts further limit the strength of the quantitative claims.","major_comments":[{"comment":"The trajectory analysis that allegedly evidences vision-in-the-loop behavior is conducted on rollouts of Doubao-2.0-Pro over EventVoyage-VL and three public datasets, not on trajectories of the fine-tuned DeepVoyager-VL policy on evaluation benchmarks. Table 1 reports accuracy only; a policy that simply imitates longer, image-tool-heavy trajectories after generic SFT could produce these gains without any intermediate visual dependency satisfying Eq. (2) or counterfactual necessity Eq. (3). Please report and annotate trajectories of the trained 8B/30B policies on held-out benchmarks, including at least one case where a visual observation at turn t resolves a variable required by a later retrieval action at t'.","section":"Section 4.3, Figure 3"},{"comment":"Counterfactual visual necessity is enforced during synthetic program construction, but it is never verified on any evaluation benchmark. Eq. (3) is a property of the task/program, not of the trained policy; without checking it on evaluation queries, the central transfer claim—that the agent learned a general vision-in-the-loop skill rather than matching the synthesized distribution—remains an assumption. Provide a counterfactual analysis on evaluation benchmarks (e.g., remove intermediate image evidence and show that answers become underdetermined or incorrect) or otherwise demonstrate that evaluation trajectories contain intermediate visual dependencies under Eqs. (2)–(3).","section":"Section 3.2, Eqs. (3), (7), (8)"},{"comment":"BrowseComp-V3 is a benchmark authored by the same group (Zhang et al., 2026a), and the paper reports the largest gain there (+23.7 for the 30B-A3B model). No overlap analysis is given between the Wikipedia/news event-graph training source (Sec. 3.2) and BrowseComp-V3's queries and images. This gain could partly reflect distributional similarity to the training distribution rather than a transferred vision-in-the-loop skill. Please quantify query/image/topic overlap between EventVoyage-VL training data and all evaluation benchmarks, and report results separately with and without the self-authored benchmark.","section":"Section 4.1, Table 1"},{"comment":"All results are single runs judged by Qwen3.6-35B-A3B, with no error bars, no multiple seeds, and no released code, data, or checkpoints. The same-backbone ablations in Tables 3–4 are load-bearing for the paper's quantitative claims, but single numbers for one run cannot establish the size of the +5.4/+6.5-point VIL-data effects or the component ablations. Please provide multiple seeds or per-item scores for bootstrap confidence intervals, and release the synthetic trajectories and trained checkpoints, or clearly label the results as preliminary.","section":"Section 4.1, Appendix C.2"}],"minor_comments":[{"comment":"The Avg. column appears to average over different benchmark subsets for different baselines. Please state the subset over which each average is computed, or remove the column to avoid incomparable aggregates.","section":"Table 1"},{"comment":"Qwen3.6-35B-A3B is used both as the ablated base model and as the LLM judge. Potential judge-family bias should be discussed or controlled, especially for the framework ablation.","section":"Table 4 / Section 4.1"},{"comment":"Define U_{L\\setminus{\\ell}} explicitly. The set-minus notation can be misread as removing an element from the universe rather than dropping a constraint from the constraint set L.","section":"Eq. (8)"},{"comment":"The relative improvements '+55.2% and +44.0%' are hard to interpret; report absolute point gains alongside, as is done earlier in the paper.","section":"Conclusion"},{"comment":"The limitations section is candid about scope, latency, and memory, but it does not mention the unverified transfer of the counterfactual property to evaluation benchmarks. This should be added as a limitation.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The main reason for major revision, rather than rejection, is that the paper's central claim is defensible and the same-backbone cumulative ablation is a strong piece of evidence. The decisive missing piece is behavioral evidence from the trained policy on held-out benchmarks, plus an overlap analysis for the self-authored benchmark. If those can be supplied, I would be willing to revisit my recommendation. I do not see evidence of deliberate circularity, but the BrowseComp-V3 issue needs to be addressed head-on rather than by relying on external-benchmark coverage alone."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real contribution: the structure-before-language event-graph synthesis (Eqs. 7-8) and the discovery/observation decoupling are genuinely new relative to the entity-substitution, front-loaded, and answer-side-grafted paradigms it positions against (Sec. 2.3, Fig. 1). Second, the paper's central mechanistic claim—that SFT on these trajectories induces a general vision-in-the-loop skill—is not actually measured. The trajectory analysis in Sec. 4.3 (Fig. 3) uses Doubao-2.0-Pro rollouts on EventVoyage-VL training questions, not the fine-tuned DeepVoyager-VL policy on held-out benchmarks. So we have accuracy gains plus an inference about the mechanism, not direct evidence of it.\n\nWhere the paper earns credit: same-backbone ablations (Tables 3-4) are well designed. The cumulative design—20K multi-source trajectories first, then +7K VIL trajectories—cleanly isolates the incremental value of the synthesized data, and the gains (5.4/6.5 points) are consistent across both model scales. The component ablation is also informative. The paper is candid about scope in Sec. D.1. The training details are thorough. This is not a paper that hides its moving parts.\n\nWhere the soft spots are, in proportion: (1) The mechanism gap above is the main one. The benchmark results are all accuracy-only, so a model that imitates longer, image-tool-heavy trajectories could produce these numbers without any actual visual dependency. The counterfactual necessity condition (Eq. 3) is enforced during synthesis but never checked on the ten evaluation distributions. I'd like to see trajectories of the trained policy on held-out benchmarks, coded for whether an image observation caused a later retrieval action. (2) BrowseComp-V3 is self-authored by this group, and there's no overlap analysis with the Wikipedia/news event-graph training source. That's a real concern, though the gains are large on other external benchmarks too (e.g., VisBrowse-Bench +30.7, BC-VL +22.6), so it doesn't look like the result hinges on that one benchmark. (3) No artifacts released, single runs, LLM judge without human agreement, many baselines self-reported. These are common in this area but they do make independent confirmation impossible.\n\nVerdict: the empirical contribution is solid enough to deserve a serious referee. The paper would benefit from either releasing code/data/checkpoints or adding a direct trajectory analysis of the trained policy. The core finding—SFT on structured multimodal trajectories gives large gains over a strong multi-source baseline—holds up; the interpretation that this is specifically a vision-in-the-loop skill is plausible but unproven.\n\nRecommendation: send to peer review. The reviewers should focus on the mechanism evidence and the self-authored benchmark overlap, not on the basics of the ablation design.","headline":"Genuinely new data-synthesis pipeline and strong same-backbone ablations, but the vision-in-the-loop mechanism is inferred from accuracy gains, not directly measured on the trained policy.","tokens_in":21351,"tokens_out":3187,"would_cite":false,"duration_ms":32424,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Supervised fine-tuning on trajectories with intermediate visual dependencies turns open multimodal agents into long-horizon vision-in-the-loop searchers, without any reinforcement learning.","keywords":["vision-in-the-loop search","multimodal deep search","long-horizon agents","supervised fine-tuning","multimodal event graph","active visual acquisition","trajectory distillation","visual tool use"],"falsifier":"Measure the same SFT recipe on a benchmark where a counterfactual check shows each question's answer is still determinable without the image discovered mid-search; if performance stays at baseline levels, the claimed skill is not vision-in-the-loop. Simpler: replace the 7K VIL trajectories with 7K similarly long trajectories from public multimodal search data matched for tool-call composition; if the +5.4/+6.5 gains vanish, the result is data-distributional rather than structural.","tokens_in":20488,"feed_emoji":"🔍","tokens_out":4221,"duration_ms":42306,"temperature":0.7,"pith_summary":"The paper proposes DeepVoyager-VL, a framework for multimodal deep search in which newly discovered images actively drive later retrieval steps. It claims that the right training data is enough: supervised fine-tuning on about 7K synthesized vision-in-the-loop trajectories adds 5.4 and 6.5 points over a strong 20K multi-source baseline for 8B and 30B models, and lifts the 30B model's ten-benchmark average from 40.7 to 58.6. The method constructs questions from a multimodal event graph so that intermediate visual evidence is necessary by construction, then trains an agent that discovers, selectively fetches, crops, and reuses images over up to 50 turns. If correct, the result means long-horizon multimodal search skill can be distilled from structure-before-language synthetic data without RL, and that open models can approach proprietary agentic baselines.","feed_headline":"SFT alone teaches long-horizon vision search, no RL needed","feed_subtitle":"Synthetic event-graph trajectories with intermediate visual steps lift an open 30B model to 58.6 average across ten benchmarks.","key_machinery":"The load-bearing mechanism is the structure-before-language synthesis pipeline, EventVoyage-VL: a multimodal event graph with entities, atomic events, macro-events, and images; subgraph extraction composes deterministic inference programs with visual interfaces, and only then are they verbalized into questions. Vision-in-the-loop is defined by two conditions: a visual observation resolves a variable needed by a later action (Eq. 2), and the same answer is not determined without retrieved images (Eq. 3). The agent side uses active visual acquisition—discovery tools returning URL–caption references, FETCHIMAGE/CROPIMAGE as the only paths for visual tokens into policy context, and goal-conditio","core_discovery":"The central claim is that vision-in-the-loop search—where an image found mid-search resolves an intermediate variable and determines the next retrieval action—can be reliably induced by trajectory supervision rather than reinforcement learning. DeepVoyager-VL synthesizes problems with this property by composing inference programs over a visually enriched multimodal event graph, enforcing that at least one internal interface is resolved by visual evidence and that every principal constraint is necessary. The agent separates image discoverability from observability: search returns lightweight references; FETCHIMAGE and CROPIMAGE materialize only selected visual evidence. On ten multimodal sear","pith_inferences":["The paper's strongest untested assumption is transfer: visual necessity is guaranteed in synthesized data by construction but never verified on evaluation benchmarks; a natural test would compare models trained on VIL trajectories against matched-length text-only or random-image trajectories on benchmarks filtered by Eq. 3.","If the transfer claim holds, the same structure-before-language pipeline could be repurposed for other modalities—audio, video, structured data—where an intermediate evidence type must trigger continued search.","The reported concentration of gains on BrowseComp-V3 is suggestive but confounded by possible source overlap between the Wikipedia/news event graph and self-authored benchmarks; an external, independently authored benchmark would sharpen the conclusion.","A complementary RL stage could push performance further; the paper frames SFT-only as sufficient, but nothing rules out RL as an add-on for even longer horizons."],"forward_implications":["SFT alone on synthetic vision-in-the-loop trajectories is a viable substitute for RL when training multimodal deep-search agents.","Gains concentrate on long-horizon, visually demanding benchmarks, indicating the data teaches the intended loop rather than generic tool use.","The framework's benefit transfers across three base models, so the recipe may generalize beyond the specific backbones tested.","Open 30B-scale models can come within 6–9 points of proprietary agentic systems on the same benchmark suite.","The ablations imply that context management, image search, and on-demand loading and cropping are complementary rather than redundant."],"fun_headline_variants":["Synthetic trajectories teach agents to search with vision mid-thought","Vision-in-the-loop search without reinforcement learning","Data synthesis turns long-horizon visual search into SFT","Image-driven reasoning: SFT beats RL for open-world search","Multimodal event graph trains agents to use visuals mid-search"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that tasks whose intermediate visual dependencies are guaranteed by construction transfer to naturally occurring multimodal search problems, so the agent learns a general vision-in-the-loop skill rather than matching the distribution of the synthesized programs.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic trajectories teach agents to search with vision mid-thought","Vision-in-the-loop search without reinforcement learning","Data synthesis turns long-horizon visual search into SFT","Image-driven reasoning: SFT beats RL for open-world search","Multimodal event graph trains agents to use visuals mid-search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000886,"raw_usage":{"total_tokens":3651,"prompt_tokens":722,"completion_tokens":2929,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":466,"completion_tokens_details":{"reasoning_tokens":2847}},"tokens_in":466,"tokens_out":2929,"duration_ms":24221,"temperature":1.0,"reasoning_tokens":2847,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T20:07:45.903359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the same SFT recipe on a benchmark where a counterfactual check shows each question's answer is still determinable without the image discovered mid-search; if performance stays at baseline levels, the claimed skill is not vision-in-the-loop. Simpler: replace the 7K VIL trajectories with 7K similarly long trajectories from public multimodal search data matched for tool-call composition; if the +5.4/+6.5 gains vanish, the result is data-distributional rather than structural.","supporting_citations":[],"review_version":1}