{"id":"e0412502-86cd-481e-8918-b3dc0bd30c8e","arxiv_id":"2607.17767","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A map-free, language-guided parking navigation system with short- and long-term memory beats adapted VLN/AD baselines on a new underground parking benchmark and in real vehicle trials.","lead":"The paper proposes VLN-AVP, a zero-shot system that uses a vision-language model plus a hybrid memory to navigate underground parking garages from natural-language instructions without pre-built maps. It also introduces a 10-scene, 1,000-episode benchmark and reports higher success rates than adapted VLN and driving baselines in simulation and on a real vehicle.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline improvement figures lack statistical reliability: no error bars, multiple seeds, or significance tests, so the reported gains over baselines may be within noise.","rationale":"The reader's verdict is CONDITIONAL, and I agree with that disposition, but my load-bearing concern differs: while the reader identifies MapTR's lane-centerline transfer as the weakest assumption, I find the absence of any uncertainty quantification more directly threatening to the central quantitative claims. The MapTR concern is genuine, but the experimental protocol partially controls for it by giving all methods the same BEV perception; if MapTR fails, all methods degrade similarly, so the relative improvement could still hold. In contrast, the reported improvements are point estimates from what appear to be single runs with a stochastic VLM. The real-world sample is only 30 trials, making the observed 10% difference between Ours and Ours w/o LTM fragile. The paper also does not release code, prompts, or thresholds, so independent verification is impossible. A concrete repeated-seed evaluation with confidence intervals and significance tests would settle whether the claimed improvements are real. If such an evaluation showed overlapping CIs or p>0.05, the central claim of consistent outperformance would fail. Until then, CONDITIONAL acceptance is appropriate: require the authors to provide this statistical evidence along with code and benchmark release.","tokens_in":10992,"tokens_out":7102,"duration_ms":88118,"concrete_test":"Run each method (VLN-AVP, MapGPT w/ BEV, ReasonNav, LaMPilot, DiLu) on a fixed set of at least 100 episodes from the VLN-AVP-Regular subset, repeated over 10 independent seeds (or with VLM temperature >0 to sample multiple responses per decision). Report mean SR with 95% bootstrap confidence intervals and perform a paired permutation test (or Wilcoxon signed-rank) between VLN-AVP and each baseline. If the confidence intervals overlap substantially or the p-value exceeds 0.05, the claimed improvements are not statistically reliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a quantitative performance comparison: 'over 25% improvement in success rate compared to VLN methods' and 'over 15% compared to other autonomous driving methods' (Abstract, Tables III and IV). The paper reports only single-point success rates (e.g., 65.2% vs 37.6% in Regular tasks) with no variance, confidence intervals, or significance testing. The system is stochastic: the VLM (Gemini-3-flash) can be sampled with nonzero temperature, object detection may vary, and trajectory generation involves randomness. A single run cannot rule out that the observed differences are due to run-to-run variation. In the real-world experiments, only 30 trials are reported; the difference between Ours (63.3% SR) and Ours w/o LTM (53.3%) is 3 episodes, which is not statistically meaningful, and the comparison with LaMPilot (36.7%) is based on a small sample. Without error bars or repeated evaluations, the universal 'improvement' claims are not supported. The reader's MapTR concern is relevant but is partly mitigated because all baselines are given the same BEV perception, so MapTR failures would affect all methods similarly; the statistical reliability issue directly undermines the comparative headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VLN-AVP, a zero-shot vision-language navigation framework for autonomous valet parking. The system replaces pre-built HD maps with MapTR BEV lane-centerline perception, uses a VLM (Gemini-3-flash) for high-level decisions, and adds a hybrid memory: a short-term perception memory based on Grounding DINO detections and a long-term topological memory with validity bits and graph-based retrieval. A hierarchical decision module selects among RS-curve candidate trajectories. The authors also introduce a 3DGS-based dataset of 10 underground garages with over 1,000 VLN episodes and report simulation and real-world experiments against VLN and AD baselines, including ablations. The headline claims are an over 25% SR improvement over VLN methods, over 15% over AD methods, and 63.3% SR in 30 real-world trials.","tokens_in":11375,"tokens_out":4410,"duration_ms":59239,"significance":"If substantiated, the paper would make a useful contribution: it is the first VLN benchmark for underground parking, it demonstrates a map-free zero-shot parking navigation pipeline with human-in-the-loop correction, and the hybrid memory reduces VLM API calls and cost. The system design is modular and mostly reproducible from existing components. However, the central empirical claim is a quantitative performance advantage, and the evidence currently consists of single-point estimates without variance, confidence intervals, significance tests, or full protocol details. The contribution is therefore promising but not yet established at the level claimed in the abstract and conclusions.","major_comments":[{"comment":"All reported success-related metrics are single-point estimates. No number of repeated runs/seeds, standard deviations, confidence intervals, or significance tests are given. The headline 'over 25% improvement' and 'over 15% improvement' are thus not statistically supported. In Table IV, for example, the difference between Ours (63.3%) and Ours w/o LTM (53.3%) on 30 trials is exactly 3 episodes, which is within plausible sampling noise. Please add repeated evaluations, per-episode counts, confidence intervals, and at least a basic significance test for the main comparisons.","section":"Abstract; Tables III, IV, V; Fig. 8"},{"comment":"The evaluation protocol is underspecified. Please state the number of episodes in the Regular and Challenging subsets, per-scene episode counts, the exact success criterion (e.g., stopping within the target bounding box, distance threshold), how instructions and targets are sampled, and whether all methods run on identical episode sets. In addition, the dataset is not currently released: Sec. V says the authors 'plan to further improve and publicly release' it, so the benchmark cannot yet be independently used by the community.","section":"Sec. IV-A, IV-B, V"},{"comment":"The method's entire action space T_t is generated from MapTR lane-centerline predictions in unseen underground garages. The paper provides no quantitative evaluation of MapTR on parking-garage scenes. If lane centerlines are sparse, disconnected, or geometrically wrong, the candidate trajectory set cannot contain a drivable path toward the requested target, and neither the VLM nor the memory modules can recover. Please report MapTR's lane extraction quality on the VLN-AVP scenes (e.g., centerline recall, trajectory reachability rate to target locations) and discuss how the system handles cases where no candidate trajectory leads to the target.","section":"Sec. III-A, Eq. (1)"},{"comment":"The ablations attribute the performance gains to the memory modules, but they inherit the statistical issue. Figure 8, which is the main evidence for LTM's 'learning ability', has no error bars or aggregation details (each point is a single episode? a window? a single scene?), and its axes are unlabeled. Moreover, the memory system depends on several free parameters introduced in Secs. III-B and III-D (K_STM, delta_theta_max, delta_t_max, theta_threshold) whose values are not reported and for which no sensitivity analysis is provided. Please add robustness experiments or justify why the chosen values are not the result of tuning that favors the proposed method.","section":"Sec. III-B, III-D, IV-E; Table VI; Fig. 8"}],"minor_comments":[{"comment":"Eq. (4) writes the prompt as [I_front, {I_detect,j}, S, g], but the STM frame defined in Sec. III-B is m = (I_detect, B, t, theta). Clarify which components of the STM frame are actually fed to the VLM.","section":"Sec. III-B, Eq. (4)"},{"comment":"The simulator is described as being built on the Real2Sim2Real simulator in REAP [32], which is an author preprint. Please specify whether the simulator code will be released, or provide enough implementation documentation for others to reproduce the benchmark.","section":"Sec. IV-B, Ref. [32]"},{"comment":"The 'Render Method' column lists '3DGS' for all datasets, which is ambiguous. Clarify how each existing dataset was rendered and whether the VLN-AVP scenes use 3DGS exclusively.","section":"Table II"},{"comment":"Add axis labels, units, and a legend to Figure 8. Also state how many episodes each plotted point aggregates and whether the curve is over a single scene or multiple scenes.","section":"Sec. IV-E, Fig. 8"},{"comment":"The real-world comparison includes only LaMPilot. State why other AD baselines were omitted, and describe the real vehicle's sensor setup and the source of ground-truth localization/target positions.","section":"Sec. IV-D, Table IV"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid system-level story and a potentially valuable benchmark, but the empirical evidence as presented is not yet at the level needed for the quantitative headline claims. The main risk is statistical: single-run numbers, no episode counts, no significance tests, and a real-world comparison sensitive to a few trials. I would like the authors to add proper evaluation statistics and protocol details, and ideally release the dataset/simulator. This is fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real contribution here is the benchmark and the architecture integration. A first VLN benchmark for underground parking with 10 garages and over 1000 episodes is useful, and the hybrid memory design — short-term perception memory plus long-term topological memory with direction-vector edges — is a sensible adaptation of VLN ideas to kinematically constrained vehicles. The baseline setup is fair: they feed all methods the same BEV perception and the same VLM, so the comparison isolates the decision-making and memory components. The ablations support the memory contributions, and the real-world case study with user feedback is a nice demonstration.\n\nThe soft spot is statistical. Every reported comparison is a single point estimate with no variance, confidence intervals, or significance tests. The headline 'over 25% improvement' is 65.2 vs 37.6 for one run of each method; the challenging subset is similar. The real-world experiment has 30 trials, and the difference between 'Ours' and 'Ours w/o LTM' is 63.3% vs 53.3%, i.e., 3 episodes — that is within noise. The paper would be much stronger with multiple seeds and variance reporting. This is an engineering publishable point, but the strong empirical claims are not yet supported.\n\nAlso, the 'zero-shot' label is a bit generous: LTM learns from past episodes within the same garage, so the system is not zero-shot in the classic sense, though it does not need pre-built maps. That is a minor labeling issue. The missing prompts and hyperparameters make reproduction hard, and the dataset is announced but not yet released; given the benchmark is half the value, that matters. The MapTR concern is real but mitigated because all methods share the same BEV backbone, so MapTR failures affect everyone equally; it caps absolute performance rather than invalidating the relative comparison.\n\nWho is this for? Anyone working on VLM-based navigation for vehicles, or on semantic memory for VLN. It deserves a serious referee — the benchmark and architecture are valuable enough that the statistical issues are addressable in revision. I would send it to peer review, not desk reject, but I would ask for error bars, more real-world trials, and the dataset release before accepting.","headline":"The benchmark and architecture are the real contributions; the headline performance numbers need error bars before they can be trusted.","tokens_in":11816,"tokens_out":2871,"would_cite":false,"duration_ms":32411,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that autonomous valet parking can be performed zero-shot in unseen underground garages by pairing a bird's-eye-view trajectory generator with a vision-language model that reads signs and follows natural-language instructio","keywords":["vision-language navigation","autonomous valet parking","zero-shot navigation","bird's-eye-view perception","long-term memory","short-term memory","underground garage benchmark","natural language instruction"],"falsifier":"Measure, across the Regular benchmark episodes, how often the generated candidate trajectory set contains at least one trajectory whose endpoint lies within the target area. If that frequency is materially below the reported 65.2% success rate, the bottleneck is BEV perception, not the VLM or memory modules; if the frequency is high but success is low, the bottleneck lies in decision-making.","tokens_in":10911,"feed_emoji":"🅿️","tokens_out":4663,"duration_ms":55520,"temperature":0.7,"pith_summary":"The paper tries to show that autonomous valet parking does not require pre-built maps or predefined slot numbers. By combining a high-frequency bird's-eye-view perception module that generates drivable candidate trajectories with a vision-language model that interprets signs and user commands, the system can navigate unfamiliar garages from natural-language instructions. To handle the VLM's low-frequency, single-frame reasoning, a short-term memory retains recent sign detections, while a long-term topological memory accumulates successful routes so repeated visits become faster and cheaper. The paper also contributes a simulation dataset of ten underground garages with over 1,000 navigation episodes, the first vision-language navigation benchmark for parking, and reports success rates of 65.2% on regular tasks, 45.4% on challenging tasks, and 63.3% in real-vehicle trials.","feed_headline":"Map-free valet parking hits 65% success in unfamiliar garages","feed_subtitle":"A vision-language model reads signs and user commands while a hybrid memory stabilizes decisions in unknown parking lots.","key_machinery":"The load-bearing mechanism is the hybrid long-short-term memory system feeding a hierarchical decision module. The short-term memory is a queue of recent sign detections with timestamps, bounding boxes, and vehicle yaw, filtered to keep only relevant, current, same-direction cues; it stabilizes VLM decisions when no sign is visible in the current frame. The long-term memory is a directed topological graph of junction nodes, landmark nodes, and direction-vector edges, updated by user feedback and reused on later episodes. The hierarchical decision module prefers a stored graph route when one exists, falls back on the cached VLM action otherwise, and maps the chosen action onto a kinematically","core_discovery":"The central claim is that the combination of kinematically filtered candidate trajectories from BEV lane perception with a low-frequency VLM decision-maker, plus two complementary memories, produces zero-shot vision-language navigation in underground parking garages. The VLM is not asked to plan in metric space; it chooses among a small set of drivable trajectories derived from predicted lane centerlines, and the memories let it act on cues absent from the current frame. The paper demonstrates that this architecture outperforms both vision-language navigation baselines and VLM-based driving baselines, that removing either memory degrades success, and that the long-term memory reduces VLM cal","pith_inferences":["The short-term-cue memory idea could transfer to other map-free indoor navigation settings, such as delivery robots or warehouse vehicles, where signage is the dominant navigation cue.","The long-term topological graph, stored as direction vectors rather than steering commands, is naturally shareable across a fleet of vehicles; pooled successful routes could form a collective map-free memory.","The directional filter that drops remembered signs when the vehicle turns assumes sign relevance is tied to yaw; multi-level or looped garages may require a position-aware or lane-aware filter.","If a cheaper or faster VLM were substituted, the memory components might partially compensate, and the challenging benchmark subset, with composite goals like 'EV charger near the exit', would directly stress-test that substitution."],"forward_implications":["Valet parking can be initiated in an unseen garage from a natural-language instruction such as 'park near the elevator' without a pre-built HD map.","A vehicle can safely follow a VLM's high-level commands by restricting its choices to kinematically feasible candidate trajectories, avoiding the frequent U-turns that make typical VLN policies impractical on real cars.","Repeated trips to the same garage build a lightweight topological graph that raises success rate and cuts VLM API calls and cost by about 75%.","The new benchmark, with ten photorealistic garage scenes and over 1,000 episodes, provides a standardized testbed for evaluating vision-language navigation in underground parking.","Caching recent sign detections in short-term memory lets the VLM make consistent decisions even when the current image contains no visible navigation cue."],"fun_headline_variants":["Zero-shot valet parking navigates unknown garages via vision-language model","First underground parking VLN benchmark with 1,000 episodes","Map-free valet navigation beats VLN by 25% in simulation","Hybrid memory boosts zero-shot parking navigation accuracy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole pipeline rests on the BEV lane-detection model, trained on open-road data, producing a correct lane centerline in an unfamiliar garage, because the candidate trajectories are generated from those lane lines and the VLM can only choose among them.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot valet parking navigates unknown garages via vision-language model","First underground parking VLN benchmark with 1,000 episodes","Map-free valet navigation beats VLN by 25% in simulation","Hybrid memory boosts zero-shot parking navigation accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000814,"raw_usage":{"total_tokens":3432,"prompt_tokens":799,"completion_tokens":2633,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":2561}},"tokens_in":543,"tokens_out":2633,"duration_ms":23514,"temperature":1.0,"reasoning_tokens":2561,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T17:03:04.019011+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, across the Regular benchmark episodes, how often the generated candidate trajectory set contains at least one trajectory whose endpoint lies within the target area. If that frequency is materially below the reported 65.2% success rate, the bottleneck is BEV perception, not the VLM or memory modules; if the frequency is high but success is low, the bottleneck lies in decision-making.","supporting_citations":[],"review_version":1}