{"id":"52073f12-898f-46c1-ba15-72ec26f5eeff","arxiv_id":"2501.08982","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A text-conditioned diffusion model with 3D Gaussian splatting refinement estimates 6DoF camera pose distributions in city-scale scenes, beating a Monte Carlo dropout baseline on five datasets.","lead":"CityLoc turns a text description, such as \"a street with a traffic light\", into a set of likely camera positions in a large 3D city scene. It couples a text-conditioned diffusion model with CLIP features and a 3D Gaussian scene, then refines candidate poses by comparing rendered views with the text.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported Gaussian-refinement gains may be an artifact of reconstructing the scene from the same views used for test poses; Appendix B never states that validation/test images were excluded from Gaussian training.","rationale":"The reader's weakest_assumption identified the same load-bearing concern: the Gaussian scene may have been built from the same images used for validation/test poses, making the refinement stage circular. My stress-test confirms that Appendix B is the exact place where this ambiguity lives and that it undermines the strongest claimed contribution. I considered whether the weaker issue of comparing only against Monte Carlo dropout is more fundamental, but that is an evidence-strength concern, whereas the Gaussian contamination, if real, invalidates the specific numerical improvement that the paper highlights. The proposed test is cheap and decisive: hold out the validation/test views from Gaussian training and re-run Table 1. Because this concern is the same as the reader's and the appropriate disposition is already CONDITIONAL (require the split check plus the previously noted missing baselines, code, and error bars), I do not move the verdict. If the test fails, the verdict should move to REJECT; if it passes, the remaining issues still warrant conditional acceptance rather than unconditional acceptance.","tokens_in":17654,"tokens_out":6478,"duration_ms":70382,"concrete_test":"Rebuild each Gaussian scene using only the 90% diffusion-training views, excluding all 10% validation/test poses from 3D Gaussian Splatting training; then rerun Algorithm 2 and recompute Table 1. If the 'Ours Refined' RDA drops to or below the 'Ours Mixup' values, the claimed refinement benefit is an artifact of view memorization. As a secondary check, render the held-out validation pose from the retrained Gaussian and measure pixel/CLIP distance to the original test image; near-zero distance would confirm that the original construction memorized the test view.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim of consistent outperformance rests substantially on the 'Ours Refined' row in Table 1, where Algorithm 2 renders the Gaussian scene at a candidate pose and maximizes CLIP similarity between the render and the input text. That input text is derived from a specific test image, and the text-to-pose pairs are built from images. Appendix B says: 'For each dataset, we first use the training views from regarding dataset to construct the Gaussian splats. Subsequently, we randomly select 10% of the poses as the validation set, while the remaining 90% are used to train the diffusion model.' This does not say that the 10% validation/test poses were held out from the Gaussian reconstruction. If a test image was used to build the 3D Gaussian scene, then rendering at that test pose can reproduce the exact test view; maximizing CLIP similarity can then succeed by memorizing the view rather than by semantic localization. The refinement improvements in Table 1 (e.g., Town k=15: 22.62 to 26.37; Street k=15: 7.96 to 15.11) are therefore not interpretable. This is a structural flaw in the evaluation, not a disagreement about baselines, and it directly affects the 'Gaussian refinement' contribution and the headline claim of superiority. The diffusion-only rows are less affected, but the final method and the paper's stated contribution 3 depend on the refinement, so the main claim is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CityLoc, a diffusion-based method for 6DoF camera pose distribution estimation conditioned on text descriptions in large-scale scenes. The method trains a DDPM-style transformer to denoise noisy poses conditioned on CLIP text/image embeddings, uses a mixup training strategy to combine textual and visual modalities, and refines sampled poses by rendering from a 3D Gaussian splatting scene and maximizing CLIP similarity between the rendered image and the text. The evaluation is conducted on five urban datasets (Small Town, UrbanScene3D Residence and SciArt, MatrixCity Street and Aerial) with a new metric RDA, and the full pipeline is reported to outperform a Monte Carlo Dropout baseline. The paper also includes a user study and qualitative visual results.","tokens_in":18007,"tokens_out":4112,"duration_ms":38495,"significance":"If the central claims hold, the paper would be a meaningful contribution to text-based localization in city-scale scenes, introducing distributional pose output and a refinement stage using 3D Gaussian rendering. The constructed pose-text pairs and the multi-granularity captioning procedure could be a useful resource for the community. The user study and the breadth of test scenes are also strengths. However, the evaluation is compromised by a potential leakage of test views into the Gaussian scene, and the baseline comparison is much weaker than the abstract claims. With the leakage resolved and stronger baselines, the contribution could be significant; as presented, the evidence does not fully support the headline claims.","major_comments":[{"comment":"The text states: 'For each dataset, we first use the training views from regarding dataset to construct the Gaussian splats. Subsequently, we randomly select 10% of the poses as the validation set, while the remaining 90% are used to train the diffusion model.' This does not assert that the validation/test poses were excluded from the Gaussian reconstruction. Since Algorithm 2 renders the Gaussian scene at the candidate pose and optimizes CLIP similarity to the text, if the test images were used to build the Gaussian, the rendered view can reproduce the test image, and the refinement improvements in Table 1 (e.g., Town k=15 from 22.62 to 26.37; Street k=15 from 7.96 to 15.11) could be due to view memorization rather than semantic localization. This is a load-bearing issue for contribution 3 and for the paper's claim of superior performance. The authors must clarify whether the validation/test poses were held out from Gaussian training and, if they were not, re-run the refinement evaluation with strictly held-out views.","section":"Appendix B, Gaussian Training"},{"comment":"The only distributional baseline reported is Monte Carlo Dropout, while Text2Loc, Text2Pos, and RET are mentioned in the related work but no quantitative comparison is given. The abstract's claim of 'consistent outperformance' against standard distribution estimation methods is therefore not supported by the presented evidence. The authors should either adapt these published methods to the same task and report their RDA numbers, or justify concretely why they cannot be applied. Additionally, no error bars or multiple-seed results are reported, so the reader cannot assess the variability of the RDA values.","section":"§4.2, Experimental Results"},{"comment":"The RDA metric in Fig. 6 and Eq. (5) is a ratio of the accuracy of the predicted distribution to the accuracy of a random distribution. The denominator is not reported anywhere, which makes the RDA values hard to interpret; for instance, a high RDA could arise from an extremely low random-accuracy baseline. The paper should report absolute metrics such as median translation/rotation error and success rate within a specified distance threshold, in addition to RDA, so that the reader can judge the absolute quality of the estimated pose distributions.","section":"§4.1, Evaluation Metrics"}],"minor_comments":[{"comment":"The line 'Random Sample embeddings: Tmix ←...' is a sentence fragment; please rephrase, for instance as 'Randomly sample the embedding Tmix from {Tt, Tf} with swap ratio β.'","section":"Algorithm 1, line 4"},{"comment":"The caption states 'Best results per row are highlighted in red'; in grayscale print red is indistinguishable from black, so please also define the best value explicitly or use a distinct formatting, and clarify in the text what 'Ours' versus 'Ours Mixup' versus 'Ours Refined' denote.","section":"Table 1 caption"},{"comment":"Two consecutive paragraphs both begin with 'By minimizing the mean squared error between the predicted and ground truth poses...' and are almost identical; they should be merged to avoid duplication.","section":"§3.1, paragraph after Multi-model Conditioning"},{"comment":"The phrase 'Translation is measured in units of 10% of the scene scale' is imprecise; please specify how the scene scale is computed and how the 10% normalization is applied.","section":"Fig. 6 caption"},{"comment":"The text says 'in 30% cases the performance of rendered images exceeds that of the ground-truth (GT) images,' but Table 3 reports match rates of 56.5% (GT) and 50.8% (Ours); the derivation of the 30% figure from the table is unclear and should be explained.","section":"§4.3, User Studies"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and timely problem, and the proposed pipeline is technically coherent. The main risk is the evaluation: the Gaussian refinement may be evaluated under data leakage, and the baseline set is too narrow. These are fixable with additional experiments and clarification, so I recommend major revision rather than rejection. I would also encourage the authors to release the code and data at the revision stage, since the current version only promises future release."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: the paper's strongest claim—consistent gains from the Gaussian refinement—rests on an evaluation setup that probably has test views inside the 3D Gaussian scene. Appendix B says they first build the splats from the dataset's training views, then randomly select 10% of poses as validation. Nothing states that the validation/test images were excluded from the splat construction. If they were included, refining a pose by rendering from a scene built with that pose's own image is template matching, not semantic localization. The diffusion-only rows are less affected, but the 'Refined' row and contribution 3 depend on this. The main claim is not yet supported.\n\nWhat is genuinely new: the task of predicting a distribution of 6DoF poses from text at city scale, using a text-conditioned DDPM with a mixup over CLIP text/image embeddings. That is a reasonable combination of existing parts, and the five large-scale scenes with multi-granularity captions are a useful resource. The qualitative results show sensible behavior—broad descriptions spread poses along the highway, specific descriptions concentrate—so the idea has merit.\n\nThe soft spots beyond the leakage: the only baseline is Monte Carlo Dropout on a simple pose regressor. Text2Loc, Text2Pos, and RET are cited but never evaluated, which makes 'superior to baseline' a low bar. There are no error bars or repeated seeds, and the RDA metric is new with no statistical characterization. The user study has 10 samples and a GT match rate of 56.5% vs 50.8%, which is weak evidence. The refinement thresholds tau1 and tau2 are fixed without sensitivity analysis.\n\nThis paper deserves a serious referee because the task is worth exploring and the architecture is plausible. But the referee should demand a re-run with Gaussian scenes built only from the 90% training split, proper baselines with their actual numbers, and variance reporting. As it stands, I would not cite the quantitative results, though the problem setup might be worth a citation once the evaluation is fixed.","headline":"The Gaussian-refinement gains likely come from test views memorized in the trained splat; the diffusion-based task formulation is reasonable but the evaluation is too weak to support the headline claims.","tokens_in":18490,"tokens_out":2387,"would_cite":false,"duration_ms":24445,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion turns text descriptions into distributions of camera poses for whole cities","keywords":["6DoF pose estimation","text-based localization","pose distribution","diffusion model","3D Gaussian splatting","CLIP","large-scale scenes","multimodal localization"],"falsifier":"Take one of the five datasets, rebuild the hierarchical 3D Gaussian scene using only the 90% of poses used to train the diffusion model, then evaluate refinement on the held-out 10%; if the RDA gain from refinement disappears or shrinks sharply, the reported improvement depends on the splat having memorized the test views.","tokens_in":17494,"feed_emoji":"📍","tokens_out":4694,"duration_ms":46849,"temperature":0.7,"pith_summary":"CityLoc claims that ambiguous text descriptions of places, such as \"a street with a traffic light and a crosswalk,\" should be localized as a distribution of camera poses rather than a single pose, because many locations can satisfy the same description. To do this, the paper trains a diffusion model conditioned on CLIP text and image embeddings to turn a noisy 6DoF pose sample into plausible poses, and then refines candidates by rendering them through a hierarchical 3D Gaussian scene and maximizing the cosine similarity between the rendered image's CLIP features and the text's CLIP features. The paper reports that this pipeline beats Monte Carlo dropout, its distributional baseline, on five large-scale urban and suburban datasets, and that including more detailed text narrows the predicted pose distribution toward the ground-truth location. If correct, the method would let robots or language models navigate by finding all places that match a natural-language description instead of requiring a precise query.","feed_headline":"Diffusion maps a text description to a spread of camera poses","feed_subtitle":"CityLoc beats Monte Carlo dropout across five large-scale scenes, and more detailed text tightens the predicted locations.","key_machinery":"The central object is a conditional denoising diffusion model over SE(3) poses: a transformer denoiser $F_\\theta$ takes a noisy pose, diffusion time step, and CLIP text or image embedding, and predicts the clean pose, trained with the DDPM denoising loss. The paper couples this with a mixup training algorithm that randomly swaps text and image embeddings in each batch, so the same pose predictor accepts either modality. The second mechanism is a Gaussian-refinement loop: a hierarchical 3D Gaussian splatting scene $G$ renders an image $\\hat{I}=G(P)$ at a candidate pose, a CLIP image encoder produces $\\hat{T}_f$, and the pose is optimized to maximize $\\hat{T}_f^\\top T_t/(\\|\\hat{T}_f\\|\\|T_t\\|)$; samples below thresholds $\\tau_1,\\tau_2$ are rejected. That coupling is what the paper claims turns coarse text-conditioned pose samples into accurate, text-consistent camera poses.","core_discovery":"On the paper's own terms, the central discovery is that text-conditioned pose distribution estimation is feasible at city scale: a diffusion model can map CLIP embeddings of descriptions to 6DoF camera pose samples whose density concentrates near locations matching the text, and the concentration improves when the text is more specific. The paper further claims that its Gaussian-splatting refinement step, which renders each coarse pose, encodes the rendered view with CLIP, and backpropagates the negative text-image cosine similarity into the pose, systematically filters out mismatched samples and increases RDA scores on most of the five datasets. A user study of rendered versus ground-truth images is reported as showing that the refined renders match text descriptions at a rate comparable to real ground-truth images.","pith_inferences":["A natural extension the paper does not pursue is zero-shot localization in a scene whose Gaussian map was built from different images; the refinement step's gain would then test whether it generalizes rather than recalls.","The CLIP-similarity refinement could be applied to any differentiable scene renderer, so the same distribution-to-pose alignment might transfer to mesh-based or other radiance-field maps.","Because the generated captions are used both to train and to cue the diffusion model, caption diversity, not just granularity, is a hidden variable; measuring RDA against manually written descriptions would separate caption quality from pose-learning quality."],"forward_implications":["A text description can return a full set of candidate poses, so an agent can plan visits to every location that matches \"traffic light with zebra crossing\" rather than being forced to pick one.","The reported granularity results imply that the quality of the caption generator directly limits localization; stronger vision-language models should translate into tighter pose distributions.","The mixup training implies the same pipeline works when only an image, only text, or both are available as the query.","The Gaussian refinement requires a prebuilt 3D Gaussian scene, so the method presupposes a prior reconstruction of the environment; it localizes within, not across, known scenes.","The RDA metric establishes a way to compare distributional pose estimators beyond single-pose error."],"supporting_citations":[{"why":"Supplies the CLIP text and image encoders that link descriptions to pose conditioning and drive the refinement similarity score.","marker":"Radford et al., 2021"},{"why":"Defines the DDPM denoising objective and sampling procedure used to model the conditional pose distribution.","marker":"Ho et al., 2020"},{"why":"Provides the hierarchical 3D Gaussian representation that renders candidate poses in large-scale scenes during refinement.","marker":"Kerbl et al., 2024"},{"why":"PoseDiffusion is the diffusion-aided pose estimation approach this work adapts and extends to text-conditioned distributions.","marker":"Wang et al., 2023b"},{"why":"Text2Loc serves as a text-to-point-cloud localization baseline that depends on detailed text and is compared against.","marker":"Xia et al., 2024"},{"why":"Text2Pos is a text-based localization baseline compared against, motivating the shift from single poses to distributions.","marker":"Kolmet et al., 2022"},{"why":"LLaVA generates the multi-granularity captions used to build the pose-text training pairs.","marker":"Liu et al., 2023c"},{"why":"iCOMA supplies the render-and-compare refinement procedure that the Gaussian refinement algorithm builds on.","marker":"Sun et al., 2023"},{"why":"MatrixCity provides two of the five large-scale evaluation scenes, including the street-level and aerial city data.","marker":"Li et al., 2023c"}],"fun_headline_variants":["CityLoc maps text to camera pose distributions at city scale","Diffusion turns text descriptions into 6DoF pose spreads","Text-conditioned poses with Gaussian splatting refinement","From text to camera poses via diffusion and CLIP at scale"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Gaussian scene used for refinement is built only from training views, so matching a rendered test pose to the text is genuine visual generalization rather than recall of an image already stored in the splat.","fun_headline_variants_meta":{"raw":{"variants":["CityLoc maps text to camera pose distributions at city scale","Diffusion turns text descriptions into 6DoF pose spreads","Text-conditioned poses with Gaussian splatting refinement","From text to camera poses via diffusion and CLIP at scale"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2604,"prompt_tokens":856,"completion_tokens":1748,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":1680}},"tokens_in":472,"tokens_out":1748,"duration_ms":13811,"temperature":1.0,"reasoning_tokens":1680,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:12:26.701211+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one of the five datasets, rebuild the hierarchical 3D Gaussian scene using only the 90% of poses used to train the diffusion model, then evaluate refinement on the held-out 10%; if the RDA gain from refinement disappears or shrinks sharply, the reported improvement depends on the splat having memorized the test views.","supporting_citations":[],"review_version":1}