Pith. sign in

REVIEW 3 major objections 5 minor 29 references

CityLoc: 6DoF Pose Distributional Localization for Text Descriptions in Large-Scale Scenes with Gaussian Representation

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Diffusion turns text descriptions into distributions of camera poses for whole cities

desk verdict The Gaussian-refinement gains likely come from test views memorized in the trained splat; the diffusion-based task formulation is reasonable but the evaluation is too weak to support the headline claims. read the letter →

arxiv 2501.08982 v3 pith:YKVVL2WY submitted 2025-01-15 cs.CV

Qi Ma , Runyi Yang , Bin Ren , Nicu Sebe , Ender Konukoglu , Luc Van Gool , Danda Pani Paudel This is my paper · ORCID
classification cs.CV
keywords 6DoFposeestimationtext-basedlocalizationdistributiondiffusionmodel3DGaussiansplattingCLIPlarge-scalescenesmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CityLoc claims that ambiguous text descriptions of places, such as "a street with a traffic light and a crosswalk," should be localized as a distribution of camera poses rather than a single pose, because many locations can satisfy the same description. To do this, the paper trains a diffusion model conditioned on CLIP text and image embeddings to turn a noisy 6DoF pose sample into plausible poses, and then refines candidates by rendering them through a hierarchical 3D Gaussian scene and maximizing the cosine similarity between the rendered image's CLIP features and the text's CLIP features. The paper reports that this pipeline beats Monte Carlo dropout, its distributional baseline, on five large-scale urban and suburban datasets, and that including more detailed text narrows the predicted pose distribution toward the ground-truth location. If correct, the method would let robots or language models navigate by finding all places that match a natural-language description instead of requiring a precise query.

What carries the argument

The central object is a conditional denoising diffusion model over SE(3) poses: a transformer denoiser $F_\theta$ takes a noisy pose, diffusion time step, and CLIP text or image embedding, and predicts the clean pose, trained with the DDPM denoising loss. The paper couples this with a mixup training algorithm that randomly swaps text and image embeddings in each batch, so the same pose predictor accepts either modality. The second mechanism is a Gaussian-refinement loop: a hierarchical 3D Gaussian splatting scene $G$ renders an image $\hat{I}=G(P)$ at a candidate pose, a CLIP image encoder produces $\hat{T}_f$, and the pose is optimized to maximize $\hat{T}_f^\top T_t/(\|\hat{T}_f\|\|T_t\|)$; samples below thresholds $\tau_1,\tau_2$ are rejected. That coupling is what the paper claims turns coarse text-conditioned pose samples into accurate, text-consistent camera poses.

What would settle it

Take one of the five datasets, rebuild the hierarchical 3D Gaussian scene using only the 90% of poses used to train the diffusion model, then evaluate refinement on the held-out 10%; if the RDA gain from refinement disappears or shrinks sharply, the reported improvement depends on the splat having memorized the test views.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that text-conditioned pose distribution estimation is feasible at city scale: a diffusion model can map CLIP embeddings of descriptions to 6DoF camera pose samples whose density concentrates near locations matching the text, and the concentration improves when the text is more specific. The paper further claims that its Gaussian-splatting refinement step, which renders each coarse pose, encodes the rendered view with CLIP, and backpropagates the negative text-image cosine similarity into the pose, systematically filters out mismatched samples and increases RDA scores on most of the five datasets. A user study of rendered versus ground-truth images is reported as showing that the refined renders match text descriptions at a rate comparable to real ground-truth images.

Load-bearing premise

The load-bearing premise is that the Gaussian scene used for refinement is built only from training views, so matching a rendered test pose to the text is genuine visual generalization rather than recall of an image already stored in the splat.

Editorial extensions

If this is right

  • A text description can return a full set of candidate poses, so an agent can plan visits to every location that matches "traffic light with zebra crossing" rather than being forced to pick one.
  • The reported granularity results imply that the quality of the caption generator directly limits localization; stronger vision-language models should translate into tighter pose distributions.
  • The mixup training implies the same pipeline works when only an image, only text, or both are available as the query.
  • The Gaussian refinement requires a prebuilt 3D Gaussian scene, so the method presupposes a prior reconstruction of the environment; it localizes within, not across, known scenes.
  • The RDA metric establishes a way to compare distributional pose estimators beyond single-pose error.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is zero-shot localization in a scene whose Gaussian map was built from different images; the refinement step's gain would then test whether it generalizes rather than recalls.
  • The CLIP-similarity refinement could be applied to any differentiable scene renderer, so the same distribution-to-pose alignment might transfer to mesh-based or other radiance-field maps.
  • Because the generated captions are used both to train and to cue the diffusion model, caption diversity, not just granularity, is a hidden variable; measuring RDA against manually written descriptions would separate caption quality from pose-learning quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CityLoc, a diffusion-based method for 6DoF camera pose distribution estimation conditioned on text descriptions in large-scale scenes. The method trains a DDPM-style transformer to denoise noisy poses conditioned on CLIP text/image embeddings, uses a mixup training strategy to combine textual and visual modalities, and refines sampled poses by rendering from a 3D Gaussian splatting scene and maximizing CLIP similarity between the rendered image and the text. The evaluation is conducted on five urban datasets (Small Town, UrbanScene3D Residence and SciArt, MatrixCity Street and Aerial) with a new metric RDA, and the full pipeline is reported to outperform a Monte Carlo Dropout baseline. The paper also includes a user study and qualitative visual results.

Significance. If the central claims hold, the paper would be a meaningful contribution to text-based localization in city-scale scenes, introducing distributional pose output and a refinement stage using 3D Gaussian rendering. The constructed pose-text pairs and the multi-granularity captioning procedure could be a useful resource for the community. The user study and the breadth of test scenes are also strengths. However, the evaluation is compromised by a potential leakage of test views into the Gaussian scene, and the baseline comparison is much weaker than the abstract claims. With the leakage resolved and stronger baselines, the contribution could be significant; as presented, the evidence does not fully support the headline claims.

major comments (3)
  1. [Appendix B, Gaussian Training] The text states: 'For each dataset, we first use the training views from regarding dataset to construct the Gaussian splats. Subsequently, we randomly select 10% of the poses as the validation set, while the remaining 90% are used to train the diffusion model.' This does not assert that the validation/test poses were excluded from the Gaussian reconstruction. Since Algorithm 2 renders the Gaussian scene at the candidate pose and optimizes CLIP similarity to the text, if the test images were used to build the Gaussian, the rendered view can reproduce the test image, and the refinement improvements in Table 1 (e.g., Town k=15 from 22.62 to 26.37; Street k=15 from 7.96 to 15.11) could be due to view memorization rather than semantic localization. This is a load-bearing issue for contribution 3 and for the paper's claim of superior performance. The authors must clarify whether the validation/test poses were held out from Gaussian training and, if they were not, re-run the refinement evaluation with strictly held-out views.
  2. [§4.2, Experimental Results] The only distributional baseline reported is Monte Carlo Dropout, while Text2Loc, Text2Pos, and RET are mentioned in the related work but no quantitative comparison is given. The abstract's claim of 'consistent outperformance' against standard distribution estimation methods is therefore not supported by the presented evidence. The authors should either adapt these published methods to the same task and report their RDA numbers, or justify concretely why they cannot be applied. Additionally, no error bars or multiple-seed results are reported, so the reader cannot assess the variability of the RDA values.
  3. [§4.1, Evaluation Metrics] The RDA metric in Fig. 6 and Eq. (5) is a ratio of the accuracy of the predicted distribution to the accuracy of a random distribution. The denominator is not reported anywhere, which makes the RDA values hard to interpret; for instance, a high RDA could arise from an extremely low random-accuracy baseline. The paper should report absolute metrics such as median translation/rotation error and success rate within a specified distance threshold, in addition to RDA, so that the reader can judge the absolute quality of the estimated pose distributions.
minor comments (5)
  1. [Algorithm 1, line 4] The line 'Random Sample embeddings: Tmix ←...' is a sentence fragment; please rephrase, for instance as 'Randomly sample the embedding Tmix from {Tt, Tf} with swap ratio β.'
  2. [Table 1 caption] The caption states 'Best results per row are highlighted in red'; in grayscale print red is indistinguishable from black, so please also define the best value explicitly or use a distinct formatting, and clarify in the text what 'Ours' versus 'Ours Mixup' versus 'Ours Refined' denote.
  3. [§3.1, paragraph after Multi-model Conditioning] Two consecutive paragraphs both begin with 'By minimizing the mean squared error between the predicted and ground truth poses...' and are almost identical; they should be merged to avoid duplication.
  4. [Fig. 6 caption] The phrase 'Translation is measured in units of 10% of the scene scale' is imprecise; please specify how the scene scale is computed and how the 10% normalization is applied.
  5. [§4.3, User Studies] The text says 'in 30% cases the performance of rendered images exceeds that of the ground-truth (GT) images,' but Table 3 reports match rates of 56.5% (GT) and 50.8% (Ours); the derivation of the 30% figure from the table is unclear and should be explained.

Circularity Check

1 steps flagged · score 7.0 of 10

Gaussian-refinement evaluation is circular: test poses are drawn from the views used to build the Gaussian scene, so refined-pose gains reduce to matching a memorized render to text generated from that same image.

  1. fitted input called prediction [Appendix B ('Experimental Setups', Gaussian Training); Algorithm 2 (Section 3.2)]
    "For each dataset, we first use the training views from regarding dataset to construct the Gaussian splats. Subsequently, we randomly select 10% of the poses as the validation set, while the remaining 90% are used to train the diffusion model. ... Render ˆI ← G(Ppred) ... Lclip ← ˆTf⊤·Tt / (∥ˆTf∥2·∥Tt∥2)."

    The validation/test poses are selected from the views used to build the Gaussian scene, and Section 3.1 generates the textual query from the image at that pose. Therefore at a test pose P, G(P) reconstructs the very source image whose caption is the query text; maximizing CLIP similarity between the caption and G(P) can succeed by retrieving the memorized training view rather than by semantic text-to-pose localization. Table 1's 'Ours Refined' gains are thus not predictions on held-out views—they are re-identification of images already fitted into the 3D representation. The diffusion-only rows are unaffected, but the paper's headline claim and Contribution 3 depend on the refined row, so the central superiority claim is not independently supported.

full rationale

The diffusion stage itself is not circular: the diffusion model is trained on 90% of the poses and evaluated on the remaining 10%, and the RDA metric is an external, independently defined measure. Self-citations in the paper (e.g., ShapeSplat, H3DGS usage) are methodological and not load-bearing for any forced conclusion. The circularity is confined to the Gaussian-refinement evaluation. As written in Appendix B, the Gaussian splats are constructed from the dataset's training views before the 10% validation poses are selected, and the text prompts are generated from the images at those poses. Algorithm 2 then optimizes a pose by maximizing CLIP similarity between the text embedding and the image rendered from the Gaussian at that pose. If the validation image is part of the Gaussian reconstruction, the render at the ground-truth pose is a near-reproduction of the caption's source image, so the refinement objective is solvable by memory rather than by generalization. The reported 'Ours Refined' improvements in Table 1 are therefore not interpretable as evidence for text-based distributional localization, and the paper's central claim of consistent outperformance is only partially supported by the non-circular diffusion-only rows. This is a genuine evaluation circularity, not merely a correctness concern, because the test signal is contained in the fitted representation by the protocol the paper describes.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the quality of generated captions, the fidelity of the Gaussian scene, a single-pose metric that sidesteps ambiguity, and the independence of the Gaussian scene from the test set. The last assumption is unverified and potentially false, which is the largest burden.

free parameters (3)
  • Swap ratio β = 0.7 (tuned by ablation, Fig. 8)
    Controls the mix of text and image embeddings during training; the paper ablates it and finds 0.7 optimal.
  • Refinement acceptance thresholds τ1, τ2 = τ1=0.17, τ2=0.2
    Hand-chosen thresholds in Algorithm 2 that decide which coarse poses to reject and which refined poses to accept; no sensitivity analysis is given.
  • Diffusion timesteps and noise schedule = T=100, β linear 1e-4 to 0.1
    Standard DDPM hyperparameters, but the final results depend on them; they are not tuned systematically.
assumptions (5)
  • domain assumption LLaVA-generated captions are accurate and discriminative descriptors for localization.
    The dataset is built by captioning images with LLaVA at different granularities; Appendix D states that on large, complex scenes LLaVA fails to generate distinct text prompts (Fig. 9), which weakens this assumption.
  • domain assumption The 3D Gaussian scene is an accurate representation that supports rendering from arbitrary poses with sufficient fidelity for CLIP matching.
    Algorithm 2 relies on G(P) rendering a useful image at any candidate pose; no evaluation of rendering quality at queried poses is provided.
  • ad hoc to paper RDA is a valid measure of distributional localization accuracy despite multiple valid poses for ambiguous text.
    The metric compares the fraction of samples near the single ground-truth pose against a random baseline; the paper acknowledges it cannot identify all positive samples for ambiguous text (Section 4.1).
  • ad hoc to paper The test poses are not used in constructing the Gaussian scene.
    Appendix B says the Gaussian splats use the dataset's training views before the 90/10 diffusion split; if test views are included, the refinement evaluation is circular. The paper never states that test views are excluded.
  • standard math DDPM reverse process and CLIP embedding spaces behave as in their original publications.
    Equations (1)-(4) and (7)-(8) rely on published DDPM/CLIP results without re-derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CityLoc: 6DoF Pose Distributional Localization for Text Descriptions in Large-Scale Scenes with Gaussian Representation." pith.science (2026). https://pith.science/paper/YKVVL2WY

@misc{pith2026250108982,
  author       = {Pith},
  title        = {Pith review of: CityLoc: 6DoF Pose Distributional Localization for Text Descriptions in Large-Scale Scenes with Gaussian Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YKVVL2WY}},
  note         = {Machine review of arXiv:2501.08982}
}
read the original abstract

Localizing textual descriptions within large-scale 3D scenes presents inherent ambiguities, such as identifying all traffic lights in a city. Addressing this, we introduce a method to generate distributions of camera poses conditioned on textual descriptions, facilitating robust reasoning for broadly defined concepts. Our approach employs a diffusion-based architecture to refine noisy 6DoF camera poses towards plausible locations, with conditional signals derived from pre-trained text encoders. Integration with the pretrained Vision-Language Model, CLIP, establishes a strong linkage between text descriptions and pose distributions. Enhancement of localization accuracy is achieved by rendering candidate poses using 3D Gaussian splatting, which corrects misaligned samples through visual reasoning. We validate our method's superiority by comparing it against standard distribution estimation methods across five large-scale datasets, demonstrating consistent outperformance. Code, datasets and more information will be publicly available at our project page.

Figures

Figures reproduced from arXiv: 2501.08982 by the authors.

Figure 1
Figure 1. CityLoc: Given an ambiguous text description, our method accurately estimates the camera pose distribution across a large-scale urban environment, pinpointing probable locations like parking spots. Using Vision-Language Models (VLMs), our approach also incorporates image inputs for more precise, context￾aware pose localization. environment, leveraging textual descriptions, where the 3D contextual environment is alre… view at source ↗
Figure 2
Figure 2. We present qualitative results of our large-scale Gaussian splats, including the number of images and the trained Gaussian memory size for each scene. as 3D representation. 2. Novel Approach for Text-Based 6DoF Localization: We present a novel diffusion-based approach that in￾tegrates mixup training with multimodal image-text features using CLIP. This method effectively bridges the gap between textual descriptions a… view at source ↗
Figure 3
Figure 3. Overview of CityLoc. In the training process, where images and multi-level of granularity text input are first converted to CLIP features. A mix algorithm combines these features to train a pose diffusion model, mapping them to a 6DoF camera pose distribution. In the inference phase, where the pose diffusion model outputs camera poses for any given text input. A pretrained Gaussian representation is used to refine t… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Qualitative results on the small town dataset: The enlarged green camera and its corresponding images represent those used to generate multiple text prompts with varying levels of granularity. We report the pose distribution conditioned on different levels of text deta…
Figure 5
Figure 5. Figure 5: An example question from the user study and its corresponding qualitative results. Ground-truth (GT) images appear on the left, while the rendered images are shown on the right. Quantitative results of user study please refer to Tab. 3. to improve distribution estimati…
Figure 6
Figure 6. Figure 6: Relative Distribution Accuracy (RDA) measures the accuracy of the sample distribution within a specified region, de￾fined by a distance k, Translation is measured in units of 10% of the scene scale, while rotation is measured in degrees. varies across levels: for low g…
Figure 8
Figure 8. Figure 8: Ablations about swap probability. User Studies. To validate the effectiveness of our method intuitively, we propose the user studies with question Which image do you think matches the text description? (The answer can be multiple selections) as shown in [PITH_FULL_IMA…
Figure 7
Figure 7. Figure 7: The bar chart compares RDA values across low, high, and maximum granularity levels for different input types for training, including varying numbers of nouns and sentences. The results indicate that increasing granularity and incorporating more descrip￾tive text genera…
Figure 9
Figure 9. Figure 9: Limitations on generated text for large scale scene: We used the Llava model for image captioning at different granularities, but we found that for larger and more complex scenes, the text generated by Llava fails to extract more effective information. For example, as …
Figure 10
Figure 10. Figure 10: Qualitative Results on the SciArt Dataset: Similarly, we use the green camera to indicate the camera pose used to generate the prompt, while high-density estimations are shown in orange camera and red camera. Providing more detailed text conditions results in a narrow…
Figure 11
Figure 11. Figure 11: Qualitative Results on the Residence Dataset: The results demonstrate excellent localization potential. The dataset primarily features a residential area with a highway passing through the left section. When using a general prompt like "The image shows a highway," the…
Figure 12
Figure 12. Figure 12: Qualitative Results on the Matrix City Aerial Dataset: We observe that our method also performs exceptionally well on large-scale scene datasets. For instance, the green camera highlights areas such as a dock and a parking lot. When using simpler or moderately detaile…
Figure 13
Figure 13. Figure 13: Qualitative Results on the Matrix City Street Dataset: On the urban street dataset, most street data is highly similar, with numerous overlapping text inputs. To address this, we used a more distinctive scenarioa garage under an overpass—as the image input. The result…
Figure 14
Figure 14. Figure 14: Qualitative results of the proposed method: Training images are shown on the left, with the rendered images on the right. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 2 canonical work pages

  1. [3]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901,

    1901

  2. [5]

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. J. Video diffusion models. arXiv preprint arXiv:2204.03458,

    arXiv

  3. [6]

    Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. d. L., Hendricks, L. A., Welbl, J., Clark, A., et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556,

    arXiv

  4. [9]

    S., Yvon, F., Gallé, M., et al

    Le Scao, T., Fan, A., Akiki, C., Pavlick, E., Ili´c, S., Hesslow, D., Castagné, R., Luccioni, A. S., Yvon, F., Gallé, M., et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100,

    arXiv

  5. [10]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    9 Li, J., Li, D., Savarese, S., and Hoi, S. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730–19742. PMLR, 2023b. Li, X. L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T. B. Diffusion-lm improves controllable text generation. arXiv...

    arXiv

  6. [11]

    Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond

    Li, Y ., Jiang, L., Xu, L., Xiangli, Y ., Wang, Z., Lin, D., and Dai, B. Matrixcity: A large-scale city dataset for city-scale neural rendering and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3205–3215, 2023c. Lin, B., Tang, Z., Ye, Y ., Cui, J., Zhu, B., Jin, P., Huang, J., Zhang, J., Pang, Y ., Ning, M., et al...

    arXiv

  7. [12]

    Spatio- temporal graph diffusion for text-driven human motion generation

    Liu, C., Zhao, M., Ren, B., Liu, M., Sebe, N., et al. Spatio- temporal graph diffusion for text-driven human motion generation. In BMVC, pp. 722–729, 2023a. Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tuning, 2023b. Liu, H., Li, C., Wu, Q., and Lee, Y . J. Visual instruction tuning, 2023c. URL https://arxiv.org/abs/ 2304.08485. Liu, H., Li...

    arXiv 2024

  8. [13]

    A conditional point diffusion-refinement paradigm for 3d point cloud completion

    Lyu, Z., Kong, Z., Xu, X., Pan, L., and Lin, D. A conditional point diffusion-refinement paradigm for 3d point cloud completion. arXiv preprint arXiv:2112.03530,

    arXiv

Show all 29 references
  1. [14]

    URL https: //doi.org/10.1145/3528223.3530127

    doi: 10.1145/3528223.3530127. URL https: //doi.org/10.1145/3528223.3530127. Peng, Z., Wang, W., Dong, L., Hao, Y ., Huang, S., Ma, S., and Wei, F. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824,

  2. [15]

    Ramesh, A., Pavlov, M., Goh, G., Gray, S., V oss, C., Rad- ford, A., Chen, M., and Sutskever, I

    URL https://arxiv.org/abs/2103.00020. Ramesh, A., Pavlov, M., Goh, G., Gray, S., V oss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation,

    arXiv

  3. [16]

    Ren, B., Liu, M., Ding, R., and Liu, H

    URL https://arxiv.org/ abs/2102.12092. Ren, B., Liu, M., Ding, R., and Liu, H. A survey on 3d skeleton-based action recognition using learning method. Cyborg and Bionic Systems, 5:0100,

    arXiv

  4. [17]

    Make-a- video: Text-to-video generation without text-video data

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al. Make-a- video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792,

    arXiv

  5. [19]

    icomma: Inverting 3d gaussians splatting for camera pose estimation via comparing and matching

    10 Sun, Y ., Wang, X., Zhang, Y ., Zhang, J., Jiang, C., Guo, Y ., and Wang, F. icomma: Inverting 3d gaussians splatting for camera pose estimation via comparing and matching. arXiv preprint arXiv:2312.09031,

    arXiv

  6. [20]

    Llama: Open and efficient foundation lan- guage models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302.13971,

    arXiv

  7. [21]

    Spectrally pruned gaus- sian fields with neural compensation

    Yang, R., Zhu, Z., Jiang, Z., Ye, B., Chen, X., Zhang, Y ., Chen, Y ., Zhao, J., and Zhao, H. Spectrally pruned gaus- sian fields with neural compensation. arXiv preprint arXiv:2405.00676,

    arXiv

  8. [22]

    mplug-owl: Modulariza- tion empowers large language models with multimodality

    Ye, Q., Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y ., Wang, J., Hu, A., Shi, P., Shi, Y ., et al. mplug-owl: Modulariza- tion empowers large language models with multimodality. arXiv preprint arXiv:2304.14178,

    arXiv

  9. [24]

    A survey on multimodal large language models

    Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., and Chen, E. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549,

    arXiv

  10. [25]

    Ferret: Refer and ground anything anywhere at any granularity

    You, H., Zhang, H., Gan, Z., Du, X., Zhang, B., Wang, Z., Cao, L., Chang, S.-F., and Yang, Y . Ferret: Refer and ground anything anywhere at any granularity. arXiv preprint arXiv:2310.07704,

    arXiv

  11. [26]

    Next-chat: An lmm for chat, detection and segmentation

    Zhang, A., Zhao, L., Xie, C.-W., Zheng, Y ., Ji, W., and Chua, T.-S. Next-chat: An lmm for chat, detection and segmentation. arXiv preprint arXiv:2311.04498,

    arXiv

  12. [27]

    Denois- ing diffusion probabilistic models for action-conditioned 3d motion generation

    Zhao, M., Liu, M., Ren, B., Dai, S., and Sebe, N. Denois- ing diffusion probabilistic models for action-conditioned 3d motion generation. In ICASSP 2024-2024 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4225–4229. IEEE,

    2024

  13. [28]

    Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping

    Zheng, Y ., Chen, X., Zheng, Y ., Gu, S., Yang, R., Jin, B., Li, P., Zhong, C., Wang, Z., Liu, L., et al. Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping. arXiv preprint arXiv:2403.09637,

    arXiv

  14. [29]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023a. Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language un...

    2020 arXiv

  15. [2015]

    Denoising diffusion implicit models

    Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502,

    2010 arXiv

  16. [2016]

    Kendall, A., Grimes, M., and Cipolla, R

    URL https: //arxiv.org/abs/1509.05909. Kendall, A., Grimes, M., and Cipolla, R. Posenet: A convo- lutional network for real-time 6-dof camera relocalization. In Proceedings of the IEEE international conference on computer vision, pp. 2938–2946,

    arXiv

  17. [2020]

    Shikra: Unleashing multimodal llm’s referential dialogue magic

    Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., and Zhao, R. Shikra: Unleashing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195,

    arXiv

  18. [2021]

    Qwen-vl: A frontier large vision- language model with versatile abilities

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A frontier large vision- language model with versatile abilities. arXiv preprint arXiv:2308.12966,

    arXiv

  19. [2022]

    Segdiff: Image segmentation with diffusion probabilistic models

    Amit, T., Nachmani, E., Shaharbany, T., and Wolf, L. Segdiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390,

    arXiv

  20. [2023]

    T., Rodriguez, A., Isola, P., and Lin, T.-Y

    Yen-Chen, L., Florence, P., Barron, J. T., Rodriguez, A., Isola, P., and Lin, T.-Y . inerf: Inverting neural radi- ance fields for pose estimation. In 2021 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems (IROS), pp. 1323–1330. IEEE,

    2021

  21. [2024]

    Kolmet, M., Zhou, Q., Ošep, A., and Leal-Taixé, L

    URL https://arxiv.org/abs/ 2406.12080. Kolmet, M., Zhou, Q., Ošep, A., and Leal-Taixé, L. Text2pos: Text-to-point-cloud cross-modal localization. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pp. 6687–6696,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.