Pith. sign in

REVIEW 4 major objections 5 minor 12 cited by

From one photo to a physics-ready 3D tabletop scene for robots

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:15 UTC pith:54V2UTPV

load-bearing objection Genuinely new system with a real engineering contribution, but the metric-layout claim rests on an untested top-view assumption that needs validation before the SOTA claims are believable. the 4 major comments →

arxiv 2512.01204 v4 pith:54V2UTPV submitted 2025-12-01 cs.CV

TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation

Ziqian Wang , Yonghao He , Licheng Yang , Wei Zou , Hongxuan Ma , Liu Liu , Wei Sui , Yuxin Guo
show 1 more author
Hu Su
This is my paper
classification cs.CV
keywords tabletop scene generation3D scene reconstructionpose alignmentscale estimationrobotic manipulationphysics simulationsingle-image 3Dcollision-free layout
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TabletopGen claims that a single 2D reference—a photo or an image synthesized from text—is enough to produce a 3D tabletop scene in which every object is an independent, physically manipulable model with a collision-free, plausible layout. The method splits the hard spatial-reasoning problem into two stages: a differentiable rotation optimizer that aligns each reconstructed object to the reference view, and a top-view spatial alignment that recovers translation and metric scale by selecting an anchor object and converting pixel positions to world coordinates. This makes automated, large-scale generation of robot-training data feasible without manual asset creation. The paper reports collision rates near zero (0.42% of object pairs, 7.69% of scenes) and a zero-shot transfer of a simulated pick-and-place policy to a real robot arm, evidence that the scenes are stable enough to train on.

Core claim

The paper's central claim is that instance-level, physically interactive 3D tabletop scenes can be generated automatically from text or a single image by composing per-object 3D reconstruction with a two-stage pose-and-scale alignment. In the first stage, each object's rotation about the vertical axis is found by differentiable rendering and minimizing a tri-modal loss over silhouette, edges, and appearance features. In the second stage, a top view of the scene is synthesized with a multimodal generative model, per-object bounding boxes are detected, and an anchor object—selected by a ratio-matched, area-weighted score—is used to convert pixel geometry into metric translations and scales; st

What carries the argument

The load-bearing mechanism is the two-stage Pose and Scale Alignment, which decouples the ill-posed single-view spatial reconstruction into rotation recovery and translation/scale recovery. Stage one, the Differentiable Rotation Optimizer (DRO), renders each canonical 3D model from a predicted viewpoint and minimizes a tri-modal loss (soft IoU silhouette, one-sided edge Chamfer, and perceptual appearance feature) to find the rotation about the vertical axis. Stage two, the Top-view Spatial Alignment (TSA), synthesizes a top view of the scene with a multimodal generative model, detects 2D bounding boxes, and selects a scaling anchor via the RMA-Score—a ratio-matched, area-weighted score that

Load-bearing premise

The synthesized top-view image must be geometrically faithful to the reference image, and the commonsense physical size priors for the anchor object must be accurate; if either is wrong, the recovered translations and scales are wrong, producing collisions or floating objects.

What would settle it

Take a reference image in which the top-view synthesis is known to fail—for instance, one object is heavily occluded from above or the top view is ambiguous—and run TabletopGen. If the pipeline still reports a near-zero collision rate, the TSA stage is not actually relying on the synthesized top view as described; if it produces collisions or floating objects, the assumption is confirmed. A second check: place a reference object of known physical size (e.g., a standard soda can) in a generated scene and measure whether the recovered metric scale matches reality.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Robotic manipulation datasets can be synthesized at scale without manual 3D assets, since each scene is assembled from independently reconstructed objects.
  • The same pipeline works from text prompts and real-world photos, so simulation environments can mirror real tabletops (real-to-sim), reducing the sim-to-real gap for policy learning.
  • Scenes become modular: an object can be swapped by replacing its canonical model and reusing the estimated pose and scale, without regenerating the whole scene.
  • Collision-free layouts at the reported rates make generated scenes directly usable for physics-based policy learning and verification in simulation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If TSA's anchor-based scale transfer generalizes, the same mechanism could apply to other dense small-object layouts such as shelves or trays, extending the method beyond tabletops without new training.
  • The reliance on a synthesized top view creates a testable boundary: when the generative model hallucinates, displaces, or omits an object in the top view, the metric scale should degrade; quantifying this would bound the method's reliability on out-of-distribution inputs.
  • Since rotation is estimated per object from the reference view, heavily occluded objects depend on the generative completion stage; the true failure modes may show up in partially hidden instances rather than in the overall collision metric.
  • A natural extension is to use the generated scenes as a self-supervised source of negative examples: annotations of where collisions would occur if alignment were wrong could train a lightweight collision-predictor for arbitrary layouts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. TabletopGen proposes a training-free, multi-stage pipeline that turns a text prompt or a single reference image into an instance-level, physics-interactive 3D tabletop scene. The pipeline extracts per-instance images via segmentation and generative completion, reconstructs each object with image-to-3D and canonical coordinate alignment, and then estimates per-instance rotation, translation, and scale using a Differentiable Rotation Optimizer (DRO) and a Top-View Spatial Alignment (TSA) mechanism. The final scene is assembled in Isaac Sim with collision geometry. The paper reports state-of-the-art visual fidelity, layout accuracy, and physical plausibility against ACDC, Gen3DSR, and MIDI, with near-zero collision rates (Col O 0.42%, Col S 7.69%), a user study favoring TabletopGen in 83.13% of cases, and a demonstration of real-to-sim-to-real policy transfer.

Significance. If the claims hold, TabletopGen would be a practically valuable contribution to embodied AI data generation: it produces fully instance-level, interactive tabletop scenes from a single image or text, sidestepping asset-retrieval limitations. The proposed two-stage pose/scale alignment (DRO followed by TSA) is a plausible decoupling of a hard single-view layout-recovery problem, and the paper includes both machine-checkable ablations and a user study. The release of code and the use of standard public components (Hunyuan3D, GroundedSAM-v2, Isaac Sim) also make the framework reproducible in principle. The significance is therefore moderate-to-high, provided the core alignment mechanism is validated against the unstated geometric-fidelity assumption of the synthesized top view, and provided the evaluation protocol is made statistically and methodologically robust.

major comments (4)
  1. [Sec. 3.3, Eq. (5)–(6), Table 3] The TSA stage is load-bearing: removing it raises Col O from 0.42% to 5.50% and Col S from 7.69% to 61.54% (Table 3). Yet the stage rests on an untested assumption: that the Seedream-synthesized top-view image I_top preserves the object set, relative positions, and size ratios of the reference view. If I_top is geometrically inconsistent with I_ref, the bounding boxes used in Eq. (5) and the RMA-Score in Eq. (6) do not correspond to the actual scene layout, so the recovered t_i and s_i are unreliable. The paper provides no quantitative validation of I_top's fidelity. I request a direct check: e.g., on synthetic scenes with known ground-truth layouts, compare the detected bounding boxes in I_top with the orthographic projections of the true 3D layout, and report object-count preservation, translation errors, and scale errors. Without this, the headline claim of accurate translation/metric
  2. [Sec. 8.1, Fig. 15, Table 1] The visual-perceptual metrics (LPIPS, DINOv2, CLIP) are reported as the best over 160 camera views per scene. The authors themselves acknowledge in Fig. 15b that this best-view protocol can mask physically implausible layouts in baselines (e.g., floating objects or models extending beyond the table boundary). Because Table 1 uses these best-view scores as evidence of state-of-the-art visual fidelity, the comparison is not neutral. Please report the mean/median over all 160 views for each method, or a fixed canonical view (e.g., 30° elevation, 1.5× distance), and show that the conclusion is unchanged. This is directly relevant to the claim of 'markedly surpassing existing methods in visual fidelity.'
  3. [Tables 1–3 and Sec. 4.1] No error bars, confidence intervals, or significance tests are reported for any quantitative result. The test set contains only 78 samples, the GPT-4o evaluation is based on a single run, and the user study (126 valid questionnaires) reports only mean scores. Some differences in Table 1 are small (e.g., LPIPS 0.4483 vs. 0.4559 for MIDI; CLIP 0.9077 vs. 0.8867). Without variance estimates or a paired test, the claim of substantial superiority over the second-best method is under-supported. Please provide per-method standard deviations, 95% CIs, and paired significance tests where applicable, and run the GPT-4o evaluation at least 3 times to report mean±std.
  4. [Sec. 4.1 and Sec. 3.3] There is a circularity concern in the evaluation. The same family of LLMs (ChatGPT) is used inside the pipeline for layout reasoning, size priors, and stacking-order analysis, while GPT-4o is used as the automated judge for Visual Fidelity, Image Alignment, and Physical Plausibility. This can inflate the reported GPT-evaluation scores because the judge and the generator share biases. The paper should either use an independent judge model (e.g., a different architecture) or de-emphasize GPT-4o scores in favor of the human study, and should report the prompt used for GPT-4o evaluation in full. Additionally, the LPIPS/DINOv2/CLIP scores are computed against the same reference image that the pipeline explicitly optimizes to match (DRO's appearance loss uses DINOv2 on the reference; the whole assembly is conditioned on I_ref); this is not a held-out test and should be stated as a limitation.
minor comments (5)
  1. [Sec. 4.1] The metrics paragraph defines Col C as the proportion of colliding object pairs, but Table 1 reports Col O and Col S. Please define Col O explicitly and keep the notation consistent.
  2. [Eq. (5)] The definitions of r_img and r_phys are incomplete. The text says r_img is the aspect ratio from I_top and r_phys comes from projecting physical size onto the XY-plane, but the units and the exact formula for projecting after applying r_i should be stated. Also, A_px should be defined as the area of the 2D bounding box in pixels, not the product of width and height without clarifying which bounding box.
  3. [Sec. 3.3] The transition from the anchor's RMA-Score to the global scale factor α (meters/pixel) is under-specified. Please state explicitly how the anchor's physical size, obtained from MLLM commonsense, is converted into α, and how s_i and t_i are then computed for all instances.
  4. [Sec. 7] The robotic manipulation demonstrations are described only in the supplementary video. Please provide quantitative success metrics (e.g., number of successful pick-and-place attempts out of total, collision counts during execution) and describe the policy used for the real-to-sim-to-real transfer. Video alone is insufficient to substantiate the claim of 'zero-shot real-to-sim-to-real policy transfer.'
  5. [Sec. 6.2] The efficiency analysis is useful, but the 32-minute runtime is dominated by sequential instance-level processing. It would be helpful to state the total GPU-hours and the parallelization assumptions more clearly, since the paper claims 'automated' but not 'real-time.'

Circularity Check

1 steps flagged

No load-bearing circularity; one evaluation metric (DINOv2) reuses the DRO optimization objective, but the core pose/scale derivation and independent collision/user-study evidence stand.

specific steps
  1. fitted input called prediction [Sec. 3.3, Eq. (4); Sec. 4.1 Metrics; Table 1]
    "Lapp(ri) = ||Φ(I render(ri))−Φ(I instance)||^2_2 ... We report LPIPS [65], DINOv2 [39], and CLIP [42] to measure perceptual similarity, visual consistency, and semantic alignment to the reference image."

    DRO fits each instance rotation by minimizing DINOv2 feature distance between the rendered instance and the reference-derived instance crop, and the reported DINOv2 score is then computed between the assembled scene and the same reference image. The DINOv2 column therefore partially reports the objective used to fit the scene, so a high DINOv2 score is partly forced by construction. This is a real evaluation confound, but it is not the central derivation: collision rates are computed in Isaac Sim, and the user study is human-based, providing independent evidence.

full rationale

TabletopGen is a compositional reconstruction pipeline: reference image -> per-instance segmentation/completion -> image-to-3D reconstruction -> DRO rotation fitting -> TSA translation/scale from a synthesized top view -> physics assembly. No load-bearing step defines its output in terms of the claimed scene-level result, and no self-citation chain is used to justify the core mechanism. The components (GroundedSAM-v2, Hunyuan3D-3.0, Seedream, ChatGPT, Isaac Sim) are external off-the-shelf tools. TSA's assumption that Seedream's top view is geometrically consistent with the reference is a genuine, untested validity risk, but an assumption is not circularity: the layout is not defined to equal the reference by equation; it is an empirical claim about a generative model. The collision metrics are computed from convex-decomposed rigid bodies in a physics simulator, independent of the generative models, and the 128-participant user study is human-based. The paper itself candidly notes in Sec. 8.1 that best-view selection can mask physically implausible baseline layouts, which is an evaluation-fairness caveat rather than a circular reduction. The one concrete overlap is DINOv2: it is used both as a DRO loss and as a headline quality metric, so that specific column is partially self-referential. The GPT-4o evaluation may also share priors with the ChatGPT-based size/layout reasoning used in the pipeline, but the human study and physical simulation prevent this from being the sole support for the central claims.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The RMA-Score is a heuristic scoring function, not an entity. The paper's load-bearing assumptions are entirely about the reliability of off-the-shelf generative models and MLLM spatial reasoning.

free parameters (5)
  • DRO loss weights λs, λe, λa = 0.5, 0.5, 2.0
    Hand-chosen weighting of silhouette, edge, and appearance terms in L_rot (Eq. 1).
  • RMA-Score tolerance τ = 0.25
    Hyperparameter controlling aspect-ratio error tolerance in anchor selection (Eq. 6).
  • DRO optimizer step count and learning rate = 140 steps, lr=3e-2
    Adam refinement hyperparameters for rotation optimization.
  • MLLM-predicted camera azimuth/elevation = e.g., (0°, 60°)
    Initial camera pose for DRO rendering, supplied by ChatGPT without independent verification.
  • MLLM commonsense physical sizes = per-object
    Used in TSA to convert pixel boxes to metric scale; effectively free parameters supplied by the LLM.
axioms (4)
  • domain assumption Generated top-view image is spatially consistent with the reference image
    TSA relies on bounding boxes in the synthesized top view; if Seedream hallucinates layout, translations and scales are wrong. Entered in Sec. 3.3 'Top-View Spatial Alignment'.
  • domain assumption Image-to-3D models produce complete, physically usable meshes
    Per-instance meshes are reconstructed from completed 2D images without geometric verification; incomplete geometry would cause instability in Isaac Sim. Entered in Sec. 3.2.
  • domain assumption MLLM reasoning about object categories, upright orientation, and stacking order is correct
    Used in instance extraction, canonical alignment, and z-order; errors propagate to the final layout. Secs. 3.1–3.3.
  • domain assumption Single-view 2D reference contains enough information to recover 3D layout
    The pipeline inverts a single image; scale and position ambiguity is resolved only via LLM priors and a synthesized top view. Sec. 3.3.

pith-pipeline@v1.3.0-alltime-deepseek · 17780 in / 11084 out tokens · 115150 ms · 2026-08-03T19:15:13.447936+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation." pith.science (2026). https://pith.science/paper/54V2UTPV

@misc{pith2026251201204,
  author       = {Pith},
  title        = {Pith review of: TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/54V2UTPV}},
  note         = {Machine review of arXiv:2512.01204}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Simulation provides a low-cost, scalable pathway to large-scale robotic manipulation data collection. However, existing 3D scene generation methods can rarely be applied directly to manipulation data synthesis, as their generated scenes often lack instance-level interactivity and physical plausibility. Focusing on tabletop manipulation, we propose TabletopGen, a training-free and automated tabletop scene generation and interactive simulation engine. Starting from text or a single image, we first obtain independent 3D object models via generative instance extraction. Second, we introduce a novel pose and scale alignment approach that recovers a collision-free scene layout using a Differentiable Rotation Optimizer and a Top-View Spatial Alignment mechanism. Finally, we assemble the generated scene in a physics simulator with collision geometry, yielding a stable, interactable environment for synthesizing multimodal manipulation data. Extensive experiments and user studies demonstrate that TabletopGen achieves state-of-the-art performance in visual fidelity, layout accuracy, and physical plausibility. Furthermore, we validate the executability of the collected trajectories on a real robotic arm via zero-shot real-to-sim-to-real policy transfer, indicating that TabletopGen can serve as a reliable data engine for robotic manipulation data synthesis.

Figures

Figures reproduced from arXiv: 2512.01204 by Hongxuan Ma, Hu Su, Licheng Yang, Liu Liu, Wei Sui, Wei Zou, Yonghao He, Yuxin Guo, Ziqian Wang.

Figure 1
Figure 1. Figure 1: We present TabletopGen, a training-free, fully automatic unified framework that generates instance-level interactive 3D tabletop scenes. As shown on the left, TabletopGen can generate visually realistic, detail-rich, plausibly arranged, and collision-free 3D scenes from either text or a single image input. As shown on the right, our framework can produce a wide variety of tabletop scenes, spanning differen… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our TabletopGen Framework. Our framework accepts either text (which is first converted into a reference image) or a single image. Starting from the image, we proceed in four stages: (1) Instance Extraction performs category analysis, segmentation, and completion to obtain clean, high-resolution per-instance images. (2) Canonical Model Generation uses Image-to-3D and MLLM￾based alignment to crea… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison under the same input images. Our method, TabletopGen, consistently outperforms all baselines. TabletopGen demonstrates strong adaptability across diverse tabletop types, delivering more realistic appearances, finer instance models, more coherent object counts and layouts, and collision-free placement. scores. We attribute this to our “instance-first, then align￾ment” design, which fi… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative ablation study on pose and scale alignment components. Compared to our full model (Ours), removing DRO yields incorrect instance rotations (yellow circles), removing TSA causes misplacements (blue parallelograms), and removing both ampli￾fies these errors, often leading to severe occlusions and collisions (red rectangles). bility (PP)—using a 7-point scale. As shown in [PITH_FULL_IMAGE:figures… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison under the same input texts. Compared with the recent tabletop generation method MesaTask, TabletopGen generates more complete and realistic scenes, includ￾ing stylistically consistent tables, more detailed instance models, and more semantically and physically reasonable layouts. realistic visual appearances, richer instance counts, more reasonable arrangements, and collision-free lay… view at source ↗
Figure 6
Figure 6. Figure 6: 3D Scenes from Real-World Images. Compared with MIDI, our method robustly reconstructs real-world photos into 3D scenes with significantly higher geometric fidelity and a more plausible, collision-free layout [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Scene Editing. Our compositional framework allows for individual instances, such as the ukulele, to be easily replaced with a new object, like the banjo, while retaining the original layout. decoupling 2D-to-3D reasoning into a Differentiable Ro￾tation Optimizer (DRO) for precise rotation recovery and a Top-view Spatial Alignment (TSA) mechanism for ro￾bust translation and metric scale inference, TabletopG… view at source ↗
Figure 8
Figure 8. Figure 8: Text-input preprocessing pipeline. For a short text (top), we first use an LLM to expand it into a detailed, layout￾aware scene description (middle). The expanded description is then fed into a text-to-image model to synthesize the reference image (bottom), which conditions all subsequent stages. Seedream to synthesize a visually realistic and plausibly ar￾ranged reference image Iref , on which all subsequ… view at source ↗
Figure 11
Figure 11. Figure 11: shows the final match for instance toy car 1, where the rendered silhouette, edges, and appearance maps closely agree with the corresponding reference images [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: DRO outputs (JSON). Estimated per-instance rotation ri (Euler yaw) about the scene z-axis, reported in degrees (◦ ). GPT provides commonsense physical sizes for each object. Subsequently, we identify the scene’s anchor object (ex￾cluding the table) via the RMA-Score; in this example, it is box 9. Using this anchor, we rescale objects whose com￾monsense size deviates significantly from the bounding box siz… view at source ↗
Figure 13
Figure 13. Figure 13: Top-view synthesis and bounding box detection. A top-view of the tabletop is synthesized with a multimodal image generation model, and 2D bounding boxes for all instances are detected on it. These boxes are used to compute the RMA-Score for anchor selection and to derive translation and scale for TSA [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: TSA outputs (JSON). Estimated per-instance scale and translation, reported in centimeters (cm). Scales are listed as [x, y, z] and translations as [x, y, z] in the tabletop frame (origin at the table center; +x left, +y forward, +z up). The anchor object selected by RMA-Score is shown at the top. using the center point of its bounding box. To handle stack￾ing situations (e.g., a pen on a book), we also us… view at source ↗
Figure 15
Figure 15. Figure 15: LPIPS-based view selection under the camera sweep protocol. We sweep the elevation from 90◦ (top) to 0 ◦ (front) in 10◦ steps and vary the camera distance from 1.0× to 2.5× the scene radius (160 renders per scene). While this protocol facilitates a relatively fair and meaningful comparison, the visual bias inherent in a single view allows physically implausible phenomena to be masked under the ”best view,… view at source ↗
Figure 16
Figure 16. Figure 16: Stylized tabletop generation from a cartoon input. TabletopGen can transfer a cartoon-style 2D input into a 3D scene. The generated scene maintains a consistent style and a plausible layout. 9.2. Visualizations and Interaction Demos To provide a more detailed and intuitive demonstration of the visual fidelity and interactive capability of our generated scenes, the supplementary video, “SceneShowcase.mp4”,… view at source ↗
Figure 17
Figure 17. Figure 17: Survey description. Overview of the study goals and instructions [PITH_FULL_IMAGE:figures/full_fig_p017_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Per-scene rating page. For each scene, participants rate Visual Fidelity, Image Alignment, and Physical Plausibility (1–7) given the input image and one method’s result. Shown here is the scoring UI for a single method, the other three methods use the same interface [PITH_FULL_IMAGE:figures/full_fig_p017_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Overall preference. After scoring four methods, participants choose a single overall favorite among the four randomized results [PITH_FULL_IMAGE:figures/full_fig_p018_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Prompt for expanding a short text input into a detailed scene description. [PITH_FULL_IMAGE:figures/full_fig_p018_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Prompt for multimodal generative completion. [PITH_FULL_IMAGE:figures/full_fig_p019_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Prompt for analyzing the commonsense size and canonical axis definition of each instance. [PITH_FULL_IMAGE:figures/full_fig_p019_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Prompt for canonicalizing the 3D model coordinate system of a single instance. [PITH_FULL_IMAGE:figures/full_fig_p020_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Prompt for synthesizing the top-view image. [PITH_FULL_IMAGE:figures/full_fig_p020_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Prompt for analyzing instance stacking order. [PITH_FULL_IMAGE:figures/full_fig_p021_25.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. One Video, One World: Turning Monocular Video into Physical 4D Scenes

    cs.CV 2026-06 unverdicted novelty 8.0

    OVOW reconstructs instance-level, simulation-ready 4D mesh scenes from monocular video via a four-stage training-free pipeline and introduces a new benchmark for structured Video-to-4D evaluation.

  2. IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoning

    cs.CV 2026-07 accept novelty 7.0

    Current VLMs top out at 62.1/100 on holistic single-image 3D indoor layout prediction, with strong recognition but weak geometric regression, and mid-tier rankings that shift relative to QA and primitive-reconstructio...

  3. REST3D: Reconstructing Physically Stable 3D Scenes from a Single Image

    cs.CV 2026-05 unverdicted novelty 7.0

    REST3D reconstructs physically stable 3D scenes from single images via agentic scene-tree understanding and physics-constrained optimization.

  4. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 accept novelty 7.0

    3D generation for embodied AI is shifting from visual realism toward interaction readiness, organized into data generation, simulation environments, and sim-to-real bridging roles.

  5. TableVerse: A Large-scale Tabletop Dataset with Real-world Grounded Layouts for Generalizable Manipulation

    cs.RO 2026-07 conditional novelty 6.0

    An automated pipeline converts in-the-wild tabletop images into 100K physically simulated MuJoCo scenes with collision-free pick-and-place trajectories.

  6. RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and Evaluation

    cs.RO 2026-07 conditional novelty 6.0

    A single RGB image is converted into a layered, simulation-ready robot scene that supports trajectory replay, synthetic data generation, and meaningful sim-real policy evaluation, plus a 564-scene DROID-Sim companion set.

  7. Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation

    cs.CV 2026-05 unverdicted novelty 6.0

    Introduces Layout-as-Policy (LaP) to turn 3D layout estimation into an iterative policy-learning refinement process for better physical coherence.

  8. STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics-Physics Dual System

    cs.CV 2026-05 unverdicted novelty 6.0

    STABLE generates simulation-ready tabletop scenes by alternating a semantic LLM reasoner for task-aligned coarse layouts with a physics corrector for physical plausibility using progressive scene expansion.

  9. V-CAGE: Vision-Closed-Loop Agentic Generation Engine for Robotic Manipulation

    cs.RO 2026-04 unverdicted novelty 6.0

    V-CAGE automates the creation of scalable, high-quality robotic manipulation datasets through context-aware scene construction, closed-loop visual verification, and perceptually-driven compression.

  10. WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

    cs.CV 2026-05 unverdicted novelty 5.0

    WorldAct activates monolithic 3D worlds into interactive scenes via multimodal agent-guided decomposition, geometrically aligned mesh reconstruction, and 3D inpainting.

  11. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 3.0

    The survey organizes 3D generation for embodied AI into data generators for assets, simulation environments for interaction, and sim-to-real bridges, noting a shift toward interaction readiness and listing bottlenecks...

  12. 3D Generation for Embodied AI and Robotic Simulation: A Survey

    cs.RO 2026-04 unverdicted novelty 2.0

    The paper surveys 3D generation techniques for embodied AI and robotics, categorizing them into data generation, simulation environments, and sim-to-real bridging while identifying bottlenecks in physical validity and...

Reference graph

Works this paper leans on

74 extracted references · 21 linked inside Pith · cited by 10 Pith papers

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Open-universe indoor scene generation using llm program synthesis and uncurated object databases.arXiv preprint arXiv:2403.09675, 2024

    Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stew- art Morris, Seung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie. Open-universe indoor scene generation using llm program synthesis and uncurated object databases.arXiv preprint arXiv:2403.09675, 2024. 2, 3

    Pith/arXiv arXiv 2024

  3. [3]

    Gen3dsr: Generalizable 3d scene reconstruction via divide and conquer from a single view

    Andreea Ardelean, Mert ¨Ozer, and Bernhard Egger. Gen3dsr: Generalizable 3d scene reconstruction via divide and conquer from a single view. In2025 International Con- ference on 3D Vision (3DV), pages 616–626. IEEE, 2025. 2, 3, 5

    2025

  4. [4]

    Towards a unified understanding of robot manipulation: A comprehensive survey.arXiv preprint arXiv:2510.10903, 2025

    Shuanghao Bai, Wenxuan Song, Jiayi Chen, Yuheng Ji, Zhide Zhong, Jin Yang, Han Zhao, Wanqi Zhou, Wei Zhao, Zhe Li, et al. Towards a unified understanding of robot manipulation: A comprehensive survey.arXiv preprint arXiv:2510.10903, 2025. 2

    arXiv 2025

  5. [5]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, S...

    1901

  6. [6]

    I-design: Personal- ized llm interior designer

    Ata C ¸ elen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-design: Personal- ized llm interior designer. InEuropean Conference on Com- puter Vision, pages 217–234. Springer, 2024. 2, 3

    2024

  7. [7]

    Housecrafter: Lifting floorplans to 3d scenes with 2d diffusion models

    Yiwen Chen, Hieu T Nguyen, Vikram V oleti, Varun Jam- pani, and Huaizu Jiang. Housecrafter: Lifting floorplans to 3d scenes with 2d diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 28440–28450, 2025. 3

    2025

  8. [8]

    Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023

    Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes.arXiv preprint arXiv:2311.13384, 2023. 2

    Pith/arXiv arXiv 2023

  9. [9]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blis- tein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025. 5

    Pith/arXiv arXiv 2025

  10. [10]

    Diffusion models in vision: A survey

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE transactions on pattern analysis and machine intelli- gence, 45(9):10850–10869, 2023. 2

    2023

  11. [11]

    Auto- mated creation of digital cousins for robust policy learning

    Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Auto- mated creation of digital cousins for robust policy learning. InConference on Robot Learning (CoRL), 2024. 2, 3, 5

    2024

  12. [12]

    Procthor: Large-scale embodied ai using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. In Advances in Neural Information Processing Systems, pages 5982–5994. Curran Associates, Inc., 2022. 2

    2022

  13. [13]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, Eli VanderBilt, Anirud- dha Kembhavi, Carl V ondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, and Ali Farhadi. Objaverse-xl: A universe of 10m+ 3d objects. InAdvances in Neural Infor- mation Proc...

    2023

  14. [14]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13142–13153, 2023. 2

    2023

  15. [15]

    Spa- tialgen: Layout-guided 3d indoor scene generation.arXiv preprint arXiv:2509.14981, 2025

    Chuan Fang, Heng Li, Yixun Liang, Jia Zheng, Yongsen Mao, Yuan Liu, Rui Tang, Zihan Zhou, and Ping Tan. Spa- tialgen: Layout-guided 3d indoor scene generation.arXiv preprint arXiv:2509.14981, 2025. 3

    arXiv 2025

  16. [16]

    Layoutgpt: Compositional visual plan- ning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Xuehai He, S Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual plan- ning and generation with large language models. InAd- vances in Neural Information Processing Systems, pages 18225–18250. Curran Associates, Inc., 2023. 2

    2023

  17. [17]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Bin- qiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 10933–10942,

  18. [18]

    3d-future: 3d fur- niture shape with texture.International Journal of Computer Vision, 129(12):3313–3337, 2021

    Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- niture shape with texture.International Journal of Computer Vision, 129(12):3313–3337, 2021. 2, 3

    2021

  19. [19]

    Any- home: Open-vocabulary generation of structured and tex- tured 3d homes

    Rao Fu, Zehao Wen, Zichen Liu, and Srinath Sridhar. Any- home: Open-vocabulary generation of structured and tex- tured 3d homes. InEuropean Conference on Computer Vi- sion, pages 52–70. Springer, 2024. 2, 3

    2024

  20. [20]

    Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024

    Daoyi Gao, D ´avid Rozenberszki, Stefan Leutenegger, and Angela Dai. Diffcad: Weakly-supervised probabilistic cad model retrieval and alignment from an rgb image.ACM Transactions on Graphics (TOG), 43(4):1–15, 2024. 3

    2024

  21. [21]

    Graphdreamer: Compositional 3d scene synthesis from scene graphs

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. Graphdreamer: Compositional 3d scene synthesis from scene graphs. InConference on Com- puter Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  22. [22]

    Artiscene: Language-driven artistic 3d scene generation through image intermediary

    Zeqi Gu, Yin Cui, Zhaoshuo Li, Fangyin Wei, Yunhao Ge, Jinwei Gu, Ming-Yu Liu, Abe Davis, and Yifan Ding. Artiscene: Language-driven artistic 3d scene generation through image intermediary. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 2891– 2901, 2025. 2, 3, 5

    2025

  23. [23]

    Roca: Ro- bust cad model retrieval and alignment from a single image

    Can G ¨umeli, Angela Dai, and Matthias Nießner. Roca: Ro- bust cad model retrieval and alignment from a single image. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4022–4031, 2022. 3

    2022

  24. [24]

    Reparo: Compositional 3d assets generation with differen- tiable 3d layout alignment

    Haonan Han, Rui Yang, Huan Liao, Jiankai Xing, Zunnan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wanhua Li. Reparo: Compositional 3d assets generation with differen- tiable 3d layout alignment. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 25367– 25377, 2025. 3, 5

    2025

  25. [25]

    Mesatask: Towards task-driven table- top scene generation via 3d spatial reasoning.arXiv preprint arXiv:2509.22281, 2025

    Jinkun Hao, Naifu Liang, Zhen Luo, Xudong Xu, Weipeng Zhong, Ran Yi, Yichen Jin, Zhaoyang Lyu, Feng Zheng, Lizhuang Ma, et al. Mesatask: Towards task-driven table- top scene generation via 3d spatial reasoning.arXiv preprint arXiv:2509.22281, 2025. 2, 3, 5

    arXiv 2025

  26. [26]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. InAdvances in Neural Infor- mation Processing Systems, pages 6840–6851. Curran Asso- ciates, Inc., 2020. 2

    2020

  27. [27]

    Flash sculptor: Modular 3d worlds from objects.arXiv preprint arXiv:2504.06178, 2025

    Yujia Hu, Songhua Liu, Xingyi Yang, and Xinchao Wang. Flash sculptor: Modular 3d worlds from objects.arXiv preprint arXiv:2504.06178, 2025. 3, 5

    Pith/arXiv arXiv 2025

  28. [28]

    Midi: Multi-instance diffusion for single image to 3d scene generation

    Zehuan Huang, Yuan-Chen Guo, Xingqiao An, Yunhan Yang, Yangguang Li, Zi-Xin Zou, Ding Liang, Xihui Liu, Yan-Pei Cao, and Lu Sheng. Midi: Multi-instance diffusion for single image to 3d scene generation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 23646–23657, 2025. 2, 3, 5

    2025

  29. [29]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 5

    Pith/arXiv arXiv 2024

  30. [30]

    Adam: A method for stochastic opti- mization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma. Adam: A method for stochastic opti- mization.arXiv preprint arXiv:1412.6980, 2014. 5

    Pith/arXiv arXiv 2014

  31. [31]

    Hunyuan3d 2.5: Towards high- fidelity 3d assets generation with ultimate details.arXiv preprint arXiv:2506.16504, 2025

    Zeqiang Lai, Yunfei Zhao, Haolin Liu, Zibo Zhao, Qingxi- ang Lin, Huiwen Shi, Xianghui Yang, Mingxin Yang, Shuhui Yang, Yifei Feng, et al. Hunyuan3d 2.5: Towards high- fidelity 3d assets generation with ultimate details.arXiv preprint arXiv:2506.16504, 2025. 5

    Pith/arXiv arXiv 2025

  32. [32]

    Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Mart ´ın-Mart´ın, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 ev- eryday activities and realistic simulation. InConference on Robot Learning, pages 80–93. PMLR, 2023. 2

    2023

  33. [33]

    Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling

    Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-hang Lee, and Peng Yuan Zhou. Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling. InEuropean Conference on Computer Vision, pages 214–230. Springer, 2024. 2, 3

    2024

  34. [34]

    Scenethesis: A language and vision agentic framework for 3d scene generation.arXiv preprint arXiv:2505.02836, 2025

    Lu Ling, Chen-Hsuan Lin, Tsung-Yi Lin, Yifan Ding, Yu Zeng, Yichen Sheng, Yunhao Ge, Ming-Yu Liu, Aniket Bera, and Zhaoshuo Li. Scenethesis: A language and vision agentic framework for 3d scene generation.arXiv preprint arXiv:2505.02836, 2025. 3, 5

    Pith/arXiv arXiv 2025

  35. [35]

    Digital twin generation from visual data: A survey.arXiv preprint arXiv:2504.13159, 2025

    Andrew Melnik, Benjamin Alt, Giang Nguyen, Artur Wilkowski, Maciej Stefa ´nczyk, Qirui Wu, Sinan Harms, Helge Rhodin, Manolis Savva, and Michael Beetz. Digital twin generation from visual data: A survey.arXiv preprint arXiv:2504.13159, 2025. 2

    arXiv 2025

  36. [36]

    Sce- negen: Single-image 3d scene generation in one feedforward pass.arXiv preprint arXiv:2508.15769, 2025

    Yanxu Meng, Haoning Wu, Ya Zhang, and Weidi Xie. Sce- negen: Single-image 3d scene generation in one feedforward pass.arXiv preprint arXiv:2508.15769, 2025. 2, 3, 5

    arXiv 2025

  37. [37]

    Isaac Sim 4.5.0, 2025

    NVIDIA. Isaac Sim 4.5.0, 2025. 5

    2025

  38. [38]

    Sceneteller: Language-to-3d scene generation

    Bas ¸ak Melis¨Ocal, Maxim Tatarchenko, Sezer Karao˘glu, and Theo Gevers. Sceneteller: Language-to-3d scene generation. InEuropean Conference on Computer Vision, pages 362–

  39. [39]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 4, 5

    Pith/arXiv arXiv 2023

  40. [40]

    Atiss: Autoregres- sive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregres- sive transformers for indoor scene synthesis. InAdvances in Neural Information Processing Systems, pages 12013– 12026. Curran Associates, Inc., 2021. 2

    2021

  41. [41]

    Advances in data- driven analysis and synthesis of 3d indoor scenes

    Akshay Gadi Patil, Supriya Gadi Patil, Manyi Li, Matthew Fisher, Manolis Savva, and Hao Zhang. Advances in data- driven analysis and synthesis of 3d indoor scenes. InCom- puter Graphics Forum, page e14927. Wiley Online Library,

  42. [42]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 5

    2021

  43. [43]

    Accelerating 3d deep learning with pytorch3d

    Nikhila Ravi, Jeremy Reizenstein, David Novotny, Tay- lor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501, 2020. 5

    Pith/arXiv arXiv 2007

  44. [44]

    Infiniteworld: A unified scalable simu- lation framework for general visual-language robot interac- tion.arXiv preprint arXiv:2412.05789, 2024

    Pengzhen Ren, Min Li, Zhen Luo, Xinshuai Song, Ziwei Chen, Weijia Liufu, Yixuan Yang, Hao Zheng, Rongtao Xu, Zitong Huang, et al. Infiniteworld: A unified scalable simu- lation framework for general visual-language robot interac- tion.arXiv preprint arXiv:2412.05789, 2024. 2

    Pith/arXiv arXiv 2024

  45. [45]

    Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-world models for diverse visual tasks.arXiv preprint arXiv:2401.14159,

    Pith/arXiv arXiv

  46. [46]

    ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, et al. Seed1. 5-thinking: Advancing su- perb reasoning models with reinforcement learning.arXiv preprint arXiv:2504.13914, 2025. 5

    arXiv 2025

  47. [47]

    Seedream 4.0: Toward next- generation multimodal image generation.arXiv preprint arXiv:2509.20427, 2025

    Team Seedream, Yunpeng Chen, Yu Gao, Lixue Gong, Meng Guo, Qiushan Guo, Zhiyao Guo, Xiaoxia Hou, Weilin Huang, Yixuan Huang, et al. Seedream 4.0: Toward next- generation multimodal image generation.arXiv preprint arXiv:2509.20427, 2025. 5

    Pith/arXiv arXiv 2025

  48. [48]

    CRC Press, 2017

    Tanya Short and Tarn Adams.Procedural generation in game design. CRC Press, 2017. 2

    2017

  49. [49]

    Layoutvlm: Differentiable optimization of 3d layout via vision-language models

    Fan-Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jia- jun Wu. Layoutvlm: Differentiable optimization of 3d layout via vision-language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 29469– 29478, 2025. 2, 3

    2025

  50. [50]

    Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation.arXiv preprint arXiv:2403.03949,

    Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim-to-real approach for robust manipulation.arXiv preprint arXiv:2403.03949,

    Pith/arXiv arXiv

  51. [51]

    Sceneformer: Indoor scene generation with transformers

    Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV), pages 106–115. IEEE, 2021. 2

    2021

  52. [52]

    Architect: Generating vivid and interactive 3d scenes with hierarchical 2d inpainting

    Yian Wang, Xiaowen Qiu, Jiageng Liu, Zhehuan Chen, Jit- ing Cai, Yufei Wang, Tsun-Hsuan Wang, Zhou Xian, and Chuang Gan. Architect: Generating vivid and interactive 3d scenes with hierarchical 2d inpainting. InAdvances in Neu- ral Information Processing Systems, pages 67575–67603. Curran Associates, Inc., 2024. 2

    2024

  53. [53]

    3d scene generation: A survey.arXiv preprint arXiv:2505.05474, 2025

    Beichen Wen, Haozhe Xie, Zhaoxi Chen, Fangzhou Hong, and Ziwei Liu. 3d scene generation: A survey.arXiv preprint arXiv:2505.05474, 2025. 2

    Pith/arXiv arXiv 2025

  54. [54]

    To- scene: A large-scale dataset for understanding 3d tabletop scenes

    Mutian Xu, Pei Chen, Haolin Liu, and Xiaoguang Han. To- scene: A large-scale dataset for understanding 3d tabletop scenes. InEuropean conference on computer vision, pages 340–356. Springer, 2022. 3

    2022

  55. [55]

    Scene synthesis via uncertainty-driven attribute syn- chronization

    Haitao Yang, Zaiwei Zhang, Siming Yan, Haibin Huang, Chongyang Ma, Yi Zheng, Chandrajit Bajaj, and Qixing Huang. Scene synthesis via uncertainty-driven attribute syn- chronization. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5630–5640, 2021. 2

    2021

  56. [56]

    Physcene: Physically interactable 3d scene synthe- sis for embodied ai

    Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthe- sis for embodied ai. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16262–16272, 2024. 2

    2024

  57. [57]

    Llplace: The 3d in- door scene layout generation and editing via large language model.arXiv preprint arXiv:2406.03866, 2024

    Yixuan Yang, Junru Lu, Zixiang Zhao, Zhen Luo, James JQ Yu, Victor Sanchez, and Feng Zheng. Llplace: The 3d in- door scene layout generation and editing via large language model.arXiv preprint arXiv:2406.03866, 2024. 2

    Pith/arXiv arXiv 2024

  58. [58]

    Holodeck: Language guided gen- eration of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Al- varo Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided gen- eration of 3d embodied ai environments. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16227–16237, 2024. 2, 3

    2024

  59. [59]

    Springer, 2018

    Georgios N Yannakakis and Julian Togelius.Artificial intel- ligence and games. Springer, 2018. 2

    2018

  60. [60]

    Cast: Component-aligned 3d scene reconstruction from an rgb image.ACM Transactions on Graphics (TOG), 44(4): 1–19, 2025

    Kaixin Yao, Longwen Zhang, Xinhao Yan, Yan Zeng, Qix- uan Zhang, Lan Xu, Wei Yang, Jiayuan Gu, and Jingyi Yu. Cast: Component-aligned 3d scene reconstruction from an rgb image.ACM Transactions on Graphics (TOG), 44(4): 1–19, 2025. 3, 5

    2025

  61. [61]

    Metascenes: Towards automated replica creation for real-world 3d scans

    Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su, Jiaxin Li, Qing Li, Wei Liang, Song-Chun Zhu, et al. Metascenes: Towards automated replica creation for real-world 3d scans. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 1667–1679, 2025. 2

    2025

  62. [62]

    Wonderworld: Interactive 3d scene generation from a single image

    Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5916–5926, 2025. 3

    2025

  63. [63]

    Lvdiffusor: Distilling functional rearrangement priors from large models into dif- fusor.IEEE Robotics and Automation Letters, 2024

    Yiming Zeng, Mingdong Wu, Long Yang, Jiyao Zhang, Hao Ding, Hui Cheng, and Hao Dong. Lvdiffusor: Distilling functional rearrangement priors from large models into dif- fusor.IEEE Robotics and Automation Letters, 2024. 3

    2024

  64. [64]

    Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion

    Guangyao Zhai, Evin Pı nar ¨Ornek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion. InAdvances in Neural Informa- tion Processing Systems, pages 30026–30038. Curran Asso- ciates, Inc., 2023. 3

    2023

  65. [65]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 5

    2018

  66. [66]

    Lohoravens: A long-horizon language-conditioned benchmark for robotic tabletop manip- ulation.arXiv preprint arXiv:2310.12020, 2023

    Shengqiang Zhang, Philipp Wicke, L ¨utfi Kerem S ¸enel, Luis Figueredo, Abdeldjallil Naceri, Sami Haddadin, Barbara Plank, and Hinrich Sch ¨utze. Lohoravens: A long-horizon language-conditioned benchmark for robotic tabletop manip- ulation.arXiv preprint arXiv:2310.12020, 2023. 2

    Pith/arXiv arXiv 2023

  67. [67]

    A survey of large language mod- els.arXiv preprint arXiv:2303.18223, 1(2), 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language mod- els.arXiv preprint arXiv:2303.18223, 1(2), 2023. 2

    Pith/arXiv arXiv 2023

  68. [68]

    Internscenes: A large-scale simulatable in- door scene dataset with realistic layouts.arXiv preprint arXiv:2509.10813, 2025

    Weipeng Zhong, Peizhou Cao, Yichen Jin, Li Luo, Wenzhe Cai, Jingli Lin, Hanqing Wang, Zhaoyang Lyu, Tai Wang, Bo Dai, et al. Internscenes: A large-scale simulatable in- door scene dataset with realistic layouts.arXiv preprint arXiv:2509.10813, 2025. 3

    Pith/arXiv arXiv 2025

  69. [69]

    Layout-your-3d: Controllable and precise 3d generation with 2d blueprint.arXiv preprint arXiv:2410.15391, 2024

    Junwei Zhou, Xueting Li, Lu Qi, and Ming-Hsuan Yang. Layout-your-3d: Controllable and precise 3d generation with 2d blueprint.arXiv preprint arXiv:2410.15391, 2024. 3 Supplementary Material

    Pith/arXiv arXiv 2024

  70. [70]

    A hobby desk with some model cars and tools

    Details of TabletopGen 6.1. Stage-wise Intermediate Outputs TabletopGen is a multi-stage 3D scene generation frame- work that decomposes the complex task of converting text or a single image into a 3D scene into several more manage- able sub-stages, including instance extraction, canonical 3D model generation, pose and scale alignment, and final 3D scene ...

  71. [71]

    Manipula- tionTaskDemo.mp4

    Robotic Manipulation Demonstrations We utilize the NVIDIA Isaac Sim 4.5.0 environment to place the generated tabletops into an indoor scene, and in- troduce a Franka Emika Panda 7-DoF arm with a parallel gripper into the scene. This widely-used, standard industrial arm is tasked with executing a series of complex pick-and- place maneuvers to demonstrate t...

  72. [72]

    Details of Experiments 8.1. Visual & Perceptual Quality Details Camera Sweep Protocol.Due to the unknown camera perspective of arbitrary input images, we established a stan- dardized camera sweep trajectory for all methods to ensure a fair and meaningful comparison. We generate a total of 160 distinct views per scene. The sweep covers the vertical range f...

  73. [73]

    SceneShowcase.mp4

    More Results 9.1. Stylized Tabletop Generation Since TabletopGen does not depend on a fixed asset library and generates each instance from the reference image, it is naturally style-agnostic. Beyond photorealistic scenes, we can generate 3D tabletop scenes from stylized inputs. Fig. 16 shows a hand-drawn, cartoon-style office table: Table- topGen reconstr...

  74. [74]

    Prompts The complete prompt templates used across all stages of TabletopGen are detailed in Figs. 20-25. Figure 17.Survey description.Overview of the study goals and instructions. Figure 18.Per-scene rating page.For each scene, participants rate Visual Fidelity, Image Alignment, and Physical Plausibility (1–7) given the input image and one method’s result...