{"id":"2ec2bfab-0f6e-45f7-b708-f28f994b98ee","arxiv_id":"2509.10441","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A trained decoder, InfGen, maps a fixed-size VAE latent to arbitrary-resolution images in one forward pass, letting diffusion models produce multi-megapixel outputs quickly.","lead":"InfGen replaces a diffusion model's image decoder with a one-step generator that turns a fixed-size latent into an image at any requested resolution. The goal is to reach 4K-quality image synthesis in seconds while leaving existing Stable Diffusion, DiT, and SiT pipelines unchanged.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing assumption is that a compact VAE latent is a complete content representation, but no experiment measures semantic/content preservation; FID/sFID patch metrics cannot distinguish faithful decoding from a generic high-resolution texture prior.","rationale":"The reader's weakest assumption already names the compact latent as a complete content representation and flags missing distributional evidence. My concern is the closely related but more specific gap: the paper never measures whether the latent's semantic content is preserved after InfGen decoding. FIDp/sFIDp are distributional patch metrics and can be improved by a model that produces locally realistic textures even if it changes object identity, count, or layout. Since the central claim is that InfGen 'decodes' a fixed latent rather than generating a new image conditioned on it, content preservation is load-bearing. The other issues noted by the reader (no 4K latency table, underspecified architecture, weak baselines, no code) are real but they weaken reproducibility and the strength of the empirical claims without directly attacking the mechanism. The proposed classification-accuracy probe on DiT-XL/2 is a clean controlled test because it uses the same z with both the original VAE decoder and InfGen, isolating the decoder's behavior. If the probe shows accuracy parity, the central claim receives independent support and the paper still warrants a conditional verdict pending artifacts and latency evidence. If the probe shows a large drop, the paradigm claim should be rejected because InfGen would be a texture prior, not a content-preserving decoder. The reader's CONDITIONAL verdict already reflects this uncertainty, so no change is needed.","tokens_in":12131,"tokens_out":9879,"duration_ms":94805,"concrete_test":"Controlled decoder comparison on DiT-XL/2 (ImageNet): sample a fixed set of 1,024 class-conditional latents z at the model's native latent size. Decode each z twice: (a) with the original SD VAE to 256x256, and (b) with InfGen to 1024x1024. Resize both outputs to 224x224 and run a pretrained ImageNet classifier. If the top-1/top-5 accuracy of InfGen outputs is substantially below that of the original VAE outputs (e.g., >5 points at 1024, not explained by resizing artifacts), then InfGen is not preserving the latent's semantic content and the central 'content representation' claim fails. A secondary check with SD1.5/SDXL would compute CLIPScore and object-count accuracy on 1k text prompts for InfGen versus the original decoder; a large drop would confirm the same failure in text-to-image mode.","verdict_should_be":"UNCHANGED","load_bearing_attack":"InfGen's central mechanism (Section 3.2) is that the fixed latent z produced by a diffusion model is a 'content representation' and that the learned one-step decoder G(z,(h,w)) expands it to any resolution. The architecture is effectively a learned super-resolution generator trained with L1, LPIPS, and adversarial losses on pairs (z, x_high), where z = VAE_encoder(resize(crop(x_high),512)). The paper validates G only with distributional metrics (FIDp, sFIDp, precision, recall) computed on 229x229 crops, plus a visual comparison. These metrics reward locally realistic texture; they do not measure whether the high-level content of z (object class, identity, count, spatial layout, text) survives decoding. A generator that partially ignores z and emits generic LAION-like texture could improve FIDp at high resolution while corrupting the content intended by the diffusion model. This matters because the abstract's core promise is 'decoding arbitrary resolution images from a compact generated latent' and 'plug-and-play' compatibility with any model sharing the latent space: if semantics are not preserved, InfGen is a high-resolution texture prior rather than a decoder. No experiment in Tables 2-4, Fig. 4, or the appendix reports content-fidelity metrics such as ImageNet classification accuracy for class-conditional DiT/SiT outputs, CLIPScore or entity-count accuracy for text-to-image SD1.5/SDXL outputs, or re-encoding consistency.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InfGen, a learned decoder that replaces the VAE decoder in latent diffusion models. InfGen takes a fixed-size latent produced by an off-the-shelf diffusion model and a target resolution (h, w), and generates an image of that resolution through a transformer decoder using cross-attention between latent tokens and learnable mask tokens. The decoder is trained on pairs (VAE-encoded resized crops, high-resolution targets) with L1, LPIPS, and adversarial losses. The authors report improved FIDp/sFIDp, precision, and recall across DiT, SiT, MDTv2, FiTv2, and SD1.5/SDXL at resolutions from 512 to 3072 pixels, alongside latency advantages over prior arbitrary-resolution methods, and claim that the method can upgrade any model sharing the SD/SDXL VAE latent space to arbitrary-resolution generation without retraining.","tokens_in":12322,"tokens_out":4164,"duration_ms":35640,"significance":"The central idea is practical and timely: if a fixed-size latent from an existing diffusion model can be decoded at arbitrary resolution by a one-step generator, then the large installed base of latent diffusion models can be upgraded to high-resolution synthesis at low computational cost. The paper provides a broad experimental study across five diffusion backbones and multiple resolutions, and the reported speed advantage is potentially substantial. However, the significance of the claim as stated is conditional on two missing pieces of evidence: a direct measurement of content/semantic preservation during decoding, and a concrete measurement of the headline 4K latency. These gaps are load-bearing because the method is explicitly positioned as a 'content representation' decoder and a 'plug-and-play' replacement for the VAE decoder. If the decoder turns out to be a locally realistic texture prior rather than a faithful content decoder, the paradigm's main practical value—seamless upgrade of existing models—would be undermined.","major_comments":[{"comment":"The central assumption that a fixed-size latent is a complete content representation is not validated by any content-preservation metric. FIDp, sFIDp, precision, and recall are computed on 229×229 patches and measure local texture statistics; they cannot distinguish faithful decoding from a generic high-resolution texture prior that ignores part of the latent. The paper should report at least one content-fidelity experiment: for example, re-encoding consistency (encode the InfGen-decoded image back to latent and compare with the original latent), or class-conditional accuracy for DiT/SiT/MDTv2, or CLIPScore for text-to-image SD1.5/SDXL. This is load-bearing because the abstract and Section 3.2 promise 'decoding arbitrary resolution images from a compact generated latent' as a general plugin, and because the plug-and-play claim for any model sharing the latent space requires demonstrating that decoder does not introduce semantic distortion across different latent distributions.","section":"Section 3.2.1 and Tables 2–4"},{"comment":"The headline latency claim—4K image generation under 10 seconds in the abstract and 7.4 seconds in Section 5—is not supported by any reported measurement. Table 4 reports latencies only for 1024×1024 and 2048×2048, with no definition of '4K' (e.g., 3840×2160 vs. 4096×4096) and no row for 4K. The contribution statement 'improving generation speed by over ten times' is also not substantiated by the Table 4 numbers: at 2048×2048, InfGen+SDXL-B-1 is 5.4+1.9 seconds versus UltraPixel's 20 seconds, which is roughly 2.7×, not 10×. Please add a dedicated 4K latency table with hardware, base model, resolution, and a breakdown of diffusion time and InfGen decoding time, and reconcile the 'over ten times' claim with the data.","section":"Abstract and Section 5"},{"comment":"The iterative extrapolation scheme encodes the previously generated image back to latent space and decodes again with a scaling factor. The paper does not analyze error accumulation across encode-decode cycles (e.g., drift in the re-encoded latent or amplification of artifacts), and Table 3 does not specify which entries use iterative extrapolation versus a single forward pass. Since the method is presented as a 'training-free resolution extrapolation' tool, the 3072×3072 results are ambiguous without this specification. An explicit comparison of iterative versus single-shot decoding at the same resolution, along with a study of the reliability limits claimed in Table 1, is necessary to support the extrapolation claims.","section":"Section 3.3"},{"comment":"The architecture description omits parameters that are essential for reproducibility and for assessing the 'lightweight generator' claim: the number of transformer blocks, hidden widths, number of attention heads, patch size, the dimension of the Fourier feature vector in Eq. (7), and the values of μ and σ used to sample the diagonal matrix B. Without these details, a reader cannot implement the method, and the claim that InfGen is significantly cheaper than the diffusion backbone cannot be verified. Please provide a complete architecture table or specify the missing hyperparameters.","section":"Sections 3.2.2 and 3.2.3"}],"minor_comments":[{"comment":"The header 'extrapolation→image: 3072×3072' appears to have a formatting issue: '3072×3072' is repeated in two adjacent columns, and the legend for the second column is missing. Please clarify the column meanings.","section":"Table 3"},{"comment":"The notation 'InfGen+DiT' and 'InfGen+ SD1.5' is inconsistent (missing subscript or formatting); please make the method naming uniform throughout.","section":"Section 4.2"},{"comment":"The state-of-the-art comparison uses different base models for the InfGen rows (SD1.5 and SDXL-B-1) without a discussion of whether the comparison is fair given the base-model difference. The caption should state which base models are used by each competitor and by InfGen.","section":"Table 4 and Section 4.3"},{"comment":"The training data description says 10 million images with resolutions exceeding 1024^2 and a further filtering to 5 million with resolutions over 2048^2. Please clarify whether the 5 million subset is contained in the 10 million set, and how the two-stage training uses these partitions.","section":"Section 4 (Dataset)"},{"comment":"The log-scale latency plot is hard to verify without a table; please include the numerical values or a supplementary table corresponding to the points in the figure.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a genuinely interesting idea and a broad set of experiments, but the missing content-preservation metrics and the unsupported 4K latency claim are significant gaps for a paper whose main selling point is 'plug-and-play arbitrary-resolution decoding.' The recommended revision is substantial but feasible: adding re-encoding or classification/CLIP-based evaluations, a 4K latency table, and a clear specification of iterative vs. single-shot decoding. I do not see a fundamental flaw that would require rejection, but the claims as currently stated outrun the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a real idea, not a repackaging. InfGen replaces the VAE decoder with a one-step transformer generator that cross-attends over a fixed latent and uses a Fourier implicit positional embedding to produce any requested resolution. That specific combination, a trained secondary generator on the latent side rather than a diffusion-side fix, is new relative to ScaleCrafter, FouriScale, Inf-DiT, and UltraPixel. If it holds up, it is a decoder swap that could apply to every model sharing the SD/SDXL latent space, with large latency wins.\n\nWhat it does well: the experiments span five different generative models (DiT, SiT, MDTv2, FiTv2, SD1.5, SDXL) and consistently show FID/sFID improvements at 1024, 2048, and 3072, with the 32x32-latent tests showing double-digit percentage FID gains. The speed numbers at 1024/2048 are credible and match the architecture's promise. The training setup is sensible: frozen VAE encoder, two-stage resolution curriculum, 5M high-res images, patch-GAN + L1 + LPIPS losses.\n\nSoft spots, in rough order of seriousness. First, the headline '4K under 10 seconds' appears in the abstract and conclusion ('7.4 seconds') but no 4K measurement table or figure exists; the largest measured resolution is 3072. That is a load-bearing mismatch between the claim and the evidence. Second, the stress-test concern is legitimate: validation uses FIDp/sFIDp on 229x229 crops plus visual inspection, which reward locally plausible texture but do not measure whether the latent's content, object identity, count, layout, or text, survives decoding. A decoder that partially ignores z and emits generic high-res texture could still score well on these metrics. The paper would be much stronger with a content-fidelity check, e.g., re-encoding consistency or classification accuracy on class-conditional outputs. That said, the visual examples do look semantically coherent, so I would call this a missing control rather than a demonstrated failure. Third, the architecture is underspecified: no layer count, width, patch size, or the distribution parameters for the Fourier matrix B, and no code/weights. Table 2 also compares against vanilla VAEs at matched resolutions where InfGen is sometimes worse, which is fine for calibration but not a headline strength. Self-citations (refs 19, 38) are background, not the source of the main result, so no circularity concern.\n\nBottom line: the central mechanism is plausible and the empirical trend is coherent. The paper deserves serious referee time, but it needs a 4K latency measurement, a semantic-preservation experiment, and full architecture/artifact release before the strong claims are settled.\n\nRecommendation: send to peer review, require those additions.","headline":"A genuinely new decoder-side idea for arbitrary-resolution LDM generation, backed by broad but patch-level-only validation; the 4K latency headline is unmeasured and architecture details are thin.","tokens_in":12980,"tokens_out":718,"would_cite":true,"duration_ms":8097,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InfGen claims that arbitrary high-resolution images can be generated from a fixed-size diffusion latent by replacing only the VAE decoder with a one-step resolution-agnostic generator.","keywords":["arbitrary resolution image generation","latent diffusion models","one-step decoder","resolution-agnostic generation","VAE decoder replacement","implicit neural positional embedding","training-free resolution extrapolation","plug-and-play diffusion upgrade"],"falsifier":"Take a latent from a diffusion model fine-tuned on a domain far from the training data, such as medical or satellite imagery, decode it with InfGen at 2048x2048, and compare semantic fidelity against the same model's native 512x512 output; if the high-resolution output introduces structures absent from or inconsistent with the low-resolution content, the 'fixed latent is complete content' assumption fails. A quantitative version is to encode the 3072x3072 output back into the latent space and measure its divergence from the original fixed latent, where large divergence indicates the decoder changed content rather than only adding detail.","tokens_in":11812,"feed_emoji":"🖼️","tokens_out":7034,"duration_ms":56034,"temperature":0.7,"pith_summary":"InfGen claims that once a diffusion model has produced its compact latent, that latent already contains the image's content, so a single trained decoder can turn it into an image at any requested resolution, without retraining the diffusion model and without high-resolution denoising. The paper replaces the VAE decoder of latent diffusion models with a transformer-based generator that uses the fixed latent as keys and values and a resolution-shaped mask token as query, trained on real high-resolution images with reconstruction, perceptual, and adversarial losses. If the claim holds, any diffusion model sharing the same latent space can be upgraded to arbitrary high-resolution generation by swapping in InfGen, with 4K images generated in about 7.4 seconds instead of over 100 seconds. The paper reports that InfGen improves FID-based quality metrics at high resolutions on all tested backbones and is faster than the previous fastest arbitrary-resolution method.","feed_headline":"One decoder swap turns diffusion models into 4K generators","feed_subtitle":"InfGen replaces the VAE decoder so one fixed latent can be decoded at any size, cutting 4K time to under 10 seconds.","key_machinery":"The central object is InfGen's arbitrary-resolution decoder, a vision transformer that treats the fixed latent $z$ as keys and values and a mask token of shape $(\\lceil h/8\\rceil, \\lceil w/8\\rceil)$ as query, with cross-attention in transformer blocks followed by upsampling to the target size. Positional information across dynamic sizes comes from an Implicit Neural Positional Embedding (INPE) that normalizes coordinates, maps them to a unit sphere, applies Fourier features, and feeds them through a small implicit neural network to produce positional encodings for queries and keys. Training uses $ℓ_1$ reconstruction loss, LPIPS perceptual loss, and a PatchGAN adversarial loss on high-resolution crops. For resolutions beyond training, a training-free iterative extrapolation scheme decodes, re-encodes, and decodes again with bounded scale factors, reaching 4K and beyond.","core_discovery":"The central discovery is that the second stage of latent diffusion, the mapping from latent to pixels, can absorb all the cost of resolution while the first stage only needs to generate a fixed, compact content latent. InfGen trains a one-step decoder, conditioned on the fixed-size latent through cross-attention, to output an image of arbitrary height and width; the decoder is trained on crops of high-resolution images at varying target resolutions, forcing it to invent plausible texture and detail beyond what the latent explicitly specifies. Because the latent is fixed and small, inference cost no longer grows with output resolution. The paper demonstrates this on several generative backbones sharing the VAE latent space and shows that replacing the VAE decoder improves quality at high resolutions compared with upsampling the original outputs, while also beating specialized arbitrary-resolution generators in speed.","pith_inferences":["Beyond the paper's claims, the plug-and-play behavior implies a testable distributional assumption: the latents of SD, SDXL, DiT, SiT, and FiTv2 must be close enough in latent space for one decoder to handle them all, and measuring that distance would predict where the patch starts to fail.","Because the decoder is trained only on real image crops, the detail it invents is bounded by the training distribution; a diffusion model fine-tuned on a very different domain may produce latents whose high-resolution decoding hallucinates plausible but semantically wrong structures.","The iterative extrapolation loop re-encodes generated pixels, so errors can compound across cycles; tracking latent drift between the original fixed latent and the re-encoded latent after a 4K decode would show whether content is preserved or only invented.","The same split, a fixed content latent plus a one-step arbitrary-size decoder, could transfer to other generative modalities with a learned tokenizer, such as video, where compact latents would make high-resolution decoding computationally cheap."],"forward_implications":["Any diffusion model trained on the same VAE latent space can be upgraded to arbitrary-resolution generation by swapping in InfGen, with no retraining of the diffusion model.","Generating a 4K image drops from over 100 seconds to roughly 7.4 seconds, because the multi-step denoising runs only on a compact fixed-size latent.","High-resolution quality improves over naively upsampling the original model outputs, with the paper reporting FID-based improvements of 41 to 44 percent at 3072x3072 on some backbones.","InfGen also functions as a competitive image tokenizer, matching or exceeding standard VAE decoders on reconstruction metrics at native resolutions.","The training-free iterative extrapolation extends the same decoder to ultra-high resolutions beyond its training range, preserving quality up to recommended scale factors."],"supporting_citations":[{"why":"Defines the latent diffusion setup and VAE latent space that InfGen trains on, and provides the SD1.5 baseline used in comparisons.","marker":"[29]"},{"why":"Supplies the SDXL VAE latent space and text-to-image backbone that InfGen upgrades and compares against.","marker":"[24]"},{"why":"Provides the DiT backbone whose fixed-size latents InfGen decodes in the plug-and-play experiments.","marker":"[23]"},{"why":"Provides the SiT backbone, another diffusion model whose latents InfGen is applied to.","marker":"[20]"},{"why":"Provides the FiTv2 backbone used to test InfGen on a flexible-resolution transformer.","marker":"[19]"},{"why":"The previous arbitrary-resolution generation method; its latency and quality are the main baselines InfGen claims to beat.","marker":"[27]"},{"why":"A training-free high-resolution generation method used as a comparison baseline for quality and speed.","marker":"[8]"},{"why":"A memory-efficient diffusion transformer for arbitrary resolution used as a comparison baseline.","marker":"[39]"},{"why":"Establishes the VAE formalism that frames the latent as a compressed content representation.","marker":"[15]"},{"why":"Provides the implicit neural representation ideas used in the dynamic positional embedding for arbitrary output sizes.","marker":"[32]"}],"fun_headline_variants":["4K generation: 100s to under 10s with InfGen","Fixed latent, any resolution: InfGen decodes 4K in seconds","Swap the decoder, keep the latent: 4K in under 10s","InfGen: one-step decode for any resolution, 4K <10s","One decoder swap, any size output: InfGen scales to 4K"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a fixed-size latent generated by a diffusion model contains enough content information that a decoder trained only on real image crops can invent coherent, semantically correct detail at any target resolution, and that latents from different diffusion models sharing the VAE encoder fall in a distribution this decoder can handle.","fun_headline_variants_meta":{"raw":{"variants":["4K generation: 100s to under 10s with InfGen","Fixed latent, any resolution: InfGen decodes 4K in seconds","Swap the decoder, keep the latent: 4K in under 10s","InfGen: one-step decode for any resolution, 4K <10s","One decoder swap, any size output: InfGen scales to 4K"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001221,"raw_usage":{"total_tokens":4984,"prompt_tokens":867,"completion_tokens":4117,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":4014}},"tokens_in":483,"tokens_out":4117,"duration_ms":23899,"temperature":1.0,"reasoning_tokens":4014,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:54:13.260910+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a latent from a diffusion model fine-tuned on a domain far from the training data, such as medical or satellite imagery, decode it with InfGen at 2048x2048, and compare semantic fidelity against the same model's native 512x512 output; if the high-resolution output introduces structures absent from or inconsistent with the low-resolution content, the 'fixed latent is complete content' assumption fails. A quantitative version is to encode the 3072x3072 output back into the latent space and measure its divergence from the original fixed latent, where large divergence indicates the decoder changed content rather than only adding detail.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the latent diffusion setup and VAE latent space that InfGen trains on, and provides the SD1.5 baseline used in comparisons."},{"cited_title":"Fit: Flexible vision trans- former for diffusion model.ICML, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the FiTv2 backbone used to test InfGen on a flexible-resolution transformer."},{"cited_title":"Ultrapixel: Advancing ultra-high-resolution image synthesis to new peaks","cited_arxiv_id":null,"evidence_quote":"The previous arbitrary-resolution generation method; its latency and quality are the main baselines InfGen claims to beat."},{"cited_title":"Scalecrafter: Tuning-free higher- resolution visual generation with diffusion models","cited_arxiv_id":null,"evidence_quote":"A training-free high-resolution generation method used as a comparison baseline for quality and speed."},{"cited_title":"Auto-encoding varia- tional bayes.In International Conference on Learning Rep- resentations, 2014","cited_arxiv_id":null,"evidence_quote":"Establishes the VAE formalism that frames the latent as a compressed content representation."},{"cited_title":"Implicit neural representa- tions with periodic activation functions.Advances in neural information processing systems, 33:7462–7473, 2020","cited_arxiv_id":null,"evidence_quote":"Provides the implicit neural representation ideas used in the dynamic positional embedding for arbitrary output sizes."}],"review_version":2}