REVIEW 6 cited by
FiT: Flexible Vision Transformer for Diffusion Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Nature is infinitely resolution-free. In the context of this reality, existing diffusion models, such as Diffusion Transformers, often face challenges when processing image resolutions outside of their trained domain. To overcome this limitation, we present the Flexible Vision Transformer (FiT), a transformer architecture specifically designed for generating images with unrestricted resolutions and aspect ratios. Unlike traditional methods that perceive images as static-resolution grids, FiT conceptualizes images as sequences of dynamically-sized tokens. This perspective enables a flexible training strategy that effortlessly adapts to diverse aspect ratios during both training and inference phases, thus promoting resolution generalization and eliminating biases induced by image cropping. Enhanced by a meticulously adjusted network structure and the integration of training-free extrapolation techniques, FiT exhibits remarkable flexibility in resolution extrapolation generation. Comprehensive experiments demonstrate the exceptional performance of FiT across a broad range of resolutions, showcasing its effectiveness both within and beyond its training resolution distribution. Repository available at https://github.com/whlzy/FiT.
Forward citations
Cited by 6 Pith papers
-
UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation
UniMC uses tokenized instance conditions (class, box, keypoints) and a timestep-aware modulator in a DiT backbone to control multi-class human and animal image generation, trained and evaluated on the new HAIG-2.9M dataset.
-
Phase-Aligned RoPE for Mixed-Resolution Diffusion Transformer
Expressing all RoPE positions on the query's grid ('one attention, one scale') plus a small boundary content-exchange step restores mixed-resolution diffusion generation that naive position interpolation destroys.
-
PixNerd: Pixel Neural Field Diffusion
PixNerd is a single-stage pixel-space diffusion transformer that uses predicted neural field weights to decode large patches, reaching 2.15 FID on ImageNet 256 without a VAE.
-
FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems
A hybrid U-Net and Transformer diffusion backbone trained in residual space with a velocity parametrization outperforms baselines on 2D turbulence super-resolution and forecasting, and generalizes zero-shot to unseen ...
-
SAC-MIL: Spatial-Aware Correlated Multiple Instance Learning for Histopathology Whole Slide Image Classification
A spatial-aware MLP-Mixer with polar rotary position embedding and channel-shifting SAC blocks achieves state-of-the-art slide-level classification on CAMELYON-16, TCGA-LUNG, and TCGA-BRCA.
-
LSDM: LLM-Enhanced Spatio-temporal Diffusion Model for Service-Level Mobile Traffic Prediction
LSDM predicts next-hour mobile traffic per app category by feeding a diffusion model with satellite imagery, POI counts, and LLM-generated text descriptions, outperforming eight baselines on a single real-world dataset.
Discussion (0). Sign in to comment.