Pith. sign in

REVIEW 6 major objections 6 minor 3 cited by

Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting

T0 review · 6 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that splitting a teacher's latent representations into high- and low-frequency bands and distilling each band separately lets a lightweight spatiotemporal forecasting model nearly match the teacher's accuracy while…

desk verdict The paper promises a spectral-distillation method it never defines and never evaluates; the abstract's 81.3% improvement is not in any table. read the letter →

arxiv 2507.02939 v2 pith:25ZCC6AX submitted 2025-06-27 cs.LG cs.AIcs.CV

Yuqi Li , Chuanguang Yang , Hansheng Zeng , Zeyu Dong , Zhulin An , Yongjun Xu , Yingli Tian , Hao Wu This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords spatiotemporalforecastingknowledgedistillationfrequencydecouplingspectralfeaturealignmentlightweightstudentmodelsNavier-StokesequationsCNN-Transformermulti-teacher
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SDKD claims that lightweight spatiotemporal forecasters can recover most of the accuracy of a large hybrid CNN–Transformer teacher if the teacher's latent representations are split into high-frequency and low-frequency bands and each band is distilled separately rather than imitated wholesale. On four benchmarks—Rayleigh-Bénard convection, Beijing taxi flows, WeatherBench, and forced two-dimensional Navier-Stokes—the paper reports that students built from U-Net, ResNet, or MLP-Mixer backbones improve clearly under this frequency-aligned distillation, with error reductions up to 81.3% in MSE and 52.3% in MAE on the Navier-Stokes dataset, while inference becomes up to 2.28× faster than the teacher. If true, this gives resource-constrained deployments a practical route to near-teacher accuracy in traffic, weather, and fluid-dynamics forecasting without the quadratic-complexity attention of the teacher.

What carries the argument

The machinery is the spectral decoupling of the teacher's latent space: a convolution block modeled as a local gradient operator $\nabla_{x,y}$ acting as a high-pass filter (Eqs. 3–4), a Transformer block whose self-attention acts as a low-pass filter via a smoothing kernel (Eqs. 5–6), and a residual fusion of the two bands (Eq. 7). The distillation loss (Eq. 10) aligns student and teacher features in each band through an operator $\Psi$ (with separate $\Psi_h$ and $\Psi_l$ for high and low frequencies, left undefined in the paper), weighted by a balance term $\alpha$. A Plancherel-theorem error decomposition (Eq. 9) justifies treating the total prediction error as a sum of high- and low-frequency errors, and multi-teacher training uses the A2D gradient-space weighting to combine teachers that specialize in different bands or regimes.

What would settle it

Reproduce the Navier-Stokes experiment and measure the teacher's actual frequency response by computing the Fourier transform of the outputs of the convolution block and the Transformer block on identical inputs; if the convolution block does not amplify high frequencies (or the Transformer does not attenuate them) relative to the other, the spectral decoupling interpretation collapses. Separately, reconcile the abstract's 81.3% MSE reduction claim with the values in Table 4 (baseline 0.141 to AEKD 0.135), since reproducing the exact evaluation protocol would settle which number is the correct headline.

Watch

Extended reading notes

Core claim

The central claim is that spatiotemporal forecasting error decomposes into a high-frequency part (local, rapidly changing gradients) and a low-frequency part (global, slowly evolving trends), and that a teacher whose latent evolution module uses convolution as a high-pass filter and Transformer attention as a low-pass filter produces interpretable, disentangled spectral priors. Distilling those priors by aligning each frequency band separately—Eq. (10), with separate operators for high and low frequencies and a weighting term—lets a lightweight student preserve both local fine-grained variations and long-term evolution patterns, approaching teacher accuracy at far lower computational cost. The contribution list additionally asserts a theoretical proof that spectral knowledge transfer is necessary for learning long-range dependencies. Experiments across four spatiotemporal benchmarks support these claims with consistent student improvements, with the largest reported gains on the Navier-Stokes dataset.

Load-bearing premise

The load-bearing premise is that a teacher's latent space separates cleanly into a high-frequency component produced by convolution and a low-frequency component produced by self-attention, and that the alignment operator $\Psi$ in the distillation loss can be concretely defined; if either fails, the spectral distillation loss in Eq. (10) has no grounding.

Editorial extensions

If this is right

  • Lightweight student backbones (U-Net, ResNet, MLP-Mixer) can be deployed in edge or real-time settings while retaining most of the teacher's accuracy, with inference up to roughly twice as fast on the Navier-Stokes benchmark.
  • Frequency-aligned distillation could become the default for compressing hybrid spatiotemporal models, replacing generic feature-matching KD that the paper argues over-smooths high-frequency detail.
  • The same decoupling could extend to other multi-scale regression tasks such as combustion dynamics, precipitation nowcasting, or ocean modeling, where high-frequency transients and long-term trends both matter.
  • Because the method accepts heterogeneous teachers (the paper uses both ST-AlterNet and SimVP), it suggests that spectral priors can be extracted even from teachers not architecturally frequency-decoupled.
  • The A2D multi-teacher extension implies that combining teachers specializing in different frequency bands or physical regimes can be done adaptively, without manual weighting or simple averaging.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The abstract's headline numbers of 81.3% MSE and 52.3% MAE reduction on Navier-Stokes do not match the main-text table (Table 4 shows baseline 0.141 to AEKD 0.135, about a 4.3% MSE drop); reconciling this discrepancy would require clarifying the exact evaluation protocol.
  • The promised theoretical proof of the necessity of spectral transfer for long-range dependency learning is listed in the contributions but does not appear in the main text; supplying the proof or removing the assertion would materially change the strength of the interpretability claim.
  • The operator $\Psi$ in Eq. (10) is never concretely defined; a natural testable extension is to instantiate $\Psi$ as a bank of band-pass filters (Fourier or wavelet) and check whether the reported gains persist under that explicit definition.
  • The frequency roles assigned to convolution (high-pass) and self-attention (low-pass) are contestable—the paper's own related-work section cites the opposite conventional assignment—so directly measuring the spectral response of the trained teacher blocks would settle whether the decoupling is real.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes Spectral Decoupled Knowledge Distillation (SDKD), a frequency-aware distillation framework that transfers high- and low-frequency latent features from a CNN-Transformer teacher to a lightweight student for spatiotemporal forecasting. The teacher is designed to decouple frequencies through convolution (claimed high-pass) and Transformer attention (claimed low-pass); the student is trained with a spectral alignment loss in Eq. (10) plus a forecasting loss. Experiments are reported on WeatherBench, TaxiBJ+, RBC, and Navier-Stokes datasets, comparing AVER-MKD, AEKD, and CAMKD baselines. The abstract claims reductions of up to 81.3% in MSE and 52.3% in MAE on the Navier-Stokes dataset.

Significance. If the claimed improvements were supported, the paper would offer a useful approach for deploying accurate spatiotemporal forecasters on resource-constrained hardware, addressing a genuine spectral-bias problem in lightweight models. The paper usefully identifies the spectral decoupling idea, provides a complexity comparison table, and includes a public code link. However, the proposed SDKD method is not evaluated in any experiment, the operators defining its loss are left unspecified, and the headline numbers in the abstract contradict the values in Table 4. As a result, the central contribution cannot be assessed from the submitted manuscript, and the significance claim is currently unsubstantiated.

major comments (6)
  1. [Abstract; Section 4.2, Table 4] The abstract claims 'reductions of up to 81.3% in MSE and in MAE 52.3% on the Navier-Stokes equation dataset.' Table 4 shows a baseline MSE of 0.141 and a best MSE of 0.135 (AEKD), corresponding to a 4.3% reduction; MAE goes from 0.239 to 0.235, a 1.7% reduction. No value in Tables 4–6 approaches the claimed 81.3% or 52.3%, so the paper's headline empirical claim is contradicted by its own reported numbers.
  2. [Section 3.4.1, Eq. (10); Section 4.2, Tables 3–6] The proposed SDKD loss is never evaluated. The method section defines LKD with high- and low-frequency extractors, but the experimental section reports only AVER-MKD, AEKD, and CAMKD; no SDKD row or column appears in any table. The improvements for which the abstract and introduction credit SDKD therefore have no supporting experiment.
  3. [Section 3.4.1, Eq. (10)] The operators Ψ_h and Ψ_l in the spectral alignment loss are never defined. The paper does not specify their architecture, input/output dimensions, or how they are computed from the latent representations, so Eq. (10) is not an implementable algorithm. This is load-bearing because the entire distillation mechanism relies on these operators.
  4. [Section 2 vs. Section 3.2, Eqs. (3)–(6)] The paper assigns contradictory frequency roles to CNNs and Transformers. In Section 3.2, convolutional layers are claimed to act as high-pass filters and Transformers as low-pass filters (Eqs. 3–6). In Section 2, however, the related work states that 'CNNs exhibit an inductive bias toward low-frequency components' and 'Transformers ... capture high-frequency details more effectively,' citing the same lines of work. The paper cannot consistently rely on both assignments; this contradiction undermines the spectral-decoupling premise of the teacher design.
  5. [Section 1, Contributions Summary; Section 2, Key Differences] The paper promises 'a theoretical proof demonstrating the necessity of spectral knowledge transfer for learning long-range dependencies' (Contributions Summary bullet 3 and Key Differences). No such proof appears in the manuscript; Eq. (9) is an error decomposition by frequency band, not a proof of necessity. This is a claimed contribution that is absent from the text.
  6. [Section 1, Contributions Summary, third bullet] The third contribution bullet reports results for 'ST-AlterNet', 'AEKD', and 'Weatherbench' (e.g., 'U-Net with AEKD (AB Loss) achieves MAE=0.8541, PSNR=31.4527, and SSIM=0.8812 on Weatherbench'), none of which are the proposed SDKD method or its headline NSE experiments. This bullet appears to have been carried over from a different manuscript and makes the contribution summary unreliable.
minor comments (6)
  1. [Table 3] The column labels are partially merged in the caption and the body (e.g., 'MAE PSNR SSIM MSE MAE SSIM'), making it difficult to tell which metrics correspond to which dataset; the numerical values in the first data row are also run together. Please reformat the table.
  2. [Tables 4–6] The method 'CAMKD' is reported as the best performer but is never defined in the method or related-work sections; please add a definition.
  3. [Table 5] The row 'SINGLE TEACHER (VAN)' is not described in the experimental setup; please explain what this baseline is.
  4. [Section 4.5] The RBC inference time comparison (0.0192 s vs 0.0189 s) is described as a 'smaller yet still notable speedup,' but the ratio is approximately 1.016, which is not notable; please soften or correct this claim.
  5. [Section 4.1, Table 2] The TaxiBJ+ entry reports a 30-minute interval in the text but Table 2 lists '1 day'; please reconcile the two descriptions.
  6. [Section 3.5] 'Inference speed increases by 2.28× compared to the teacher' is consistent with the NS numbers in Section 4.5, but no parameter or FLOPs comparison is given for the student models, despite Table 1 motivating the work with parameter counts.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction in the derivation chain; the central issues are missing specification and untested claims, not equivalence-by-construction.

full rationale

The derivation chain does not reduce to its own inputs. The teacher's latent-space split into high- and low-frequency branches (Eqs. 3-7) is presented as an architectural choice; the distillation loss Eq. (10) compares student features Ψ(G(X)) to teacher features Ψ(F(X)) while the overall objective includes the task loss ∥G(X)-Y∥², and the forecasting target Y provides external ground truth, so the student's improvement is not forced by the teacher's self-definition. The A2D multi-teacher weighting is adopted from an external prior work (Du et al. [7]), and the cited spectral-bias results ([14], [25]) are from outside the author set; although the paper's use of those citations is internally inconsistent (the related work assigns CNNs low-frequency bias and Transformers high-frequency bias, contradicting Section 3.2), that is a correctness issue rather than a circular one. No uniqueness theorem or load-bearing self-citation appears. The more serious problems—the operators Ψ_h and Ψ_l in Eq. (10) are never defined, no experiment in Tables 3-6 evaluates SDKD (rows are AVER-MKD, AEKD, CAMKD), and the claimed 81.3% MSE / 52.3% MAE reductions are not traceable to any reported number—are failures of specification and evidence, not circularity, and do not raise the circularity score under the hard rules.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. Its load-bearing assumptions are behavioral claims about CNN and Transformer frequency responses, which the paper itself contradicts elsewhere, plus the entirely unspecified spectral feature extractors Psi_h and Psi_l. The free hyperparameters lambda, alpha, and C are not given values, making the distillation loss and multi-teacher weighting impractical to reproduce.

free parameters (3)
  • lambda = not reported
    Balances the forecasting loss and the spectral distillation loss in Eq. (11). No value is given in the text.
  • alpha = not reported
    Balances high-frequency versus low-frequency alignment in Eq. (10). No value is specified.
  • C = not reported
    Tolerance hyperparameter for multi-teacher gradient weighting in Eq. (13). No value is stated.
assumptions (4)
  • domain assumption Convolution layers act as high-pass filters via local gradient operators (Eqs. 3-4).
    Invoked as the basis for the teacher's high-frequency extraction. The paper cites Park et al. [25], but this is a contested behavioral claim and the related work section itself describes CNNs as low-frequency biased.
  • domain assumption Self-attention acts as a low-pass filter with the response in Eq. (6).
    Invoked as the basis for the teacher's low-frequency modeling. The paper cites Jacot et al. [14], but the exact functional form in Eq. (6) is not derived or shown to follow from that reference.
  • ad hoc to paper There exist spectral feature extractors Psi_h and Psi_l that isolate high- and low-frequency components of the latent space (Eq. 10).
    The entire distillation loss depends on these operators, but the paper never defines them or describes how they are computed. This is central to the method and is left entirely unspecified.
  • standard math Parseval/Plancherel decomposition allows splitting the total squared error into high- and low-frequency components around a cutoff omega_c (Eq. 9).
    This is a standard Fourier-analytic identity and is the only mathematically solid assumption in the derivation, assuming the signals are square-integrable and the cutoff is well defined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting." pith.science (2026). https://pith.science/paper/25ZCC6AX

@misc{pith2026250702939,
  author       = {Pith},
  title        = {Pith review of: Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25ZCC6AX}},
  note         = {Machine review of arXiv:2507.02939}
}
read the original abstract

Spatiotemporal forecasting tasks, such as traffic flow, combustion dynamics, and weather forecasting, often require complex models that suffer from low training efficiency and high memory consumption. This paper proposes a lightweight framework, Spectral Decoupled Knowledge Distillation (termed SDKD), which transfers the multi-scale spatiotemporal representations from a complex teacher model to a more efficient lightweight student network. The teacher model follows an encoder-latent evolution-decoder architecture, where its latent evolution module decouples high-frequency details and low-frequency trends using convolution and Transformer (global low-frequency modeler). However, the multi-layer convolution and deconvolution structures result in slow training and high memory usage. To address these issues, we propose a frequency-aligned knowledge distillation strategy, which extracts multi-scale spectral features from the teacher's latent space, including both high and low frequency components, to guide the lightweight student model in capturing both local fine-grained variations and global evolution patterns. Experimental results show that SDKD significantly improves performance, achieving reductions of up to 81.3% in MSE and in MAE 52.3% on the Navier-Stokes equation dataset. The framework effectively captures both high-frequency variations and long-term trends while reducing computational complexity. Our codes are available at https://github.com/itsnotacie/SDKD

Figures

Figures reproduced from arXiv: 2507.02939 by the authors.

Figure 1
Figure 1. Architecture Overview of SDKD. Stage 1: Teacher model pretraining with frequency decoupling. Stage 2: Lightweight student architecture construction using parameter-efficient backbones like ResNet or U-Net. Stage 3: Offline Training with Knowledge Distillation. Stage 4: Online Prediction on New Inputs. Fhigh(Z)(x, y) = X |α|≤k aα · ∂ αZ(x, y). (4) The term α is the differentiation order, and aα is a learn￾able parame… view at source ↗
Figure 2
Figure 2. Spectral characteristics analysis in the Weatherbench [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. Inference speedup comparison between teacher and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Should Active RAG Retrieve? A Budget-Aware Evaluation of Utility, Calibration, and Cost

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Active RAG evaluation should be budget-aware: report exact and deployable frontiers, realized usage, harm rates, and cost decompositions instead of single-point accuracy.

  2. SWA-SOP: Spatially-aware Window Attention for Semantic Occupancy Prediction in Autonomous Driving

    cs.CV 2025-06 conditional novelty 6.0 of 10

    SWA-SOP shows that sliding-window attention with per-slot spatial embeddings and a center query improves semantic occupancy prediction on LiDAR and camera inputs, but the headline benchmark numbers are second-best rat...

  3. ${C}^{3}$-GS: Learning Context-aware, Cross-dimension, Cross-scale Feature for Generalizable Gaussian Splatting

    cs.CV 2025-08 conditional novelty 4.0 of 10

    C3-GS improves generalizable Gaussian Splatting by adding coordinate-aware attention, cross-dimensional attention, and cross-scale opacity fusion to the MVSGaussian baseline.

Reference graph

Works this paper leans on

54 extracted references · 43 canonical work pages · cited by 3 Pith papers

  1. [1]

    Dissecting the high-frequency bias in convolutional neural networks

    Antonio A Abello, Roberto Hirata, and Zhangyang Wang. Dissecting the high-frequency bias in convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 863–871,

    work page

  2. [2]

    Accurate medium-range global weather forecasting with 3d neural networks

    Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):533– 538, 2023. 1

    work page 2023

  3. [3]

    Choose a transformer: Fourier or galerkin

    Shuhao Cao. Choose a transformer: Fourier or galerkin. Advances in neural information processing systems, 34:24924– 24940, 2021. 3

    work page 2021

  4. [4]

    Towards understanding the spectral bias of deep learning

    Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, and Quanquan Gu. Towards understanding the spectral bias of deep learning. arXiv preprint arXiv:1912.01198, 2019. 2

    arXiv 1912

  5. [5]

    Selective frequency network for image restoration

    Yuning Cui, Yi Tao, Zhenshan Bing, Wenqi Ren, Xinwei Gao, Xiaochun Cao, Kai Huang, and Alois Knoll. Selective frequency network for image restoration. In The Eleventh International Conference on Learning Representations, 2023. 3

    work page 2023

  6. [6]

    Deep learning for spatio-temporal modeling: dynamic traffic flows and high frequency trading

    Matthew F Dixon, Nicholas G Polson, and Vadim O Sokolov. Deep learning for spatio-temporal modeling: dynamic traffic flows and high frequency trading. Applied Stochastic Models in Business and Industry, 35(3):788–807, 2019. 1

    work page 2019

  7. [7]

    Agree to disagree: Adap- tive ensemble knowledge distillation in gradient space

    Shangchen Du, Shan You, Xiaojie Li, Jianlong Wu, Fei Wang, Chen Qian, and Changshui Zhang. Agree to disagree: Adap- tive ensemble knowledge distillation in gradient space. ad- vances in neural information processing systems, 33:12345– 12355, 2020. 5, 7

    work page 2020

  8. [8]

    Neuralom: Neural ocean model for subseasonal-to-seasonal simulation

    Yuan Gao, Ruiqi Shu, Hao Wu, Fan Xu, Yanfei Xiang, Rui- jian Gou, Qingsong Wen, Xian Wu, and Xiaomeng Huang. Neuralom: Neural ocean model for subseasonal-to-seasonal simulation. arXiv preprint arXiv:2505.21020, 2025. 1

    arXiv 2025

Show all 54 references
  1. [9]

    Simvp: Simpler yet better video prediction

    Zhangyang Gao, Cheng Tan, Lirong Wu, and Stan Z Li. Simvp: Simpler yet better video prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3170–3180, 2022. 6

    2022

  2. [10]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  3. [11]

    Knowledge transfer via distillation of activation bound- aries formed by hidden neurons

    Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi. Knowledge transfer via distillation of activation bound- aries formed by hidden neurons. In Proceedings of the AAAI conference on artificial intelligence, pages 3779–3787, 2019. 7

    2019

  4. [12]

    Distilling the knowledge in a neural network

    Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 3, 7

    2015 arXiv

  5. [13]

    Knowledge distillation on spatial-temporal graph convolu- tional network for traffic prediction

    Mohammad Izadi, Mehran Safayani, and Abdolreza Mirzaei. Knowledge distillation on spatial-temporal graph convolu- tional network for traffic prediction. International Journal of Computers and Applications, pages 1–12, 2024. 3

    2024

  6. [14]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. Advances in neural information processing systems, 31, 2018. 3, 4

    2018

  7. [15]

    Show, attend and distill: Knowledge distillation via attention-based fea- ture matching

    Mingi Ji, Byeongho Heo, and Sungrae Park. Show, attend and distill: Knowledge distillation via attention-based fea- ture matching. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7945–7952, 2021. 3

    2021

  8. [16]

    Graph neural network for traf- fic forecasting: A survey

    Weiwei Jiang and Jiayun Luo. Graph neural network for traf- fic forecasting: A survey. Expert Systems with Applications, 207:117921, 2022. 1

    2022

  9. [17]

    Frequency-guided masking for enhanced vision self- supervised learning

    Amin Karimi Monsefi, Mengxi Zhou, Nastaran Karimi Mon- sefi, Ser-Nam Lim, Wei-Lun Chao, and Rajiv Ram- nath. Frequency-guided masking for enhanced vision self- supervised learning. arXiv e-prints, pages arXiv–2409, 2024. 3

    2024

  10. [18]

    Fourier neural operator for parametric par- tial differential equations

    Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and An- ima Anandkumar. Fourier neural operator for parametric par- tial differential equations. arXiv preprint arXiv:2010.08895,

    2010 arXiv

  11. [19]

    Curriculum temperature for knowledge distillation

    Zheng Li, Xiang Li, Lingfeng Yang, Borui Zhao, Renjie Song, Lei Luo, Jun Li, and Jian Yang. Curriculum temperature for knowledge distillation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 1504–1512, 2023. 3

    2023

  12. [20]

    Promptkd: Unsupervised prompt distillation for vision-language models

    Zheng Li, Xiang Li, Xinyi Fu, Xin Zhang, Weiqiang Wang, Shuo Chen, and Jian Yang. Promptkd: Unsupervised prompt distillation for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26617–26626, 2024. 3

    2024

  13. [21]

    Conditional local convolution for spatio- temporal meteorological forecasting

    Haitao Lin, Zhangyang Gao, Yongjie Xu, Lirong Wu, Ling Li, and Stan Z Li. Conditional local convolution for spatio- temporal meteorological forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7470–7478,

  14. [22]

    Investigating and explaining the frequency bias in image classification

    Zhiyu Lin, Yifei Gao, and Jitao Sang. Investigating and explaining the frequency bias in image classification. arXiv preprint arXiv:2205.03154, 2022. 2

    2022 arXiv

  15. [23]

    Machine learn- ing for geographically differentiated climate change mitiga- tion in urban areas.Sustainable Cities and Society, 64:102526,

    Nikola Milojevic-Dupont and Felix Creutzig. Machine learn- ing for geographically differentiated climate change mitiga- tion in urban areas.Sustainable Cities and Society, 64:102526,

  16. [24]

    Urban traffic prediction from spatio- temporal data using deep meta learning

    Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. Urban traffic prediction from spatio- temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1720–1730, 2019. 1

    2019

  17. [25]

    How do vision transformers work? arXiv preprint arXiv:2202.06709, 2022

    Namuk Park and Songkuk Kim. How do vision transformers work? arXiv preprint arXiv:2202.06709, 2022. 2, 3

    2022 arXiv

  18. [26]

    Frequency attention for 9 knowledge distillation

    Cuong Pham, Van-Anh Nguyen, Trung Le, Dinh Phung, Gus- tavo Carneiro, and Thanh-Toan Do. Frequency attention for 9 knowledge distillation. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 2277–2286, 2024. 3

    2024

  19. [27]

    Do vision trans- formers see like convolutional neural networks? Advances in neural information processing systems, 34:12116–12128,

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks? Advances in neural information processing systems, 34:12116–12128,

  20. [28]

    On the spectral bias of neural networks

    Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International conference on machine learning, pages 5301–

  21. [29]

    Fitnets: Hints for thin deep nets

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550 ,

    arXiv

  22. [30]

    U- net: Convolutional networks for biomedical image segmenta- tion

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmenta- tion. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Pa...

    2015

  23. [31]

    Understanding depthwise separable convolu- tions and the efficiency of mobilenets

    Arjun Sarkar. Understanding depthwise separable convolu- tions and the efficiency of mobilenets. Towards Data Science,

  24. [32]

    Convolutional lstm network: A machine learning approach for precipitation now- casting, 2015

    Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai kin Wong, and Wang chun Woo. Convolutional lstm network: A machine learning approach for precipitation now- casting, 2015. 1, 3

    2015

  25. [33]

    Ocean-e2e: Hybrid physics-based and data-driven global fore- casting of extreme marine heatwaves with end-to-end neural assimilation

    Ruiqi Shu, Yuan Gao, Hao Wu, Ruijian Gou, Yanfei Xiang, Fan Xu, Qingsong Wen, Xian Wu, and Xiaomeng Huang. Ocean-e2e: Hybrid physics-based and data-driven global fore- casting of extreme marine heatwaves with end-to-end neural assimilation. arXiv preprint arXiv:2505.22071, 2025. 1

    2025 arXiv

  26. [34]

    Estimating low-frequency variabil- ity and trends in atmospheric temperature using era-interim

    AJ Simmons, P Poli, DP Dee, P Berrisford, H Hersbach, S Kobayashi, and C Peubey. Estimating low-frequency variabil- ity and trends in atmospheric temperature using era-interim. Quarterly Journal of the Royal Meteorological Society, 140 (679):329–353, 2014. 1

    2014

  27. [35]

    Mlp- mixer: An all-mlp architecture for vision

    Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lu- cas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp- mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:...

    2021

  28. [36]

    High- frequency component helps explain the generalization of con- volutional neural networks

    Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High- frequency component helps explain the generalization of con- volutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8684–8694, 2020. 2

    2020

  29. [37]

    Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms

    Yunbo Wang, Mingsheng Long, Jianmin Wang, Zhifeng Gao, and Philip S Yu. Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. Advances in neural information processing systems, 30, 2017. 1, 3

    2017

  30. [38]

    Earthfarsser: Versatile spatio-temporal dynamical systems modeling in one model

    Hao Wu, Yuxuan Liang, Wei Xiong, Zhengyang Zhou, Wei Huang, Shilong Wang, and Kun Wang. Earthfarsser: Versatile spatio-temporal dynamical systems modeling in one model. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 15906–15914, 2024. 1, 3, 6

    2024

  31. [39]

    Pure: Prompt evolution with graph ode for out-of-distribution fluid dynamics model- ing

    Hao Wu, Changhu Wang, Fan Xu, Jinbao Xue, Chong Chen, Xian-Sheng Hua, and Xiao Luo. Pure: Prompt evolution with graph ode for out-of-distribution fluid dynamics model- ing. Advances in Neural Information Processing Systems, 37: 104965–104994, 2024. 1

    2024

  32. [40]

    Neural manifold operators for learning the evolu- tion of physical dynamics

    Hao Wu, Kangyu Weng, Shuyi Zhou, Xiaomeng Huang, and Wei Xiong. Neural manifold operators for learning the evolu- tion of physical dynamics. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Min- ing, pages 3356–3366, 2024. 6

    2024

  33. [41]

    Pastnet: Introducing physical inductive biases for spatio-temporal video prediction

    Hao Wu, Fan Xu, Chong Chen, Xian-Sheng Hua, Xiao Luo, and Haixin Wang. Pastnet: Introducing physical inductive biases for spatio-temporal video prediction. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 2917–2926, 2024. 1

    2024

  34. [42]

    Turb-l1: Achieving long-term turbulence tracing by tackling spectral bias

    Hao Wu, Yuan Gao, Ruiqi Shu, Zean Han, Fan Xu, Zhihong Zhu, Qingsong Wen, Xian Wu, Kun Wang, and Xiaomeng Huang. Turb-l1: Achieving long-term turbulence tracing by tackling spectral bias. arXiv preprint arXiv:2505.19038, 2025. 1

    2025

  35. [43]

    Advanced long-term earth system fore- casting by learning the small-scale nature

    Hao Wu, Yuan Gao, Ruiqi Shu, Kun Wang, Ruijian Gou, Chuhan Wu, Xinliang Liu, Juncai He, Shuhao Cao, Jun- feng Fang, et al. Advanced long-term earth system fore- casting by learning the small-scale nature. arXiv preprint arXiv:2505.19432, 2025. 1

    2025

  36. [44]

    Hierarchical self-supervised augmented knowledge dis- tillation

    Chuanguang Yang, Zhulin An, Linhang Cai, and Yongjun Xu. Hierarchical self-supervised augmented knowledge dis- tillation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pages 1217–1223, 2021. 1

    2021

  37. [45]

    Mutual contrastive learning for visual representation learning

    Chuanguang Yang, Zhulin An, Linhang Cai, and Yongjun Xu. Mutual contrastive learning for visual representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3045–3053, 2022. 1

    2022

  38. [46]

    Cross-image relational knowl- edge distillation for semantic segmentation

    Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, and Qian Zhang. Cross-image relational knowl- edge distillation for semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12319–12328, 2022. 3

    2022

  39. [47]

    Online knowledge distillation via mutual contrastive learning for visual recognition

    Chuanguang Yang, Zhulin An, Helong Zhou, Fuzhen Zhuang, Yongjun Xu, and Qian Zhang. Online knowledge distillation via mutual contrastive learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):10212–10227, 2023. 1

    2023

  40. [48]

    Clip-kd: An empirical study of clip model distillation

    Chuanguang Yang, Zhulin An, Libo Huang, Junyu Bi, Xin- qiang Yu, Han Yang, Boyu Diao, and Yongjun Xu. Clip-kd: An empirical study of clip model distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15952–15962, 2024. 3

    2024

  41. [49]

    Multi-teacher knowledge distillation with reinforcement learning for visual recognition

    Chuanguang Yang, Xinqiang Yu, Han Yang, Zhulin An, Chengqing Yu, Libo Huang, and Yongjun Xu. Multi-teacher knowledge distillation with reinforcement learning for visual recognition. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, pages 9148–9156, 2025. 5

    2025

  42. [50]

    Decoupling dark 10 knowledge via block-wise logit distillation for feature-level alignment

    Chengting Yu, Fengzhao Zhang, Ruizhe Chen, Aili Wang, Zuozhu Liu, Shurun Tan, and Er-Ping Li. Decoupling dark 10 knowledge via block-wise logit distillation for feature-level alignment. IEEE Transactions on Artificial Intelligence, 2024. 2

    2024

  43. [51]

    Ginar+: A robust end-to-end framework for multivariate time series forecasting with missing values

    Chengqing Yu, Fei Wang, Zezhi Shao, Tangwen Qian, Zhao Zhang, Wei Wei, Zhulin An, Qi Wang, and Yongjun Xu. Ginar+: A robust end-to-end framework for multivariate time series forecasting with missing values. IEEE Transactions on Knowledge and Data Engineering, 2025. 1

    2025

  44. [52]

    Merlin: Multi-view representation learning for robust multivariate time series forecasting with unfixed missing rates

    Chengqing Yu, Fei Wang, Chuanguang Yang, Zezhi Shao, Tao Sun, Tangwen Qian, Wei Wei, Zhulin An, and Yongjun Xu. Merlin: Multi-view representation learning for robust multivariate time series forecasting with unfixed missing rates. arXiv preprint arXiv:2506.12459, 2025. 1

    2025 arXiv

  45. [53]

    Skilful nowcasting of extreme precipitation with nowcastnet

    Yuchen Zhang, Mingsheng Long, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I Jordan, and Jianmin Wang. Skilful nowcasting of extreme precipitation with nowcastnet. Nature, 619(7970):526–532, 2023. 1

    2023

  46. [54]

    Freekd: Knowledge distillation via semantic frequency prompt

    Yuan Zhang, Tao Huang, Jiaming Liu, Tao Jiang, Kuan Cheng, and Shanghang Zhang. Freekd: Knowledge distillation via semantic frequency prompt. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15931–15940, 2024. 2 11

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.