{"id":"4d7a00c8-737e-4577-a919-b1f02eea5fd8","arxiv_id":"2501.11043","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"BF-STVSR combines B-spline based temporal interpolation and Fourier based spatial encoding to achieve state-of-the-art continuous space-time video super-resolution without optical flow networks.","lead":"This paper introduces two new modules for continuous video super-resolution, a B-spline temporal mapper and a Fourier spatial mapper, and shows they improve interpolation quality while removing the need for a pre-trained optical flow network. The method achieves slightly better PSNR and SSIM than prior state-of-the-art on standard datasets, with lower computational cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The undefined 'ground-truth forward motion' warmup in Sec. 4.1 may reintroduce a pre-trained optical flow dependency during training, undercutting the no-optical-flow claim and the SOTA attribution.","rationale":"I read the paper as aiming to make C-STVSR self-contained by replacing RAFT with internal B-spline and Fourier motion/spatial mappers, while still achieving SOTA. For that claim to hold, the learning procedure must not secretly rely on the external flow component it says it eliminates. The reader's weakest-assumption analysis targets exactly this: Sec. 4.1 invokes 'ground-truth forward motion' without definition, and since none of the evaluation datasets provide true physical flow, the only concrete way to obtain such motion is to estimate it with a pre-trained flow network on ground-truth frames. This is not an internal inconsistency in the forward pass, but it is a load-bearing gap in the evidence for the central novelty. If confirmed, the paper's statement that it 'removes the need for an external optical flow network' would be true only at inference, and the reported SOTA margins could be partly attributed to a warmup that uses RAFT-like flow as a teacher. The released code makes this check feasible, so I would not reject the paper; instead I would keep the reader's CONDITIONAL verdict and require either a clear definition of the warmup source or a no-warmup retraining ablation. If code inspection shows the warmup actually uses an analytic or dataset-provided motion signal, the concern is refuted and the claim stands. The reader identified the same load-bearing concern, so my assessment agrees with the reader's weakest-assumption and verdict.","tokens_in":14537,"tokens_out":6436,"duration_ms":65806,"concrete_test":"Inspect the released training code (github.com/Eunjnnn/bfstvsr) for how 'ground-truth forward motion' is produced in the Sec. 4.1 warmup. Concretely, search the data pipeline and training loop for calls to RAFT or precomputed flow files; if the warmup loads or generates flow fields with a pre-trained network and feeds them into the warping operator, the hidden dependency is confirmed. A complementary decisive ablation is to retrain BF-STVSR from scratch with the substitution probability fixed at 0 for all 600k iterations and compare GoPro-Adobe PSNR/SSIM to Table 1; if metrics drop materially or training diverges, the warmup is load-bearing and must be disclosed as a training-time optical-flow dependency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Sec. 3.1) is that BF-STVSR 'removes the need for an external optical flow network (e.g., RAFT)' via a learnable internal B-spline motion model, and Eq. (6) drops the L_RAFT loss. However, Sec. 4.1 states: 'To ensure training stability, we substitute the predicted forward motion with the ground-truth forward motion with a certain probability, starting from 1.0 and gradually reducing to 0 over the first 150,000 iterations.' The term 'ground-truth forward motion' is never defined, and Adobe240, Vid4, and GoPro do not provide ground-truth flow. The only plausible source is a pre-trained flow estimator (e.g., RAFT) run on ground-truth frames to produce the motion fields used during the 150k-iteration warmup. If so, the model is trained with a strong teacher signal from the very external component the paper claims to eliminate: it is only true that no flow is used at inference. This hidden dependency could explain part of the reported gains over MoTIF, since the warmup may initialize or regularize motion learning. It also makes Table 4's last row ('no O·F, no L_RAFT') potentially misleading because the warmup substitution is not one of the ablated factors. The claim is not internally inconsistent, but the missing definition blocks verification of the paper's central contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BF-STVSR, a continuous spatial-temporal video super-resolution (C-STVSR) method that replaces the external optical flow network (RAFT) used in MoTIF with two learned modules: a B-spline Mapper for smooth temporal motion interpolation and a Fourier Mapper for spatial high-frequency detail. The network takes two low-resolution frames and produces a high-resolution frame at an arbitrary time t and spatial scale, using forward warping with predicted motion and reliability maps. The training objective drops the optical-flow supervision term L_RAFT of MoTIF, retaining only a Charbonnier reconstruction loss. Experiments on Vid4, GoPro, and Adobe240 report PSNR/SSIM and video-quality metrics (VFIPS, FloLPIPS, tOF, VMAF), as well as FLOPs and inference-time comparisons. The paper claims state-of-the-art performance in most settings, with acknowledged exceptions (Vid4, one out-of-distribution scale, and FloLPIPS on GoPro).","tokens_in":14844,"tokens_out":4151,"duration_ms":44712,"significance":"If the central claim holds, BF-STVSR is a meaningful step for C-STVSR: it replaces a bulky pre-trained optical-flow component with a lightweight, self-contained motion model, while improving or matching accuracy and reducing inference cost. The paper is generally honest about its exceptions, provides code, and reports multiple perceptual metrics, which strengthens reproducibility. The main significance hinges on whether the training procedure truly avoids external optical flow; the undefined 'ground-truth forward motion' warmup in Section 4.1 is a load-bearing ambiguity that must be resolved before the no-optical-flow claim can be accepted.","major_comments":[{"comment":"The sentence 'To ensure training stability, we substitute the predicted forward motion with the ground-truth forward motion with a certain probability, starting from 1.0 and gradually reducing to 0 over the first 150,000 iterations' is not backed by a definition of 'ground-truth forward motion.' The training datasets (Adobe240, Vid4, GoPro) do not provide ground-truth optical flow, so the only plausible source is an external flow estimator applied to ground-truth frames, e.g., RAFT. If this is the case, the claim in Sec. 3.1 that BF-STVSR 'removes the need for an external optical flow network' is only true at inference; the training procedure is still teacher-forced with external flow estimates. Moreover, Table 4's last row, labeled as the variant without O·F and without L_RAFT, still includes this warmup substitution, so it does not demonstrate the effect of removing external flow entirely. Please specify the exact source of the ground-truth motion and, if it comes from a pre-trained network, provide an ablation or a re-training without the warmup to verify that the reported performance is not attributable to this hidden dependency.","section":"Sec. 4.1"},{"comment":"The reported PSNR/SSIM improvements over MoTIF are small (e.g., GoPro-Average 30.22 vs. 30.04 dB; Adobe-Average 30.12 vs. 29.82 dB). No error bars, confidence intervals, or multiple-seed statistics are provided. Given typical run-to-run variance in deep learning, these margins may be within statistical noise. Please report the standard deviation or the range over at least three independent training runs for the main comparison, or otherwise justify that the improvements are stable and not due to a single lucky run.","section":"Sec. 4.2, Tables 1-3"}],"minor_comments":[{"comment":"The abstract states 'Our approach achieves state-of-the-art in various metrics, including PSNR and SSIM,' but Table 1 shows that on Vid4, TMNet outperforms BF-STVSR, and Table 2 shows that at ×16 temporal and ×4 spatial scale, LIIF/LTE are better. The paper explicitly acknowledges these exceptions in the text, but the abstract and conclusion would benefit from a qualifier, such as 'on most evaluated settings.'","section":"Abstract and Sec. 4.2"},{"comment":"The text says 'our model consistently outperforms the baselines across all metrics, except for the FloLPIPS on GoPro dataset,' but the reported FloLPIPS for GoPro is 0.151 for both VideoINR and BF-STVSR, i.e., a tie rather than a clear exception. Please clarify whether this is a tie or whether one is strictly better.","section":"Table 3"},{"comment":"For the FLOPs and inference-time comparison, please specify the input resolution and whether the reported numbers include the encoder, decoders, and warping operations. This would make the efficiency comparison more reproducible.","section":"Sec. 4.4, Fig. 5"},{"comment":"In Eq. (1), d = p_d(g) is written as a scalar-dependent estimate, but it is used as a denominator with the B-spline argument (t̂ - k_r)/d. Since p_d outputs are per-channel, please clarify the broadcasting or the shape of d to avoid ambiguity.","section":"Eq. (1)"},{"comment":"References [34] and [35] appear to refer to the same paper (Su et al., 'Deep video deblurring for hand-held cameras', CVPR 2017) with different page ranges. Please consolidate or correct the duplicate citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the undefined 'ground-truth forward motion' warmup in Sec. 4.1. This is not a subtle point; it directly bears on the paper's central novelty and on the interpretation of Table 4. Please request the authors to define the term and, if it relies on an external flow estimator, to provide an ablation without this warmup. The small performance margins also warrant a request for variance estimates. The rest of the methodology appears sound and the paper is otherwise clearly written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a reasonable incremental step for continuous space-time video super-resolution. What's actually new: a B-spline temporal mapper plus a Fourier spatial mapper, both adapted from prior image-SR work (BTC, LTE), combined to replace the pre-trained optical flow network at inference. The empirical observation that naive position encoding degrades performance in this setting, especially with flow, is interesting and not in the prior C-STVSR papers. The paper ships code, reports FLOPs and latency, and includes honest limitations (large motion, training cost). That's real evidence and should be credited. The soft spots are mostly about verification, not method. The biggest one: Section 4.1 says training substitutes predicted forward motion with 'ground-truth forward motion' starting from probability 1.0 and decaying over 150k iterations. The term is never defined. Adobe240, Vid4, and GoPro don't provide ground-truth flow, so the only plausible source is a pre-trained flow estimator like RAFT run on the training frames. If that's the case, the paper's central claim of removing the external optical flow dependency is only true at inference, and the training still leans on the very component the authors say they eliminate. This also makes Table 4's last row (no O·F, no L_RAFT) misleading if the warmup still uses flow-generated targets. This isn't necessarily fatal—inference is still flow-free and the efficiency numbers hold—but it blocks verification of the paper's main story. Second, the reported gains over MoTIF are modest: around 0.1–0.3 dB PSNR on the key benchmarks. With no error bars or significance tests, that margin is within typical run-to-run variation. The SOTA claim is plausible but fragile. The citation pattern is fine; the adaptations from LTE and BTC are properly cited, and self-citation is not an issue given the code and reproducible setup. Who gets value from this: researchers working on C-STVSR or implicit neural video representations. It deserves a serious referee, but the referee should be asked to pin down the warmup's source of 'ground-truth' motion and to provide variance information. My own verdict is conditional: the method is promising, but the training dependency needs clarification before I'd trust the no-optical-flow claim.","headline":"Solid incremental C-STVSR work with code and honest experiments, but the undefined 'ground-truth forward motion' in the training warmup is a verification gap that must be closed before the no-optical-flow claim is taken at face value.","tokens_in":647,"tokens_out":838,"would_cite":false,"duration_ms":23025,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A continuous video super-resolution framework using B-spline and Fourier mappers achieves state-of-the-art space-time interpolation without any optical flow network at inference.","keywords":["video super-resolution","frame interpolation","implicit neural representation","B-spline basis","Fourier features","optical flow","continuous space-time","arbitrary-scale upsampling"],"falsifier":"Inspect the released code or checkpoints to identify the source of the 'ground-truth forward motion' supervision schedule; alternatively, retrain the model from scratch without any such substitution and compare GoPro-Average PSNR/SSIM to the reported 30.22 / 0.8802. A substantial drop would show that the hidden flow supervision is load-bearing, while a match would confirm the no-flow claim.","tokens_in":14347,"feed_emoji":"🎬","tokens_out":9225,"duration_ms":86190,"temperature":0.7,"pith_summary":"The paper claims that continuous spatial-temporal video super-resolution—upsampling video to arbitrary spatial and temporal scales in one model—can be done without any pre-trained optical flow network by replacing motion and feature estimation with two learned basis-function mappers. A B-spline Mapper predicts spline coefficients and knots from encoded frame features and evaluates them at the target time to produce smooth motion vectors, while a Fourier Mapper predicts dominant frequencies and amplitudes to capture fine spatial detail. The authors report that naive position encoding degrades performance in prior implicit neural representation models, and that their mappers outperform these baselines on Vid4, GoPro, and Adobe240 benchmarks in PSNR, SSIM, and perceptual video metrics, at lower FLOPs and faster inference. If the paper is right, arbitrary-scale video upsampling becomes a simpler, faster, and more self-contained operation.","feed_headline":"B-splines and Fourier bases beat optical flow for video upsampling","feed_subtitle":"One model interpolates video in space and time at any scale, with lower FLOPs","key_machinery":"The B-spline Mapper parameterizes temporal motion: for each reference frame, estimators predict coefficient $c_r$, knot $k_r$, and dilation $d$; the spline basis $\\beta^n((\\hat{t}-k_r)/d)$ is evaluated at the relative time $\\hat{t}$, and a linear projection $f_{\\theta_b}$ converts the weighted basis into motion vectors $M^{H}_{t_r\\to t}$ and reliability maps $Z^{H}_{t_r\\to t}$. The Fourier Mapper represents spatial features by predicting amplitude $A_r$ and frequency $F_r$ from the nearest latent feature $z_r$ and evaluating $A_r \\odot [\\cos(\\pi F_r \\delta_r); \\sin(\\pi F_r \\delta_r)]$, then projecting with a linear layer. These two mappers replace the coordinate-concatenation MLPs and the RAFT-based flow features of prior C-STVSR models; softmax splatting carries the warped features forward, and the system is trained end-to-end with only the frame-reconstruction loss.","core_discovery":"BF-STVSR is a one-stage continuous space-time video super-resolution framework. Given two low-resolution frames, an encoder produces latent features; the B-spline Mapper predicts, for each reference frame, B-spline coefficients, knots, and a dilation parameter so that a linear projection of the spline basis evaluated at the relative time yields the forward motion vectors and reliability maps at any target time $t$. The Fourier Mapper predicts amplitude and frequency fields from the nearest latent feature and evaluates cosine and sine of the frequency times the spatial offset to build high-resolution spatial features. These features are forward-warped to $t$ via softmax splatting, concatenated with the target time coordinate, and decoded into the high-resolution intermediate frame. The training objective is only a Charbonnier loss between the predicted and ground-truth frames; the optical-flow supervision loss used by MoTIF is dropped. The paper's central claim is that this design achieves state-of-the-art results while eliminating the external optical flow network at inference, with lower FLOPs and latency.","pith_inferences":["If the unwritten 'ground-truth forward motion' used for the first 150,000 training steps comes from a pre-trained flow estimator such as RAFT, then the 'no optical flow' claim is true only at inference; the training budget would still include a flow computation, and the reported FLOPs and latency savings would not cover training.","A clean test of the design principle would swap the Fourier spatial basis for another local basis (e.g., wavelets or learned features) while keeping the B-spline temporal mapper, isolating whether the gains come from basis choice or from removing flow features.","The same spatial-temporal basis split could transfer to other continuous video prediction tasks—novel view synthesis, video inpainting, or video compression—where smooth motion and high-frequency texture are both essential.","The authors' own limitation section concedes that large motion remains unsolved; a stress-test benchmark with fast camera or object motion would show whether the B-spline parameterization is a general motion prior or only a smooth-motion prior."],"forward_implications":["If the central claim holds, one model can produce video at any spatial and temporal scale (e.g., ×4 space and ×8 time, or unseen ×12 time) without an optical flow network, making deployment cheaper and simpler.","The negative result on position encoding suggests that generic Fourier encodings on input coordinates can hurt in video INR tasks; task-specific mappers (splines for time, Fourier for space) may be a more reliable design pattern.","The B-spline motion parameterization appears to improve out-of-distribution temporal interpolation: the model generalizes to ×6 and ×12 time scales despite training at ×8, matching or exceeding baselines.","Dropping the optical-flow supervision term simplifies training to a single reconstruction loss without sacrificing accuracy; the authors report the model remains competitive or better than the version trained with flow supervision."],"supporting_citations":[{"why":"MoTIF: the base C-STVSR pipeline and primary baseline; supplies the encoder–warp–decoder structure this work modifies and the numbers it compares against.","marker":"[6]"},{"why":"VideoINR: the first continuous space-time video super-resolution method; baseline for INR-based motion and feature modeling.","marker":"[7]"},{"why":"RAFT: the pre-trained optical flow network used by MoTIF; the paper's central claim is that this component can be removed.","marker":"[37]"},{"why":"LTE: supplies the Fourier-basis strategy for estimating dominant spatial frequencies from latent features.","marker":"[16]"},{"why":"B-spline Texture Coefficients Estimator: supplies the B-spline basis idea for continuous signal representation in super-resolution.","marker":"[27]"},{"why":"Softmax splatting: the forward warping operation used to propagate features and reliability maps to the target time.","marker":"[26]"},{"why":"SIREN: the periodic activation architecture used in the coefficient, knot, amplitude, and frequency estimators.","marker":"[33]"},{"why":"ZoomingSlowMo: the encoder architecture adopted to extract latent features from the input low-resolution frames.","marker":"[40]"}],"fun_headline_variants":["B-splines and Fourier mappers replace optical flow for video SR","One-stage video upsampling with B-spline and Fourier features","No optical flow needed: B-spline and Fourier video super-resolution","B-spline time interpolation plus Fourier spatial detail for video SR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper never defines the 'ground-truth forward motion' it substitutes into training for the first 150,000 iterations; if that supervision comes from a pre-trained optical flow network, then the claim of eliminating optical flow applies only at inference, not to training.","fun_headline_variants_meta":{"raw":{"variants":["B-splines and Fourier mappers replace optical flow for video SR","One-stage video upsampling with B-spline and Fourier features","No optical flow needed: B-spline and Fourier video super-resolution","B-spline time interpolation plus Fourier spatial detail for video SR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1220,"prompt_tokens":948,"completion_tokens":272,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":197}},"tokens_in":564,"tokens_out":272,"duration_ms":3210,"temperature":1.0,"reasoning_tokens":197,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:40:47.687464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released code or checkpoints to identify the source of the 'ground-truth forward motion' supervision schedule; alternatively, retrain the model from scratch without any such substitution and compare GoPro-Average PSNR/SSIM to the reported 30.22 / 0.8802. A substantial drop would show that the hidden flow supervision is load-bearing, while a match would confirm the no-flow claim.","supporting_citations":[{"cited_title":"Motif: Learning motion trajectories with local implicit neural functions for continuous space-time video super- resolution","cited_arxiv_id":null,"evidence_quote":"MoTIF: the base C-STVSR pipeline and primary baseline; supplies the encoder–warp–decoder structure this work modifies and the numbers it compares against."},{"cited_title":"Videoinr: Learning video implicit neural represen- tation for continuous space-time super-resolution","cited_arxiv_id":null,"evidence_quote":"VideoINR: the first continuous space-time video super-resolution method; baseline for INR-based motion and feature modeling."},{"cited_title":"Local texture estimator for implicit representation function","cited_arxiv_id":null,"evidence_quote":"LTE: supplies the Fourier-basis strategy for estimating dominant spatial frequencies from latent features."},{"cited_title":"B- spline texture coefficients estimator for screen content im- age super-resolution","cited_arxiv_id":null,"evidence_quote":"B-spline Texture Coefficients Estimator: supplies the B-spline basis idea for continuous signal representation in super-resolution."},{"cited_title":"Softmax splatting for video frame interpolation","cited_arxiv_id":null,"evidence_quote":"Softmax splatting: the forward warping operation used to propagate features and reliability maps to the target time."},{"cited_title":"Martel, Alexander W","cited_arxiv_id":null,"evidence_quote":"SIREN: the periodic activation architecture used in the coefficient, knot, amplitude, and frequency estimators."},{"cited_title":"Allebach, and Chenliang Xu","cited_arxiv_id":null,"evidence_quote":"ZoomingSlowMo: the encoder architecture adopted to extract latent features from the input low-resolution frames."}],"review_version":1}