REVIEW 2 major objections 6 minor 42 references
MVP: Winning Solution to SMP Challenge 2025 Video Track
T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that fusing XCLIP video embeddings with user statistics and metadata in a CatBoost regressor produced the winning video-popularity predictor at SMP Challenge 2025, with a MAPE of 0.1754.
desk verdict A credible SMP Challenge winner with code and a sensible ablation, but the test/validation MAPE ambiguity needs to be fixed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the fused multimodal feature vector: visual features $v_{\text{video}}$ obtained by average-pooling frame embeddings from XCLIP and projecting with PCA, concatenated with user-related features $v_{\text{user}}$, and temporal-metadata-text features $v_{\text{meta}}$ into a single vector. This vector is fed to a CatBoost gradient-boosted decision tree regressor with categorical feature support, trained with Huber loss and averaged over five cross-validation folds. The concatenation is what lets the tree model discover interactions between, say, a user's follower count and the video's visual content, and the ablation isolates how much each block contributes.
What would settle it
On the official SMPD-Video test set, run the MVP pipeline with all user-profile features removed; if the MAPE does not degrade substantially from the reported 0.1754 toward the 0.3010 seen in the validation ablation, then the claim that user statistics drive the result would be contradicted, and the practical requirement that such metadata be available at inference would be loosened.
Extended reading notes
Core claim
The central claim is that video popularity on short-form social platforms can be accurately predicted by a relatively simple multimodal pipeline: sample video frames, encode them with a pretrained XCLIP model, average-pool the frame embeddings and compress them with PCA, concatenate the result with log-transformed user statistics, posting-time features, metadata, and Word2Vec-based text features, and regress the log-scaled popularity score with a five-fold averaged CatBoost model using Huber loss. The paper reports a MAPE of 0.1754 on the official test set, first place in the challenge's Video Track. The ablation study shows that removing user profile statistics is far more damaging than removing any other feature group, raising MAPE to 0.3010, which the authors read as evidence that historical user behavior and social reach are the dominant drivers of content virality in this dataset.
Load-bearing premise
The whole result depends on user profile statistics (follower counts, like counts, video counts, and similar) being available and informative at prediction time, because removing them from the model raises MAPE from 0.1754 to 0.3010.
Editorial extensions
If this is right
- If the reported result holds, future video-popularity systems on similar platforms should treat user engagement history as a primary input rather than an auxiliary signal.
- The MAPE gap between the full model and the no-user-features variant implies that on platforms without public user statistics, purely content-based prediction would be notably less accurate.
- Outlier removal and K-fold averaging each contribute measurable gains (MAPE 0.1839 and 0.1786 without them), so preprocessing and ensembling matter as much as individual feature groups.
- The framework's modular design suggests that swapping the visual encoder or the regressor would preserve the overall pipeline, but the paper does not test that transfer empirically.
Reading between the lines
- Editorial inference: Because user features alone account for most of the signal, the method's competitive edge may not carry over to platforms that hide or aggregate user statistics; there, visual and textual features would have to carry more weight.
- Editorial inference: The model concatenates modalities without cross-modal attention, so it leaves open whether an end-to-end or attentional fusion could recover the lost signal when user metadata is absent.
- Editorial inference: The reported smoothing at the low and high ends of the popularity distribution suggests that label-distribution or cost-sensitive training could improve extremes more than adding more features.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper describes MVP, the authors' entry to the SMP Challenge 2025 Video Track for predicting the popularity of social media videos. The pipeline extracts XCLIP visual features, combines them with user profile statistics, temporal/metadata features, and Word2Vec-based text features, applies log transformations and IQR-based outlier removal, and trains a CatBoost regressor with five-fold ensemble averaging. The authors report that the system ranked first in the official evaluation and achieved a MAPE of 0.1754, and they provide ablations showing the contribution of each feature group, with user profile features being the most influential.
Significance. If the reported ranking is accurate, the paper documents a strong, reproducible baseline for the SMPD-Video benchmark: it is backed by an official external evaluation, the source code is linked, and the ablation study is internally consistent and directionally sensible. The methodological novelty is modest—the main ingredients are feature concatenation, preprocessing, and gradient boosting—but the value of the paper lies in a validated, public pipeline and in the empirical evidence that user metadata dominates performance. The paper would be strengthened by clearer reporting of the test-versus-validation split and by variance estimates, but the core competitive result is plausible and verifiable from the public artifacts.
major comments (2)
- [Section 4.3.1 and Table 1] The headline claim that MVP 'achieves a MAPE of 0.1754' on the official evaluation set is not unambiguously substantiated, because Table 1 is captioned 'Ablation study results on the validation set' and reports the identical value of 0.1754 for the full model, while Sections 4.3.3 and 4.4 analyze only validation-set behavior. No distinct official test-set MAPE is reported anywhere in the paper. Since the central claim is first place on the official evaluation, the exact test-set number is load-bearing; please report the validation and official test MAPE separately, or explicitly state that the two numbers coincide and explain why.
- [Section 3.2 and Section 4.3.2] Section 3.2 states that the IQR-based outlier filter is applied to 'training labels,' but the paper does not clarify whether the same filtering is applied to the validation or test labels when computing MAPE. If filtered labels are used at evaluation time, the reported MAPE would be optimistic relative to a deployment scenario in which extreme popularity values are present. Please state explicitly which labels are filtered and, if any filtering is applied to evaluation labels, report both filtered and unfiltered metrics so that the effect can be assessed.
minor comments (6)
- [Figures 2 and 3] In the manuscript version I received, the contents of Figures 2 and 3 appear as uninterpretable '/uni...' character sequences rather than rendered plots; please verify that the figures are correctly embedded and legible in the submitted PDF.
- [References] The reference list is duplicated after the 'References' heading; remove the duplicate set.
- [Section 4.4] The text contains 'Figure Figure 3'; it should read 'Figure 3.'
- [Equation (6)] Equation (6) is ambiguous: 'log2 r / d + 1' could be read as (log2 r)/d + 1. The intended formula appears to be log2(r/d + 1); please add parentheses.
- [Section 3.2] The Word2Vec model used for caption and keyword embeddings is not specified (training corpus, embedding dimension, or pretrained source); a brief specification would improve reproducibility.
- [Table 1] The ablation results are reported as point estimates without per-fold variance or significance tests; for differences as small as 0.1758 vs. 0.1754, reporting the per-fold standard deviation would help the reader judge which differences are meaningful.
Circularity Check
No significant circularity: the winning result rests on official held-out evaluation, and the self-citations are not load-bearing.
full rationale
The central result is a held-out official-evaluation number from an external benchmark, not a fitted quantity. Features are engineered from inputs (Eqs. 1-2), training uses 5-fold cross-validation (Eqs. 4-5), and MAPE (Eq. 7) is computed against test labels. No equation reduces to an input by construction, and the self-citations appear in related work about tracking and video analysis, not as load-bearing support for the popularity prediction. The identical 0.1754 in Table 1 and Section 4.3.1 is a presentational ambiguity about which split is reported, not a circular derivation.
Assumptions & free parameters
free parameters (7)
- Huber loss delta (δ)
- PCA output dimension
- Number of sampled frames N
- CatBoost hyperparameters
- Word2Vec embedding dimension
- Log transform base for user features
- IQR outlier threshold multiplier =
1.5
assumptions (4)
- domain assumption The popularity score is defined by the dataset as s = log2(r/d+1), and MAPE on this score is the correct evaluation metric.
- domain assumption Pretrained XCLIP features provide useful visual semantics for popularity prediction.
- domain assumption User profile statistics are available and reliable at prediction time.
- domain assumption CatBoost with Huber loss is an appropriate regression model for this task.
Cite this review
Pith. "Pith review of MVP: Winning Solution to SMP Challenge 2025 Video Track." pith.science (2026). https://pith.science/paper/CNDML4GY
@misc{pith2026250700950,
author = {Pith},
title = {Pith review of: MVP: Winning Solution to SMP Challenge 2025 Video Track},
year = {2026},
howpublished = {\url{https://pith.science/paper/CNDML4GY}},
note = {Machine review of arXiv:2507.00950}
}
read the original abstract
Social media platforms serve as central hubs for content dissemination, opinion expression, and public engagement across diverse modalities. Accurately predicting the popularity of social media videos enables valuable applications in content recommendation, trend detection, and audience engagement. In this paper, we present Multimodal Video Predictor (MVP), our winning solution to the Video Track of the SMP Challenge 2025. MVP constructs expressive post representations by integrating deep video features extracted from pretrained models with user metadata and contextual information. The framework applies systematic preprocessing techniques, including log-transformations and outlier removal, to improve model robustness. A gradient-boosted regression model is trained to capture complex patterns across modalities. Our approach ranked first in the official evaluation of the Video Track, demonstrating its effectiveness and reliability for multimodal video popularity prediction on social platforms. The source code is available at https://anonymous.4open.science/r/SMPDVideo.
Figures
Reference graph
Works this paper leans on
-
[1]
Hervé Abdi and Lynne J Williams. 2010. Principal component analysis. Wiley interdisciplinary reviews: computational statistics 2, 4 (2010), 433–459
2010
-
[2]
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. 2021. ViViT: A Video Vision Transformer. arXiv:2103.15691 [cs] doi:10.48550/arXiv.2103.15691
-
[3]
Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2018. Multi- modal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence 41, 2 (2018), 423–443
2018
-
[4]
Gedas Bertasius, Heng Wang, and Lorenzo Torresani. 2021. Is Space-Time Attention All You Need for Video Understanding? arXiv:2102.05095 [cs] doi:10.48550/arXiv.2102.05095
-
[5]
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794
2016
-
[6]
Zhangtao Cheng, Jienan Zhang, Xovee Xu, Goce Trajcevski, Ting Zhong, and Fan Zhou. 2024. Retrieval-augmented hypergraph for multimodal social media popularity prediction. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining . 445–455
work page 2024
-
[7]
Cisco. 2020. Cisco Annual Internet Report - Cisco Annual Internet Report (2018–
work page 2020
-
[8]
Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. 2018. CatBoost: gra- dient boosting with categorical features support. arXiv preprint arXiv:1810.11363 (2018)
arXiv 2018
Show all 42 references
-
[9]
Wenhao Hu, Weilong Chen, Weimin Yuan, Yan Wang, Shimin Cai, and Yanru Zhang. 2024. Dual-Stream Pre-Training Transformer to Enhance Multimodal Learning for Social Media Prediction. InProceedings of the 32nd ACM International Conference on Multimedia. 11450–11456
2024
-
[10]
Yangliu Hu, Zikai Song, Na Feng, Yawei Luo, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2025. Sf2t: Self-supervised fragment finetuning of video-llms for fine-grained understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference. 29108–29117
2025
-
[11]
Xin Lai, Yihong Zhang, and Wei Zhang. 2020. HyFea: Winning Solution to Social Media Popularity Prediction for Multimedia Grand Challenge 2020. InProceedings of the 28th ACM International Conference on Multimedia (MM ’20) . Association for Computing Machinery, New York, NY, USA...
2020 doi
-
[12]
Wenbing Li, Hang Zhou, Junqing Yu, Zikai Song, and Wei Yang. 2024. Coupled mamba: Enhanced multi-modal fusion with coupled state space model. arXiv preprint arXiv:2405.18014 (2024)
2024 arXiv
- [13]
-
[14]
Yang Liu, Samuel Albanie, Arsha Nagrani, and Andrew Zisserman. 2019. Use what you have: Video retrieval using representations from collaborative experts. arXiv preprint arXiv:1907.13487 (2019)
2019 arXiv
-
[15]
Run Luo, Zikai Song, Lintao Ma, Jinlin Wei, Wei Yang, and Min Yang. 2024. Diffusiontrack: Diffusion model for multi-object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 3991–3999
2024
- [16]
-
[17]
Shijian Mao, Wudong Xi, Lei Yu, Gaotian Lü, Xingxing Xing, Xingchen Zhou, and Wei Wan. 2023. Enhanced CatBoost with Stacking Features for Social Media Prediction. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23). Association for Computing Machiner...
2023
- [18]
-
[19]
Antoine Miech, Ivan Laptev, and Josef Sivic. 2018. Learning a text-video embed- ding from incomplete and heterogeneous data. arXiv preprint arXiv:1804.02516 (2018)
2018 arXiv
- [20]
-
[21]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...
2021
-
[22]
Zikai Song, Run Luo, Lintao Ma, Ying Tang, Yi-Ping Phoebe Chen, Junqing Yu, and Wei Yang. 2025. Temporal Coherent Object Flow for Multi-Object Tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 6978–6986
2025
-
[23]
Zikai Song, Run Luo, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2023. Compact transformer tracker with correlative masked modeling. In Proceedings of the AAAI conference on artificial intelligence , Vol. 37. 2321–2329
2023
-
[24]
Zikai Song, Ying Tang, Run Luo, Lintao Ma, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2024. Autogenic language embedding for coherent point tracking. In Proceedings of the 32nd ACM International Conference on Multimedia . 2021–2030
2024
-
[25]
Zikai Song, Zhiwen Wan, Wei Yuan, Ying Tang, Junqing Yu, and Yi-Ping Phoebe Chen. 2021. Distractor-aware tracker with a domain-special optimized benchmark for soccer player tracking. In Proceedings of the 2021 International Conference on Multimedia Retrieval. 276–284
2021
-
[26]
Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2022. Transformer tracking with cyclic shifting window attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 8791–8800
2022
-
[27]
Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid
-
[28]
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. 2022. VideoMAE: Masked Autoencoders Are Data-Efficient Learners for Self-Supervised Video Pre-Training. https://arxiv.org/abs/2203.12602v3
2022 arXiv
-
[29]
Mingsheng Tu, Tianjiao Wan*, Qisheng Xu, Xinhao Jiang, Kele Xu, and Cheng Yang. 2024. Higher-Order Vision-Language Alignment for Social Media Pre- diction. In Proceedings of the 32nd ACM International Conference on Multimedia . ACM, Melbourne VIC Australia, 11457–11463. doi:10...
2024
-
[30]
Caroline Violot, Tuğrulcan Elmas, Igor Bilogrevic, and Mathias Humbert. 2024. Shorts vs. regular videos on YouTube: a comparative analysis of user engage- ment and content creation trends. In Proceedings of the 16th ACM Web Science Conference. 213–223
2024
-
[31]
Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, et al. 2022. Language models with image descriptors are strong few-shot video-language learners. Advances in Neural Information Processing Systems ...
2022
-
[32]
Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, and Jiebo Luo
-
[33]
Bo Wu, Peiye Liu, Wen-Huang Cheng, Bei Liu, Zhaoyang Zeng, Jia Wang, Qiushi Huang, and Jiebo Luo. 2023. SMP Challenge: An overview and analysis of social media prediction challenge. In Proceedings of the 31st ACM International Conference on Multimedia. 9651–9655
2023
-
[34]
Bo Wu, Peiye Liu, Qiushi Huang, Zhaoyang Zeng, Jia Wang, Bei Liu, Jiebo Luo, and Wen-Huang Cheng. 2024. SMP Challenge Summary: Social Media Prediction Challenge. In Proceedings of the 32nd ACM International Conference on Multimedia. 11442–11444
2024
-
[35]
In Proceedings of the 27th ACM International Conference on Multimedia
Smp challenge: An overview of social media prediction challenge 2019. In Proceedings of the 27th ACM International Conference on Multimedia . 2667–2671
2019
-
[36]
Kele Xu, Zhimin Lin, Jianqiao Zhao, Peicang Shi, Wei Deng, and Huaimin Wang
-
[37]
Hang Zhou, Jiale Cai, Yuteng Ye, Yonghui Feng, Chenxing Gao, Junqing Yu, Zikai Song, and Wei Yang. 2025. Video anomaly detection with motion and appearance guided patch diffusion model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 10761–10769
2025
-
[38]
Bo Wu, Tao Mei, Wen-Huang Cheng, and Yongdong Zhang. 2016. Unfolding temporal dynamics: Predicting social media popularity using multi-scale temporal decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 30
2016
-
[42]
Yaochen Zhu, Jiayi Xie, and Zhenzhong Chen. 2020. Predicting the Popular- ity of Micro-videos with Multimodal Variational Encoder-Decoder Framework. https://arxiv.org/abs/2003.12724v1. doi:10.1109/TMM.2021.3120537
2020 arXiv
-
[2019]
In Proceedings of the IEEE/CVF international conference on computer vision
Videobert: A joint model for video and language representation learning. In Proceedings of the IEEE/CVF international conference on computer vision . 7464– 7473
-
[2020]
In Proceedings of the 28th ACM International Conference on Multimedia
Multimodal deep learning for social media popularity prediction with attention mechanism. In Proceedings of the 28th ACM International Conference on Multimedia. 4580–4584
-
[2023]
https://www.cisco.com/c/en/us/solutions/collateral/executive- perspectives/annual-internet-report/white-paper-c11-741490.html
White Paper. https://www.cisco.com/c/en/us/solutions/collateral/executive- perspectives/annual-internet-report/white-paper-c11-741490.html
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.