REVIEW 4 major objections 5 minor 1 cited by
TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read TimeMaster claims a 3B multimodal model can learn expert time-series reasoning from ~1k GPT-4o examples per task plus token-level GRPO, reaching 75.29% average accuracy on TIMERBED.
desk verdict Worth a serious look: the RL recipe is coherent and cheap, but the SOTA claim currently rests on an unstated train/test split for the SFT/RL data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the coupling of a rigid three-tag output grammar with a three-term reward. The model must emit <think>…</think> (chain-of-thought over the plotted signal), <class>…</class> (the predicted label), and <extension>…</extension> (optional actionable advice); the format reward is a regex check on that structure, the hard reward is exact label match, and the soft reward delegates open-ended extension quality to a GPT-4o judge scoring four dimensions on [0,1]. GRPO (group size G=5) computes group-normalized advantages so tokens in better-scoring rollouts are pushed up without a separate value network, while the SFT warm-up supplies the temporal priors that keep RL exploration anchored. The same mechanism yields both the accuracy jump and the qualitative shift in reasoning behavior.
What would settle it
Re-run the full SFT+GRPO pipeline with strict split discipline: partition each TIMERBED dataset into train and test folds before sampling the GPT-4o warm-up examples and before collecting RL rollouts, holding the test fold out entirely. If the headline accuracy drops substantially below 75.29% or falls behind the VL-Time few-shot baseline, the result is an artifact of training/test overlap. A second check is to measure agreement between TimeMaster's <think> reasoning and its <class> label on the test set; the paper's own failure cases admit reasoning–label mismatches, so a high mismatch rate would undercut the 'expert-like reasoning' claim even if accuracy holds.
Extended reading notes
Core claim
The central claim is that a time-series multimodal LLM can acquire expert-style reasoning without large curated datasets: instead of hundreds of thousands of paired examples, TimeMaster warms a 3B vision-language model on about one thousand GPT-4o-generated reasoning/classification samples per task, then optimizes a composite reward with GRPO. The reward has three parts: a regex-based format reward that enforces the ordered <think>, <class>, and <extension> tags; a hard reward that gives 1 only on an exact label match; and a soft reward, active when the extension is enabled, in which a GPT-4o judge scores extension quality on specificity, appropriateness, relevance, and depth. On TIMERBED — whale-call detection, satellite lightning events, ECG arrhythmia, EMG neuromuscular diagnosis, activity recognition, and device-type inference — the full SFT+RL model scores 75.29% average accuracy, beating the strongest classical baselines and the VL-Time few-shot GPT-4o prompting pipeline, with qualitative evidence that RL-trained reasoning incorporates uncertainty, contrastive elimination of rival classes, and multi-scale pattern synthesis absent from the SFT-only model.
Load-bearing premise
The model is trained on data sampled from the same TIMERBED datasets it is then tested on, and the paper never states that the roughly 1,000 GPT-4o-generated SFT samples and the RL rollouts are confined to a training split disjoint from the test split; if any of them come from test instances, the reported 75.29% average accuracy is inflated and the state-of-the-art claim collapses.
Editorial extensions
If this is right
- If the 75.29% result holds, roughly 1k warm-up examples per task substitutes for the ~200k curated pairs used by prior SFT-only time-series QA models, changing the data economics of building temporal reasoners.
- The format-hard-soft reward stack is task-agnostic: the same recipe should transfer to forecasting-with-explanation, anomaly diagnosis, or time-series QA without architectural changes.
- Because outputs are parseable tags, downstream systems can automatically audit structure, grade extension quality, and reject format-violating answers — a practical requirement for clinical or industrial deployment.
- The paper's ablations show format and hard rewards are complementary (39% to 85% to 100% accuracy on EMG as rewards are added), implying reward composition rather than model scale explains most of the gain.
- A 3B model beating frontier models at prompting time-series plots suggests the binding constraint in time-series MLLMs is task-specific temporal priors, not parameter count.
Reading between the lines
- The soft reward's judge is the same model family that generated the SFT data, so the reward may optimize for what GPT-4o finds convincing rather than clinically true; swapping in an independent judge or human-rated extensions would reveal whether this bias exists.
- Because inputs are rendered plots, the model's reasoning could be tracking rendering artifacts such as axis ranges, colors, or resolution rather than the underlying waveform; perturbing axes or occluding plot regions would test perceptual faithfulness.
- The SFT-then-GRPO recipe with a composite reward is a general template for other visual reasoning domains — medical imaging, spectral analysis, industrial waveforms — where a small model plus structured rewards may rival large prompting-only models.
- The train/test provenance question is resolvable by inspection or re-run; any downstream comparison should first verify that GPT-4o sampling and GRPO rollouts used only training-fold instances.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces TimeMaster, a two-stage training method that combines supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO) to make a 3B-parameter multimodal LLM (Qwen2.5-VL-3B-Instruct) perform structured reasoning over visualized time-series inputs. The model is trained to output three XML-tagged segments—<think>, <class>, and <extension>—and is optimized with a composite reward comprising format, hard (classification accuracy), and soft (LLM-as-a-Judge extension quality) components. On the TIMERBED benchmark, the paper reports an average accuracy of 75.29%, claiming state-of-the-art results that surpass classical time-series models by 14.6% and few-shot GPT-4o/VL-Time by 7.3%. The paper also includes ablations of reward components and group size, plus qualitative case studies of the model's reasoning behavior.
Significance. If the empirical results are verified, the paper makes a valuable contribution: it suggests that a relatively small amount of SFT warm-up (about 1k examples per task) combined with token-level RL can elicit interpretable, expert-like reasoning from a 3B multimodal model, at a fraction of the data cost of prior approaches such as Time-MQA. The structured-output design and the decomposition into format, hard, and soft rewards are simple and potentially reusable across time-series reasoning tasks. The claimed gains over strong baselines are substantial, and the paper provides reproducible-looking hyperparameter settings. However, the central empirical claim is currently contingent on an unverified dataset-provenance assumption (train/test disjointness), and several secondary comparisons lack variance information. These issues do not affect the soundness of the RL formulation, but they do affect the credibility of the SOTA claim.
major comments (4)
- [Section 4.1 (Table 1), Appendices G.2 and I, Table 4] The central claim of 75.29% average accuracy rests on the unstated assumption that the TIMERBED samples used for GPT-4o SFT generation and GRPO rollout contexts are disjoint from the evaluation split. The paper never specifies a train/test split for these training samples, and for the small datasets (EMG: 205 samples, CTU: 500, TEE: 143, Table 4) the described class-balanced, ground-truth-filtered collection procedure in Appendix I could plausibly draw from the same pool used for evaluation. If any overlap exists, the hard reward in Eq. (3) and the GRPO objective in Eq. (7) directly optimize the reported test accuracy, inflating the SOTA claim. The authors must specify the per-dataset split used for SFT and RL, confirm disjointness from the evaluation set, and release the code and data needed to verify this.
- [Section 4.1 (Table 1), Table 3] Baseline accuracies are reported without standard deviations or repeated-seed runs, while Table 3 provides variance only for TimeMaster. Several reported gaps are small relative to plausible seed variation (e.g., TEE: 68.49 for TimeMaster vs. 64.29 for VL-Time few-shot; EMG: 100.00 for TimeMaster vs. 91.67 for VL-Time few-shot). The comparison should either include error bars or multiple seeds for the strongest baselines, or explicitly state that those numbers are single-run values inherited from prior work; otherwise the robustness of the margin over baselines is unclear.
- [Appendix G.2 and Table 2] The main-result configuration sets (λfmt, λhard, λsoft) = (0.1, 0.9, 0), so the reported 75.29% accuracy does not exercise the soft reward at all. The paper's abstract, introduction, and Takeaway III (Section 4.2) nevertheless present soft reward as a core component that improves extension quality; the only quantitative evidence for this is the EMG-only ablation in Table 2 (soft reward score improving from 0.45 to 0.72). This claim should be either demonstrated on at least one additional dataset or explicitly scoped as an EMG-only result, and the main text should clarify that the SOTA accuracy is achieved without the soft-reward term.
- [Appendix D.2 (Figure 5)] The group-size ablation on TEE and CTU reports accuracy curves without stating whether the evaluation set is disjoint from the RL training contexts. If the same provenance issue described in the first major comment applies here, the observed monotonic improvement with increasing G could reflect memorization of overlapping examples rather than better advantage estimation. This ablation should be re-run or re-reported on a confirmed disjoint split before it is used to justify the default choice of G=5.
minor comments (5)
- [Appendix G.1] The phrase 'Following the methodology of VL-Following Time [17]' appears to be a typo; it should read 'VL-Time [17]'.
- [Figure 14] In the VL-Time output, the text contains the string 'NoWhale/class>' which is missing the opening '<'; this is likely a typesetting error and should be corrected.
- [Table 4 vs. Appendix G.2] The datasets are listed in inconsistent orders across Table 4 and Appendix G.2; please unify the ordering (e.g., by reasoning type) for readability.
- [References] Several references contain spacing artifacts such as '[ 17]' and '[ 27]'; the bibliography should be formatted consistently.
- [Table 6] The hyperparameter table lists 'Lmax' but the text refers to 'max sequence length'; please rename the parameter for clarity.
Circularity Check
No significant circularity: TimeMaster's RL training optimizes ground-truth accuracy directly, and the TIMERBED train/test provenance concern is a correctness issue, not a circular reduction.
full rationale
TimeMaster's derivation chain is self-contained with respect to the circularity patterns checked. The reported 75.29% is an accuracy on TIMERBED, and the training signal used to produce the model is the hard reward rhard = I[hat-c = c*] (Eq. 3), which is a direct function of ground-truth labels rather than of the evaluation numbers; RL optimizes this reward through the GRPO objective in Eq. (7). The SFT warm-up data are generated by GPT-4o and filtered by matching ground-truth labels (Appendix I), so no parameter is fitted to a 'prediction' that is then reported as independent; the evaluation metric is the same accuracy that the reward encourages, which is ordinary supervised/RL training rather than circularity. The soft reward uses GPT-4o as judge while GPT-4o also generated SFT data, but soft reward is disabled in the main experiments (lambda_soft = 0 in Appendix G.2), so it is not load-bearing for the headline accuracy. Self-citations ([2], [8]) appear in related-work context and do not carry any load-bearing premise; no uniqueness theorem or ansatz is imported from the authors' prior work. The one substantive risk, that SFT/RL contexts are sampled from TIMERBED without a stated disjoint train/test split (Appendix G.2, I), is a dataset-provenance and correctness concern, not a circularity reduction: even if leakage occurred, the reported numbers would be inflated by test-set fitting, not derived from the method's equations by construction. Since no specific Eq. X = Eq. Y or fitted-parameter-renamed-as-prediction step can be exhibited, the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Reward weights (lambda_format, lambda_hard, lambda_soft) =
(0.1, 0.9, 0) for main results; (0.1, 0.9, 1) for extension case studies
- GRPO group size G =
5
- Per-dataset RL epochs =
40 (RCW, ECG, HAR); 100 (EMG, CTU); 300 (TEE)
- SFT data collection scale and filter =
About 1,000 GPT-4o samples per dataset, temperature 1.0, only label-correct samples retained
assumptions (4)
- standard math GRPO clipped surrogate objective and KL regularization are valid optimizers for this policy
- domain assumption A line-plot image of a time series preserves enough temporal structure for classification and reasoning
- domain assumption TIMERBED labels are correct and are the correct supervision for expert-like reasoning
- ad hoc to paper The SFT and RL training samples from TIMERBED are drawn from a split disjoint from the evaluation split
Cite this review
Pith. "Pith review of TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning." pith.science (2026). https://pith.science/paper/4AZXZF4L
@misc{pith2026250613705,
author = {Pith},
title = {Pith review of: TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/4AZXZF4L}},
note = {Machine review of arXiv:2506.13705}
}
read the original abstract
Time-series reasoning remains a significant challenge in multimodal large language models (MLLMs) due to the dynamic temporal patterns, ambiguous semantics, and lack of temporal priors. In this work, we introduce TimeMaster, a reinforcement learning (RL)-based method that enables time-series MLLMs to perform structured, interpretable reasoning directly over visualized time-series inputs and task prompts. TimeMaster adopts a three-part structured output format, reasoning, classification, and domain-specific extension, and is optimized via a composite reward function that aligns format adherence, prediction accuracy, and open-ended insight quality. The model is trained using a two-stage pipeline: we first apply supervised fine-tuning (SFT) to establish a good initialization, followed by Group Relative Policy Optimization (GRPO) at the token level to enable stable and targeted reward-driven improvement in time-series reasoning. We evaluate TimeMaster on the TimerBed benchmark across six real-world classification tasks based on Qwen2.5-VL-3B-Instruct. TimeMaster achieves state-of-the-art performance, outperforming both classical time-series models and few-shot GPT-4o by over 14.6% and 7.3% performance gain, respectively. Notably, TimeMaster goes beyond time-series classification: it also exhibits expert-like reasoning behavior, generates context-aware explanations, and delivers domain-aligned insights. Our results highlight that reward-driven RL can be a scalable and promising path toward integrating temporal understanding into time-series MLLMs.
Figures
Figures from the paper (11 more)
Forward citations
Cited by 1 Pith paper
-
A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models
The authors organize LLM-based time series reasoning into three exclusive topologies (direct, chain, branch) crossed with four objectives, and use them to label 125 papers, benchmarks, and resources.
Reference graph
Works this paper leans on
-
[1]
Deep learning in human activity recognition with wearable sensors: A review on advances.Sensors, 22(4):1476, 2022
Shibo Zhang, Yaxuan Li, Shen Zhang, Farzad Shahabi, Stephen Xia, Yu Deng, and Nabil Alshurafa. Deep learning in human activity recognition with wearable sensors: A review on advances.Sensors, 22(4):1476, 2022
2022
-
[2]
Diverse intra-and inter-domain activity style fusion for cross-person generalization in activity recognition
Junru Zhang, Lang Feng, Zhidan Liu, Yuhan Wu, Yang He, Yabo Dong, and Duanqing Xu. Diverse intra-and inter-domain activity style fusion for cross-person generalization in activity recognition. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4213–4222, 2024
2024
-
[3]
Sensor alignment for multivariate time-series unsupervised domain adaptation
Yucheng Wang, Yuecong Xu, Jianfei Yang, Zhenghua Chen, Min Wu, Xiaoli Li, and Lihua Xie. Sensor alignment for multivariate time-series unsupervised domain adaptation. InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 10253–10261, 2023
2023
-
[4]
Conditional contrastive domain generalization for fault diagnosis.IEEE Transactions on Instrumentation and Measurement, 71:1–12, 2022
Mohamed Ragab, Zhenghua Chen, Wenyu Zhang, Emadeldeen Eldele, Min Wu, Chee-Keong Kwoh, and Xiaoli Li. Conditional contrastive domain generalization for fault diagnosis.IEEE Transactions on Instrumentation and Measurement, 71:1–12, 2022
2022
-
[5]
Understanding electricity-theft behavior via multi-source data
Wenjie Hu, Yang Yang, Jianbo Wang, Xuanwen Huang, and Ziqiang Cheng. Understanding electricity-theft behavior via multi-source data. InProceedings of The Web Conference 2020, pages 2264–2274, 2020
2020
-
[6]
Tactis: Transformer-attentional copulas for time series
Alexandre Drouin, Étienne Marcotte, and Nicolas Chapados. Tactis: Transformer-attentional copulas for time series. InInternational Conference on Machine Learning, pages 5447–5493. PMLR, 2022
work page 2022
-
[7]
Tslanet: Rethinking transformers for time series representation learning
Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. Tslanet: Rethinking transformers for time series representation learning. InInternational Conference on Machine Learning, pages 12409–12428. PMLR, 2024
work page 2024
-
[8]
Adacket: Adaptive convolutional kernel transform for multivariate time series classification
Junru Zhang, Lang Feng, Haowen Zhang, Yuhan Wu, and Yabo Dong. Adacket: Adaptive convolutional kernel transform for multivariate time series classification. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 189–204. Springer, 2023
work page 2023
Show all 85 references
-
[9]
Times- net: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Times- net: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022
2022 arXiv
-
[10]
Recurrent neural networks for time series classification
Michael Hüsken and Peter Stagge. Recurrent neural networks for time series classification. Neurocomputing, 50:223–235, 2003
2003
-
[11]
Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
2023 arXiv
-
[12]
Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
2023 arXiv
-
[13]
Qwen technical report.arXiv preprint arXiv:2309.16609, 2023
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023
2023 arXiv
-
[14]
Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023
2023 arXiv
-
[15]
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 10
2022
-
[16]
Position: Empowering time series reasoning with multimodal llms.arXiv preprint arXiv:2502.01477, 2025
Yaxuan Kong, Yiyuan Yang, Shiyu Wang, Chenghao Liu, Yuxuan Liang, Ming Jin, Stefan Zohren, Dan Pei, Yan Liu, and Qingsong Wen. Position: Empowering time series reasoning with multimodal llms.arXiv preprint arXiv:2502.01477, 2025
2025
-
[17]
A picture is worth a thousand numbers: Enabling llms reason about time series via visualization.arXiv preprint arXiv:2411.06018, 2024
Haoxin Liu, Chenghao Liu, and B Aditya Prakash. A picture is worth a thousand numbers: Enabling llms reason about time series via visualization.arXiv preprint arXiv:2411.06018, 2024
2024 arXiv
-
[18]
Explainable multi-modal time series prediction with llm-in-the-loop.arXiv preprint arXiv:2503.01013, 2025
Yushan Jiang, Wenchao Yu, Geon Lee, Dongjin Song, Kijung Shin, Wei Cheng, Yanchi Liu, and Haifeng Chen. Explainable multi-modal time series prediction with llm-in-the-loop.arXiv preprint arXiv:2503.01013, 2025
2025
-
[19]
Language models still struggle to zero-shot reason about time series.arXiv preprint arXiv:2404.11757, 2024
Mike A Merrill, Mingtian Tan, Vinayak Gupta, Tom Hartvigsen, and Tim Althoff. Language models still struggle to zero-shot reason about time series.arXiv preprint arXiv:2404.11757, 2024
2024 arXiv
-
[20]
Gpt-4o, 2024
OpenAI. Gpt-4o, 2024. Accessed: 2025-04-21
2024
-
[21]
Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning.arXiv preprint arXiv:2412.03104, 2024
Zhe Xie, Zeyan Li, Xiao He, Longlong Xu, Xidao Wen, Tieying Zhang, Jianjun Chen, Rui Shi, and Dan Pei. Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning.arXiv preprint arXiv:2412.03104, 2024
2024
-
[22]
Timecap: Learning to contextualize, augment, and predict time series events with large language model agents
Geon Lee, Wenchao Yu, Kijung Shin, Wei Cheng, and Haifeng Chen. Timecap: Learning to contextualize, augment, and predict time series events with large language model agents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18082–18090, 2025
2025
-
[23]
From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection.Advances in Neural Information Processing Systems, 37:58118–58153, 2024
Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection.Advances in Neural Information Processing Systems, 37:58118–58153, 2024
2024
-
[24]
Beyond trend and periodicity: Guiding time series forecasting with textual cues.arXiv preprint arXiv:2405.13522, 2024
Zhijian Xu, Yuxuan Bian, Jianyuan Zhong, Xiangyu Wen, and Qiang Xu. Beyond trend and periodicity: Guiding time series forecasting with textual cues.arXiv preprint arXiv:2405.13522, 2024
2024 arXiv
-
[25]
Time-mmd: A new multi-domain multimodal dataset for time series analysis.arXiv preprint arXiv:2406.08627, 2024
Haoxin Liu, Shangqing Xu, Zhiyuan Zhao, Lingkai Kong, Harshavardhan Kamarthi, Aditya B Sasanur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, et al. Time-mmd: A new multi-domain multimodal dataset for time series analysis.arXiv preprint arXiv:2406.08627, 2024
2024 arXiv
-
[26]
Multi-modal forecaster: Jointly predicting time series and textual data.arXiv preprint arXiv:2411.06735, 2024
Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das, Zihao Zhou, Abhishek Tanpure, Mathew Luo, and Rose Yu. Multi-modal forecaster: Jointly predicting time series and textual data.arXiv preprint arXiv:2411.06735, 2024
2024 arXiv
-
[27]
Time-mqa: Time series multi-task question answering with context enhancement.arXiv preprint arXiv:2503.01875, 2025
Yaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du, Stefan Zohren, Zhangyang Wang, Ming Jin, and Qingsong Wen. Time-mqa: Time series multi-task question answering with context enhancement.arXiv preprint arXiv:2503.01875, 2025
2025 arXiv
-
[28]
Gpt4mts: Prompt-based large language model for multimodal time-series forecasting
Furong Jia, Kevin Wang, Yixiang Zheng, Defu Cao, and Yan Liu. Gpt4mts: Prompt-based large language model for multimodal time-series forecasting. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23343–23351, 2024
2024
-
[29]
MIT press, 2018
Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press, 2018
2018
-
[30]
Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023
2023
-
[31]
Time-llm: Time series forecasting by reprogramming large language models.arXiv preprint arXiv:2310.01728, 2023
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models.arXiv preprint arXiv:2310.01728, 2023. 11
-
[32]
Autotimes: Au- toregressive time series forecasters via large language models.Advances in Neural Information Processing Systems, 37:122154–122184, 2024
Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Au- toregressive time series forecasters via large language models.Advances in Neural Information Processing Systems, 37:122154–122184, 2024
2024
-
[33]
Calf: Aligning llms for time series forecasting via cross-modal fine-tuning
Peiyuan Liu, Hang Guo, Tao Dai, Naiqi Li, Jigang Bao, Xudong Ren, Yong Jiang, and Shu-Tao Xia. Calf: Aligning llms for time series forecasting via cross-modal fine-tuning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18915–18923, 2025
2025
-
[34]
Test: Text prototype aligned embedding to activate llm’s ability for time series
Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. Test: Text prototype aligned embedding to activate llm’s ability for time series. InThe Twelfth International Conference on Learning Representations
-
[35]
Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019
2019
-
[36]
How can time series analysis benefit from multiple modalities? a survey and outlook.arXiv preprint arXiv:2503.11835, 2025
Haoxin Liu, Harshavardhan Kamarthi, Zhiyuan Zhao, Shangqing Xu, Shiyu Wang, Qingsong Wen, Tom Hartvigsen, Fei Wang, and B Aditya Prakash. How can time series analysis benefit from multiple modalities? a survey and outlook.arXiv preprint arXiv:2503.11835, 2025
2025
-
[37]
Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment
Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecma: Towards llm-empowered multivariate time series forecasting via cross-modality alignment. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, page...
2025
-
[38]
Meit: Multi-modal electrocardiogram instruction tuning on large language models for report generation.arXiv preprint arXiv:2403.04945, 2024
Zhongwei Wan, Che Liu, Xin Wang, Chaofan Tao, Hui Shen, Zhenwu Peng, Jie Fu, Rossella Arcucci, Huaxiu Yao, and Mi Zhang. Meit: Multi-modal electrocardiogram instruction tuning on large language models for report generation.arXiv preprint arXiv:2403.04945, 2024
2024 arXiv
-
[39]
Multi-modal deep learning for credit rating prediction using text and numerical data streams.Applied Soft Computing, page 112771, 2025
Mahsa Tavakoli, Rohitash Chandra, Fengrui Tian, and Cristián Bravo. Multi-modal deep learning for credit rating prediction using text and numerical data streams.Applied Soft Computing, page 112771, 2025
2025
-
[40]
Terra: A multimodal spatio-temporal dataset spanning the earth.Advances in Neural Information Processing Systems, 37:66329– 66356, 2024
Wei Chen, Xixuan Hao, Yuankai Wu, and Yuxuan Liang. Terra: A multimodal spatio-temporal dataset spanning the earth.Advances in Neural Information Processing Systems, 37:66329– 66356, 2024
2024
-
[41]
Bjtt: A large-scale multimodal dataset for traffic prediction.IEEE Transactions on Intelligent Transportation Systems, 2024
Chengyang Zhang, Yong Zhang, Qitan Shao, Jiangtao Feng, Bo Li, Yisheng Lv, Xinglin Piao, and Baocai Yin. Bjtt: A large-scale multimodal dataset for traffic prediction.IEEE Transactions on Intelligent Transportation Systems, 2024
2024
-
[42]
Event traffic forecasting with sparse multimodal data
Xiao Han, Zhenduo Zhang, Yiling Wu, Xinfeng Zhang, and Zhe Wu. Event traffic forecasting with sparse multimodal data. InProceedings of the 32nd ACM International Conference on Multimedia, pages 8855–8864, 2024
2024
-
[43]
Evaluating system 1 vs
Haoxin Liu, Zhiyuan Zhao, Shiduo Li, and B Aditya Prakash. Evaluating system 1 vs. 2 reasoning approaches for zero-shot time-series forecasting: A benchmark and insights.arXiv preprint arXiv:2503.01895, 2025
2025 arXiv
-
[44]
Fine-tuning language models from human preferences
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019
1909 arXiv
-
[45]
Learning to summarize with human feedback
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in neural information processing systems, 33:3008–3021, 2020
2020
-
[46]
Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...
2022
-
[47]
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. 12
2024
-
[48]
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025
2025 arXiv
-
[49]
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024
2024 arXiv
-
[50]
Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025
2025 arXiv
-
[51]
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025
2025 arXiv
-
[52]
itransformer: Inverted transformers are effective for time series forecasting.arXiv preprint arXiv:2310.06625, 2023
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting.arXiv preprint arXiv:2310.06625, 2023
2023 arXiv
-
[53]
A time series is worth 64 words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022
2022 arXiv
-
[54]
Beyond numbers: A survey of time series analysis in the era of multimodal llms.Authorea Preprints, 2025
Xiongxiao Xu, Yue Zhao, S Yu Philip, and Kai Shu. Beyond numbers: A survey of time series analysis in the era of multimodal llms.Authorea Preprints, 2025
2025
-
[55]
Attention is all you need.Advances in neural information processing systems, 30, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017
2017
-
[56]
Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting.Advances in neural information processing systems, 34:22419–22430, 2021
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting.Advances in neural information processing systems, 34:22419–22430, 2021
2021
-
[57]
Informer: Beyond efficient transformer for long sequence time-series forecasting
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11106–11115, 2021
2021
-
[58]
Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. InInternational conference on machine learning, pages 27268–27286. PMLR, 2022
2022
-
[59]
Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? InProceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128, 2023
2023
-
[60]
STANDING
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 13 A Limitations Our work still has some limitations. We have only provided an initial ...
2025 arXiv
-
[61]
The x -axis should reflect forward motion , while the y and z axes can show lateral and vertical changes
WALKING: walking involves significant changes in acceleration due to the forward and side movements . The x -axis should reflect forward motion , while the y and z axes can show lateral and vertical changes . Observing the trends in the chart , we see small fluctuations across...
-
[62]
The fluctuations would be larger and more pronounced in z -axis data due to the changes in vertical motion
WALKING_UPSTAIRS/DOWNSTAIRS: Both activities involve rhythmic changes typically seen in walking , but they include vertical motion . The fluctuations would be larger and more pronounced in z -axis data due to the changes in vertical motion
-
[63]
SITTING/LAYING: These activities show very low variability , with minimal changes in all axes due to the lack of forward movement
-
[64]
LAYING” based on vague cues like “relatively small movements
STANDING: This involves very low -paced changes due to stability and lack of dynamic activity . Typical accelerometer data for standing will show minuscule fluctuations and low variability on all axes . In the provided data , the acceleration values for x , y, and z axes range...
-
[65]
This is typical in neuropathy due to reinnervation and the presence of motor units with abnormal recruitment patterns
Waveform Morphology : The waveform is polyphasic , meaning it has multiple peaks . This is typical in neuropathy due to reinnervation and the presence of motor units with abnormal recruitment patterns
-
[66]
The polyphasic nature of the waveform is indicative of reinnervation, where motor units are recruited in a different manner than in a healthy state
Temporal Patterns : The waveform shows a consistent pattern of sharp onset and brief duration , which is characteristic of neuropathy . The polyphasic nature of the waveform is indicative of reinnervation, where motor units are recruited in a different manner than in a healthy...
-
[67]
</think> <class>Neuropathy</class> < t h i n k>1
Conclusion: The waveform's characteristics—polyphasic morphology, increased amplitude, and long duration —are consistent with neuropathy. </think> <class>Neuropathy</class> < t h i n k>1. Amplitude Analysis : The waveform shows a range of amplitudes , with peaks reaching appro...
-
[68]
The waveform is polyphasic, meaning it has multiple peaks and troughs within the waveform
Morphology Analysis : The waveform has sharp onset and brief duration , indicating a rapid rise and fall of the electrical activity. The waveform is polyphasic, meaning it has multiple peaks and troughs within the waveform
-
[69]
Temporal Patterns : The waveform shows consistent amplitude and morphology , with no indication of variability or changes in waveform morphology over time
-
[70]
- Myopathy: Typically shows small amplitude and short duration , indicating a loss or dysfunction of muscle fibers
Comparison with Conditions : - Healthy: Typically shows normal recruitment and morphology of motor unit potentials, sharp onset , brief duration , and consistent amplitude. - Myopathy: Typically shows small amplitude and short duration , indicating a loss or dysfunction of mus...
-
[71]
</think> <class>Neuropathy</class> <think > 1
Conclusion: The waveform shows sharp onset and brief duration, consistent amplitude , and polyphasic morphology, which aligns with the characteristics of neuropathy. </think> <class>Neuropathy</class> <think > 1. Amplitude Analysis: The waveform shows a range of amplitudes fro...
-
[72]
The waveform morphology is consistent with normal recruitment and morphology of motor unit potentials
Morphology and Temporal Patterns : The waveform appears to have sharp onset and brief duration, which are typical of motor unit potentials . The waveform morphology is consistent with normal recruitment and morphology of motor unit potentials
-
[73]
WALKING_UPSTAIRS
Conclusion: The waveform shows sharp onset and brief duration, consistent with normal recruitment and morphology of motor unit potentials . The amplitude range is consistent with normal motor unit potentials. </think> <class>Healthy</class> Figure 8: Comparison of reasoning ou...
-
[74]
walking up stairs
Laying: Laying involves minimal acceleration , with all axes showing low values . Given the plot , the x and y axes show moderate values , while the z -axis shows significant values. This suggests that the activity involves significant vertical movement, which is characteristi...
-
[75]
atrial fibrillation
Pattern Analysis : The pattern shows a sustained high energy usage, which aligns with desktop usage . Desktops often have consistent power usage due to their larger components and higher power requirements . 4. Conclusion: The sustained high energy usage pattern suggests the d...
-
[76]
IR Negative
Approximate Value Ranges : The minimum value is approximately -1, and the maximum value is around 3. 4. Conclusion: The signal shows a gradual ramp -up followed by fluctuations, matching the characteristics of a Gradual Intra - Cloud event. </think> <class>Gradual Intra-Cloud<...
-
[77]
NOWHALE,
Frequency Consideration: Right whale calls would manifest as regular up -calls with distinct peaks and valleys over approximately one second . 3. Signal Density : The waveform 's variations do not exhibit clear periodicity that matches the expected 1-second duration of right w...
-
[78]
Specificity – Is the extension clearly tailored to the predicted class or context, avoiding generic language?
-
[79]
Appropriateness – Is the recommendation suitable given the prediction and reasoning?
-
[80]
Relevance – Does the extension logically follow from the preceding reasoning and predicted class?
-
[81]
be careful
Depth – Does the extension demonstrate domain knowledge, including detailed actions or next-step considerations? Instructions: - Penalize vague or generic outputs (e.g., "be careful") unless well-justified. - Reward informative, actionable, and context-aware suggestions. - Ret...
-
[82]
Since the sampling rate is 2kHz, any frequency components within this range should be detectable
**Frequency Range**: Right whale up-calls typically range from 60Hz to 250Hz. Since the sampling rate is 2kHz, any frequency components within this range should be detectable
-
[83]
Given the 2-second duration of the waveform, any call should be visible if it exists
**Duration**: Right whale calls usually last around 1 second. Given the 2-second duration of the waveform, any call should be visible if it exists
-
[84]
This would likely appear as a consistent pattern or peak within the correct frequency range over the duration of the call
**Signal Characteristics**: The waveform should show a distinct modulation corresponding to the up-call frequency. This would likely appear as a consistent pattern or peak within the correct frequency range over the duration of the call
-
[85]
] Generated Reasoning Sample (HAR) [
**Noise Levels**: Low-frequency noise from other activities may obscure the call, making it less visible. In the provided waveform: - The waveform appears mostly consistent without distinct peaks that would stand out as a right whale up-call. - The signal amplitude ranges with...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.