Pith. sign in

REVIEW 4 major objections 5 minor 66 references

HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Compact language models match cloud-scale LLMs on wearable health prediction.

desk verdict Useful benchmark, but the 'SLMs match LLMs' headline rests on a comparison to Health-LLM baselines that isn't apples-to-apples. read the letter →

arxiv 2509.07260 v5 pith:ALBC42MA submitted 2025-09-08 cs.AI cs.HCcs.LG

Xin Wang , Ting Dang , Xinyu Zhang , Vassilis Kostakos , Michael J. Witbrock , Hong Jia This is my paper
classification cs.AIcs.HCcs.LG
keywords smalllanguagemodelswearablehealthcaremonitoringhealthpredictionbenchmarkzero-shotlearningfew-shotinstructiontuningLoRAon-devicedeployment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether small language models (SLMs, roughly 1–4 billion parameters) can substitute for cloud-based large language models in predicting health conditions from wearable sensor data. Its central claim is yes: across three public wearable datasets and eight prediction tasks, the nine SLMs tested perform on par with or better than the larger LLM baselines under zero-shot, few-shot, and instruction-tuned settings, while using far less memory and responding faster. The authors build HealthSLM-Bench, a benchmark that turns 14-day summaries of steps, calories, heart rate, sleep, and mood into prompts, and deploy two fine-tuned SLMs on a phone to measure real-world latency and memory use. They also show the main remaining weakness: SLMs collapse toward majority classes on imbalanced tasks and struggle in some few-shot and calorie-regression cases. If the claim holds, privacy-preserving on-device health monitoring becomes feasible without sacrificing prediction quality.

What carries the argument

HealthSLM-Bench, a prompt-based benchmark that converts 14-day wearable sensor summaries (steps, calories, resting heart rate, sleep, self-reported mood) into instruction prompts, evaluates nine SLMs under zero-shot, few-shot, and instruction-tuning protocols, and compares their mean accuracy and mean absolute error against published LLM numbers. Instruction tuning uses LoRA, low-rank adaptation that trains small injected matrices while freezing the base model. On-device evaluation uses 4-bit quantized models run through a lightweight inference engine, with latency and memory metrics borrowed from the mobile-benchmark literature.

What would settle it

Re-run one LLM and one SLM on the same split of one dataset with an identical prompt template, greedy decoding, a 30-token cap, and a single parser; then recompute means with failed predictions counted as errors rather than omitted. If the LLM's mean error then drops below the SLM's, the paper's comparable-performance claim is not reproducible.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that compact language models are not a compromise version of LLMs for this benchmark: across the three datasets, SLM means are better than LLM means for stress, readiness, and fatigue in zero-shot settings, and after low-rank instruction tuning they beat LLMs on fatigue accuracy and calorie error by wide margins. The deployment experiment shows two top SLMs producing complete responses in about 1.8 to 7.6 seconds on a phone, versus about 57 seconds for a 7-billion-parameter LLM, while using 9–28% less RAM. The paper also documents where parity fails: one SLM consistently underperforms, calorie regression is much harder for SLMs in zero-shot settings, a

Load-bearing premise

The headline comparison assumes the earlier LLM results were obtained under the same prompts, same greedy decoding, same token cap, and same handling of unparseable outputs as the new SLM runs; if any of these differ, the parity claim could be an artifact.

Editorial extensions

If this is right

  • On-device SLMs can handle privacy-sensitive health inference without sending raw physiological data to a server.
  • A phone can run a fine-tuned SLM end-to-end in seconds, making real-time monitoring feasible; the paper reports about 1.8 seconds total for the smallest model versus about 57 seconds for a 7B LLM.
  • LoRA instruction tuning is enough to bring SLMs to LLM-level health accuracy, avoiding the cost of full fine-tuning.
  • Certain tasks, notably fatigue and calorie estimation, favor SLMs after tuning, while stress, readiness, and activity remain somewhat better handled by LLMs.
  • Class imbalance and few-shot collapse are the main remaining barriers to adoption, and the benchmark quantifies exactly where each model fails.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper aggregates 14 days into summary statistics, so it cannot see recovery dynamics or circadian patterns; prompting with raw daily sequences could let larger models pull ahead, which would bracket the parity claim to aggregated inputs.
  • Because the labels are self-reported fatigue, stress, mood, and sleep quality, a natural test is whether the same parity survives on objective clinical endpoints such as polysomnography or blood pressure readings.
  • The majority-class collapse after fine-tuning suggests that imbalance-aware LoRA, such as class-weighted loss or minority oversampling, might convert the current comparable result into a clear SLM advantage on imbalanced tasks.
  • If the few-shot collapse on some tasks stems from prompt format rather than model capacity, instruction-formatting search could be a cheaper fix than scaling model size.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces HealthSLM-Bench, a benchmark that evaluates nine small language models (SLMs) in the 1B-4B range on eight health prediction tasks drawn from three wearable/mobile datasets (PMData, GLOBEM, AW-FB). The evaluation covers zero-shot, few-shot (N=1,3,5,10), and LoRA-based instruction tuning. The authors compare SLM results with LLM baselines from Health-LLM [23], report that SLMs match or exceed LLM performance on several tasks, and deploy two fine-tuned SLMs on an iPhone 15 Pro Max to measure TTFT, token throughput, CPU, and RAM usage against Llama-2-7b. They conclude that SLMs are a promising privacy-preserving and efficient alternative to cloud-based healthcare LLMs, while acknowledging limitations from class imbalance and few-shot collapse.

Significance. If the claims hold, this is a useful contribution: it extends prior SLM health-monitoring work to a broader set of models, datasets, and adaptation protocols, and it includes rare on-device deployment measurements. The efficiency measurements and the systematic comparison of nine SLMs are concrete assets for the community. However, the central SLM-vs-LLM comparison currently depends on cross-paper baseline numbers whose protocol equivalence is not demonstrated, and the averaging over different subsets of models makes the headline means difficult to interpret. The paper does not yet provide code/checkpoints despite promising them, so reproducibility is currently limited to the textual protocol.

major comments (4)
  1. [§5, 5.1; Table 3; Appendix A] The headline claim that SLMs match or outperform LLMs rests on comparing SLM runs with LLM numbers taken from Health-LLM [23]. Section 5 says 'under the same settings as in [23]', but Appendix A specifies greedy decoding, max 30 generated tokens, Alpaca-style prompts, and output-constraint parsing for SLMs; no evidence is given that Health-LLM used the same decoding, max-token limit, prompt template, or failed-output handling. In Table 3, the LLM mean for stress (0.64) is computed over 10 models excluding GPT-3.5/GPT-4, whereas the SLM mean (0.61) averages all 9 models. If failure handling or decoding differs, the observed SLM advantage on stress/readiness/fatigue could be an artifact of averaging over different subsets. This is the load-bearing part of the benchmark and must be resolved by reproducing the baselines or reporting the exact Health-LLM protocol.
  2. [Table 4; §5.1 (Few-shot learning)] The few-shot comparison is asymmetric. LLM rows are labeled 'FS-best' (the best result across shot counts from [23]), while SLM rows are fixed FS-1, FS-3, FS-5, FS-10. Comparing the best-of-k LLM result with a single one-shot SLM result inflates the LLM baseline, not the SLM one, and conflates model capability with prompt-selection budget. For a fair benchmark, either report the same N for both families or provide the full shot-count sweep for LLMs.
  3. [Tables 3–5; dash handling] No explicit rule is given for how '-' (failed predictions) are treated in mean rows. In Table 3 the stress mean 0.64 excludes GPT-3.5/GPT-4 but the SLM mean 0.61 includes all 9 models; in Table 5 Gemma-2-2b-it has dashes for three PMData tasks, and the SLM mean likely drops it. Averages over different model sets are not comparable. The paper should report the number of valid predictions per model/task and compute means either over the same set or with an explicit imputation/failure penalty.
  4. [§5.2; Table 6] The efficiency comparison is narrower than implied. Only two SLMs are deployed and compared with Llama-2-7b as a proxy for the fine-tuned LLM baseline, and the summary statistics come from 10 randomly selected samples with no confidence intervals or variance. The conclusion that SLMs preserve latency and memory advantages is reasonable for these specific models, but it does not support a general efficiency claim across the nine SLMs in the benchmark. Reporting per-model variance and the prompt-length distribution would strengthen the claim.
minor comments (5)
  1. [§4.1] The AW-FB dataset heading appears as 'A W_FB' in several places; use 'AW-FB' consistently.
  2. [Appendix A] Typographical issues: 're-productiveness' should be 'reproducibility' and 'with batch size of 128 with 3 number of epochs' should be rephrased.
  3. [Table 4] Header contains 'across across eight healthcare monitoring tasks'; remove the duplicate word.
  4. [References] References [57] and [58] refer to the same llama.cpp repository; unify and correct the year/attribution.
  5. [§5.2] Table 6 title 'Efficiency & Utilization of LLMs & SLMs' is misleading because only one LLM (Llama-2-7b) is included.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose central claim rests on external LLM baselines, not on fitted parameters or self-citation chains.

full rationale

This is an empirical evaluation, not a derivation chain. The headline claim (Section 5.1: 'SLMs achieve comparable or better performance than LLMs across the three health datasets') is supported by two independent inputs: (i) the authors' own zero-shot, few-shot, and LoRA-tuned evaluations of SLMs, described concretely in Appendix A (greedy decoding, max 30 generated tokens, Alpaca-style templates, LoRA fine-tuning), and (ii) LLM baseline numbers taken from Health-LLM [23], an external paper by different authors. The LLM numbers are not outputs of the present paper's pipeline, not fitted parameters, and not defined in terms of the SLM results, so the comparison cannot be circular by construction. No fitted input is renamed as a prediction: the instruction-tuned models are evaluated on test data, and no parameter is tuned to a subset and then reported as a prediction of a closely related quantity. The self-citations [18] and [55] are used only for prompt construction, data-processing conventions, and related-work context; they are not invoked as a uniqueness theorem, a forbidden-alternative argument, or the justification for the central claim. The Appendix also openly discloses the SLM inference protocol, which is a transparency feature rather than a circular step. The legitimate concern raised by the reader—whether the 'same settings as in [23]' claim truly holds given Appendix A's greedy decoding and 30-token cap, and whether Table 3's means are computed over different model subsets—is a threat to external validity and fair comparison, not circularity: even if the protocols differ, the conclusion would be an artifact of uneven evaluation, not a tautology or a fitted-input-renamed-as-prediction. Since the central result depends on independent external baselines and direct held-out evaluation, the paper is self-contained against the circularity failure modes considered here. Score 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

This benchmark does not introduce a theoretical model, so there are no fitted equations. The load-bearing inputs are hand-chosen protocol and hyperparameter settings (LoRA rank, learning rate, max tokens, 14-day window, split seed, few-shot example selection) plus assumptions that self-reports are valid ground truth and that copied LLM numbers are comparable. The five settings listed as free parameters are all required to reproduce the numbers in Tables 3 to 6.

free parameters (6)
  • LoRA rank
    Not reported in the paper; chosen by hand, it controls the capacity of instruction tuning and thus all fine-tuned results in Table 5.
  • Learning rate = 5e-5
    Appendix A: 'learning rate as 5e-5', a hand-chosen hyperparameter for LoRA fine-tuning.
  • Max generated tokens = 30
    Appendix A caps generation at 30 tokens; this can truncate outputs and affect whether predictions are parsed as valid, influencing invalid-prediction counts.
  • 14-day data window = 14 days
    Section 4.3 follows prior work in standardizing daily sequences to 14-day windows; window length is a modeling choice that affects all predictions.
  • Data split seed
    Section 4.3 says data is 'randomly shuffled and split' 80/20 but no seed is given, so the exact splits underlying every table are not reproducible.
  • Few-shot example selection
    The paper varies N in {1,3,5,10} but does not report how the N examples are chosen or whether selection is seeded; this affects few-shot results in Table 4.
assumptions (4)
  • domain assumption Self-reported labels in PMData, GLOBEM, and AW-FB are treated as ground truth for health status.
    Section 4.1 uses self-reported fatigue, stress, readiness, sleep quality, anxiety and depression scores as training and evaluation targets. If these are noisy or biased, all accuracy and MAE numbers inherit that noise.
  • ad hoc to paper The LLM baseline numbers from Health-LLM [23] are directly comparable to the authors' SLM evaluations.
    Section 5.1 says the comparison is 'under the same settings as in [23]', but no evidence is given that prompts, decoding settings, output parsing, or handling of '-' failures match the SLM protocol described in Appendix A.
  • domain assumption Accuracy and MAE are sufficient summary metrics for the health prediction tasks.
    The paper acknowledges class imbalance and models predicting only majority classes (Section 5.1, Appendix C), yet it reports only accuracy and MAE, which do not quantify per-class performance or balanced accuracy.
  • domain assumption Greedy decoding at max 30 tokens yields valid, complete predictions for all tasks.
    Appendix A sets greedy decoding and caps generated tokens at 30; long or complex responses may be truncated, and invalid predictions are marked '-' in tables, meaning some models were excluded from means.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring." pith.science (2026). https://pith.science/paper/ALBC42MA

@misc{pith2026250907260,
  author       = {Pith},
  title        = {Pith review of: HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ALBC42MA}},
  note         = {Machine review of arXiv:2509.07260}
}
read the original abstract

Mobile and wearable healthcare monitoring play a vital role in facilitating timely interventions, managing chronic health conditions, and ultimately improving individuals' quality of life. Previous studies on large language models (LLMs) have highlighted their impressive generalization abilities and effectiveness in healthcare prediction tasks. However, most LLM-based healthcare solutions are cloud-based, which raises significant privacy concerns and results in increased memory usage and latency. To address these challenges, there is growing interest in compact models, Small Language Models (SLMs), which are lightweight and designed to run locally and efficiently on mobile and wearable devices. Nevertheless, how well these models perform in healthcare prediction remains largely unexplored. We systematically evaluated SLMs on health prediction tasks using zero-shot, few-shot, and instruction fine-tuning approaches, and deployed the best performing fine-tuned SLMs on mobile devices to evaluate their real-world efficiency and predictive performance in practical healthcare scenarios. Our results show that SLMs can achieve performance comparable to LLMs while offering substantial gains in efficiency and privacy. However, challenges remain, particularly in handling class imbalance and few-shot scenarios. These findings highlight SLMs, though imperfect in their current form, as a promising solution for next-generation, privacy-preserving healthcare monitoring.

Figures

Figures reproduced from arXiv: 2509.07260 by the authors.

Figure 1
Figure 1. The label distribution of the four tasks in PMData [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. The label distribution of the two tasks in GLOBEM [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. The label distribution of the two tasks in AW_FB [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Distribution of predictions for the four tasks in PMData under FS setting [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Distribution of predictions for the four tasks in PMData [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Distribution of predictions for the two tasks in GLOBEM [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 51 canonical work pages

  1. [23]

    Health-llm: Large language models for health prediction via wearable sensor data

    Yubin Kim, Xuhai Xu, Daniel McDuff, Cynthia Breazeal, and Hae Won Park. Health-llm: Large language models for health prediction via wearable sensor data. In Tom Pollard, Edward Choi, Pankhuri Singhal, Michael Hughes, Elena Sizikova, Bobak Mortazavi, Irene Chen, Fei Wang, Tasmie Sarker, Matthew McDermott, and Marzyeh Ghassemi, editors,Proceedings of the fi...

    work page 2024

  2. [1]

    Wearable health technology and electronic health record integration: scoping review and future directions.Journal of Medical Internet Research, 21(9):e12861, 2019

    Cecilia Dinh-Le, Rebecca Chuang, Sonia Chokshi, and Devin Mann. Wearable health technology and electronic health record integration: scoping review and future directions.Journal of Medical Internet Research, 21(9):e12861, 2019

    work page 2019

  3. [2]

    Pros: an efficient pattern-driven compressive sensing framework for low-power biopotential-based wearables with on-chip intelligence

    Nhat Pham, Hong Jia, Minh Tran, Tuan Dinh, Nam Bui, Young Kwon, Dong Ma, Phuc Nguyen, Cecilia Mascolo, and Tam Vu. Pros: an efficient pattern-driven compressive sensing framework for low-power biopotential-based wearables with on-chip intelligence. InProceedings of the 28th Annual International Conference on Mobile Computing And Networking, pages 661–675, 2022

    work page 2022

  4. [3]

    Ur2m: Uncertainty and resource-aware event detection on microcontrollers

    Hong Jia, Young D Kwon, Dong Mat, Nhat Pham, Lorena Qendro, Tam Vu, and Cecilia Mascolo. Ur2m: Uncertainty and resource-aware event detection on microcontrollers. In2024 IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 1–10. IEEE, 2024

    work page 2024

  5. [4]

    Udama: Unsupervised domain adaptation through multi-discriminator adversarial training with noisy labels improves cardio-fitness prediction

    Yu Wu, Dimitris Spathis, Hong Jia, Ignacio Perez-Pozuelo, Tomas I Gonzales, Soren Brage, Nicholas Wareham, and Cecilia Mascolo. Udama: Unsupervised domain adaptation through multi-discriminator adversarial training with noisy labels improves cardio-fitness prediction. In Machine Learning for Healthcare Conference, pages 863–883. PMLR, 2023

    work page 2023

  6. [5]

    AI on the Pulse: Real-Time Health Anomaly Detection with Wearable and Ambient Intelligence

    Lucas Gabrielli et al. Ai on the pulse: Integrating wearable sensors, ambient intelligence, and large language models for continuous health monitoring.arXiv preprint arXiv:2508.03436,

    work page Pith review arXiv

  7. [6]

    Wearable eeg and ai for real-time personalized health monitoring and intervention.Journal of Cloud Computing, 14(1):1–15, 2025

    Yassir Ghadi et al. Wearable eeg and ai for real-time personalized health monitoring and intervention.Journal of Cloud Computing, 14(1):1–15, 2025

    work page 2025

  8. [7]

    A survey on large language models for sensor-based human activity recognition and health monitoring.Sensors, 24(15):5045, 2024

    Emilio Ferrara. A survey on large language models for sensor-based human activity recognition and health monitoring.Sensors, 24(15):5045, 2024

    work page 2024

Show all 66 references
  1. [8]

    Llasa: Multimodal large language models for interpreting human activity from inertial sensor data

    Muhammad Imran et al. Llasa: Multimodal large language models for interpreting human activity from inertial sensor data. InProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  2. [9]

    Personal health llms: Towards context-aware and adaptive health monitoring from wearable sensor data.Nature Medicine, 2025

    Amir Khasentino et al. Personal health llms: Towards context-aware and adaptive health monitoring from wearable sensor data.Nature Medicine, 2025. In press

    2025

  3. [10]

    Security and privacy challenges of large language models.ACM Computing Surveys, 58(2):1–38, 2025

    Anish Das. Security and privacy challenges of large language models.ACM Computing Surveys, 58(2):1–38, 2025. doi: 10.1145/3712001. URL https://dl.acm.org/doi/10. 1145/3712001

    2025 doi

  4. [11]

    H. Li, Y . Chen, J. Luo, Y . Kang, X. Zhang, Q. Hu, C. Chan, and Y . Song. Privacy in large language models: Attacks, defences and future directions.arXiv, 2024. URL https://arxiv. org/pdf/2310.10383. Available: https://arxiv.org/pdf/2310.10383

    2024 arXiv

  5. [12]

    Camel: Energy-aware llm inference on resource-constrained devices.arXiv preprint arXiv:2508.09173, 2025

    Tianqi Xu, Wei Zhang, Chen Li, and Yifan Wang. Camel: Energy-aware llm inference on resource-constrained devices.arXiv preprint arXiv:2508.09173, 2025

    2025 arXiv

  6. [13]

    Large language models on edge devices: Challenges and opportunities for intelligent data analysis.Frontiers in Computer Science, 7:1538277, 2025

    Hui Wang, Qiang Liu, and Mei Chen. Large language models on edge devices: Challenges and opportunities for intelligent data analysis.Frontiers in Computer Science, 7:1538277, 2025

    2025

  7. [14]

    Phi-3 technical report: A highly capable language model locally on your phone

    Microsoft. Phi-3 technical report: A highly capable language model locally on your phone. Technical Report, 2024. URLhttps://arxiv.org/pdf/2404.14219v4

    2024 arXiv

  8. [15]

    Tinyllama: An open-source small language model

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model. https://github.com/jzhang38/TinyLlama, 2024. TinyLlama achieves approximately 70-80% of LLaMA2’s performance on commonsense reasoning tasks such as HellaSwag and ARC-Challenge

    2024

  9. [16]

    Qwen2-1.5b: A new series of large language models, 2024

    Qwen. Qwen2-1.5b: A new series of large language models, 2024. URL https:// huggingface.co/Qwen/Qwen2-1.5B. Accessed: 2024-09-01. 10

    2024

  10. [17]

    Gemma 2: Improving open language models at a practical size

    Gemma Team and Google DeepMind. Gemma 2: Improving open language models at a practical size. Technical report, Google DeepMind, 2024. For full author list, see Contributions and Acknowledgments section. Correspondence togemma-2-report@google.com

    2024

  11. [18]

    Efficient and personalized mobile health event prediction via small language models

    Xin Wang, Ting Dang, Vassilis Kostakos, and Hong Jia. Efficient and personalized mobile health event prediction via small language models. InProceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’24, page 2353–2358, New York, N...

    2024

  12. [19]

    Marcus, Suchi Saria, and Daniel Halperin

    Brandon Ballinger, Joy Hsieh, Avesh Singh, Nitish Sohoni, Jae Wang, Fangfei Li, Amit Sharma, Akshay Sharma, Gregory M. Marcus, Suchi Saria, and Daniel Halperin. Deepheart: Semi-supervised sequence learning for cardiovascular risk prediction. InProceedings of the Thirty-Second ...

  13. [20]

    Kristján Hallgrímsson, Tom Goodwin, Sujit Ghosh, Peter Bühlmann, Christian Mathys, Vincent Lefort, Ara Darzi, Lionel Tarassenko, and David A. Clifton. Learning individualized cardio- vascular responses from large-scale wearable sensors data. InProceedings of the Thirty-Third A...

    2019

  14. [21]

    Mullick, A

    T. Mullick, A. Radovic, S. Shaaban, and A. Doryab. Predicting depression in adolescents using mobile and wearable sensors: Multimodal machine learning–based exploratory study.JMIR Formative Research, 6(6):e35807, 2022. doi: 10.2196/35807. URL https://formative. jmir.org/2022/6/e35807

    2022 doi

  15. [22]

    Holtzman, Md

    Sebastian Kasl, Nathanael S. Holtzman, Md. Masudul Islam Shandhi, Tanishq Gupta, Jiang Kuang, Gregory D. Hager, Shawn S. Lam, and Suchi Saria. On the generalizability of wearable- based machine learning for respiratory virus detection. InProceedings of the 9th Machine Learning...

    2024

  16. [24]

    Exploring large-scale language models to evaluate eeg-based multimodal data for mental health.arXiv preprint arXiv:2408.07313, 2024

    Yongquan Hu, Shuning Zhang, Ting Dang, Hong Jia, Flora D Salim, Wen Hu, and Aaron J Quigley. Exploring large-scale language models to evaluate eeg-based multimodal data for mental health.arXiv preprint arXiv:2408.07313, 2024

    2024 arXiv

  17. [25]

    Dey, and Dakuo Wang

    Xuhai Xu, Bingsheng Yao, Yuanzhe Dong, Saadia Gabriel, Hong Yu, James Hendler, Marzyeh Ghassemi, Anind K. Dey, and Dakuo Wang. Mental-llm: Leveraging large language models for mental health prediction via online text data. InProceedings of the ACM on Interactive, Mobile, Weara...

    2024 doi

  18. [26]

    Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    2023

  19. [27]

    Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    2023 arXiv

  20. [28]

    Physiollm: Supporting personalized health insights with wearables and large language models.arXiv preprint arXiv:2406.19283, 2024

    Cathy Mengying Fang, Valdemar Danry, Nathan Whitmore, Andria Bao, Andrew Hutchison, Cayden Pierce, and Pattie Maes. Physiollm: Supporting personalized health insights with wearables and large language models.arXiv preprint arXiv:2406.19283, 2024. 11

    2024 arXiv

  21. [29]

    Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    2024 arXiv

  22. [30]

    Lane, and Mengwei Xu

    Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Wei Liu, Jian Luan, Xiwen Zhang, Nicholas D. Lane, and Mengwei Xu. Demystifying small language models for edge deployment. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol...

    2025

  23. [31]

    Mobileaibench: Benchmarking llms and lmms for on-device use cases

    Rithesh Murthy, Liangwei Yang, Juntao Tan, Tulika Manoj Awalgaonkar, Yilun Zhou, Shelby Heinecke, Sachin Desai, Jason Wu, Ran Xu, Sarah Tan, et al. Mobileaibench: Benchmarking llms and lmms for on-device use cases. 2023

    2023

  24. [32]

    Phi-3-mini-4k-instruct: A lightweight, state-of-the-art open model, 2024

    Microsoft. Phi-3-mini-4k-instruct: A lightweight, state-of-the-art open model, 2024. URL https://phi.microsoft.com/phi-3-mini-4k-instruct. Accessed: 2024-09-01

    2024

  25. [33]

    Tinyllama-1.1b-chat-v1.0: A compact llama model with 1.1b parameters, 2024

    TinyLlama. Tinyllama-1.1b-chat-v1.0: A compact llama model with 1.1b parameters, 2024. URLhttps://github.com/jzhang38/TinyLlama. Accessed: 2024-09-01

    2024

  26. [34]

    Gemma 2: A lightweight, state-of-the-art open model family, 2024

    Google. Gemma 2: A lightweight, state-of-the-art open model family, 2024. URL https: //huggingface.co/google/gemma-2-2b. Accessed: 2024-09-01

    2024

  27. [35]

    Smollm-1.7b-instruct: A series of small language models, 2024

    HuggingFaceTB. Smollm-1.7b-instruct: A series of small language models, 2024. URL https://huggingface.co/HuggingFaceTB/SmolLM-1.7B-Instruct . Accessed: 2024- 09-01

    2024

  28. [36]

    Llama 3.2 model card

    Meta AI. Llama 3.2 model card. Hugging Face, 2024. Release date: September 25, 2024. Includes lightweight text-only (1 B, 3 B) and multimodal (11 B, 90 B) models

    2024

  29. [37]

    Pmdata: A sports logging dataset

    Vajira Thambawita, Steven Alexander Hicks, Hanna Borgli, Håkon Kvale Stensland, Debesh Jha, Martin Kristoffer Svensen, Svein-Arne Pettersen, Dag Johansen, Håvard Dagenborg Johansen, Susann Dahl Pettersen, Simon Nordvang, Sigurd Pedersen, Anders Gjerdrum, Tor-Morten Grønli, Per...

    2020

  30. [38]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff...

    2020 arXiv

  31. [39]

    Chain-of-thought prompting elicits reasoning in large language models

    John Wei, Michael Bosma, Dale Schuurmans, and et al. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of NeurIPS 2022, 2022. URL https: //arxiv.org/abs/2201.11903

    2022 arXiv

  32. [40]

    Self-consistency improves chain-of-thought reasoning in language models

    Xinyang Wang, Hinrich Schütze, and et al. Self-consistency improves chain-of-thought reasoning in language models. InProceedings of NeurIPS 2022, 2022. URL https: //arxiv.org/abs/2203.11171

    2022 arXiv

  33. [41]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y . Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V . Le. Finetuned language models are zero-shot learners. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. URL https://ic...

    2022

  34. [42]

    Stanford alpaca: An instruction-following llama model

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. Stanford alpaca: An instruction-following llama model. https://crfm.stanford.edu/2023/03/13/alpaca.html, 2023. Accessed: 2025-04-27. 12

    2023

  35. [43]

    Smith, Iz Beltagy, and Hannaneh Hajishirzi

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources. InAd- vances in Neural...

  36. [44]

    Dolly: The first truly open-source instruction-tuned model

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Sam Shah, Ali Ghodsi, Patrick Wendell, and Matei Zaharia. Dolly: The first truly open-source instruction-tuned model. Databricks blog,

  37. [45]

    Vicuna: An open-source chatbot impressing gpt-4 with 90 LMSYS blog post,

    Vicuna Team. Vicuna: An open-source chatbot impressing gpt-4 with 90 LMSYS blog post,

  38. [46]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In Proceedings of the Tenth International Conference on Learning Representations (ICLR), 2022. URLhttps://openrevi...

    2022

  39. [47]

    Fine-tuned on the Stanford Alpaca dataset

  40. [48]

    Replication data for: Using machine learning methods to predict physical activity types with apple watch and fitbit data using indirect calorimetry as the criterion, 2020

    Daniel Fuller. Replication data for: Using machine learning methods to predict physical activity types with apple watch and fitbit data using indirect calorimetry as the criterion, 2020. URL https://doi.org/10.7910/DVN/ZS2Z2J. Accessed: 2025-04-29

    2020 doi

  41. [49]

    Instruction tuning inspired by Alpaca’s methodology

  42. [50]

    Geneactiv: Raw data accelerometer for physical activity and sleep research

    Activinsights Ltd. Geneactiv: Raw data accelerometer for physical activity and sleep research. https://www.activinsights.com/products/geneactiv/, 2015. Accessed: 2025-08- 30

    2015

  43. [51]

    Morris, Eve Riskin, Jennifer Mankoff, and Anind K

    Xuhai Xu, Han Zhang, Yasaman Sefidgar, Yiyi Ren, Xin Liu, Woosuk Seo, Jennifer Brown, Kevin Kuehn, Mike Merrill, Paula Nurius, Shwetak Patel, Tim Althoff, Margaret E. Morris, Eve Riskin, Jennifer Mankoff, and Anind K. Dey. Globem dataset: Multi-year datasets for longitudinal h...

    2023 arXiv

  44. [52]

    Fitbit charge 2: Heart rate + fitness wristband

    Fitbit Inc. Fitbit charge 2: Heart rate + fitness wristband. https://www.fitbit.com/ global/us/products/trackers/charge2, 2016. Accessed: 2025-08-30

    2016

  45. [53]

    Fitbit versa 2: Health & fitness smartwatch

    Fitbit Inc. Fitbit versa 2: Health & fitness smartwatch. https://www.fitbit.com/global/ us/products/smartwatches/versa2, 2019. Accessed: 2025-08-30

    2019

  46. [54]

    Tinyllama: A distilled version of llama for efficient language tasks

    LLaMA Open Source Team. Tinyllama: A distilled version of llama for efficient language tasks. https://github.com/TinyLlama, 2024. Highlights the use of knowledge distillation for TinyLlama-1.1B derived from LLaMA-13B

    2024

  47. [55]

    Apple watch series 2

    Apple Inc. Apple watch series 2. https://support.apple.com/kb/SP745, 2016. Accessed: 2025-08-30

    2016

  48. [56]

    Gguf, 2023

    Hugging Face. Gguf, 2023. URL https://huggingface.co/docs/hub/en/gguf. Ac- cessed: 2025-09-05

    2023

  49. [57]

    Fine-tune small language model (slm) phi-3 using azure machine learning

    Microsoft Corporation. Fine-tune small language model (slm) phi-3 using azure machine learning. https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/ fine-tune-small-language-model-slm-phi-3-using-azure-machine/ba-p/ 4130399, 2024. Accessed: 2025-06-09

    2024

  50. [58]

    llama.cpp: Efficient llm inference in c/c++

    Georgi Gerganov and community. llama.cpp: Efficient llm inference in c/c++. https:// github.com/ggml-org/llama.cpp, 2023. Released March 10, 2023; accessed 2025-09-06. 13

    2023

  51. [59]

    Beyond scale: Small language models are comparable to gpt-4 in mental health understanding.arXiv preprint arXiv:2507.08031, 2025

    Hong Jia, Shiya Fu, Feng Xia, Vassilis Kostakos, and Ting Dang. Beyond scale: Small language models are comparable to gpt-4 in mental health understanding.arXiv preprint arXiv:2507.08031, 2025

    2025 arXiv

  52. [60]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  53. [61]

    Ggerganov/llama.cpp: Llm inference in c/c++

    Ggerganov. Ggerganov/llama.cpp: Llm inference in c/c++. URL https://github.com/ ggerganov/llama.cpp

  54. [62]

    3” when the true label is “4

    Trevor Hastie, Robert Tibshirani, and Jerome Friedman.The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009. 14 Appendix A Implementation Details We fine-tune our SLMs on a NVIDIA A100 80GB GPUs with a batch size of 128 with 3 number of e...

    2009

  55. [63]

    Kwon, Alessio Orsino, Ting Dang, Domenico Talia, and Cecilia Mascolo

    Hong Jia, Young D. Kwon, Alessio Orsino, Ting Dang, Domenico Talia, and Cecilia Mascolo. Tinytta: Efficient test-time adaptation via early-exit ensembles on edge devices. InAdvances in Neural Information Processing Systems, volume 37, pages 43274–43299, 2024

    2024

  56. [65]

    Bishop.Pattern Recognition and Machine Learning

    Christopher M. Bishop.Pattern Recognition and Machine Learning. Springer, 2006

    2006

  57. [2023]

    URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/ ec6413875e4ab08d7bc4d8e225263398-Abstract-Datasets_and_Benchmarks.html

    2023

  58. [2025]

    Accessed: 2025-09-04

    2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.