TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML

Chaymae Yahyati; Ibrahim Ouahbi; Ismail Lamaakal; Khalid El Makkaoui; Yassine Maleh

REVIEW 5 major objections 4 minor 28 references

TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML

T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A TinyML monitor turns short-horizon temporal instability into calibrated abstention without labels or extra forward passes.

desk verdict The temporal-consistency uncertainty signals and MCU measurements are useful, but the paper's central 'streaming conformal calibration' claim is not established by the evidence it provides. read the letter →

arxiv 2508.12905 v1 pith:P4KVNSTF submitted 2025-08-18 cs.LG cs.CL

Ismail Lamaakal , Chaymae Yahyati , Khalid El Makkaoui , Ibrahim Ouahbi , Yassine Maleh This is my paper · ORCID

classification cs.LGcs.CL

keywords temporalconsistencyuncertaintyquantificationTinyMLstreamingconformalcalibrationabstentionlabel-freemonitoringaccuracy-dropdetection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TCUQ proposes that a lightweight device can monitor its own reliability in a streaming setting by watching how its predictions and features wobble over a short window. Four cheap signals—multi-lag predictive divergence, feature instability, label-flip rate, and a confidence-margin proxy—are combined into a single uncertainty score with $O(W)$ ring-buffer state and $O(1)$ per-step updates, and a streaming conformal quantile converts that score into a budgeted accept/abstain rule. The paper argues this yields calibrated behavior without online labels or extra forward passes, fits in kilobyte-scale memory on microcontrollers, and detects accuracy drops under corrupted streams earlier and more reliably than early-exit and deep-ensemble baselines (up to 0.86 AUPRC at high severity). If right, it gives TinyML deployments a practical way to refuse predictions when conditions degrade, without the memory and latency cost of ensembles or multi-pass methods.

What carries the argument

The load-bearing object is the streaming conformal layer combined with the temporal-consistency signal. The signal vector $\mathbf{s}_t = [D_t, (1-S_t), (1-c_t), m_t]^\top$ aggregates multi-lag posterior divergence, feature instability, decision inconsistency, and a near-tie-sensitive confidence proxy; the logistic combiner $\sigma(\mathbf{w}^\top \mathbf{s}_t + b)$ trained once offline maps it to an uncertainty score $U_t$. The streaming conformal layer maintains a memory-constant estimate of the $(1-\alpha)$ quantile $q_{\alpha,t}$ of the nonconformity scores $r_t = \lambda U_t + (1-\lambda)(1-C_\phi(x_t))$ and abstains when $r_t \ge q_{\alpha,t}$; this is what converts the score into a calibrated, budgeted accept/abstain rule without online labels or extra forward passes. The ring buffer of size $W$ supplies the history for the lagged signals, keeping all per-step updates $O(1)$ and total state $O(W(d+L))$.

What would settle it

On a labeled stream that interleaves ID, CID, and OOD segments, record the empirical error rate among predictions the rule accepts; if that rate exceeds $\alpha$ (say, 0.1) on any segment, or if the rejection rate drifts away from $\alpha$ because the quantile tracker does not follow the distribution, the streaming conformal calibration claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that short-horizon temporal consistency is a sufficient label-free signal for calibrated uncertainty monitoring on microcontrollers. Concretely, TCUQ computes, from a frozen backbone's posteriors and features over a small ring buffer, four signals—multi-lag Jensen–Shannon divergence, cosine feature instability, decision-persistence violation, and a blended inverse-confidence/inverse-margin proxy—and merges them through a logistic combiner fitted once offline into a scalar $U_t$. The nonconformity score $r_t = \lambda U_t + (1-\lambda)(1-C_\phi(x_t))$ is fed to a streaming conformal layer that maintains an online estimate of the $(1-\alpha)$ quantile; the device abstains when $r_t \ge q_{\alpha,t}$, subject to a rate controller for the abstention budget $b$. The paper reports that this single-pass monitor fits in kilobytes, reduces flash and latency by roughly 50–60% and 30–45% versus early-exit ensembles and deep ensembles while keeping accuracy parity, and improves corrupted-stream accuracy-drop detection by 3–7 AUPRC points, reaching 0.66 on MNIST-C and up to 0.86 at high severity on TinyImageNet-C, with failure detection up to 0.92 AUROC.

Load-bearing premise

The central premise is that an online, memory-constant estimate of the $(1-\alpha)$ quantile of the label-free scores yields a calibrated accept/abstain rule—but the paper never specifies the tracker, and the measured exceedance deviation $\Delta_\alpha$ matches $\alpha$ by the quantile's definition rather than by demonstrating control of the error rate among accepted predictions.

Editorial extensions

If this is right

If TCUQ's claims hold, a TinyML device can run a calibrated abstention policy in a single forward pass, needing only a few kilobytes of extra state, where early-exit ensembles and deep ensembles are out of memory.
Corrupted-stream accuracy drops would be detectable 25–35% sooner than with the compared baselines, with AUPRC gains of 3–7 points and up to 0.86 at high corruption severity.
The monitor would remain label-free online: calibration comes from the streaming quantile, so no ground-truth labels are needed after deployment.
The same recipe extends to audio keyword spotting and to larger backbones (ResNet-50 experiments) without architectural changes, and the authors argue it transfers to transformer-based backbones.
Budgeted abstention becomes practical: a user-set abstention budget $b$ can be enforced while still responding to bursts of high uncertainty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The calibration guarantee is only as strong as the assumption that the online quantile estimator actually tracks the $(1-\alpha)$ quantile under stream shift; the paper does not specify the memory-constant tracker, so a stress test on a drifting stream would settle whether the exceedance rate matches $\alpha$ over time.
The measured $\Delta_\alpha$ (Eq. 11) is the deviation of the rejection rate from $\alpha$, which follows from the quantile definition; it is not a measure of error control among accepted predictions. A fair reader should infer that the paper establishes quantile exceedance calibration, not selective-risk control, unless the rate controller also bounds accepted-prediction error.
The paper defers the rate controller that reconciles $\alpha$ with the abstention budget $b$ to Appendix B.5, which actually discusses the number of temporal-assistance exits, not the controller; the budgeted-abstention guarantee is therefore not fully demonstrated in the main text.
A testable extension: use TCUQ on a long, labeled deployment stream and compute the empirical error rate among accepted predictions; if it exceeds $\alpha$ when the stream drifts, the conformal layer needs explicit risk-controlling updates rather than a drifting threshold.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

TCUQ is a plausible TinyML uncertainty monitor with a genuinely useful temporal-consistency feature set, but the headline streaming conformal calibration claim does not survive contact with the paper's own equations.

What is actually new: combining four lightweight temporal signals — multi-lag JSD divergence, cosine feature stability, label persistence, and a confidence-margin proxy — into a single logistic score maintained with an O(W) ring buffer and O(1) updates. That specific combination is new for TinyML, and the ablations (Tables 6 and 7) suggest the temporal signals carry real signal. The MCU footprint and latency results (Tables 11–12) also look plausible: single-pass inference, small state, and roughly 50% smaller binaries than ensemble baselines. Those are useful engineering contributions worth acknowledging.

Where it falls down: the paper claims the streaming quantile gives 'calibrated' abstention without online labels. The only calibration metric offered is Eq. 11, the exceedance deviation between r_t and its own online quantile. That quantity is approximately alpha by construction; it says nothing about whether the accepted predictions are correct. The paper never reports selective risk or accepted-set error rates, and the 'memory-constant quantile tracker' from Section 3 is never specified. The budget controller is also missing: Section 3 points to Appendix B.5, but that appendix is about the number of temporal-assistance exits, not a rate controller. On top of that, the numbers have internal contradictions: Big-MCU latency reductions differ between Section 5.1 and Table 11 (27% vs. 43%), CIFAR-10-C severity-5 AUPRC is 0.80 in Section 5.2 but 0.77 in Tables 4 and 6, and Appendix D.2 reports Big-MCU latencies roughly seven times smaller than Table 11 for the same setup. These do not necessarily invalidate the temporal feature, but they make it hard to trust the headline empirical claims.

Who this is for: TinyML practitioners interested in lightweight uncertainty signals, and UQ researchers who want a cautionary example of conflating quantile exceedance with risk control. A serious referee could get the authors to fix the framing — drop the word 'conformal' or define precisely what is being calibrated, specify the quantile tracker and the budget controller, report accepted-set risk, and reconcile the tables. As is, the central calibration claim is the centerpiece and it is unsupported. I would not desk-reject it, but I would not accept it in this form.

Referee Report

5 major / 4 minor

Summary. The paper proposes TCUQ, an on-device uncertainty monitor for streaming TinyML. It computes four temporal-consistency signals from a short ring buffer (multi-lag predictive divergence, feature stability, decision persistence, and a confidence/margin proxy), combines them with a logistic model fitted offline on a labeled development set, forms a scalar nonconformity score r_t, and maintains an online quantile q_{α,t} to decide between prediction and abstention. The authors claim that this yields a calibrated, budgeted accept/abstain rule without online labels, and they report MCU size/latency comparisons, CID accuracy-drop detection AUPRC, failure-detection AUROC, and proper-score/ECE comparisons on MNIST, SpeechCommands, CIFAR-10, and TinyImageNet.

Significance. If the calibration claim were valid, TCUQ would be a meaningful TinyML contribution: a single-pass, kilobyte-scale, label-free abstention mechanism with O(W) memory and O(1) per-step updates is genuinely useful, and the temporal-consistency score shows promising empirical performance as a ranking signal (best CID AUPRC in Tables 1 and 4, competitive AUROC in Table 2). However, the central novelty—streaming conformal calibration—is not established. The only streaming calibration metric is tautological, the quantile tracker and budget controller are unspecified, no accepted-set risk or coverage is reported, and the resource tables contain internal contradictions. The empirical detection results may survive a reframing as a lightweight error-ranking score, but the current claims about calibrated, budgeted abstention are unsupported.

major comments (5)

[Section 3; Eq. (11)] The only streaming calibration metric reported, Δ_α = |(1/T)Σ 1[r_t ≥ q_{α,t}] − α|, measures the rate at which the label-free nonconformity score r_t exceeds its own maintained quantile. For any reasonable quantile estimator this rate is approximately α by construction, so Eq. (11) is a consistency check on the estimator, not an independent test of calibrated abstention. Calibrated selective classification requires controlling the error rate among accepted predictions (equivalently, coverage of the accepted set), and the paper reports no such quantity; Section 4 says risk–coverage curves are plotted, but no such figure appears in the results. The abstract's claim that the streaming conformal layer yields 'calibrated behavior' is therefore unsupported.
[Section 3; Appendix B.5] The 'simple rate controller' said to enforce the abstention budget b is never specified, and the cross-reference to Appendix B.5 is incorrect: Appendix B.5 analyzes the number K of temporal-assistance exits and does not discuss an abstention-rate controller. Without the controller's rule, the interaction between quantile-based abstention and budget enforcement cannot be analyzed, and the claimed budgeted abstention is not reproducible.
[Section 3] The paper relies on a 'memory-constant quantile tracker' to maintain q_{α,t}, but no algorithm, state size, update rule, or convergence property is given; the only description elsewhere is 'a lightweight stochastic estimator' in Section 4. Because this tracker is the entire basis of the streaming conformal layer, the central mechanism cannot be implemented or verified from the manuscript.
[Abstract; Section 5; Table 11] The resource-savings numbers are internally inconsistent. The Section 5 summary claims flash reductions of 50%/52% versus EE-ens and 38%/62% versus DEEP on Big-MCU, and the abstract says 'typically about 50 to 60% smaller', but Table 11 shows reductions of 33% (300 vs 450 KB) and 34% (356 vs 541 KB) versus EE-ens, and 28% (300 vs 414 KB) and 38% (356 vs 578 KB) versus DEEP. Section 5.1's latency claims (27% and 22%) also disagree with Table 11 (43%/31% and 29%/38%). These discrepancies affect the headline deployment claim.
[Table 12; Table 13] The Small-MCU latency measurements are mutually contradictory: Table 12 reports SpeechCommands latency for TCUQ of 169 ms (with EE-ens at 352 ms and DEEP at 296 ms), while Table 13 reports 48.0 ± 0.7 ms for TCUQ on Small-MCU SpeechCommands (EE-ens 98 ms, DEEP 86 ms). The relative speedups are similar, but the absolute numbers differ by about a factor of 3.5, so at least one table cannot be correct. The energy/latency protocol in Appendix D.2 must be reconciled with the main MCU results.

minor comments (4)

[Equation (2); Section 4] The symbol α is used both for the confidence-margin blend in Eq. (2) and for the target quantile level in the streaming calibration setup; rename one to avoid ambiguity.
[Table 1; Appendix Table 4] Appendix Table 4 reports TCUQ MNIST-C AUPRC of 0.62 and SpeechCmd-C AUPRC of 0.62, while Table 1 reports 0.66 and 0.63 for the same settings; the discrepancy should be explained.
[Section 4; Tables 11–12] The text says latency measurements are averaged over 1,000 inferences and the standard deviation is reported, but Tables 11 and 12 contain no standard deviations; either add the intervals or adjust the text.
[Appendix D.2] The reproducibility statement says measurement scripts and board configurations are provided as an artifact bundle, but no link or repository is given.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline claim of streaming conformal calibration reduces to a quantile-definition identity; the detection results themselves are not circular.

self definitional [Section 3 (Eq. 4 and the budget-aware accept/abstain rule) and Appendix E.3 (Eq. 11)]
"maintain an online estimate of the (1−α) quantile qα,t of {r1,...,rt} using a memory-constant quantile tracker. ... when rt ≥ qα,t and the abstention controller allows it, the system outputs ABSTAIN; otherwise, it emits ŷt. ... Δα = |1/T Σ_{t=1}^T 1{rt ≥ qα,t} − α| (11), measuring how closely the calibrated rejection rate matches the target risk level α"

The paper offers Eq. 11 as the streaming calibration check that underpins the claim of 'calibrated behavior.' But qα,t is defined as the (1−α) quantile of the same nonconformity scores r_t whose exceedance rate Eq. 11 measures. An empirical quantile exceeds (or is exceeded by) the corresponding fraction of the data by construction, so Δα ≈ 0 is dictated by the definition of the tracker rather than by any independent property of the abstain rule. This does not establish that the accepted subset is correct or that error risk among accepted predictions is controlled; the conformal-calibration contribution reduces to a definitional identity.

full rationale

TCUQ's temporal-consistency signal, its AUPRC accuracy-drop detection (Tables 1-2, severity curves), and its AUROC failure detection are self-contained and benchmarked against external baselines; those results are not circular. The logistic combiner (Eq. 3) is explicitly fitted on a labeled development split, so its fitted status is disclosed rather than smuggled in as a prediction. The circularity is concentrated in the second headline contribution: the streaming conformal layer is said to yield a calibrated accept/abstain rule, but the only reported calibration metric, Eq. 11, measures the exceedance rate of r_t against qα,t, which is the very quantile that defines the threshold; that rate equals α by construction and says nothing about risk on accepted predictions. A related missing support is also flagged: the 'simple rate controller' for the abstention budget is deferred to Appendix B.5, which actually studies the number of temporal-assistance exits, not a rate controller, so the budgeted-abstention mechanism is not specified. Because one headline 'prediction' is definitionally forced while the temporal/detection contributions carry independent content, the overall circularity score is 6.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several fitted or hand-chosen parameters (W, L, lambda, alpha, b, combiner weights, Eq. 2's alpha) and on domain assumptions about temporal instability and quantile-based calibration. The most consequential is the assumption that an unspecified streaming quantile of an unlabeled score provides calibrated risk control; this is where the circularity burden is highest.

free parameters (8)

logistic combiner weights (w,b) = not reported (learned offline on dev set)
Equation 3; fitted once offline on a small labeled development set mixing ID and shifted examples to predict misclassification.
nonconformity blend lambda = 0.7 default
Equation 4; selected on development split, controls balance between uncertainty score and inverse confidence.
target risk level alpha = 0.1 default
Used as the conformal quantile level; selected on dev split; its meaning conflates rejection rate with error risk.
abstention budget b = 0.15 default
Long-run abstention target in the accept/abstain rule; the controller enforcing it is not described.
ring buffer window W = 16 (vision), 20 (audio)
Chosen on dev split under RAM/latency budgets; memory scales with W.
lag set L = {1,2,4} with weights proportional to 1/l
Hand-chosen; small lags for short-horizon consistency.
confidence-margin blend alpha in Eq. 2 = not reported
Equation 2 blends inverse confidence and inverse margin but its value is not specified anywhere in the paper.
temporal-assistance weights w_TA,0 and delta = 3 and 0.5
Equations 5-7; training-only auxiliary exit loss weights that affect the backbone and the downstream signals.

assumptions (5)

domain assumption Short-horizon temporal instability of posteriors, features, and labels is a useful label-free proxy for misclassification under corruption and shift.
Underpins the four signals in Section 3; the paper's ablations give some empirical support but it is an assumption about the deployment regime.
ad hoc to paper A memory-constant online quantile tracker can estimate the (1-alpha) quantile of an unlabeled score stream well enough to make accept/abstain decisions.
Section 3 and E.3; the tracker is never specified, and the 'calibration' reduces to matching the exceedance rate to alpha.
domain assumption The offline logistic combiner, trained on dev data with representative CID/OOD examples, generalizes to the corruption types used in evaluation.
Section A.3; if dev corruptions overlap test corruptions, this is fitting rather than prediction.
domain assumption Comparisons against baselines are controlled despite TCUQ's backbone being trained with temporal-assistance exits.
A.3 describes TA training only for TCUQ's backbone; baselines are said to use the same backbones, but it is unclear whether they received the same TA training; Table 7 shows TA changes NLL by up to about 20% on TinyImageNet.
domain assumption The moving-window accuracy drop threshold (mu_ID minus 3 sigma_ID) defines ground-truth corruption events.
Section 4 and A.6; this operationalizes 'accuracy drop' but is a heuristic about event timing.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML." pith.science (2026). https://pith.science/paper/P4KVNSTF

@misc{pith2026250812905,
  author       = {Pith},
  title        = {Pith review of: TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyML},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P4KVNSTF}},
  note         = {Machine review of arXiv:2508.12905}
}

read the original abstract

We introduce TCUQ, a single pass, label free uncertainty monitor for streaming TinyML that converts short horizon temporal consistency captured via lightweight signals on posteriors and features into a calibrated risk score with an O(W ) ring buffer and O(1) per step updates. A streaming conformal layer turns this score into a budgeted accept/abstain rule, yielding calibrated behavior without online labels or extra forward passes. On microcontrollers, TCUQ fits comfortably on kilobyte scale devices and reduces footprint and latency versus early exit and deep ensembles (typically about 50 to 60% smaller and about 30 to 45% faster), while methods of similar accuracy often run out of memory. Under corrupted in distribution streams, TCUQ improves accuracy drop detection by 3 to 7 AUPRC points and reaches up to 0.86 AUPRC at high severities; for failure detection it attains up to 0.92 AUROC. These results show that temporal consistency, coupled with streaming conformal calibration, provides a practical and resource efficient foundation for on device monitoring in TinyML.

Figures

Figures reproduced from arXiv: 2508.12905 by the authors.

**Figure 1.** TCUQ for streaming TinyML. A compact backbone produces final features and posteriors for the current input, while a small ring buffer retains a short history. From this window, TCUQ extracts four lightweight temporal signals—predictive divergence, feature stability, decision persistence, and a confidence proxy and merges them (via weights learned once offline) into a single uncertainty score without extra forward p… view at source ↗

**Figure 2.** TCUQ micro-view. The final features fD feed a light logistic map w and activation σ(·) to produce the stepwise uncertainty Ut (top path). In parallel, the model outputs posteriors pϕ(y | xt) and, together with recent history, forms a compact signal set st (left) that informs w through a merge node for clean routing. A streaming conformal layer maintains an online quantile qα for calibrated accept/abstain decisions. … view at source ↗

**Figure 3.** Microcontroller results for SpeechCmd (SpCmd) and CIFAR-10 (cfr10) in one row. Lower is better. In Small-MCU panels, methods without bars for cfr10 are OOM. Fashion-MNIST (Xiao et al., 2017) as OOD for MNIST, SVHN (Netzer et al., 2011) for CIFAR10, unseen non-keyword audio and background noise for SpeechCommands, and non-overlapping TinyImageNet classes for the TinyImageNet model. All OOD sets are disjoint from tra… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Accuracy-drop detection (b,c). AUPRC vs. corruption severity for CIFAR-10-C and TinyImageNet-C. Higher is better. TCUQ (blue, solid) leads across datasets and severities. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Convergence with/without TA. Batch loss at a TA block for ResNet-8 on CIFAR-10. B.5 UNCERTAINTY QUALITY VS. NUMBER OF TEMPORAL-ASSISTANCE EXITS We study how the number of temporal-assistance exits K used during training (inference remains single-pass) affects accuracy…

**Figure 6.** Figure 6: Effect of temporal-assistance exit count |K| on MobileNetV2/TinyImageNet. Left: top-1 accuracy (higher is better). Right: NLL (lower is better). Red dashed line marks the BASE model. Accuracy is non-monotonic with K, while NLL improves up to K=8 and then slightly worse…

**Figure 7.** Figure 7: Accuracy-drop detection on RESNET-50/TINYIMAGENET-C. AUPRC averaged over all corruptions at each severity. TCUQ dominates across severities and shows the largest gap at high severity, while remaining single-pass at inference. C MCU RESULTS AND FURTHER DISCUSSION We ben…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 14 canonical work pages

[1]

Angelopoulos and Stephen Bates

Anastasios N. Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv:2107.07511,

arXiv
[3]

Abstention budget adherence

We track the exceedance deviation ∆α = 1 T TX t=1 ⊮ rt≥qα,t − α , (11) measuring how closely the calibrated rejection rate matches the target risk levelα on ID segments (lower is better). Abstention budget adherence. With a desired long-run budgetb and observed abstention rateˆb, we report|ˆb−b| over the full stream and short windows, assessing both avera...

work page 2023
[4]

Nikhil P Ghanathe and Steven J. E. Wilton. Qute: Quantifying uncertainty in tinyml with early-exit- assisted ensembles for model monitoring. arXiv:2404.12599,

arXiv
[5]

org/abs/2106.00170

URL https://arxiv. org/abs/2106.00170. W Brier Glenn et al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3,

arXiv
[8]

Branchynet: Fast inference via early exiting from deep neural networks

Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pp. 2464–2469. IEEE,

2016
[9]

(2019) in our discussion but do not optimize for them

For completeness, we additionally reference adaptive/static variants (ACE/SCE) from Nixon et al. (2019) in our discussion but do not optimize for them. E.3 S TREAMING METRICS FOR TCUQ Because TCUQ operates online, we complement static metrics with streaming criteria that capture drift response and budgeted abstention: Quantile risk control. Let the noncon...

work page 2019
[11]

Accessed: 2025-08-

URL https: //www.st.com/resource/en/datasheet/stm32f767zi.pdf. Accessed: 2025-08-

work page 2025
[15]

Fashion-mnist: A novel image dataset for benchmark- ing machine learning algorithms

12 Preprint Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: A novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv:1708.07747,

arXiv

Show all 28 references

[16]

Yue Zheng, Yuhao Chen, Bin Qian, Xiufang Shi, Yuanchao Shu, and Jiming Chen

arXiv:1711.07128 (2017). Yue Zheng, Yuhao Chen, Bin Qian, Xiufang Shi, Yuanchao Shu, and Jiming Chen. A review on edge large language models: Design, execution, and applications. ACM Computing Surveys, 57(8):1–35,

2017 arXiv
[18]

Early-exit Ensembles (EE-ensemble) (Qendro et al., 2021). We add multiple early-exit heads to the shared backbone (matching our TCUQ exit locations) and train all heads jointly with the sum of per-exit cross-entropy losses on the same labels (equal weights unless stated). At i...

2020
[19]

14 Preprint A.2 D ATASETS We evaluate TCUQ and all baseline methods on four in-distribution datasets spanning both vision and audio, following standard TinyML evaluation practice

to ensure parity. 14 Preprint A.2 D ATASETS We evaluate TCUQ and all baseline methods on four in-distribution datasets spanning both vision and audio, following standard TinyML evaluation practice. These datasets are selected to reflect the scale, modality, and complexity of t...

2018
[20]

For SpeechCmd, we convert raw WA V to Mel spectrograms (49×10×1) using a fixed front end; for vision datasets we apply standard per-dataset normalization

for 15 Preprint CIFAR-10, and a MobileNetV2 for TinyImageNet (Howard et al., 2017). For SpeechCmd, we convert raw WA V to Mel spectrograms (49×10×1) using a fixed front end; for vision datasets we apply standard per-dataset normalization. Optimization and schedules. We train w...

2017
[21]

While TS can improve calibration in static settings, it does not exploit temporal structure, streaming adaptation, or feature-level consistency, which are central to TCUQ

is a standard post-hoc calibration method that learns a single scalar to rescale logits and reduce overconfidence. While TS can improve calibration in static settings, it does not exploit temporal structure, streaming adaptation, or feature-level consistency, which are central...

2021
[22]

Recent approaches such as Meronen et al

remove the OOD data requirement but primarily target OOD detec- tion, sometimes sacrificing in-distribution accuracy. Recent approaches such as Meronen et al. (2024), which address overconfidence in early-exit networks, rely on resource-intensive approximations (e.g., Laplace)...

2024
[23]

PostN employs normalizing flows to model a predictive distribution for each input without increasing runtime memory requirements

is the most relevant comparator for TCUQ in the single-pass deterministic category. PostN employs normalizing flows to model a predictive distribution for each input without increasing runtime memory requirements. We evaluate PostN on MNIST and CIFAR-10 by substituting its enc...

2020
[24]

typically includes the original final output block in that aggregation. To understand its effect under our TinyML setting, we run an ablation on ResNet-8/CIFAR-10 comparing (i) averaging with the final exit and (ii) averaging without it, keeping all training and evaluation pro...
[25]

wait for more context

trained on TinyImageNet (50 epochs, batch size 128; other settings as in Section 3). We then evaluate accuracy-drop detection on TINY IMAGE NET-C by averaging AUPRC over all corruption types at each severity level. As shown in Figure 7, TCUQ consistently outperforms EE- ENS an...
[26]

Let the prediction confidence be c(x) = maxℓpϕ(y =ℓ| x)

for comparability with prior work. Let the prediction confidence be c(x) = maxℓpϕ(y =ℓ| x). Partition [0, 1] intoM bins {Bm}; with acc(Bm) and conf(Bm) denoting empirical accuracy and mean confidence in bin m, ECE is ECE = MX m=1 |Bm| N acc(Bm)− conf(Bm) . (10) While low ECE i...

2019
[1998]

Shiyu Liang, Yixuan Li, and R

doi: 10.1109/5.726791. Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR,

doi
[2016]

Hydra: Preserving ensemble diversity for model distillation

Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans, Sebastian Nowozin, and Rodolphe Jenatton. Hydra: Preserving ensemble diversity for model distillation. arXiv preprint arXiv:2001.04694,

2001 arXiv
[2017]

Learning multiple layers of features from tiny images

Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto,

2009
[2018]

Accessed: 2025-08-

URL https: //www.st.com/resource/en/datasheet/stm32l432kc.pdf. Accessed: 2025-08-

2025
[2019]

Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam

Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861,

arXiv
[2020]

Speech commands: A dataset for limited-vocabulary speech recognition

Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209,

arXiv
[2021]

arXiv:2006.06848

URL https://arxiv.org/abs/2006.06848. arXiv:2006.06848. Colby Banbury, Vijay Janapa Reddi, and et al. Mlperf tiny benchmark. In Proceedings of MLSys,

2006 arXiv
[2023]

Imagenet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee,

2009
[2024]

Mnist-c: A robustness benchmark for computer vision

Norman Mu and Justin Gilmer. Mnist-c: A robustness benchmark for computer vision. arXiv preprint arXiv:1906.02337,

1906 arXiv
[2025]

13 Preprint APPENDIX A T RAINING AND DATASET DETAILS In this section, we detail the models and training configurations used throughout our experiments, along with the specific baselines against which we compare TCUQ. A.1 B ASELINES We compare TCUQ against a set of standard and...

2016

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Angelopoulos and Stephen Bates

Anastasios N. Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv:2107.07511,

arXiv

[2] [3]

Abstention budget adherence

We track the exceedance deviation ∆α = 1 T TX t=1 ⊮ rt≥qα,t − α , (11) measuring how closely the calibrated rejection rate matches the target risk levelα on ID segments (lower is better). Abstention budget adherence. With a desired long-run budgetb and observed abstention rateˆb, we report|ˆb−b| over the full stream and short windows, assessing both avera...

work page 2023

[3] [4]

Nikhil P Ghanathe and Steven J. E. Wilton. Qute: Quantifying uncertainty in tinyml with early-exit- assisted ensembles for model monitoring. arXiv:2404.12599,

arXiv

[4] [5]

org/abs/2106.00170

URL https://arxiv. org/abs/2106.00170. W Brier Glenn et al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3,

arXiv

[5] [8]

Branchynet: Fast inference via early exiting from deep neural networks

Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pp. 2464–2469. IEEE,

2016

[6] [9]

(2019) in our discussion but do not optimize for them

For completeness, we additionally reference adaptive/static variants (ACE/SCE) from Nixon et al. (2019) in our discussion but do not optimize for them. E.3 S TREAMING METRICS FOR TCUQ Because TCUQ operates online, we complement static metrics with streaming criteria that capture drift response and budgeted abstention: Quantile risk control. Let the noncon...

work page 2019

[7] [11]

Accessed: 2025-08-

URL https: //www.st.com/resource/en/datasheet/stm32f767zi.pdf. Accessed: 2025-08-

work page 2025

[8] [15]

Fashion-mnist: A novel image dataset for benchmark- ing machine learning algorithms

12 Preprint Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: A novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv:1708.07747,

arXiv