TS-Insight: Visualizing Thompson Sampling for Verification and XAI

\'Eloi Durant; Jun Pang; Mohammad Ghoniem; Nicolas M\'edoc; Parsa Vares

REVIEW 3 major objections 4 minor 13 references

TS-Insight: Visualizing Thompson Sampling for Verification and XAI

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TS-Insight lets developers verify and explain Thompson Sampling decisions by turning each arm's posterior, evidence, and draw history into synchronized visual panels.

desk verdict Useful TS visual debugging tool, but its 'rare event' signal is statistically broken; worth a revision. read the letter →

arxiv 2507.19898 v2 pith:ABXLL4LD submitted 2025-07-26 cs.HC cs.AIcs.LGstat.ML

Parsa Vares , Éloi Durant , Jun Pang , Nicolas Médoc , Mohammad Ghoniem This is my paper · ORCID

classification cs.HCcs.AIcs.LGstat.ML

keywords ThompsonSamplingmulti-armedbanditsvisualanalyticsexplainableAIalgorithmvisualizationDiscountedHighestDensityRegionactivelearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Thompson Sampling and its variants make decisions by drawing a random sample from each arm's posterior belief and picking the arm with the highest draw, but their behavior is usually inspected only through aggregate metrics such as cumulative regret. TS-Insight is introduced as a visual analytics tool that makes these internal mechanics inspectable: for each arm it plots the evolving Beta posterior as a shaded Highest Density Region band, tracks the raw success and failure evidence counts, and records a barcode of when the arm was chosen and whether the pull succeeded. A separate XAI Snapshot view shows, at any step, every arm's posterior mean alongside its posterior draw, so a developer can see whether a selection was driven by strong belief or by an optimistic rare draw. The authors apply the tool to Discounted Thompson Sampling in an active-learning task on a fully labeled corpus of scientific papers, where the decay of the evidence counts on idle arms makes the discounting mechanism visible. The claim that grounds the paper is that with these views, a developer can verify the intended update and discounting logic, explain why a particular arm was selected, and judge when a choice falls outside the algorithm's own certainty region.

What carries the argument

The load-bearing object is the Highest Density Region (HDR) band drawn around each arm's posterior mean $\mu_k = \alpha_k/(\alpha_k+\beta_k)$. For a target probability $\rho$, the tool approximates a symmetric interval $[a_\rho,b_\rho]$ by bisection on the Beta cumulative distribution function, clipping the endpoints to $[0,1]$ so that the interval is meant to contain probability mass $\rho$, and shades that interval as the arm's certainty region. Each posterior draw $\hat{\theta}_k$ from the current Beta posterior is marked against the band, so a draw outside the band is read as a rare event signaling high epistemic uncertainty. The three subplots are synchronized: the HDR band is regenerated from the evidence counts, the barcode records which arm was selected and whether the reward was a success or failure, and the XAI Snapshot view compares $\mu_k$ and $\hat{\theta}_k$ across all arms at one step to expose whether selection followed strong belief or a high but unlikely draw.

What would settle it

Take the posterior Beta(0.5,10) with target probability 0.5 and compute the symmetric interval that the paper's bisection returns around the mean 0.5/10.5; compute the probability mass actually inside that interval and compare the density at its endpoints with densities just outside it. If the mass is below 0.5, or if values outside the interval have higher density than values inside it, then the symmetric HDR is not a valid highest-density region and the rare-event flag is unreliable for skewed posteriors.

Watch

Extended reading notes

Core claim

The paper's central claim is that the opacity of Thompson Sampling can be removed by decomposing each arm's state into synchronized visual signals rather than by looking only at outcome-level aggregates. In TS-Insight, each arm has a row of three subplots: a Highest Density Region band around the posterior mean with each posterior draw marked against it, the cumulative success and failure counts, and a barcode of selection outcomes. At a selected step, the XAI Snapshot view juxtaposes every arm's posterior mean with its posterior draw, making it directly visible that the chosen arm is the one with the highest draw even when its mean is not the highest; in the running example at step 228, Arm 8 has the highest mean but Arm 7 is selected because its draw is higher, which the tool labels as exploration. The design is demonstrated on Discounted Thompson Sampling, where the decay of the evidence counts on idle arms becomes visible in the evidence subplot, allowing a developer to trace a single decision back to the belief and evidence that produced it. In the authors' reading, this is what makes verification, diagnosis, and explainability of exploration/exploitation dynamics possible.

Load-bearing premise

The tool's ability to flag a posterior draw as a rare event assumes that the symmetric interval it draws around each arm's average belief is the region where the belief's density is actually highest; for strongly skewed Beta posteriors that interval is not the highest-density region, so an ordinary draw can be mislabeled as rare, and when the average sits at 0 or 1 the code erases the band entirely.

Editorial extensions

If this is right

A developer can verify the update logic by locating a pull in the barcode and seeing the corresponding change in the success or failure count and the resulting narrowing or widening of the HDR band on that arm.
In Discounted Thompson Sampling, the forgetting mechanism becomes visible as idle arms' evidence counts decay toward zero, letting users confirm that recent evidence is being prioritized as intended.
At every step, the XAI Snapshot view distinguishes exploration from exploitation: if the chosen arm is not the one with the highest posterior mean, the tool attributes the choice to a higher posterior draw.
A draw falling outside the HDR band gives a per-arm reliability signal, warning that the algorithm acted on a low-probability outcome while the posterior was uncertain or unstable.
Hiding or showing arms and step ranges lets a user compare arm trajectories across phases of the run, which can reveal non-stationarity in the environment as shifts in which arms are selected.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the tool's value would be a controlled debugging study with injected faults, such as a wrong discount factor, a reversed reward mapping, or a skipped posterior update; the paper lists user studies as future work but does not propose this specific fault-injection protocol.
The symmetric HDR approximation should be stress-tested on strongly skewed Beta posteriors, for example $\alpha=0.5, \beta=10$ with $\rho=0.5$; because the true highest-density region is one-sided there, the symmetric band can misclassify ordinary draws as rare, which is an implication of the design rather than a claim the paper makes.
Since the visual machinery only needs a posterior, an evidence count, and a sampled outcome per arm, the same decomposition could be applied to other Thompson Sampling variants such as Sliding-Window or Batched TS, which the paper names but does not visualize.
The snapshot view suggests a cheap diagnostic not stated in the paper: if many selections are attributed to draws far outside the HDR band, the algorithm may be exploring more aggressively than its posterior uncertainty justifies, and TS-Insight could aggregate this into a summary statistic.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

TS-Insight is a genuinely useful idea: the first tool I know of that lets a developer watch Thompson Sampling step by step, with per-arm posterior draws, evidence counts, and selection history. The three-subplot decomposition and the XAI snapshot are well chosen, and the authors correctly summarize the TS/DTS mechanics. The code is on GitHub, which helps reproducibility. For a poster contribution, the paper is honest: it explicitly says no user study has been done and calls the SYNERGY usage a preliminary validation without showing results. I'd happily cite it as a starting point for TS visualization work.

The soft spot is not the lack of a study; it's the HDR Evolution Plot's statistical logic. The stress-test note is right. The paper computes a symmetric interval around the posterior mean, not a true Highest Density Region. For a Beta(0.5, 100), the density is monotone decreasing, so the real HDR is one-sided; a symmetric interval mislabels the most probable draws as rare. And because the target is 50%, exactly half the draws lie outside the band by construction. The text's phrase 'when a draw falls outside this region, it signals a rare event' is misleading — a 50% interval will be outside half the time. The bisection can also silently fail when the symmetric interval's max mass is below ρ. These flaws directly undercut the 'reliability' claim in Section 4. They don't break the barcode or alpha/beta tracing, which remain useful for verification and diagnosis.

Also minor: the exploration/exploitation label is a bit too simple — a draw above the mean isn't necessarily 'exploration' in a meaningful sense; it's just the sampling mechanism. That's a nitpick, though.

If this came to me for peer review, I would send it out, because the tool fills a real gap and the central design is sound. But I'd ask for a major revision before acceptance: either implement a true HDR (Hyndman's algorithm is a few lines) or stop calling it an HDR and frame the band explicitly as a symmetric credible interval, and soften or remove the 'rare event' language. With that, this is a solid tool note.

Referee Report

3 major / 4 minor

Summary. The paper introduces TS-Insight, an open-source visual analytics tool for inspecting Thompson Sampling (TS) and Discounted Thompson Sampling (DTS) at the level of individual arms and steps. For each arm, the tool provides three synchronized subplots: an HDR Evolution Plot showing the posterior distribution and the location of posterior draws, an Alpha/Beta plot tracing the pseudo-counts of successes and failures, and a Barcode view summarizing the arm's selection history and outcomes. A separate XAI Snapshot view decomposes each selection into the posterior mean versus the posterior draw to explain why a particular arm was chosen. The paper claims these views support verification of the algorithm's mechanics, diagnosis of its behavior, and explanation of individual decisions. The contribution is illustrated with a walkthrough of a DTS-based active learning run on the SYNERGY systematic review dataset, and a preliminary validation on that dataset is mentioned but not reported. The paper is an accepted IEEE VIS 2025 poster, and its main text is accompanied by appendices containing additional references, a scope statement, key concept explanations, and a description of the HDR computation.

Significance. If the claims hold, TS-Insight addresses a real gap: existing visualizations of bandit algorithms typically show aggregate metrics such as cumulative regret, whereas developers debugging TS-based systems need step-level, per-arm traces of posterior updates and decisions. The paper explicitly provides machine-checkable code (the tool is open source), a concrete walkthrough, and a clear articulation of the tool's intended questions. The three-part decomposition of posterior belief, evidence, and outcome history is a sensible design, and the XAI Snapshot view's contrast between the posterior mean and the realized draw gives a concise, faithful explanation of why Thompson Sampling selected a particular arm. The major caveat is that the HDR Evolution Plot, which carries the paper's reliability/uncertainty analysis, does not in fact compute a Highest Density Region as claimed, and its 'rare event' signal is statistically unreliable as implemented. The paper also provides no empirical validation of the claimed verification, diagnosis, and explanatory benefits, beyond a single illustrative example.

major comments (3)

[Appendix D, Eq. (1); Section 3, HDR Evolution Plot] The interval computed by the bisection in Appendix D is a symmetric interval around the posterior mean, not a Highest Density Region. For skewed Beta posteriors (e.g., α=0.5, β=100) the density is monotone decreasing, so the true HDR is one-sided, while the symmetric interval excludes high-density values near 0 and includes low-density values near 1. The density-ordering condition stated in Appendix D (f(x) ≥ f(y) for all x in the interval and y outside) is therefore violated for such posteriors. Consequently, the plot can label the most probable draws as lying outside the 'HDR' and low-probability draws as inside, inverting the intended 'rare event' signal that underlies the third key question in Section 4 ('When was the algorithm's choice outside the certainty region?'). The authors should either implement a true HDR computation (e.g., via the algorithm in Hyndman 1996) or rename the band as a 'central interval' and revise the interpretation accordingly.
[Appendix D, bisection loop; Section 3, 'rare event' wording] The implementation targets a 50% coverage (Appendix D: 'The resulting 50% HDR...'). By definition, 50% of posterior draws fall outside any 50% central interval, so the HDR Evolution Plot will mark roughly half of all draws as 'rare events' on every step. This makes the reliability signal uninformative at best and actively misleading at worst: a developer will see a 'rare event' marker at half the steps, not only for genuinely low-probability draws. In addition, the bisection loop in Appendix D can silently fail to reach the target probability ρ when the maximum enclosed mass, attained at δmax = min(μ, 1−μ), is less than ρ; the algorithm returns a narrower band than requested without warning. The authors should document the target coverage, justify the choice of 50%, and add a check for the case where the requested ρ is unattainable within the symmetric interval.
[Abstract and Section 1] The abstract and introduction claim that TS-Insight 'enabl[es] verification, diagnosis, and explainability' of TS/DTS, but the paper provides no empirical evaluation: no user study, no quantitative task-based evaluation, and no results from the SYNERGY validation that is mentioned in Section 1. The only evidence is a single illustrative walkthrough (Fig. 1 and the surrounding text in Section 4). Given that the paper proposes a visual analytics tool with specific cognitive claims (verification, diagnosis, reliability assessment), the authors should either report the results of the mentioned SYNERGY validation or explicitly scope the paper as a design proposal and temper the abstract's language. At minimum, a task-oriented evaluation with a small number of developers would substantiate the central claim.

minor comments (4)

[Title page] The header contains a typo: 'T echnology' appears in the author affiliation lines; please correct this to 'Technology'.
[Figure 2] The caption of Fig. 2 states the example is at 'Step t = 228' and the text describes Arm 7 being chosen with a posterior draw near 1.0, but the figure itself is not referenced in the body text with a specific analysis of all arms; consider adding a more detailed explanation of the log-scale y-axis and the relationship between the bars for μ and θ to help readers interpret the snapshot view.
[Section 2] The definitions of 'exploitation' and 'exploration' in Section 2 are outcome-based (whether the selected arm had the highest posterior mean), which conflates the realized event with the algorithm's latent random strategy. This is acceptable as a pedagogical simplification, but the text should note that the classification is an ex-post interpretation, not a property of the decision rule itself.
[Appendix C] The definition of Highest Density Region ('the range of values where we are 50% certain the arm's true success rate lies') is correct in spirit but would benefit from a mention that for skewed distributions the HDR need not be symmetric, anticipating the issue raised in Appendix D.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TS-Insight visualizes the algorithm's own state, and the apparent restatements are the intended explanatory function rather than derived predictions.

full rationale

TS-Insight does not derive predictions from fitted parameters, nor does it rely on load-bearing self-citations. Its central function is to render the internal state of Thompson Sampling (posterior parameters alpha/beta, posterior draws, selections, and updates) in synchronized views. The XAI Snapshot showing that the selected arm has the highest posterior draw is a direct display of the TS selection rule, not a prediction derived from the visualization; the tool's explanatory value is precisely that it restates the algorithm's definition in an inspectable form. The HDR band in Appendix D is computed from the Beta posterior using a standard symmetric-interval approximation; the paper's phrasing that a draw outside a 50% HDR 'signals a rare event' is a statistical-validity concern (a 50% HDR leaves 50% of probability mass outside), but it is not a circular derivation: no equation is fitted to the output it is used to explain, and no cited result is supplied by the authors' own prior work. Appendix B appropriately limits scope to TS variants, which is a scope statement rather than circular reasoning. The paper is self-contained as a visualization design; the verification and explanation claims are grounded in the algorithm's own update and selection mechanics, not in the visualization's outputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no physical or mathematical entities. Its assumptions are about the correctness of the TS update model, the validity of the symmetric HDR approximation, and the meaningfulness of its strategy labels. The HDR-level parameter rho is a free design choice that affects the tool's certainty-region interpretation.

free parameters (1)

HDR probability level rho = 0.50 (50%)
The dashboard shades a 50% central band for each posterior. This author-chosen threshold determines which draws count as rare or outside the certainty region; it is not fitted to data.

assumptions (4)

domain assumption The Beta posterior update and Thompson Sampling selection rule, draw theta from each arm, choose argmax, update alpha or beta, are the correct model of the executed algorithm.
The whole visualization is generated from these definitions; if the implementation deviates, all plots misrepresent the algorithm.
ad hoc to paper A symmetric interval around the posterior mean is a valid Highest Density Region for the Beta posteriors displayed.
Appendix D uses [mu-delta, mu+delta]; for skewed Beta posteriors the true HDR is one-sided, so the outside HDR equals rare event reading can be wrong.
ad hoc to paper The bisection over delta, with delta_max = min(mu, 1-mu), always finds an interval with enclosed probability rho.
For posteriors concentrated near the boundary the central interval may not contain probability rho before clamping; the fallback a_rho = b_rho = mu collapses the band and removes the rare-event signal.
domain assumption Exploitation versus Exploration can be read from whether the selected arm has the highest posterior mean.
Section 2 defines exploration as selecting a lower-mean arm due to a higher draw, but a high draw on the highest-mean arm is still sampling-driven; the label oversimplifies the mechanism.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TS-Insight: Visualizing Thompson Sampling for Verification and XAI." pith.science (2026). https://pith.science/paper/ABXLL4LD

@misc{pith2026250719898,
  author       = {Pith},
  title        = {Pith review of: TS-Insight: Visualizing Thompson Sampling for Verification and XAI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ABXLL4LD}},
  note         = {Machine review of arXiv:2507.19898}
}

read the original abstract

Thompson Sampling (TS) and its variants are powerful Multi-Armed Bandit algorithms used to balance exploration and exploitation strategies in active learning. Yet, their probabilistic nature often turns them into a "black box", hindering debugging and trust. We introduce TS-Insight, a visual analytics tool explicitly designed to shed light on the internal decision mechanisms of Thompson Sampling-based algorithms, for model developers. It comprises multiple plots, tracing for each arm the evolving posteriors, evidence counts, and sampling outcomes, enabling the verification, diagnosis, and explainability of exploration/exploitation dynamics. This tool aims at fostering trust and facilitating effective debugging and deployment in complex binary decision-making scenarios especially in sensitive domains requiring interpretable decision-making.

Figures

Figures reproduced from arXiv: 2507.19898 by the authors.

**Figure 1.** Excerpt from the TS-Insight dashboard, illustrating the behavior of multiple arms over time, each decomposed in three [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** The XAI Snapshot view explains the algorithm’s choice at [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 2 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....
[2]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....
[3]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....
[4]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....
[5]

De Bruin , Y

J. De Bruin , Y. Ma, G. Ferdinands, J. Teijema, and R. Van de Schoot . SYNERGY - O pen machine learning dataset on study selection in systematic reviews. DataverseNL, 2023. https://doi.org/10.34894/HE6NAQ doi:10.34894/HE6NAQ

doi:10.34894/he6naq 2023
[6]

Golovin, B

D. Golovin, B. Solnik, S. Moitra, G. Kochanski, J. Karro, and D. Sculley. G oogle V izier: A service for black-box optimization. In Proc. the 23rd SIGKDD Conference on Knowledge Discovery and Data Mining , pp.\ 1487--1495. ACM, 2017. https://doi.org/10.1145/3097983.3098043 doi:10.1145/3097983.3098043

arXiv 2017
[7]

R. J. Hyndman. Computing and graphing highest density regions. The American Statistician , 50(2):120--126, 1996. https://doi.org/10.2307/2684423 doi:10.2307/2684423

doi:10.2307/2684423 1996
[8]

Raj and S

V. Raj and S. Kalyani. Taming non-stationary bandits: A B ayesian A pproach, 2017. http://arxiv.org/abs/1707.09727 arXiv:1707.09727

arXiv 2017

Show all 13 references

[9]

D. J. Russo, B. V. Roy, A. Kazerouni, I. Osband, and Z. Wen. A tutorial on T hompson S ampling. Foundations and Trends in Machine Learning , 11(1):1--96, 2018. https://doi.org/10.1561/2200000070 doi:10.1561/2200000070

2018 doi
[10]

W. R. Thompson. O n the L ikelihood that O ne U nknown P robability E xceeds A nother in V iew of the E vidence of T wo S amples. Biometrika , 25(3/4):285--294, 1933. https://doi.org/10.2307/2332286 doi:10.2307/2332286

1933 doi
[11]

Zhang, S

J. Zhang, S. Shao, S. Verma, and R. Nowak. Algorithm selection for deep active learning with imbalanced datasets. In Proc. the 37th International Conference on Neural Information Processing Systems , article 422, 2023

2023
[12]

S. S. Villar, J. Bowden, and J. Wason. Multi-armed bandit models for the optimal design of clinical trials: Benefits and challenges. Statistical Science , 30(2):199--215, 2015. https://doi.org/10.1214/14-STS504 doi:10.1214/14-STS504

2015 doi
[13]

S. L. Scott. A modern B ayesian look at the multi-armed bandit. Applied Stochastic Models in Business and Industry , 26(6):639--658, 2010. https://doi.org/10.1002/asmb.874 doi:10.1002/asmb.874

2010 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....

[2] [2]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....

[3] [3]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....

[4] [4]

write newline

" write newline "" before.all 'output.state := STRINGS d dc pc.str doi.urlstr INTEGERS nth.dash doi.string.pos doilen FUNCTION add.doi doi empty "\ doi 'd := "" 'doi.urlstr := #0 'nth.dash := #1 'doi.string.pos := d text.length 'doilen := nth.dash #3 < d doi.string.pos #1 substring 'dc := #1 doi.string.pos + 'doi.string.pos := dc "/" = #1 nth.dash + 'nth....

[5] [5]

De Bruin , Y

J. De Bruin , Y. Ma, G. Ferdinands, J. Teijema, and R. Van de Schoot . SYNERGY - O pen machine learning dataset on study selection in systematic reviews. DataverseNL, 2023. https://doi.org/10.34894/HE6NAQ doi:10.34894/HE6NAQ

doi:10.34894/he6naq 2023

[6] [6]

Golovin, B

D. Golovin, B. Solnik, S. Moitra, G. Kochanski, J. Karro, and D. Sculley. G oogle V izier: A service for black-box optimization. In Proc. the 23rd SIGKDD Conference on Knowledge Discovery and Data Mining , pp.\ 1487--1495. ACM, 2017. https://doi.org/10.1145/3097983.3098043 doi:10.1145/3097983.3098043

arXiv 2017

[7] [7]

R. J. Hyndman. Computing and graphing highest density regions. The American Statistician , 50(2):120--126, 1996. https://doi.org/10.2307/2684423 doi:10.2307/2684423

doi:10.2307/2684423 1996

[8] [8]

Raj and S

V. Raj and S. Kalyani. Taming non-stationary bandits: A B ayesian A pproach, 2017. http://arxiv.org/abs/1707.09727 arXiv:1707.09727

arXiv 2017