Pith. sign in

REVIEW 4 major objections 5 minor 24 references

Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adversarial image tweaks can make AI assistants leave a statistical fingerprint when students copy them.

desk verdict The attack-transfer side is solid and the statistics are honestly conditioned; the false-flag bound is a computed bound under an unvalidated student model, not a measured rate. read the letter →

arxiv 2608.01112 v1 pith:ODPRNEFE submitted 2026-08-02 cs.AI

Tobias Braun , Jonas Grebe , Louis Rethfeld , Marcus Rohrbach This is my paper · ORCID
classification cs.AI
keywords AIcheatingdetectionadversarialperturbationsmultimodalmultiple-choicequestionsstatisticalfingerprintinglikelihood-ratioblack-boxtransfereducationalintegritytargetedattacks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that homework can be engineered so that sustained blind copying from an AI assistant leaves a detectable statistical trace, without relying on after-the-fact detection of AI-generated text. It protects multimodal multiple-choice questions with subtle visual perturbations that steer AI solvers toward designated wrong answers; students who copy those answers reproduce a pattern genuine students almost never produce. On a shared 20-question assignment, the paper reports that the detector catches at least 95% of modeled blind-copying cases while falsely flagging fewer than 8 in 10,000 genuine students under an educator-provided student model. If this holds, educators gain a preventive instrument: assessment design itself, not post-hoc classification, creates evidence of outsourcing.

What carries the argument

The load-bearing object is the target fingerprint: each question's randomly chosen incorrect answer $S_i$ is kept secret, and the visual perturbation is optimized so that the assistant returns $S_i$ with high probability. The detector uses the indicator $Y_i$ that a submitted answer equals $S_i$, and the item-weighted log-likelihood ratio $L=\sum_i[Y_i\log(p_i/q_i)+(1-Y_i)\log((1-p_i)/(1-q_i))]$, where $p_i$ is a conservative lower bound on the assistant's target rate from 30 repeated queries and $q_i$ is an educator-provided conservative upper bound on a genuine student's chance of choosing that distractor. Questions are retained only when $p_i>q_i$; the threshold is set to the most stringent value that keeps 95% power, and the exact weighted-sum distribution under conditional independence supplies the Type-I error.

What would settle it

Give a proctored cohort of genuine students the protected 20-question assignment with no AI access, run the paper's detector with its reported $p_i$ and $q_i$ bounds, and count flags. If the empirical false-flag rate materially exceeds 8 in 10,000, or if students' response vectors show correlated target matches that lift the likelihood ratio over threshold, the conditional-independence and educator-bound assumptions are refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that adversarial machine learning can be turned into a preventive guard for educational assessments. For each protected question, the educator secretly chooses an incorrect answer, optimizes a small visual perturbation against an ensemble of open-weight surrogate models, and confirms by repeated black-box queries that the deployed assistant reliably returns that wrong answer. Blind-copying students thereby inherit an assignment-specific error pattern, while genuine students pick wrong answers independently. Using an item-weighted log-likelihood ratio over these target matches, the paper reports at least 95% detection power with a familywise false-flag bound below 8 in 10,000 under its educator-guided student model, and shows the shared 20-question assignment covers three major assistant families simultaneously and transfers to one assistant outside the construction set. It also reports that the guarantees degrade sharply under a data-free fallback student model and that some assistants outside the set are poorly separated.

Load-bearing premise

The reported false-flag rate rests on the assumptions that a genuine student's chance of picking each secret wrong answer is independent across questions and no higher than the educator's estimates; if students' errors are correlated through shared misconceptions or a single attractive distractor, the 8-in-10,000 figure could be far too low.

Editorial extensions

If this is right

  • A 20-question protected assignment can simultaneously cover assistants from three major families, so an educator does not need to know which assistant a student would use.
  • Sustained blind copying across most of the assignment is the behavior that gets flagged; isolated or critically checked use falls outside the threat model.
  • Because the secret targets are assigned per question, the fingerprint survives the student's answer being a single letter and does not require analyzing free-form text.
  • If educators lack reliable estimates of how often genuine students pick each distractor, the fallback $q_i=0.5$ raises the false-flag bound to roughly one in fifty, so the reported guarantees depend on educator judgment.
  • The candidate pool admits at most three disjoint protected assignments under the analytical bound and two verified disjoint ones, so the approach is reusable but limited by the supply of steerable questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The visible deterrent may be as valuable as the detector itself: once students know that blind copying leaves a reviewable statistical trace, the expected cost of outsourcing rises even when no flag is issued.
  • The same fingerprint mechanism could be carried by numerical canaries, questions whose answer is an unlikely value, which the paper names as future work and which may separate genuine students from assistants even more sharply than multiple-choice distractors.
  • Deployment would need per-cohort validation of the educator-provided bounds before quoting the 8-in-10,000 figure, because real student cohorts may share misconceptions that induce correlated target matches.
  • The poor separation for some assistants outside the construction set suggests that coverage is model-dependent; a practical system should re-run calibration whenever an assistant provider updates a model.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a preventive approach to detecting sustained blind copying from AI assistants in multimodal multiple-choice assessments. Each protected question carries a subtle visual perturbation, optimized on an open-weight surrogate ensemble, that steers black-box assistants toward a secret designated incorrect answer. Across an assignment, the induced answer pattern acts as a statistical fingerprint: students who copy assistant outputs reproduce the target pattern more often than genuine students. The authors construct a shared 20-question assignment covering three a priori assistant endpoints (Claude Opus 4.8, Gemini 3.5 Flash, GPT-5.6 Sol), calibrate target-response probabilities with repeated API queries and Clopper-Pearson lower bounds, model genuine-student target-selection probabilities either from five PhD researchers' elicited upper bounds or from a data-free q_i=0.5 fallback, and report >95% modeled power with a modeled familywise false-flag rate below 8 in 10,000 under the educator model. They also report transfer to one out-of-set assistant and weaker separation for two others, and include a detailed supplement documenting assumptions, calibration protocols, and limitations.

Significance. If the quantitative claims hold, the paper offers a genuinely new direction: instead of post-hoc detection of AI-generated text, assessments are designed so that blind copying leaves an auditable statistical trace. The paper is internally careful in several ways that deserve emphasis: development and holdout calibration calls are separated (Supp. C.4), the reported bounds use exact Clopper-Pearson intervals and a union bound (Supp. C.2, Eq. 10), and the supplement explicitly lists the modeling assumptions behind the headline numbers. The central risk is not the adversarial steering but the genuine-student null model: the Type-I and posterior numbers are model computations, not measurements on real student response vectors. Because the paper frames its contribution as feasibility, this is addressable, but it is load-bearing for the main quantitative claims.

major comments (4)
  1. [Table 3 / Supp. A.2, Eq. (7)] The familywise Type-I bound alpha <= 7.21e-4 is a model computation, not an empirical false-flag rate: it is the upper-tail probability of the product-Bernoulli statistic in Supp. Eq. (7) evaluated at q_i bounds elicited from five PhD researchers, and no complete genuine-student response vectors are reported. A violation of q_i on a few high-weight items, or any positive cross-question dependence in target-match indicators, can move that upper tail by orders of magnitude. The headline 'fewer than 8 in 10,000 genuine students' should therefore be stated as conditional on the unvalidated null model, and the paper should either supply a proctored pilot or a sensitivity analysis under correlated response models.
  2. [Supp. D.2] The educator-provided q_i are explicitly described as 'modeling assumptions rather than empirical response frequencies or frequentist confidence bounds.' Taking the maximum of five researcher estimates is not a confidence bound for the intended cohort; if the target distractor is more attractive to actual students than to the assessors, the retained-pair condition p_i > q_i and the threshold tau* in Table 3 are no longer conservative. Because this is load-bearing for the central Type-I claim, the paper should either provide controlled cohort data on the protected assignment or restrict the quantitative claim to a clearly labeled conditional feasibility analysis.
  3. [Supp. B.3] Conditional independence is acknowledged as 'a substantive modeling assumption' but is not tested on real response patterns. Correlated guessing, fatigue, shared misconceptions, or a single attractive distractor on a high-weight question can produce tail probabilities far above the product-Bernoulli values used in Eq. (7). The authors' justification is plausible for match indicators, but the central guarantee depends on this assumption, so a stronger deployment should calibrate the null from full-assignment student responses or use block/bootstrap procedures that preserve observed dependence.
  4. [Supp. B.4] Because the design-based randomization detector is abandoned (retention conditioning invalidates the design-based null), the method has no model-free component for the genuine-student distribution. This makes the q_i and independence assumptions in the previous comments load-bearing for every reported Type-I number. I recommend that the main text state this limitation more prominently and provide a concrete validation protocol, rather than leaving it only in the supplement.
minor comments (5)
  1. [Abstract / Introduction] The sentence 'falsely flagging fewer than 8 in 10,000 genuine students' should consistently include the qualifier 'modeled' or 'under the educator-provided student model,' because the number is computed, not measured on genuine-student submissions.
  2. [Table 3, out-of-set rows] The out-of-set transfer claim for Gemini 3 Flash should be accompanied by its own Type-I value of 1.24e-3, which exceeds the 8-in-10,000 headline bound for the covered set; the text should clarify that transfer does not inherit the same false-flag guarantee.
  3. [Supp. D.3] The q_i=0.5 fallback relies on the structural assumption that the correct answer is at least as likely as any individual distractor for genuine students; this assumption should be stated in the main text wherever the fallback operating point is cited.
  4. [Figure 11] The label 'crysalis' appears to be a typo; the intended word is likely 'chrysalis.'
  5. [References] The author name 'Trie.u' in the Bayer, Trieu, and Ellison reference appears to be a formatting artifact and should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the detector's operating characteristics are explicitly conditional on measured assistant probabilities and educator-provided student-model bounds, with independent holdout calls and out-of-set evaluation.

full rationale

The paper's central derivation chain is self-contained rather than circular. Assistant target probabilities p_i are measured empirically via repeated black-box queries and bounded below by Clopper–Pearson confidence bounds (Supp. C.2, Eq. 16); genuine-student target-selection probabilities q_i are elicited from blinded PhD researchers and taken as conservative upper bounds (Supp. D.2). The likelihood-ratio statistic (Eq. 1) and threshold selection (Eq. 9) are then exact calculations under the stated product-Bernoulli model. The paper repeatedly and explicitly qualifies the headline numbers as conditional: 'Under the educator-provided student model and the assumed response structure, the detector identifies at least 95% of blind-copying cases while falsely flagging fewer than 8 in 10,000 genuine students.' This is a modeled operating point, not an empirical claim about unmeasured students, and the supplemeent is transparent about the dependence: Supp. D.2 states 'Educator-provided values are modeling assumptions rather than empirical response frequencies or frequentist confidence bounds. The resulting Type-I bounds are therefore conditional on these assessments being valid upper bounds for the intended student population.' Supp. B.3 likewise flags conditional independence as 'a substantive modeling assumption.' These are explicitly acknowledged limitations on assumption validity, not circular reductions. The power guarantee is by construction in the sense that the threshold is chosen to achieve at least 95% modeled power, but that is standard statistical design and is not presented as an external prediction. Independent holdout calibration calls (Supp. C.4) avoid in-sample reuse, and evaluation on assistants outside the construction set (Gemini 3 Flash, GPT-5.4, GPT-5.4 Nano) provides an external transfer check. No load-bearing self-citation or imported uniqueness theorem appears. The only substantive weakness is that the false-flag bound rests on unvalidated q_i bounds and independence, which the authors themselves flag; this is an assumption-validity concern, not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central quantitative claims rest on four hand-chosen parameters (most importantly epsilon and the per-question q_i bounds) and five domain assumptions. The most fragile are the conditional-independence assumption and the validity of educator-provided q_i as true upper bounds; both are acknowledged in the supplement but are not empirically established with real student response vectors.

free parameters (4)
  • epsilon (adversarial perturbation budget) = 16 (8-bit pixel scale)
    Chosen through a development sweep over epsilon, T, and alpha in Supp E.6; the main transfer results depend on this choice and no first-principles value is given.
  • calibration query count K = 30
    Chosen so that 30/30 target hits yields a 95% Clopper-Pearson lower bound near 0.905 (main text, Supp C.2); this affects the tightness of every assistant target-probability bound.
  • per-question genuine-student target probabilities q_i = educator max estimates (values per question, not tabulated)
    Hand-set upper bounds from five assessors (Supp D.2); the reported Type-I error and posterior probabilities are conditional on these being valid upper bounds.
  • detector operating parameters = beta_max=0.05, eta_p=eta_q=0.05, Delta=0.05, priors pi=0.01 and 0.05
    Required power, confidence levels, equivalence margin, and prior sensitivity values are chosen by the authors; results change substantially under the target-fallback model.
assumptions (5)
  • domain assumption Conditional independence of fingerprint-match indicators across questions under both H0 and H1 (product-Bernoulli response model).
    Required for exact weighted-sum Type-I and power calculations in Eqs. 7-8; the authors state in Supp B.3 that student ability, fatigue, and shared misconceptions may correlate responses, and assistant responses may correlate through shared preprocessing.
  • domain assumption Stationary assistant endpoints and independent repeated calls.
    Clopper-Pearson binomial bounds in Supp C.2 require i.i.d. calls from a fixed model, prompt, and decoding configuration; model drift or correlated API sampling would invalidate the bounds.
  • domain assumption Educator-provided q_i are valid conservative upper bounds for the intended student cohort.
    The false-positive guarantee is conditional on human estimates (Supp D.2); if a distractor is more attractive than estimated, the Type-I error exceeds the reported value.
  • domain assumption For the target fallback, the correct answer is at least as likely as any individual distractor, so q_i <= 0.5.
    Justifies the data-free q_i=0.5 bound in Supp D.3; this fails for ambiguous or miskeyed items or systematic misconceptions.
  • domain assumption Protection does not materially change genuine-student behavior.
    Supported by the blinded educator equivalence analysis (mean difference 0.0275, 90% CI within +/-0.05) and annotator judgments, but not by real student data; semantic preservation is assessed indirectly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating." pith.science (2026). https://pith.science/paper/ODPRNEFE

@misc{pith2026260801112,
  author       = {Pith},
  title        = {Pith review of: Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODPRNEFE}},
  note         = {Machine review of arXiv:2608.01112}
}
read the original abstract

The widespread adoption of generative AI enables students to outsource cognitive effort to increasingly capable assistants, creating an illusion of competence while undermining the independent reasoning that education aims to cultivate. We investigate whether adversarial machine learning can be repurposed to protect educational exercises against such corrosive reliance. Our approach uses multimodal multiple-choice questions whose visual components can be protected with subtle visual perturbations that steer AI solvers toward designated incorrect answers. These responses form a statistical fingerprint: students who blindly copy a solver reproduce the induced answer pattern more frequently than genuine students. We study the feasibility of this paradigm under realistic black-box assistant assumptions using three of the most common state-of-the-art multimodal language models: Anthropic's Claude, Google's Gemini, and OpenAI's ChatGPT. By using accessible surrogate models, we optimize adversarial perturbations that induce consistent response patterns. Those patterns enable principled detection through statistical hypothesis testing. These findings establish both the promise and the limitations of fighting machine-assisted reasoning with the vulnerabilities of the machines themselves.

Figures

Figures reproduced from arXiv: 2608.01112 by the authors.

Figure 1
Figure 1. We use an ensemble of surrogate models to opti [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Before intervention, all student-solvable ques [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the proposed framework. First, an incorrect target answer [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Overview of the perturbation-generation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Calibration of assistant-specific target probabilities across datasets. Each triplet represents one assistant-question pair: [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Example protected question from MMMU (Yue [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Empirical stability of probability-valued estimates across different numbers of repeated API calls. Mean and [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Representative interface used to elicit genuine-student response estimates. Five PhD-level researchers independently [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: White-box convergence of targeted steering across the six surrogate models in Table 6. At each attack step, the figure [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Answer-state transition matrix for GPT-5.4 Nano on 150 questions, comprising 50 questions each from MMMU, [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Successful examples that assistants initially answer correctly but are consistently redirected toward the designated [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Representative failure cases in which adversarial steering does not induce the designated target. Because these [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 19 canonical work pages

  1. [2]

    Chen, L.; Bian, Y.; Deng, Y.; Cai, D.; Li, S.; Zhao, P.; and Wong, K.-F

    Linking Artificial Intelli- gence Facilitated Academic Misconduct to Existing Preven- tion Frameworks.CrimRxiv. Chen, L.; Bian, Y.; Deng, Y.; Cai, D.; Li, S.; Zhao, P.; and Wong, K.-F. 2024a. WatME: Towards lossless watermark- ing through lexical redundancy. InProceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volum...

    work page 2024

  2. [3]

    H.; Postma, D.; Hickerson, D.; McGaughran,J.;andKhuat,H.Q.2024

    Perkins, M.; Roe, J.; Vu, B. H.; Postma, D.; Hickerson, D.; McGaughran,J.;andKhuat,H.Q.2024. SimpleTechniques to Bypass GenAI Text Detectors: Implications for Inclusive Education. 21(1):

    work page 2024

  3. [6]

    Under the conditionally independent Bernoulli model, if the true null probabilities satisfy qfp i,j≤qfp i,j, calibrating the upper-tail threshold underqfp j is conservative

    Retaining only pairs satisfying pfp i,j >qfp i,j ensures that a fingerprint match has positive relative weight and that the statistic is increasing inYi,j. Under the conditionally independent Bernoulli model, if the true null probabilities satisfy qfp i,j≤qfp i,j, calibrating the upper-tail threshold underqfp j is conservative. Likewise, if pfp i,j≥p fp i...

    work page 2026

  4. [8]

    Kapur,M.2014

    Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Align- ment.CoRR, abs/2505.21494. Kapur,M.2014. ProductiveFailureinLearningMath.Cog- nitive Science, 38(5): 1008–1022. Kirchenbauer, J.; Geiping, J.; Wen, Y.; Shu, M.; Saifullah, K.; Kong, K.; Fernando, K.; Saha, A.; Goldblum, M.; and Goldstein,T.2024. Onthereliabilityofwatermarksforlarge...

    arXiv 2014

  5. [9]

    Surrogate parameters remain frozen, and gradients update onlyδi

    DatasetTgt prot (%) MMBench 5.5 MMMU 4.3 ScienceQA 3.1 The loss targets the answer letter at the first generation step. Surrogate parameters remain frozen, and gradients update onlyδi. We optimize Equation 19 using MI-FGSM (Dong et al. 2018). After each step, the perturbation is projected onto the permitted L∞ region and the image is clipped to the valid ...

    work page 2018

  6. [10]

    Learn to Ex- plain:MultimodalReasoningviaThoughtChainsforScience QuestionAnswering. InKoyejo,S.;Mohamed,S.;Agarwal, A.;Belgrave,D.;Cho,K.;andOh,A.,eds.,AdvancesinNeu- ral Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9,

    work page 2022

  7. [11]

    McGill,S.2012.PlagiarisminLatinLiterature

    Smolvlm: Redefining small and efficient multimodal models.arXiv preprint arXiv:2504.05299. McGill,S.2012.PlagiarisminLatinLiterature. Cambridge University Press. ISBN 978-1-107-01937-9. Nicks, C.; Mitchell, E.; Rafailov, R.; Sharma, A.; Manning, C. D.; Finn, C.; and Ermon, S

    arXiv 2012

  8. [12]

    InThe Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    Language Model Detectors Are Easily Optimized Against. InThe Twelfth In- ternational Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    work page 2024

Show all 24 references
  1. [15]

    In Calandrino, J

    Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models. In Calandrino, J. A.; and Troncoso, C., eds.,32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023, 2187–2204. USENIX Association. Shan, S.; Ding, W.; Passananti, J.; Wu, ...

    2023

  2. [16]

    InIEEESym- posium on Security and Privacy, SP 2024, San Francisco, CA, USA, May 19-23, 2024, 807–825

    Nightshade: Prompt-Specific Poisoning AttacksonText-to-ImageGenerativeModels. InIEEESym- posium on Security and Privacy, SP 2024, San Francisco, CA, USA, May 19-23, 2024, 807–825. IEEE. Soderstrom, N. C.; and Bjork, R. A

    2024

  3. [18]

    Team,G.;Mesnard,T.;Hardin,C.;Dadashi,R.;Bhupatiraju, S.; Pathak, S.; Sifre, L.; Rivière, M.; Kale, M

    Gemma 4 technical report.arXiv preprint arXiv:2607.02770. Team,G.;Mesnard,T.;Hardin,C.;Dadashi,R.;Bhupatiraju, S.; Pathak, S.; Sifre, L.; Rivière, M.; Kale, M. S.; Love, J.; etal.2024. Gemma:Openmodelsbasedongeminiresearch and technology.arXiv preprint arXiv:2403.08295. Tsipra...

    2024 arXiv

  4. [19]

    In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, 4951–4961

    Fooling Vision and Language Models De- spite Localization and Attention Mechanism. In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, 4951–4961. Computer Vision Foundation / IEEE Computer Society. Yue, X.; N...

    2018 arXiv

  5. [21]

    (15) Writingρi =|E i|−1, a conservative threshold is tα = min{t:P ρ (Ttgt≥t)≤α}

    = ( |Ei|−1, A i∈E i, 0, A i /∈Ei. (15) Writingρi =|E i|−1, a conservative threshold is tα = min{t:P ρ (Ttgt≥t)≤α}. This null does not require students to distribute their errors uniformly. The randomization is introduced by the educator’s target-selection procedure. The requir...

    1934

  6. [22]

    The reported detector instead usesK= 30returned-answer calls for provider-independent calibration, so these probability-valued estimates do not enter the reported operating points

    Figure 7 shows how probability-valued estimates converge as the number of repeated calls increases. The reported detector instead usesK= 30returned-answer calls for provider-independent calibration, so these probability-valued estimates do not enter the reported operating poin...

    2024

  7. [24]

    Adherence is the fraction of responses that contain a valid capital letter corresponding to an available option

    Table 10: Prompt-template adherence of the surrogate models in Table 6, evaluated on 50 questions from each of MMMU, MMBench, and ScienceQA. Adherence is the fraction of responses that contain a valid capital letter corresponding to an available option. Model Adherence (%) Gem...

  8. [2014]

    In Bengio, Y.; and LeCun, Y., eds.,2nd International Conference on Learning Represen- tations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings

    Intriguing prop- erties of neural networks. In Bengio, Y.; and LeCun, Y., eds.,2nd International Conference on Learning Represen- tations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. Team, G.; Abd, S. E.; Aggarwal, V.; Algayres, R.; Andreev, ...

    2014

  9. [2017]

    InInternational Conference on Learning Representations

    Delving into Transferable Adversarial Examples and Black-Box Attacks. InInternational Conference on Learning Representations. Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; Chen, K.; and Lin, D. 2024.MMBench:IsYourMulti-modalMod...

    2024

  10. [2018]

    In2018 IEEE Conference on Computer Vision and Pattern Recognition,CVPR2018,SaltLakeCity,UT,USA,June18- 22, 2018, 9185–9193

    Boosting Adversarial Attacks With Momentum. In2018 IEEE Conference on Computer Vision and Pattern Recognition,CVPR2018,SaltLakeCity,UT,USA,June18- 22, 2018, 9185–9193. Computer Vision Foundation / IEEE Computer Society. Gehrmann, S.; Strobelt, H.; and Rush, A

    2018

  11. [2019]

    In Costa-jussà, M

    GLTR: Sta- tistical Detection and Visualization of Generated Text. In Costa-jussà, M. R.; and Alfonseca, E., eds.,Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics: System Demonstrations, 111–116. Flo- rence, Italy: Association for Compu...

    2026

  12. [2022]

    InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, 10758–10766

    Pro- tectingintellectualpropertyoflanguagegenerationapiswith lexical watermark. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, 10758–10766. Herbert, F. 1965.Dune (Dune Chronicles, #1). Hodder & Stoughton. Hoeffding, W

    1965

  13. [2023]

    Raising the Cost of Malicious AI-Powered Image Editing. In Krause, A.; Brunskill, E.; Cho, K.; Engelhardt, B.;Sabato,S.;andScarlett,J.,eds.,InternationalConference on Machine Learning, ICML 2023, 23-29 July 2023, Hon- olulu,Hawaii,USA,volume202ofProceedingsofMachine Learning R...

    2023

  14. [2024]

    In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 38745–38768

    On the Learnability of Watermarks for Language Models. In Kim, B.; Yue, Y.; Chaudhuri, S.; Fragkiadaki, K.; Khan, M.;andSun,Y.,eds.,InternationalConferenceonLearning Representations, volume 2024, 38745–38768. He, X.; Xu, Q.; Lyu, L.; Wu, F.; and Wang, C

    2024

  15. [2025]

    Bayer,J.B.;Trie.u,P.;andEllison,N.B.2020

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631. Bayer,J.B.;Trie.u,P.;andEllison,N.B.2020. SocialMedia Elements, Ecologies, and Effects.Annual Review of Psy- chology, 71: 471–497. Birks, D.; and Clare, J

    2020 arXiv

  16. [2026]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 42341–42351

    Omni-Attack: Adversarial At- tacksonOpen-EndedVQAinBlack-BoxMultimodalLLMs. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 42341–42351. Huang, J.; Chen, L.; Guo, T.; Zeng, F.; Zhao, Y.; Wu, B.; Yuan,Y.;Zhao,H.;Guo,Z.;Zhang,Y.;etal.2025. Mm...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.