pith. sign in

arxiv: 2607.01280 · v1 · pith:NF7AO76Enew · submitted 2026-07-01 · 💻 cs.LG · cs.PL

Fixed-Set Robustness in Programming by Example: Example Corruption and Semantic Partition Recovery

Yuan Si , Jialu Zhang This is my paper

Pith reviewed 2026-07-03 21:40 UTC · model grok-4.3

classification 💻 cs.LG cs.PL
keywords programming by exampleadversarial robustnessversion spacesemantic partition aggregationexample corruptionSyGuSPBEstring transformation
0
0 comments X

The pith

Low-margin PBE tasks are vulnerable to fixed-set adversarial example corruption that random noise tests miss, and semantic partition voting recovers only when clean vote margins already exist.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper studies a failure mode in programming-by-example systems where an adversary who sees the synthesizer deliberately chooses which examples to corrupt. It contrasts this targeted attack with the usual modeling of errors as random noise and formalizes worst-case corruption over finite version spaces. The authors implement exact and heuristic search methods for a string-transformation DSL and introduce version-space partition aggregation as a defense that splits examples and votes on semantic signatures. Evidence from curated tasks, generated margin-1 cases, public SyGuS slices, and Playgol shows that single targeted edits can flip results while random controls cannot, yet the defense succeeds only when the clean semantics already produce a decisive vote margin. The claim is deliberately bounded: current robustness evaluations therefore understate the risk on realistic low-margin tasks.

Core claim

The central claim is that low-margin PBE tasks have an adversarial robustness dimension missed by random-typo and noisy-PBE evaluations. Version-space partition aggregation helps only when the clean semantics keep a partition vote margin, which often fails on realistic tasks. On public SyGuS PBE_SLIA slices the vote margin is near one, so an adaptive attacker drives VPA accuracy to zero; one curated edit flips all eight spike tasks while 200-trial random controls succeed on at most 16.7 percent; generated margin-1 rows flip under budget 1 yet VPA recovers them; and Playgol shows positive paired-bootstrap gaps against controls on the 141 accepted rows.

What carries the argument

Version-space partition aggregation (VPA), which synthesizes on disjoint example groups and votes by semantic signatures.

If this is right

  • One targeted edit flips all eight spike tasks while random-typo, DSL-pool, and distance-matched controls succeed on 10.3 percent, 11.0 percent, and 16.7 percent respectively.
  • Generated margin-1 rows flip under corruption budget 1, yet VPA recovers them when the margin condition holds.
  • On accepted public SyGuS slices the vote margin is near one, allowing an adaptive attacker to drive VPA accuracy to zero.
  • Playgol v2 exhibits positive paired-bootstrap gaps for VPA against typo and same-pool random controls on the 141 accepted rows.
  • A small exact-output prompt harness over 20 controlled margin-1 tasks reproduces the same clean-to-attacked accuracy drop across local and API models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Robustness claims for PBE systems should be re-evaluated using bounded adversarial search rather than random perturbation alone.
  • If semantic signatures fail to produce high margins outside the tested benchmarks, VPA may require larger partitions or alternative aggregation rules.
  • The gap between random and adversarial performance implies that existing noisy-PBE loss objectives underestimate risk from knowledgeable attackers.
  • Extending the exact-within-pool corruption search to additional DSLs could identify similar low-margin vulnerabilities in other domains.

Load-bearing premise

The near-one vote margins observed on the chosen SyGuS and Playgol slices are representative of realistic PBE tasks.

What would settle it

Finding a broader collection of realistic PBE tasks where clean semantic partitions already produce decisive vote margins and VPA maintains accuracy against adaptive attackers would falsify the negative conclusion on VPA.

Figures

Figures reproduced from arXiv: 2607.01280 by Jialu Zhang, Yuan Si.

Figure 1
Figure 1. Figure 1: Threat model and VPA overview. The attack searches a budget- [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

Programming-by-example systems infer programs from a small set of input-output examples. Robust PBE work usually models wrong examples as samples from a stochastic noise process and then minimizes an expected or empirical loss. This paper studies a different failure mode: an adversary who sees the synthesizer and chooses the examples whose corruption most damages the returned program. We formalize fixed-set worst-case corruption for finite PBE version spaces, implement exact-within-bounded-pool and heuristic corruption searches for a string-transformation DSL, and introduce version-space partition aggregation (VPA), a defense that synthesizes on disjoint example groups and votes by semantic signatures. The central claim is deliberately bounded and partly negative: low-margin PBE tasks have an adversarial robustness dimension that random-typo and noisy-PBE evaluations miss, while semantic partition aggregation helps only when the clean semantics keep a partition vote margin, which often fails on realistic tasks. Evidence from curated/generated DSL tasks, accepted public SyGuS PBE_SLIA slices, SYNTRA Playgol v2, and noisy-PBE objective baselines supports that boundary. One curated edit flips all 8 spike tasks while 200-trial typo, DSL-pool, and distance-matched random controls succeed on 10.3%, 11.0%, and 16.7%; generated margin-1 rows flip under budget 1 yet VPA recovers them; on public SyGuS the vote margin is near one, so an adaptive attacker drives VPA accuracy to zero; accepted public SyGuS slices move across exact-within-pool budget boundaries; and Playgol shows positive paired-bootstrap gaps against typo and same-pool random controls on the 141 accepted rows. A small exact-output prompt harness over 20 controlled margin-1 tasks shows the same qualitative clean-to-attacked pattern across local and API models, while it is treated as a scope check, not a broad LLM benchmark.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper formalizes fixed-set worst-case corruption attacks on finite version spaces in programming-by-example (PBE) synthesizers, implements exact-within-bounded-pool and heuristic searches over a string-transformation DSL, and introduces version-space partition aggregation (VPA) as a defense that aggregates syntheses over disjoint example groups and votes by semantic signatures. Its central bounded claim is that low-margin PBE tasks possess an adversarial robustness dimension missed by random-typo or noisy-PBE evaluations, while VPA recovers accuracy only when clean semantics already produce a sufficient partition vote margin—an outcome that frequently fails on realistic tasks. Supporting evidence includes curated edit attacks that flip all 8 spike tasks (vs. 10-17% success for controls), generated margin-1 rows, public SyGuS PBE_SLIA slices where margins are near one and adaptive attacks drive VPA accuracy to zero, Playgol v2 results on 141 rows, and a small LLM prompt harness.

Significance. If the bounded claim holds, the work usefully distinguishes adversarial fixed-set corruption from stochastic noise models and supplies controlled experiments across curated, generated, and public benchmarks (SyGuS PBE_SLIA, Playgol v2) that demonstrate the practical limits of semantic aggregation. The explicit negative result on VPA under near-unit margins, together with the positive paired-bootstrap gaps against typo and same-pool random baselines, supplies a falsifiable boundary condition that future PBE robustness papers can test directly.

major comments (3)
  1. [Abstract] Abstract and experimental sections: the partly negative conclusion that VPA 'often fails on realistic tasks' rests on the observation that vote margins are near one on the accepted SyGuS PBE_SLIA slices and the 141 Playgol rows; the manuscript must demonstrate that these slices are not biased toward low-margin cases by the acceptance filter or task curation, or else the generalization does not follow.
  2. [Abstract] The claim that an adaptive attacker drives VPA accuracy to zero on public SyGuS requires both that the reported margins are correctly computed and that semantic signatures produce comparable margins on other realistic string tasks; the current evidence leaves open whether the near-one margins are an artifact of the chosen slices.
  3. Gaps remain around the exact implementation of the corruption search (exact-within-bounded-pool) and margin calculation; these details are load-bearing for reproducing the reported attack success rates and VPA recovery thresholds.
minor comments (2)
  1. [Abstract] Clarify the precise definition of 'partition vote margin' and how it is computed from semantic signatures, including any threshold used to decide when VPA is expected to help.
  2. The 20-task LLM prompt harness is presented as a scope check; if retained, it should be moved to an appendix with explicit controls for prompt variation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thoughtful review and constructive feedback. We address each of the major comments below, providing clarifications and indicating where revisions will be made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract and experimental sections: the partly negative conclusion that VPA 'often fails on realistic tasks' rests on the observation that vote margins are near one on the accepted SyGuS PBE_SLIA slices and the 141 Playgol rows; the manuscript must demonstrate that these slices are not biased toward low-margin cases by the acceptance filter or task curation, or else the generalization does not follow.

    Authors: The PBE_SLIA slices are drawn from the publicly accepted SyGuS benchmark tasks, which represent standard realistic string transformation problems used in the synthesis community. The Playgol v2 evaluation on 141 rows offers an independent dataset not subject to the same acceptance criteria. In the revision, we will explicitly qualify our claims to these evaluated benchmarks and add a discussion of potential curation effects as a limitation, while noting that the low-margin phenomenon is consistent across both sources. We maintain that the evidence supports the bounded claim without overgeneralization. revision: partial

  2. Referee: [Abstract] The claim that an adaptive attacker drives VPA accuracy to zero on public SyGuS requires both that the reported margins are correctly computed and that semantic signatures produce comparable margins on other realistic string tasks; the current evidence leaves open whether the near-one margins are an artifact of the chosen slices.

    Authors: Margin computation is performed on the clean version spaces prior to any corruption, as described in the methods. We will include explicit formulas and pseudocode for both margin calculation and the semantic signature voting in the revised appendix to facilitate verification. The Playgol v2 results provide supporting evidence on a separate collection of tasks where similar low margins lead to VPA failure under attack. We agree that additional datasets would be valuable and will note this as future work. revision: yes

  3. Referee: [—] Gaps remain around the exact implementation of the corruption search (exact-within-bounded-pool) and margin calculation; these details are load-bearing for reproducing the reported attack success rates and VPA recovery thresholds.

    Authors: We will expand the supplementary material with detailed pseudocode for the exact-within-bounded-pool corruption search, including the pool construction, enumeration bounds, and the precise definition of semantic signatures used in VPA voting. This will ensure full reproducibility of the attack success rates and recovery thresholds reported. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical claims rest on external benchmarks

full rationale

The paper evaluates fixed-set robustness and VPA using public external benchmarks (SyGuS PBE_SLIA slices, Playgol v2) plus generated/curated tasks, with no equations or claims that reduce by construction to fitted parameters from the same data, self-citations that bear the central load, or ansatzes smuggled via prior work. All reported margins, attack successes, and recovery rates are computed directly from those independent sources rather than being renamed or forced by internal definitions.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 1 invented entities

The paper relies on standard domain assumptions of PBE (finite version spaces for the chosen DSL) and introduces one new method (VPA) without new physical entities or fitted constants that drive the central claim.

free parameters (2)
  • corruption budget
    Bounds the number of example changes an adversary may make; experiments reference budget 1 for flipping tasks.
  • margin threshold = 1
    Margin-1 tasks are singled out as recoverable by VPA.
axioms (1)
  • domain assumption PBE version spaces are finite for the string-transformation DSL
    Invoked to formalize fixed-set worst-case corruption.
invented entities (1)
  • version-space partition aggregation (VPA) no independent evidence
    purpose: Defense that partitions examples into disjoint groups, synthesizes per group, and votes by semantic signatures
    Newly proposed method without independent evidence outside this paper.

pith-pipeline@v0.9.1-grok · 5876 in / 1452 out tokens · 41030 ms · 2026-07-03T21:40:32.810210+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 67 canonical work pages · 4 internal anchors

  1. [1]

    arXiv preprint arXiv:2509.11065

    Y . Si, D. Li, H. Shi, and J. Zhang, “ViScratch: Using large language models and gameplay videos for automated feedback in Scratch,” arXiv:2509.11065, 2025

    work page arXiv 2025

  2. [2]

    Stitch: Step-by-step LLM guided tutoring for Scratch,

    Y . Si, K. Qi, D. Li, H. Shi, and J. Zhang, “Stitch: Step-by-step LLM guided tutoring for Scratch,” arXiv:2510.26634, 2025

    work page arXiv 2025

  3. [3]

    ScratchEval: A multi- modal evaluation framework for LLMs in block-based programming,

    Y . Si, S. Han, D. Li, H. Shi, and J. Zhang, “ScratchEval: A multi- modal evaluation framework for LLMs in block-based programming,” arXiv:2602.00757, 2026

    work page arXiv 2026

  4. [4]

    EcoScratch: Cost- effective multimodal repair for Scratch using execution feedback,

    Y . Si, M. Wang, D. Li, H. Shi, and J. Zhang, “EcoScratch: Cost- effective multimodal repair for Scratch using execution feedback,” arXiv:2603.29624, 2026

    work page arXiv 2026

  5. [5]

    Raven: Rethinking Automated Assessment for Scratch Programs via Video-Grounded Evaluation

    D. Li, D. Li, H. Shi, and J. Zhang, “Raven: Rethinking auto- mated assessment for Scratch programs via video-grounded evaluation,” arXiv:2604.17820, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  6. [6]

    ScratchLens: Lens-Parametric Behavioral Equivalence for Scratch Programs

    Y . Si and J. Zhang, “ScratchLens: Lens-parametric behavioral equiva- lence for Scratch programs,” arXiv:2606.15817, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  7. [7]

    ScratchWorld: Evaluating If World Models Compute Executable Consequences

    Y . Lin and J. Zhang, “ScratchWorld: Evaluating if world models compute executable consequences,” arXiv:2606.31689, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  8. [8]

    A systematic study of time limit exceeded errors in online programming assignments,

    J. Zhang, J. Gu, W. Zhang, J. P. Cambronero, J. C. Kolesar, R. Piskac, D. Li, and H. Shi, “A systematic study of time limit exceeded errors in online programming assignments,” arXiv:2510.14339, 2025

    work page arXiv 2025

  9. [9]

    Automated feedback generation for competition-level code,

    J. Zhang, D. Li, J. C. Kolesar, H. Shi, and R. Piskac, “Automated feedback generation for competition-level code,” inProc. IEEE/ACM International Conference on Automated Software Engineering (ASE), 2022, Article 13, pp. 1–13

    work page 2022

  10. [10]

    PyDex: Repairing bugs in introductory Python assignments using LLMs,

    J. Zhang, J. P. Cambronero, S. Gulwani, V . Le, R. Piskac, G. Soares, and G. Verbruggen, “PyDex: Repairing bugs in introductory Python assignments using LLMs,”Proc. ACM Program. Lang., vol. 8, no. OOPSLA1, pp. 1100–1124, 2024

    work page 2024

  11. [11]

    Using pre-trained language models to resolve textual and semantic merge conflicts,

    J. Zhang, T. Mytkowicz, M. Kaufman, R. Piskac, and S. K. Lahiri, “Using pre-trained language models to resolve textual and semantic merge conflicts,” inProc. ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2022, pp. 77–88

    work page 2022

  12. [12]

    Learning CI configuration correctness for early build feedback,

    M. Santolucito, J. Zhang, E. Zhai, J. Cito, and R. Piskac, “Learning CI configuration correctness for early build feedback,” inProc. IEEE Inter- national Conference on Software Analysis, Evolution and Reengineering (SANER), 2022, pp. 1006–1017

    work page 2022

  13. [13]

    Static detection of silent misconfigurations with deep interaction analysis,

    J. Zhang, R. Piskac, E. Zhai, and T. Xu, “Static detection of silent misconfigurations with deep interaction analysis,”Proc. ACM Program. Lang., vol. 5, no. OOPSLA, pp. 1–30, 2021

    work page 2021

  14. [14]

    Inductive program synthesis over noisy data,

    S. Handa and M. C. Rinard, “Inductive program synthesis over noisy data,” inProc. ACM Joint European Software Engineering Confer- ence and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2020, pp. 87–98

    work page 2020

  15. [15]

    Program synthesis over noisy data with guarantees,

    S. Handa and M. Rinard, “Program synthesis over noisy data with guarantees,” arXiv:2103.05030, 2021

    work page arXiv 2021

  16. [16]

    Inductive program synthesis over noisy datasets using abstraction refinement based optimization,

    S. Handa and M. Rinard, “Inductive program synthesis over noisy datasets using abstraction refinement based optimization,” arXiv:2104.13315, 2021

    work page arXiv 2021

  17. [17]

    RobustFill: Neural program learning under noisy I/O,

    J. Devlin, J. Uesato, S. Bhupatiraju, R. Singh, A.-r. Mohamed, and P. Kohli, “RobustFill: Neural program learning under noisy I/O,” inProc. International Conference on Machine Learning (ICML), 2017, pp. 990– 998

    work page 2017

  18. [18]

    Learning programs from noisy data,

    V . Raychev, P. Bielik, M. Vechev, and A. Krause, “Learning programs from noisy data,” inProc. ACM SIGPLAN Symposium on Principles of Programming Languages (POPL), 2016, pp. 761–774

    work page 2016

  19. [19]

    Adversarial robustness of program synthesis models,

    M. Anand, P. Kayal, and M. Singh, “Adversarial robustness of program synthesis models,” inNeurIPS 2021 Workshop on Advances in Program- ming Languages and Neurosymbolic Systems, 2021

    work page 2021

  20. [20]

    Is programming by example solved by LLMs?

    W.-D. Li and K. Ellis, “Is programming by example solved by LLMs?” arXiv:2406.08316, 2024

    work page arXiv 2024

  21. [21]

    Exploring the security threats of knowledge base poisoning in retrieval-augmented code generation,

    B. Lin, S. Wang, L. Chen, and X. Mao, “Exploring the security threats of knowledge base poisoning in retrieval-augmented code generation,” arXiv:2502.03233, 2025

    work page arXiv 2025

  22. [22]

    Practical Poisoning Attacks against Retrieval-Augmented Generation.arXiv preprint arXiv:2504.03957, 2025

    B. Zhang, Y . Chen, Z. Liu, L. Nie, T. Li, Z. Liu, and M. Fang, “Practical poisoning attacks against retrieval-augmented generation,” arXiv:2504.03957, 2025

    work page arXiv 2025

  23. [23]

    PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models,

    W. Zou, R. Geng, B. Wang, and J. Jia, “PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models,” inProc. USENIX Security Symposium, 2025

    work page 2025

  24. [24]

    Data poisoning for in-context learning,

    P. He, H. Xu, Y . Xing, H. Liu, M. Yamada, and J. Tang, “Data poisoning for in-context learning,” inFindings of the Association for Computational Linguistics: NAACL 2025, 2025

    work page 2025

  25. [25]

    PHE statement on delayed reporting of COVID-19 cases,

    Public Health England, “PHE statement on delayed reporting of COVID-19 cases,” GOV .UK, Oct. 2020. [Online]. Available: https://www.gov.uk/government/news/ phe-statement-on-delayed-reporting-of-covid-19-cases

    work page 2020

  26. [26]

    JPMorgan Chase whale trades: A case history of derivatives risks and abuses,

    U.S. Senate Permanent Subcommittee on Investigations, “JPMorgan Chase whale trades: A case history of derivatives risks and abuses,” Mar. 2013

    work page 2013

  27. [27]

    Certified defenses for data poi- soning attacks,

    J. Steinhardt, P. W. Koh, and P. Liang, “Certified defenses for data poi- soning attacks,” inAdvances in Neural Information Processing Systems, 2017

    work page 2017

  28. [28]

    Deep partition aggregation: Provable defense against general poisoning attacks,

    A. Levine and S. Feizi, “Deep partition aggregation: Provable defense against general poisoning attacks,” inInternational Conference on Learning Representations (ICLR), 2021

    work page 2021

  29. [29]

    Improved certified defenses against data poisoning with deterministic finite aggregation,

    W. Wang, A. J. Levine, and S. Feizi, “Improved certified defenses against data poisoning with deterministic finite aggregation,” inInternational Conference on Machine Learning (ICML), 2022, pp. 22769–22783

    work page 2022

  30. [30]

    Syntax- guided synthesis,

    R. Alur, R. Bodik, G. Juniwal, M. M. K. Martin, M. Raghothaman, S. Seshia, R. Singh, A. Solar-Lezama, E. Torlak, and A. Udupa, “Syntax- guided synthesis,” inProc. Formal Methods in Computer-Aided Design (FMCAD), 2013, pp. 1–8

    work page 2013

  31. [31]

    Generalization as search,

    T. M. Mitchell, “Generalization as search,”Artificial Intelligence, vol. 18, no. 2, pp. 203–226, 1982

    work page 1982

  32. [32]

    Programming by demonstration using version space algebra,

    T. A. Lau, S. A. Wolfman, P. Domingos, and D. S. Weld, “Programming by demonstration using version space algebra,”Machine Learning, vol. 53, no. 1–2, pp. 111–156, 2003

    work page 2003

  33. [33]

    Automating string processing in spreadsheets using input- output examples,

    S. Gulwani, “Automating string processing in spreadsheets using input- output examples,” inProc. ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), 2011, pp. 317–330

    work page 2011

  34. [34]

    Disjunctive program synthesis: A robust approach to programming by example,

    M. Raza and S. Gulwani, “Disjunctive program synthesis: A robust approach to programming by example,” inProc. AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018

    work page 2018

  35. [35]

    PROSE framework: Microsoft Program Synthesis using Examples,

    Microsoft Research, “PROSE framework: Microsoft Program Synthesis using Examples,” accessed Jun. 22, 2026. [Online]. Available: https: //www.microsoft.com/en-us/research/project/prose-framework/

    work page 2026

  36. [36]

    FlashMeta: A framework for inductive program synthesis,

    O. Polozov and S. Gulwani, “FlashMeta: A framework for inductive program synthesis,” inProc. ACM SIGPLAN Conference on Object- Oriented Programming, Systems, Languages, and Applications (OOP- SLA), 2015, pp. 107–126

    work page 2015

  37. [37]

    Combinatorial sketching for finite programs,

    A. Solar-Lezama, L. Tancau, R. Bodik, S. A. Seshia, and V . A. Saraswat, “Combinatorial sketching for finite programs,” inProc. International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2006, pp. 404–415

    work page 2006

  38. [38]

    Oracle-guided component-based program synthesis,

    S. Jha, S. Gulwani, S. A. Seshia, and A. Tiwari, “Oracle-guided component-based program synthesis,” inProc. International Conference on Software Engineering (ICSE), 2010, pp. 215–224

    work page 2010

  39. [39]

    Type-and-example-directed program synthesis,

    P.-M. Osera and S. Zdancewic, “Type-and-example-directed program synthesis,” inProc. ACM SIGPLAN Conference on Programming Lan- guage Design and Implementation (PLDI), 2015, pp. 619–630

    work page 2015

  40. [40]

    Synthesizing data structure transformations from input-output examples,

    J. K. Feser, S. Chaudhuri, and I. Dillig, “Synthesizing data structure transformations from input-output examples,” inProc. ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2015, pp. 229–239

    work page 2015

  41. [41]

    Program synthesis from polymorphic refinement types,

    N. Polikarpova, I. Kuraj, and A. Solar-Lezama, “Program synthesis from polymorphic refinement types,” inProc. ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2016, pp. 522–538

    work page 2016

  42. [42]

    Program synthesis,

    S. Gulwani, O. Polozov, and R. Singh, “Program synthesis,”Foundations and Trends in Programming Languages, vol. 4, no. 1–2, pp. 1–119, 2017

    work page 2017

  43. [43]

    DeepCoder: Learning to write programs,

    M. Balog, A. L. Gaunt, M. Brockschmidt, S. Nowozin, and D. Tarlow, “DeepCoder: Learning to write programs,” inInternational Conference on Learning Representations (ICLR), 2017

    work page 2017

  44. [44]

    Leveraging grammar and reinforcement learning for neural program synthesis,

    R. Bunel, M. Hausknecht, J. Devlin, R. Singh, and P. Kohli, “Leveraging grammar and reinforcement learning for neural program synthesis,” in International Conference on Learning Representations (ICLR), 2018

    work page 2018

  45. [45]

    Representing partial programs with blended abstract semantics,

    M. Nye, Y . Pu, M. Bowers, J. Andreas, J. B. Tenenbaum, and A. Solar-Lezama, “Representing partial programs with blended abstract semantics,” inInternational Conference on Learning Representations (ICLR), 2021

    work page 2021

  46. [46]

    DreamCoder: Boot- strapping inductive program synthesis with wake-sleep library learning,

    K. Ellis, C. Wong, M. Nye, M. Sable-Meyer, L. Cary, L. Morales, L. Hewitt, A. Solar-Lezama, and J. B. Tenenbaum, “DreamCoder: Boot- strapping inductive program synthesis with wake-sleep library learning,” inProc. ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2021

    work page 2021

  47. [47]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Good- fellow, and R. Fergus, “Intriguing properties of neural networks,” in International Conference on Learning Representations (ICLR), 2014

    work page 2014

  48. [48]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” inInternational Conference on Learning Repre- sentations (ICLR), 2015

    work page 2015

  49. [49]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” inInternational Conference on Learning Representations (ICLR), 2018

    work page 2018

  50. [50]

    Poisoning attacks against sup- port vector machines,

    B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against sup- port vector machines,” inProc. International Conference on Machine Learning (ICML), 2012

    work page 2012

  51. [51]

    Using machine teaching to identify optimal training-set attacks on machine learners,

    S. Mei and X. Zhu, “Using machine teaching to identify optimal training-set attacks on machine learners,” inProc. AAAI Conference on Artificial Intelligence, 2015, pp. 2871–2877

    work page 2015

  52. [52]

    Poison frogs! Targeted clean-label poisoning attacks on neural networks,

    A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, and T. Goldstein, “Poison frogs! Targeted clean-label poisoning attacks on neural networks,” inAdvances in Neural Information Processing Systems, 2018

    work page 2018

  53. [53]

    BadNets: Evaluating backdooring attacks on deep neural networks,

    T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “BadNets: Evaluating backdooring attacks on deep neural networks,”IEEE Access, vol. 7, pp. 47230–47244, 2019

    work page 2019

  54. [54]

    Spectral signatures in backdoor attacks,

    B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” inAdvances in Neural Information Processing Systems, 2018

    work page 2018

  55. [55]

    Weight poisoning attacks on pretrained models,

    K. Kurita, P. Michel, and G. Neubig, “Weight poisoning attacks on pretrained models,” inProc. Annual Meeting of the Association for Computational Linguistics (ACL), 2020, pp. 2793–2806

    work page 2020

  56. [56]

    Certified adversarial robust- ness via randomized smoothing,

    J. Cohen, E. Rosenfeld, and J. Z. Kolter, “Certified adversarial robust- ness via randomized smoothing,” inProc. International Conference on Machine Learning (ICML), 2019, pp. 1310–1320

    work page 2019

  57. [57]

    Certified robustness to label-flipping attacks via randomized smoothing,

    E. Rosenfeld, E. Winston, P. Ravikumar, and J. Z. Kolter, “Certified robustness to label-flipping attacks via randomized smoothing,” inProc. International Conference on Machine Learning (ICML), 2020, pp. 8230– 8241

    work page 2020

  58. [58]

    Robust estimation of a location parameter,

    P. J. Huber, “Robust estimation of a location parameter,”Annals of Mathematical Statistics, vol. 35, no. 1, pp. 73–101, 1964

    work page 1964

  59. [59]

    Robust estimators in high dimensions without the computational intractability,

    I. Diakonikolas, G. Kamath, D. Kane, J. Li, A. Moitra, and A. Stew- art, “Robust estimators in high dimensions without the computational intractability,” inProc. IEEE Symposium on Foundations of Computer Science (FOCS), 2016, pp. 655–664

    work page 2016

  60. [60]

    You autocomplete me: Poisoning vulnerabilities in neural code completion,

    R. Schuster, C. Song, E. Tromer, and V . Shmatikov, “You autocomplete me: Poisoning vulnerabilities in neural code completion,” inProc. USENIX Security Symposium, 2021

    work page 2021

  61. [61]

    arXiv preprint arXiv:2301.02344 (2023)

    H. Aghakhani, W. Dai, A. Manoel, X. Fernandes, A. Kharkar, C. Kruegel, G. Vigna, D. Evans, B. Zorn, and R. Sim, “TrojanPuzzle: Covertly poisoning code-suggestion models,” arXiv:2301.02344, 2023; inProc. IEEE Symposium on Security and Privacy (S&P), 2024

    work page arXiv 2023

  62. [62]

    Poisoning web- scale training datasets is practical,

    N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramer, “Poisoning web- scale training datasets is practical,” arXiv:2302.10149, 2023; inProc. IEEE Symposium on Security and Privacy (S&P), 2024

    work page arXiv 2023

  63. [63]

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real- world LLM-integrated applications with indirect prompt injection,” arXiv:2302.12173, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  64. [64]

    On the complexity of teaching,

    S. A. Goldman and M. J. Kearns, “On the complexity of teaching,” Journal of Computer and System Sciences, vol. 50, no. 1, pp. 20–31, 1995

    work page 1995

  65. [65]

    Machine teaching: An inverse problem to machine learning and an approach toward optimal education,

    X. Zhu, “Machine teaching: An inverse problem to machine learning and an approach toward optimal education,” inProc. AAAI Conference on Artificial Intelligence, 2015

    work page 2015

  66. [66]

    Queries and concept learning,

    D. Angluin, “Queries and concept learning,”Machine Learning, vol. 2, no. 4, pp. 319–342, 1988

    work page 1988

  67. [67]

    Active learning literature survey,

    B. Settles, “Active learning literature survey,” University of Wisconsin– Madison, Computer Sciences Technical Report 1648, 2009

    work page 2009