$\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection

Daniil Orel; Indraneil Paul; Iryna Gurevych; Preslav Nakov

REVIEW 4 major objections 6 minor 1 cited by

$\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current AI-generated code detectors fail on diverse code and are easily evaded by humanised output; training on the new DroidCollection corpus restores their recall.

desk verdict The dataset is a genuine resource and the OOD stress tests are solid, but the uncertainty-resampling step as written is circular, so the in-domain F1 numbers and the resampling ablation need to be redone before the detector claims can be trusted. read the letter →

arxiv 2507.10583 v3 pith:SCUJF5TW submitted 2025-07-11 cs.SE cs.AIcs.CY

Daniil Orel , Indraneil Paul , Iryna Gurevych , Preslav Nakov This is my paper

classification cs.SEcs.AIcs.CY

keywords AI-generatedcodedetectionmachine-generateddatasetconstructionadversarialrobustnesshumanisedencoder-onlytransformermetriclearninguncertaintyresampling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that existing AI-generated code detectors fail in realistic conditions—they generalise poorly across programming languages and coding domains, and they are easily evaded when generated code is humanised by prompting or preference-tuning a model to write like a person. To fix this, the authors build DroidCollection, an open dataset of over a million code samples spanning seven languages, 43 generators, three domains, and four classes (human, AI-generated, human-AI refined, adversarially humanised), and DroidDetect, two encoder-only detectors trained on it. Their experiments show that adding a modest amount of adversarial data restores detector recall from near zero to above 0.9, and that metric learning plus uncertainty-based resampling improve robustness when training labels are noisy. If true, the result matters because it makes AI-code detection a tractable data-breadth problem rather than an unsolvable arms-race problem.

What carries the argument

The load-bearing object is DroidCollection, a corpus of over a million code samples across seven languages, 43 generators from 11 model families, three coding domains (general-use GitHub code, algorithmic problems, research/data-science code), and four classes including human-AI co-authored and adversarially humanised code. It is generated with varied decoding strategies, inverse-instruction prompting, persona-conditioned synthetic tasks, and preference-tuning (DPO) to produce human-like outputs, giving detectors the distributional breadth and the adversarial exposure they need. The DroidDetect models are ModernBERT encoders trained with a multi-task classification objective, supervised contrastive (triplet) loss to tighten class separability, and MC-Dropout based uncertainty resampling that removes the most ambiguous human-written samples from training.

What would settle it

Audit the human-written split of DroidCollection against the outputs of its 43 generators and against pre-copilot repositories, then retrain DroidDetect on the uncontaminated subset: if its adversarial recall and out-of-domain F1 drop materially on this verified set, the claim that a small amount of adversarial data suffices is overstated.

Watch

Extended reading notes

Core claim

The central claim is that the brittleness of machine-generated code detectors is a data problem, not an architecture problem. Detectors trained on narrow, single-domain corpora lose to diverse real-world code and collapse against humanised outputs, but the same encoder-only backbones, fine-tuned on DroidCollection's full breadth, reach near-ceiling in-domain weighted F1 (99.23 binary, 93.66 ternary) and recall above 0.9 on adversarial samples while GPTZero falls to 0.10. The paper further shows that a small amount of adversarially humanised data is sufficient to repair evasion vulnerability, and that supervised contrastive learning and MC-Dropout-based resampling of uncertain human-written samples improve accuracy when the human labels are noisy. The authors therefore present DroidCollection and DroidDetect as the first resource suite that makes robust detection of AI-generated code practical across languages, domains, and adversarial settings.

Load-bearing premise

The entire evaluation rests on the assumption that the human-written samples in DroidCollection really were written by people; the authors admit that code collected after coding assistants became popular could itself contain LLM-assisted passages, so the detector's clean separation may partly be learned from labels that are not as clean as they appear.

Editorial extensions

If this is right

If the corpus results hold, training on diverse adversarial data lets open detectors catch humanised AI code, closing the gap shown by GPTZero's 0.10 recall on such samples.
Only a modest amount of adversarial data is needed to repair evasion vulnerability, so hardening an existing detector is cheap once humanised samples are available.
Positive transfer across syntactically related languages (for example C/C++ to C# and Java) means language coverage can be prioritised, while typologically isolated languages like Python and JavaScript need their own training data.
Single-domain benchmarks overstate real-world readiness: models trained on algorithmic problems fall to roughly half their F1 on general or research code, so evaluation must be domain-diverse.
Public contamination flags on the human-written split give downstream users a mechanism to filter dubious labels, making the dataset usable in stricter settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the acknowledged contamination risk is real, the reported near-ceiling numbers may be optimistic: an independent audit of post-copilot human-written samples could reveal a harder separation than the paper's ablation suggests.
The 'small amount of adversarial data fixes it' result implies an arms-race dynamic in which detectors must be continuously retrained against new humanisers, and the corpus will need regular refresh as new model families appear.
The recipe generalises beyond code: inverse instructions, persona-conditioned unconditional generation, and DPO-based humanisation are transferable to detecting AI-generated essays, reviews, or social-media text.
The consistent cross-language transfer hints at shared stylistic signatures of machine-written code; probing which features the encoder attends to could yield explainable detectors rather than black boxes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

I read the Droid paper with the reader's summary in hand. The reader is right on both counts: the dataset is the real contribution, and the circularity concern is real.

What's genuinely new is DroidCollection: the largest open suite for AI-generated code detection, with 43 generators, seven languages, three domains, varied decoding strategies, human-AI co-authored code, and a new adversarial humanized split. Prior datasets like GPT-Sniffer, CodeGPTSensor, CoDet-M4, and CodeMirage are much narrower. The external stress tests on Swallow-Code and The Heap are independent and support the claim that the final DroidDetect models generalize out-of-distribution. That is solid. The paper is also transparent about the contamination risk in Section 3.6, and the released dataset includes flags for suspicious samples, which is responsible.

The soft spots are real, and one is load-bearing. Section 5 says they used MC Dropout to estimate model uncertainty on the human-written portion of the test set, removed the top 7% most uncertain samples from the training set, and retrained. That is circular if literal: the test set determines what is excluded from training, and the retrained model is evaluated on that same test set. The near-ceiling F1 scores in Tables 3 and 6 become hard to interpret, and the resampling ablation in Table 6 loses its meaning. Since this resampling is the paper's explicit mitigation for the Section 3.6 label contamination, the acknowledged contamination risk lands harder. No training code is provided to check whether a validation set was actually used. Even if this is a typo, the paper must say so and recompute.

Second, there are no error bars or multiple seeds. The gains from the supervised contrastive loss and resampling are fractions of a point (e.g., 99.18 vs 99.14 vs 99.15). Those differences are not statistically grounded. Third, the adversarial robustness evaluation is in-distribution: the adversarial samples are generated by models tuned on the same human code the detector trains on, so the 'easily amended' claim is only shown for the seen attack pipeline, not for an unseen humanizer.

None of this breaks the resource contribution. But the detector evaluation, and the specific claims about metric learning and resampling, need to be redone with a proper split and seeds. I would send this to a serious referee, conditional on the authors clarifying the resampling step and providing code. The dataset alone earns the review.

The paper is for anyone building or evaluating code detectors, and for educators, hiring, and security contexts. Bring it to the reading group; the dataset is worth knowing about even if the detector numbers are pending.

My recommendation: engage, but require the fix. The resource is valuable and the flaws are addressable.

Referee Report

4 major / 6 minor

Summary. The paper introduces DroidCollection, a large open dataset for AI-generated code detection, with over one million samples spanning seven languages, three domains, 43 generative models, and including human-AI co-authored and adversarially humanized code. It also presents DroidDetect, ModernBERT-based detectors trained with a multi-task objective, and evaluates them against zero-shot and fine-tuned baselines in binary, ternary, and four-class settings. The main empirical claims are that existing detectors fail to generalize out-of-domain and are easily evaded by adversarial humanization, while DroidDetect reaches near-ceiling in-domain F1, transfers across languages and domains, and benefits from metric learning and uncertainty-based resampling. The paper also reports stress tests on Swallow-Code and The Heap supporting generalization to external data.

Significance. If the results hold, this would be a substantial contribution: the dataset is an order of magnitude broader than prior benchmarks in generator and language coverage, the models are released, and the external stress tests on Swallow-Code and The Heap provide valuable evidence of generalization beyond the training pipeline. The adversarial corpus and the uncertainty-filtering idea are potentially useful to the community. However, the uncertainty-resampling procedure as described in §5 is circular, and the adversarial evaluation is partially in-distribution, so the main performance claims cannot currently be taken at face value. With a corrected evaluation protocol, the resource and the detector comparison would be significant for the field.

major comments (4)

[Section 5] The uncertainty-resampling step is described as using the test set to select training samples: 'we applied MC Dropout to estimate the model uncertainty on the human-written portion of the test set... removing them from the training set.' If this is literal, then the test set is used to decide which training samples are removed, and the same test set is then used to produce the in-domain F1 scores in Tables 3, 4, and 6. That is an information leak and makes the resampling ablation and the near-ceiling in-domain numbers uninterpretable. Please clarify whether 'test set' is a typo for a separate validation split, document the split construction, recompute all affected results, and release the training/evaluation code so the split logic is verifiable.
[Section 3.6] The paper acknowledges that human-written samples sourced after coding copilots became popular were not filtered, so some 'human' labels may actually be AI-generated. The uncertainty filter is the only explicit defense against this contamination, but its validity depends on the non-circular split logic raised in the previous comment. Please provide independent evidence for the filtering step, for example by running it on a held-out validation split, manually auditing a sample of removed instances, or comparing detector decisions on pre-copilot vs post-copilot human code. Without such evidence, the label-contamination risk weakens the interpretation of the reported human/AI separation.
[Section 3.4 / Table 5] The adversarial robustness evaluation is in-distribution: the adversarial samples are generated by DPO-tuned models trained on DroidCollection-Pref built from the same human-written pool, and DroidDetect is trained on these same adversarial samples. The external stress tests in Appendix D.2 cover human-written and machine-refined code but do not test unseen adversarial generators. To support the abstract's claim that the evasion problem 'can be easily amended,' please add an evaluation on adversarial samples from held-out generators or from an independently tuned evasion model, or otherwise temper the generalization claim.
[Section 4.4 / Abstract] The abstract and Section 4.4 claim that the adversarial-evasion problem can be fixed by training on 'a small amount' of adversarial data, but no experiment varies the amount of adversarial training data. Table 5 only compares models trained on the full DroidCollection with and without adversarial exposure. Please add a size-sweep ablation (e.g., 1%, 5%, 10% of the adversarial split) or revise the claim to reflect the actual experiment.

minor comments (6)

[Appendix E.1] The text contains the typo 'DridDetect models'; please correct it to 'DroidDetect models.'
[Table 5] Table 5 reports recall only; please also report precision or the full confusion matrix to contextualize the 0.10 recall of GPTZero and the high false positives of M4 and CoDet-M4 mentioned in Section 4.4.
[Tables 3-6] The naming is inconsistent: 'DroidDetectCLS-Base/Large' is used in Tables 3-4, while 'DroidDetect-Base/Large' is used elsewhere, and Table 6 uses unclear row labels such as '- Triplet Loss'; please unify the notation.
[Tables 3-5] No variance or multiple-seed results are reported for the main tables; given near-ceiling scores, a seed or confidence-interval analysis would help assess the stability of the comparisons.
[Appendix C.3 / Figure 2] Figure 2 is referenced, but the 'interquartile range within the model's predictions' is not defined; please specify the bootstrap or aggregation procedure used to obtain these intervals.
[Section 5] The 'top 7%' uncertainty threshold is presented as a fixed choice; please justify it or provide a sensitivity analysis.

Circularity Check

1 steps flagged · score 6.0 of 10

Section 5's uncertainty resampling is described as using the test set to filter the training set; the reported resampling gains and final DroidDetect numbers are therefore not independent of the evaluation set.

fitted input called prediction [Section 5 (Detector Training and Ablations), Table 6; mitigation for Section 3.6 contamination]
"To address this, we applied MC Dropout (Hasan et al., 2022) to estimate the model uncertainty on the human-written portion of the test set. Specifically, we identified the top 7% most uncertain samples– those for which a pre-trained model exhibited low prediction confidence–and resampled the dataset, removing them from the training set. We then retrained the model on the remaining data, thereby getting rid of the influence of potentially mislabelled or ambiguous samples."

The uncertainty used to choose which training samples to remove is computed on the human-written portion of the test set. The test set therefore determines the training distribution, and the same test set then produces the F1 scores in Tables 3, 4, and 6. The 'DroidDetect vs - Resampling' ablation in Table 6 compares a model whose training set was filtered using test-set information against one without that information; the improvement is expected because the filter was chosen to remove samples the model is least confident about on that very test set. This makes the resampling gain and the final DroidDetect numbers non-independent of the evaluation set.

full rationale

The dataset construction and most detector comparisons are self-contained: DroidCollection is an independent resource, the zero-shot baseline evaluations use external detectors, and the OOD stress tests on Swallow-Code and The Heap are external. The core circularity is confined to the uncertainty-resampling step: the text states the MC Dropout uncertainty is estimated on the test set and that the top 7% uncertain samples are then removed from the training set before retraining, after which the same test set is used for the reported scores. If taken literally, this is test-set leakage that inflates the resampling ablation and the final in-domain numbers. If it is a typo for training set, the paper provides no code or commit hash to verify which split was used, so the claim remains unverified. Because the final DroidDetect models incorporate this filter and achieve their best results with it, the circularity is partial but load-bearing for the final-model numbers; the dataset resource and external stress tests remain independent evidence.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several unchecked assumptions about label quality, the representativeness of the adversarial attacks, and the validity of the uncertainty-based cleaning. The only hand-set numeric parameters that materially affect the results are the dataset filtering bounds and the 7% uncertainty threshold. No genuinely new entities are postulated.

free parameters (2)

top_7%_uncertainty_threshold = 0.07 (top 7% of MC Dropout uncertainty estimates)
Section 5: samples with the highest predictive uncertainty are removed from the human-written training set. The threshold is chosen by hand without sensitivity analysis, and the paper states the uncertainty is computed on the test set, which would leak test information into training.
dataset_filtering_bounds = AST depth 2-31; max line length 12-400 chars; avg line length 5-140; 6-300 lines; alphanumeric fraction 0.2-0.75…
Section 3.6: these bounds define DroidCollection and therefore shape detector performance. They are hand-picked, and no ablation shows how sensitive the reported F1 scores are to them.

assumptions (5)

domain assumption Human labels are correct for pre-copilot-era corpora and the cleaning is sufficient.
Section 3.6 explicitly acknowledges that human-written code collected after copilots became popular may contain AI-generated samples; the paper treats remaining labels as clean. The uncertainty resampling is intended to mitigate this, but the main results depend on this assumption.
domain assumption The English-language and parseability filters preserve the target distribution.
Section 3.6: samples are retained only if they parse and have English docstrings with >99% confidence; this may bias the dataset away from real-world code with mixed languages, but is treated as representative.
ad hoc to paper MC Dropout uncertainty on human-written code identifies mislabelled or ambiguous samples.
Section 5: the paper assumes that the top 7% uncertain human-written samples are the noisy ones; this is a heuristic without validation that the removed samples are actually mislabelled.
domain assumption The DPO-tuned 'humanised' models produce code that is representative of adversarial evasion.
Section 3.4: the adversarial attack is defined by the authors' own preference-tuning pipeline; the paper equates a detector failing on these samples with 'almost entirely unusable' against adversarial humanisation, which is only one family of attacks.
domain assumption Inverse instructions and persona-based tasks generate prompts representative of real user demands.
Section 3.2: inverse instructions derived from existing code and persona-based task generation are assumed to simulate real user-LM interactions; no external validation of this coverage is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of $\texttt{Droid}$: A Resource Suite for AI-Generated Code Detection." pith.science (2026). https://pith.science/paper/SCUJF5TW

@misc{pith2026250710583,
  author       = {Pith},
  title        = {Pith review of: $\textttDroid$: A Resource Suite for AI-Generated Code Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCUJF5TW}},
  note         = {Machine review of arXiv:2507.10583}
}

abstract

In this work, we compile $\textbf{$\texttt{DroidCollection}$}$, the most extensive open data suite for training and evaluating machine-generated code detectors, comprising over a million code samples, seven programming languages, outputs from 43 coding models, and over three real-world coding domains. Alongside fully AI-generated samples, our collection includes human-AI co-authored code, as well as adversarial samples explicitly crafted to evade detection. Subsequently, we develop $\textbf{$\texttt{DroidDetect}$}$, a suite of encoder-only detectors trained using a multi-task objective over $\texttt{DroidCollection}$. Our experiments show that existing detectors' performance fails to generalise to diverse coding domains and programming languages outside of their narrow training data. Additionally, we demonstrate that while most detectors are easily compromised by humanising the output distributions using superficial prompting and alignment approaches, this problem can be easily amended by training on a small amount of adversarial data. Finally, we demonstrate the effectiveness of metric learning and uncertainty-based resampling as means to enhance detector training on possibly noisy distributions.

Figures

Figures reproduced from arXiv: 2507.10583 by the authors.

**Figure 1.** Feature importances Model Truncation Length 128 256 512 GptSniffer 57.05 57.20 56.64 M4 59.69 53.10 51.13 CoDet-M4 72.28 70.62 61.68 DroidDetect-Base 91.90 96.25 99.18 DroidDetect-Large 94.91 98.31 99.25 [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗

**Figure 2.** Weighted F1-score comparison between mod [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗

**Figure 3.** DroidDetect-Base classifiers confusion matrix [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

MultiAIGCD: A Comprehensive dataset for AI Generated Code Detection Covering Multiple Languages, Models,Prompts, and Scenarios
cs.SE 2025-07 conditional novelty 6.0 of 10

A new multilingual benchmark dataset for AI-generated code detection, spanning three languages, six LLMs, three prompts, and three usage scenarios, with baseline detector evaluation.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages · cited by 1 Pith paper

[1]

AI, :, Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, and 14 others. 2025. Yi: Open Foundation Models by 01.AI. Preprint, arXiv:2403.04652. Mistral AI. 2025. Mistral Small – A new balance of perform...

arXiv 2025
[2]

b a s i c a l l y say i ti s 1−D

** Prompt Creation **: Write a clear and specific LLM prompt that , if provided to a language model , would generate code with similar functionality and structure . Enclose the LLM prompt within [ LLM_PROMPT ] and [/ LLM_PROMPT ] tags . Interaction will be in the following way : # ## INPUT : [ CODE ] {{ code }} [/ CODE ] # ## OUTPUT : [ SUMMARY ] {{ summa...

work page 2024
[3]

In Proceedings of the 21st In- ternational Conference on Mining Software Reposito- ries, MSR ’24, page 394–406, New York, NY , USA

Whodunit: Classifying Code as Human Authored or GPT-4 Generated - A case study on CodeChef problems. In Proceedings of the 21st In- ternational Conference on Mining Software Reposito- ries, MSR ’24, page 394–406, New York, NY , USA. Association for Computing Machinery. Daphne Ippolito, Daniel Duckworth, Chris Callison- Burch, and Douglas Eck. 2020. Automa...

work page 2020
[6]

Technical report, Center for Security and Emerging Technology

Cybersecurity Risks of AI-Generated Code. Technical report, Center for Security and Emerging Technology. Center for Security and Emerging Tech- nology. JianWang, Shangqing Liu, Xiaofei Xie, and Yi Li. 2024. An Empirical Study to Evaluate AIGC Detectors on Code Content. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software En- ...

arXiv 2024
[7]

OUTFOX: LLM-Generated Essay Detec- tion Through In-Context Learning with Adversarially Generated Examples. In Thirty-Eighth AAAI Con- ference on Artificial Intelligence, AAAI 2024, Thirty- Sixth Conference on Innovative Applications of Ar- tificial Intelligence, IAAI 2024, Fourteenth Sympo- sium on Educational Advances in Artificial Intelli- gence, EAAI 2...

work page 2024
[9]

In Proceedings of the 22nd An- nual Workshop of the Australasian Language Tech- nology Association, pages 197–202, Canberra, Aus- tralia

Overview of the 2024 ALTA Shared Task: De- tect Automatic AI-Generated Sentences for Human- AI Hybrid Articles. In Proceedings of the 22nd An- nual Workshop of the Australasian Language Tech- nology Association, pages 197–202, Canberra, Aus- tralia. Association for Computational Linguistics. Vijayaraghavan Murali, Chandra Shekhar Maddila, Imad Ahmad, Mich...

arXiv 2024
[11]

Enclose this summary within [ SUMMARY ] and [/ SUMMARY ] tags

** Code Summary **: Analyze the given code and summarize its purpose , logic , and functionality . Enclose this summary within [ SUMMARY ] and [/ SUMMARY ] tags

work page
[2019]

In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 2801–2807

The dangers of post-hoc interpretability: Un- justified counterfactual explanations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 2801–2807. ijcai.org. Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, and Zhaoxiang Zhang. 2023a. SheetCopilot: Bringing Softwa...

arXiv 2019

Show all 12 references

[2023]

In Ad- vances in Neural Information Processing Systems , volume 36, pages 53728–53741

Direct Preference Optimization: Your Lan- guage Model is Secretly a Reward Model. In Ad- vances in Neural Information Processing Systems , volume 36, pages 53728–53741. Curran Associates, Inc. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Bala- subramanian, Wenxiao Wang, and Soh...

2023 arXiv
[2024]

Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and Mark Chen

OpenReview.net. Mohammad Bavarian, Heewoo Jun, Nikolas Tezak, John Schulman, Christine McLeavey, Jerry Tworek, and Mark Chen. 2022. Efficient Training of Language Models to Fill in the Middle. CoRR, abs/2207.14255. A.Z. Broder. 1997. On the resemblance and con- tainment of doc...

2022 arXiv
[2025]

In The Thirteenth In- ternational Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025

TestGenEval: A real world unit test generation and test completion benchmark. In The Thirteenth In- ternational Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenRe- view.net. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks V . S. Lakshmanan....

2025
[2309]

Jessica Ji, Jenny Jun, Maggie Wu, and Rebecca Gelles

International Committee on Computational Linguistics. Jessica Ji, Jenny Jun, Maggie Wu, and Rebecca Gelles

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

AI, :, Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, and 14 others. 2025. Yi: Open Foundation Models by 01.AI. Preprint, arXiv:2403.04652. Mistral AI. 2025. Mistral Small – A new balance of perform...

arXiv 2025

[2] [2]

b a s i c a l l y say i ti s 1−D

** Prompt Creation **: Write a clear and specific LLM prompt that , if provided to a language model , would generate code with similar functionality and structure . Enclose the LLM prompt within [ LLM_PROMPT ] and [/ LLM_PROMPT ] tags . Interaction will be in the following way : # ## INPUT : [ CODE ] {{ code }} [/ CODE ] # ## OUTPUT : [ SUMMARY ] {{ summa...

work page 2024

[3] [3]

In Proceedings of the 21st In- ternational Conference on Mining Software Reposito- ries, MSR ’24, page 394–406, New York, NY , USA

Whodunit: Classifying Code as Human Authored or GPT-4 Generated - A case study on CodeChef problems. In Proceedings of the 21st In- ternational Conference on Mining Software Reposito- ries, MSR ’24, page 394–406, New York, NY , USA. Association for Computing Machinery. Daphne Ippolito, Daniel Duckworth, Chris Callison- Burch, and Douglas Eck. 2020. Automa...

work page 2020

[4] [6]

Technical report, Center for Security and Emerging Technology

Cybersecurity Risks of AI-Generated Code. Technical report, Center for Security and Emerging Technology. Center for Security and Emerging Tech- nology. JianWang, Shangqing Liu, Xiaofei Xie, and Yi Li. 2024. An Empirical Study to Evaluate AIGC Detectors on Code Content. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software En- ...

arXiv 2024

[5] [7]

OUTFOX: LLM-Generated Essay Detec- tion Through In-Context Learning with Adversarially Generated Examples. In Thirty-Eighth AAAI Con- ference on Artificial Intelligence, AAAI 2024, Thirty- Sixth Conference on Innovative Applications of Ar- tificial Intelligence, IAAI 2024, Fourteenth Sympo- sium on Educational Advances in Artificial Intelli- gence, EAAI 2...

work page 2024

[6] [9]

In Proceedings of the 22nd An- nual Workshop of the Australasian Language Tech- nology Association, pages 197–202, Canberra, Aus- tralia

Overview of the 2024 ALTA Shared Task: De- tect Automatic AI-Generated Sentences for Human- AI Hybrid Articles. In Proceedings of the 22nd An- nual Workshop of the Australasian Language Tech- nology Association, pages 197–202, Canberra, Aus- tralia. Association for Computational Linguistics. Vijayaraghavan Murali, Chandra Shekhar Maddila, Imad Ahmad, Mich...

arXiv 2024

[7] [11]

Enclose this summary within [ SUMMARY ] and [/ SUMMARY ] tags

** Code Summary **: Analyze the given code and summarize its purpose , logic , and functionality . Enclose this summary within [ SUMMARY ] and [/ SUMMARY ] tags

work page

[8] [2019]

In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 2801–2807

The dangers of post-hoc interpretability: Un- justified counterfactual explanations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 2801–2807. ijcai.org. Hongxin Li, Jingran Su, Yuntao Chen, Qing Li, and Zhaoxiang Zhang. 2023a. SheetCopilot: Bringing Softwa...

arXiv 2019