{"id":"daba68dc-7fe6-4e4c-a308-b6e539e2bd3d","arxiv_id":"2505.11449","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMs can already mine personal data for blackmail, find real vulnerabilities in obscure software, and automate in-browser and server-side attacks, changing the economics of cybercrime.","lead":"This paper argues that large language models will shift cyberattacks from broad, generic attacks toward cheap, personalized ones, such as tailored blackmail and long-tail software exploits. It backs this with proof-of-concept attacks on public datasets and real software, and predicts these attacks will become economical as LLM prices fall.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Long-tail exploitation is not demonstrated: §3.2 yields candidate bugs, not working exploits, contradicting §1's 'find and exploit' claim.","rationale":"The reader identified the cost-trend extrapolation as the weakest assumption; that is a real forecast risk. I find a more immediate, internal issue: the paper's own §3.2 contradicts §1 on whether LLMs can exploit long-tail software. This gap is loaded because the economic model counts vulnerability discovery and exploit development together, and the experiments only reduce the discovery term. The paper is transparent about the limitation and about the need for future capability gains, so the appropriate response is not rejection but a conditional acceptance requiring either an automated end-to-end exploit harness for at least some validated cases, or a downgrade of the 'imminently practical' / 'find and exploit' wording. The data-mining, authenticated-action, and password-logging PoCs stand independently and are valuable; the issue is specifically the scope of Direction 1. I therefore keep the reader's CONDITIONAL verdict unchanged.","tokens_in":26030,"tokens_out":9137,"duration_ms":97748,"concrete_test":"Automate the missing step: take the 19 human-validated vulnerabilities from §3.2, give a current frontier LLM agent the source code plus the LLM-generated vulnerability report, and require it to produce a working proof-of-concept exploit (e.g., a crafted image or config that triggers the XSS) with no human assistance. Measure the success rate and cost per working exploit. Also compute the §2.2 break-even using only measured costs (LLM discovery plus human validation/exploitation at current rates) and current API prices. If the agent cannot produce working exploits at low cost, revise Direction 1 from 'find and exploit' to 'find', and soften the §1 and abstract claims accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central economic argument for Direction 1 (long-tail exploitation) requires that LLMs reduce the combined cost of identifying a vulnerability and developing an exploit. Section 1 states LLMs 'can already find and exploit simple vulnerabilities in software with small user counts (see Section 3.2)', but Section 3.2.2 reports the opposite: when asked to produce proof-of-concept exploits, 'current models do not yet have the ability to achieve this step'. The 54 candidate vulnerabilities were manually processed by the authors; humans validated 19 as exploitable, and no LLM-generated exploit is shown. Thus the measured cost reduction applies only to the 'find' half of the equation in §2.2. The claim that the exploit-development cost will become 'near-zero' (Section 3.2.2) is an unmeasured capability forecast, not a result of this paper. As a consequence, even with the cited LLM price declines, a human-in-the-loop exploitation cost remains, and the headline conclusion that long-tail attacks are 'imminently practical' is stronger than the evidence. A parallel gap appears in §3.5, where the XSS orchestration was run by manually copying LLM function requests into a JS console rather than by an autonomous agent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper argues that large language models (LLMs) will change the economics of cyberattacks by lowering the cost of vulnerability discovery in long-tail software and by enabling per-victim attack customization. The authors formalize a toy economic model (Section 2.2) and present proof-of-concept experiments: PII extraction from Enron emails, audio, JFK files, LAION images, and personal photos (Section 3.1); automated auditing of 200 low-popularity Chrome extensions (Section 3.2); generation of phishing pages that mimic network devices (Section 3.3); autonomous actions on an authenticated Facebook account (Section 3.4); in-browser XSS orchestration (Section 3.5); and injection of password-logging code into five web server frameworks (Section 3.6). The paper concludes that these attacks, although mostly not yet profitable at current prices, will become economically efficient as LLM inference costs decline, and that this shift requires new defense-in-depth strategies.","tokens_in":26240,"tokens_out":7167,"duration_ms":63028,"significance":"Assuming the results hold, this is a timely and important contribution to the security community's understanding of LLM-enabled threats. The paper's strengths are its concrete proof-of-concept artifacts (public transcripts, prompts, and responsible-disclosure details), its use of public datasets for reproducibility, and its honest acknowledgment of limitations, including the current cost barriers and the absence of fully autonomous exploit generation. The paper also makes falsifiable predictions about inference-cost trajectories, which is a useful feature for future empirical work. However, the central claim that LLMs already lower the joint cost of finding and exploiting vulnerabilities is under-supported by the long-tail experiment, and the autonomy of the XSS demonstration is weaker than the surrounding prose suggests. These issues are correctable with more careful framing or additional experiments.","major_comments":[{"comment":"Section 1 states that LLMs 'can already find and exploit simple vulnerabilities in software with small user counts (see Section 3.2)', but Section 3.2.2 reports that when asked to produce proof-of-concept exploits, 'current models do not yet have the ability to achieve this step'. The 54 candidate vulnerabilities from the Chrome-extension study were manually validated by the authors, and 19 were confirmed exploitable; no LLM-generated working exploit is presented. Because the toy model in Section 2.2 treats 'cost to identify vulnerability and develop exploit' as a single term, the measured cost reduction covers only the discovery half, while the exploit-development half remains an unmeasured capability forecast. Please reconcile the Section 1 claim with this result, or supply evidence that the exploit-development cost is also reduced by LLMs.","section":"Section 1 vs. Section 3.2.2"},{"comment":"The XSS orchestration experiment is performed by manually copying the LLM's function requests into a browser JavaScript console and returning the results to the LLM. This human-in-the-loop setup contradicts the Section 3.5 claim that 'LLMs bypass these defenses because they can autonomously run an attack directly on the exploited machine'. The demonstration shows that an LLM can produce a plausible sequence of DOM operations, but not that an autonomous agent can perform the attack end-to-end; the feasibility of replacing the human with an automated harness is an unverified assumption. In addition, the e-banking application used in the experiment 'is not actually vulnerable to such XSS attacks' because of a 2-factor confirmation, so the end-to-end attack would not succeed in the described setting. Please either add an automated agent experiment or weaken the autonomy and practicality claims.","section":"Section 3.5.1"},{"comment":"The paper's conclusion that the demonstrated attacks are 'likely to become economically efficient in the near future' is based on an external extrapolation of LLM inference-cost reductions (9x-900x over three years, citing Epoch [15] and a16z [2]). None of the paper's experiments measure this trajectory, and no sensitivity analysis is provided for the case where price-performance improvements slow. Since several of the attacks are presently uneconomical (e.g., $270 for 200 Chrome extensions; $60-$100 for the Enron PII task), the central 'will soon alter the economics' thesis rests on an unvalidated forecast. Please state this dependence explicitly and discuss how the conclusions would change under slower cost declines.","section":"Section 8 / Section 2.2"},{"comment":"The cost-benefit comparison for LLM-based PII extraction assumes 'the cost of a human filtering out false positives is zero'. This assumption is not justified and materially affects the break-even values ($0.40 per password, $4.48 per credit card) that the paper uses to argue LLMs can be economically valuable for PII mining. If human review of LLM outputs has nonzero cost, the advantage over the $1.50 traditional-DLP baseline narrows or disappears at current prices. Please provide a sensitivity analysis with respect to human-review cost or support the assumption.","section":"Section 3.1.1"}],"minor_comments":[{"comment":"The phrase 'patched patched the extension' should be corrected to 'patched the extension'.","section":"Section 3.2.2"},{"comment":"The typo 'DCHP modification' should be 'DHCP modification'.","section":"Section 3.3"},{"comment":"In the threat-model discussion, 'resource they could need' should be 'resources they could need'.","section":"Section 2"},{"comment":"The contributions section says Edoardo performed the password-stealing experiments 'in Section 4.2', but the experiments appear in Section 3.6; please update the cross-reference.","section":"Contributions"},{"comment":"Reference [61] is typeset as 'Tomekcce et al.' while the body text writes 'Tömekçe'; please unify the spelling.","section":"References"},{"comment":"The footnote contains the typo 'we one should not read'; it should read 'one should not read'.","section":"Section 3.1.1"}],"recommendation":"major_revision","confidential_remarks":"This manuscript comes from well-known authors and is likely to attract significant attention; the editorial process should encourage the authors to tone down the 'imminently practical' language in the abstract and Section 1 unless the human-in-the-loop gaps are addressed. The responsible-disclosure practices are commendable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth reading. The breadth-versus-depth economic framing is the real contribution: LLMs make personalized attacks cheaper and also cut the fixed cost of finding bugs in long-tail software. That lens organizes several attack trends that were previously discussed separately, and several proof-of-concept results are genuinely new. The autonomous extraction of SSNs from handwritten JFK documents, the semantic mining of Enron emails that surfaces an extramarital affair, and the single-prompt password logger that works across five different server frameworks are all concrete and reproducible in spirit. The Chrome extension study is also valuable: 19 real vulnerabilities found in low-popularity extensions is a meaningful data point, not a toy.\n\nThe paper is also unusually transparent. It reports costs, gives prompts and transcripts, discusses responsible disclosure, and openly states where automation failed. That honesty is a real strength.\n\nThe soft spots are real but mostly fixable. The biggest one is the gap between the abstract's promise and the long-tail experiment. Section 1 says LLMs can already \"find and exploit\" simple vulnerabilities, but Section 3.2.2 says the model could not produce proof-of-concept exploits and that the authors manually validated the 54 candidates down to 19. So the measured cost reduction applies to finding candidates, not to producing working exploits. The claim that exploit-development cost becomes \"near-zero\" is an unmeasured forecast, not a result. The XSS orchestration in Section 3.5.1 has a similar issue: the LLM's function calls were manually copied into a JavaScript console, so the demonstration is of the reasoning, not of an autonomous agent. Both are acknowledged in the paper, but the abstract and introduction still push an \"imminently practical\" reading that the evidence does not quite support.\n\nThe economic conclusion also depends on continued LLM inference cost declines, cited from Epoch and a16z. That is a plausible external trend, but it is an extrapolation. If price-performance stalls, the demonstrated attacks stay too expensive to scale today. The paper's own framing as a forecast, rather than a measurement, would be easier to defend.\n\nNone of this sinks the paper. The individual demonstrations stand on their own, and the framing is worth taking seriously. I would send it to peer review and then ask for a revised version that aligns the headline claims with the evidence: say \"candidate vulnerability discovery\" where that is what happened, soften \"imminently practical\" to \"conditional on cost trends,\" and ideally release the XSS harness and prompts so the autonomy claim can be checked. The citation pattern looks fine. For security researchers and anyone thinking about AI-enabled threats, this is a useful paper to engage with.\n\nSend it out. The overclaims are fixable and the core content merits referee time.","headline":"A real, well-transparent set of LLM attack demos wrapped in a compelling economics frame, but the 'imminently practical' long-tail claim overstates what Section 3.2 actually shows.","tokens_in":26789,"tokens_out":1843,"would_cite":true,"duration_ms":21931,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLMs will make per-victim tailored cyberattacks cheap enough to be profitable.","keywords":["LLM-enabled cyberattacks","exploit economics","long-tail software vulnerabilities","targeted blackmail","ransomware monetization","personal information extraction","LLM inference cost trends","defense-in-depth"],"falsifier":"Watch the price of running a fixed capability level — a model that matches today's performance at finding passwords in email and auditing small applications — over the next three years while holding measured yields constant. If that price stops falling, or falls far more slowly than the 9x to 900x per three years the paper cites, then the extrapolation that these attacks become 'economically efficient in the near future' is falsified; the capability demonstrations would stand, but the monetization claim would not.","tokens_in":25832,"feed_emoji":"🔓","tokens_out":12470,"duration_ms":106164,"temperature":0.7,"pith_summary":"Large language models, the paper argues, will change the economics of cyberattacks by removing the need for expert humans in the loop. Today an attacker either strikes broadly with generic ransomware or spends heavily on one high-value target; LLMs, the authors claim, permit both at once — finding easy bugs in software with few users, then customizing how each infected device is monetized. The paper demonstrates this with proof-of-concept attacks using current frontier models: an unassisted model recovers blackmail-worthy secrets from a public corporate email corpus, identifies real vulnerabilities in obscure Chrome extensions, rewrites web-server code to steal passwords, and orchestrates a simulated bank transfer through a browser flaw. Most of these attacks are still too expensive to scale widely today, but the authors argue that falling LLM prices will flip the profitability calculation, so defenders should plan for a world in which ordinary users, not just high-value targets, are plausible victims.","feed_headline":"LLMs make per-victim cyberattacks profitable","feed_subtitle":"Finding easy bugs in obscure software and tailoring ransoms per victim makes once-unprofitable attacks pay off.","key_machinery":"The load-bearing mechanism is the paper's toy economic equation for a financially motivated adversary: $$\\text{value} = (\\text{profit per exploit}) \\times (\\#\\text{ impacted}) - (\\text{cost to identify vulnerability and develop exploit}).$$ In the pre-LLM world this identity forces a choice between depth and breadth: the fixed cost of finding and weaponizing a bug must be amortized over many victims, so attackers target popular software and monetize in generic ways. LLMs press on both terms at once — they drive the discovery cost low enough that even a few hundred victims can justify an exploit, and they raise the profit per exploit by letting malware personalize what it steals and what it demands. Each case study in the paper is organized around showing that one of these two terms is now within an LLM's reach.","core_discovery":"The paper's central claim is that LLMs commodify the adaptive 'intelligence' needed to understand and act on arbitrary data, and that this changes both sides of the exploit equation. On the discovery side, a model can read source code and find simple vulnerabilities in products with only hundreds or thousands of users — software never worth a human attacker's fixed cost — so the long tail of applications becomes a viable target. On the monetization side, a model that can read a victim's emails, photos, and audio can identify the personally most damaging secret and tailor a ransom, replacing the lowest-common-denominator ransomware model. The authors support this with measured demonstrations: an LLM finds twice as many passwords in the Enron emails as a commercial data-loss-prevention tool, at higher precision; it independently concludes that one employee is having an extramarital affair; it flags 54 of 200 audited low-usage Chrome extensions as vulnerable, with 19 confirmed exploitable; and one prompt makes it add password-exfiltrating code to five different web-server frameworks. The conclusion is that it is no longer a question of whether models have the capability, but whether profit exceeds cost — and cost is falling.","pith_inferences":["A natural extension the paper does not compute is the cost curve: measure the price per confirmed vulnerability as a function of model capability and API price, and locate the crossover where auditing the long tail of an application category becomes cheaper than the expected payout; the Chrome-extension study supplies one data point ($270 for 200 extensions, 3 high and 16 medium severity confirmed","The same logic is dual-use: if attackers can cheaply audit neglected software, so can defenders, which suggests automated security review of unmaintained open-source packages could become viable at a fraction of human cost — a consequence the authors mention only in passing.","If LLM inference prices plateau instead of continuing their recent decline, the capability demonstrations stand on their own but the monetization timeline stretches; the case for new defenses does not depend on the extrapolation, only its urgency does."],"forward_implications":["The depth-versus-breadth trade-off that has structured cybercrime for decades erodes: attacks can be both wide and tailored, so ordinary users face threats previously reserved for high-value targets.","Software with small user bases — browser extensions, niche web applications, uncommon network devices — becomes an attractive target, because the fixed cost of finding a bug no longer dominates the economics.","Ransomware's business model shifts from encryption toward targeted blackmail: the value of a compromised device becomes whatever sensitive material the model finds on it, and the ransom demand can be set per victim.","Defense-in-depth must be rethought: defenses aimed at generic attacks, such as monitoring disk writes for encryption or blocking cookie exfiltration, lose value, while detecting on-device LLM inference and large-scale information mining becomes central.","Because LLM prices fall while human labor does not, the authors expect these attacks to cross the profitability threshold in the near future, so the threat is a matter of timing rather than of missing capability."],"supporting_citations":[{"why":"Supplies the toy economic model — value equals profit per exploit times number impacted minus development cost — that frames the entire argument.","marker":"[41]"},{"why":"Provides the price-trend data (9x to 900x cost reductions over three years) that supports the claim that attacks become economically efficient.","marker":"[15]"},{"why":"Supports the assertion that LLM inference costs are falling rapidly, the load-bearing extrapolation for future profitability.","marker":"[2]"},{"why":"The classical data-loss-prevention baseline whose precision and recall the LLM is measured against in the PII-mining experiments.","marker":"[27]"},{"why":"The Enron email dataset used as the testbed for syntactic and semantic secret mining, including the affair discovery.","marker":"[40]"},{"why":"The Enron audio corpus whose transcripts the multimodal model mines for incriminating business content.","marker":"[1]"},{"why":"The public web-scale image collection used to show generalized recovery of sensitive financial information from personal photos.","marker":"[53]"},{"why":"The OCR baseline that the LLM outperforms at recovering social security numbers from scanned government documents.","marker":"[36]"},{"why":"Gives the average XSS bounty the authors compare against to argue that long-tail exploit economics are shifting.","marker":"[4]"}],"fun_headline_variants":["LLMs make targeted cyberattacks cheap and easy","LLMs find easy bugs in niche software for profit","LLMs tailor ransoms to each victim's secrets","LLMs turn blackmail and exploits into automated tools","LLMs shift hacking toward long-tail targets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion that these attacks will soon be economically efficient rests on the forecast that LLM inference prices will keep falling at their recent rate (the paper cites 9x to 900x reductions in three years); if that price-performance curve stalls, the demonstrated attacks remain too expensive to scale widely today.","fun_headline_variants_meta":{"raw":{"variants":["LLMs make targeted cyberattacks cheap and easy","LLMs find easy bugs in niche software for profit","LLMs tailor ransoms to each victim's secrets","LLMs turn blackmail and exploits into automated tools","LLMs shift hacking toward long-tail targets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000619,"raw_usage":{"total_tokens":2909,"prompt_tokens":1021,"completion_tokens":1888,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":1813}},"tokens_in":637,"tokens_out":1888,"duration_ms":12679,"temperature":1.0,"reasoning_tokens":1813,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:52:44.303640+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Watch the price of running a fixed capability level — a model that matches today's performance at finding passwords in email and auditing small applications — over the next three years while holding measured yields constant. If that price stops falling, or falls far more slowly than the 9x to 900x per three years the paper cites, then the extrapolation that these attacks become 'economically efficient in the near future' is falsified; the capability demonstrations would stand, but the monetization claim would not.","supporting_citations":[{"cited_title":"On the economics of ran- somware","cited_arxiv_id":null,"evidence_quote":"Supplies the toy economic model — value equals profit per exploit times number impacted minus development cost — that frames the entire argument."},{"cited_title":"LLM inference prices have fallen rapidly but unequally across tasks","cited_arxiv_id":null,"evidence_quote":"Provides the price-trend data (9x to 900x cost reductions over three years) that supports the claim that attacks become economically efficient."},{"cited_title":"Sensitive data protection pricing","cited_arxiv_id":null,"evidence_quote":"The classical data-loss-prevention baseline whose precision and recall the LLM is measured against in the PII-mining experiments."},{"cited_title":"The Enron corpus: A new dataset for email classification research","cited_arxiv_id":null,"evidence_quote":"The Enron email dataset used as the testbed for syntactic and semantic secret mining, including the affair discovery."},{"cited_title":"LAION-5B: An open large-scale dataset for training next generation image-text models","cited_arxiv_id":null,"evidence_quote":"The public web-scale image collection used to show generalized recovery of sensitive financial information from personal photos."},{"cited_title":"Tesseract: an open-source optical character recognition engine","cited_arxiv_id":null,"evidence_quote":"The OCR baseline that the LLM outperforms at recovering social security numbers from scanned government documents."},{"cited_title":"Bug bounty hunters earned over $4m for xss flaws reported via hackerone in 2020","cited_arxiv_id":null,"evidence_quote":"Gives the average XSS bounty the authors compare against to argue that long-tail exploit economics are shifting."}],"review_version":1}