{"id":"57d695dc-64af-4194-8500-da588715a3ab","arxiv_id":"2505.08204","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A large-scale study of Android apps found 70% of a 181-app sample had bypassable AI usage restrictions, and an automated tool discovered 120 exploitable apps in a 2,950-app scan.","lead":"This security study examined Android apps that use AI language models and found that most of them let attackers bypass the limits the app developer set on how the AI can be used. The researchers also built an automated tool that found 120 exploitable apps and generated scripts that could call the AI directly.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The '120 apps with unrestricted access' claim is not established: exploit verification only checks a trivia answer, and only 46 apps' LMs were jailbroken.","rationale":"The reader's weakest assumption is that the automated '120 unrestricted' claim depends on a narrow success criterion, and the reader's CONDITIONAL verdict reflects exactly the concern I identify. My stress-test adds specificity: the paper's own Section VII-C verification only checks for the presence of the trivia answer in script logs, and Section VIII reports that jailbreaking succeeded for only 46 apps. Therefore, for most of the 120 apps, the paper has verified only that a script can send a query and receive a response, not that the LM is free, unlimited, and topic-unrestricted. This is an internally supported concern, not a disagreement with consensus: the paper's manual findings (127/181) and the automated lower-bound are coherent, and the taxonomy and case studies are valuable. However, the abstract and contributions claim more than the automated verification supports. I would keep the CONDITIONAL verdict: the paper should be accepted only after the claims are re-labeled to match the verification (e.g., 'automatically generated scripts that reach the LM' with a separate count for full unrestricted access), the 65+61=126 versus 120 discrepancy is explained, and per-app breakdowns or artifacts are provided. No ad hominem is intended; this is a measurement-validity issue in the paper's central quantitative claim.","tokens_in":25504,"tokens_out":3526,"duration_ms":37362,"concrete_test":"Take the 120 apps for which LM-Scout claims working exploits and rerun each generated script under a three-probe battery: (1) an out-of-domain topic query (e.g., a binary-search request in a travel-focused app), (2) a harmful-content query from the Jailbreak Chat set used in Section V-B, and (3) a quota-exceeding sequence that sends more than the app's advertised free-query limit. Count how many scripts pass all three probes and additionally leak or expose the pre-prompt when one exists. If that count is materially below 120, the paper should relabel the result as 'endpoint reachable' or 'R-App bypassed' rather than 'unrestricted access,' and all prevalence statements should be reworded accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central automated result is that LM-Scout generated working scripts granting 'unrestricted access' to the LM for 120 of 2,950 apps, meaning 'free, unlimited, answering queries about any topic' (Contributions, Section I). The verification described in Section VII-C does not measure this property. An exploit is deemed successful if the script's logs contain the answer to a fixed trivia query ('Tell only in three words, the capital of Country A, Country B, Country C'). That only demonstrates that the script can reach and receive a response from the LM endpoint; it does not demonstrate that quota limits, topic restrictions, moderation, or pre-prompt protections are bypassed. The paper itself reports that, after the R-App exploit succeeds, LM-Scout 'proceeds to test the LM against R-LM' with jailbreak prompts and 'successfully jailbreaks LMs of 46 apps' (Section VIII). Thus, for roughly 74 of the 120 claimed apps, the paper has not shown that the LM answers out-of-domain or harmful queries, nor that response-length or quota limits are absent. The number 120 is also not reconciled with the reported breakdown: 65 static plus 61 dynamic scripts equals 126 script-generation successes, not 120. Because the phrase 'unrestricted access' is the headline claim, the measurement criterion is load-bearing: if verification had instead required out-of-topic queries, harmful-content queries, and quota-exceeding queries to succeed, the 120 number would likely shrink substantially. This is not an objection to the manual reconnaissance results or the taxonomy; it is a precise mismatch between the paper's strongest claim and its own verification procedure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies the security of language model (LM) integration in Android apps. It first manually analyzes 181 LM-using apps, develops a taxonomy of LM restrictions (quota, topic, moderation, proprietary-information protection), and reports that at least one restriction can be bypassed in 127 of those apps. The authors then build LM-Scout, an automated pipeline combining static analysis, LLM-guided dynamic UI interaction, network traffic analysis, and LLM-based exploit-script synthesis, and run it on 2,950 PlayStore apps. The paper's headline result is that LM-Scout automatically generates scripts granting 'unrestricted access' to the LM for 120 apps. The paper also reports root causes for the insecure integrations and recommends server-side enforcement of restrictions. A manual-follow-up experiment yields 18 additional scripts when the UI-interaction step is performed by a human.","tokens_in":25695,"tokens_out":9398,"duration_ms":90938,"significance":"The study addresses a timely and under-explored problem and provides one of the first large-scale measurements of LM-integration security in mobile apps. Its strengths include a clear threat model, a useful taxonomy, a substantial measurement infrastructure, and responsible-disclosure efforts that led to fixes in 16 apps. The manual reconnaissance data (127/181) is credible and the case studies are informative. However, the automated headline claim of 120 apps with 'unrestricted access' is currently not supported by the verification methodology: the script success criterion only checks a single benign trivia response, and the paper's own jailbreak testing succeeds for only 46 apps. If the claim is narrowed or re-verified, the paper would be a valuable contribution to the security literature; in its current form, the central quantitative claim overstates what is measured.","major_comments":[{"comment":"The verification criterion does not match the definition of 'unrestricted access' used in the paper. Section I defines unrestricted access as the ability to query any topic (including harmful topics), with no limits on query number or length, and to access proprietary data. Section VII-C states that an exploit script is considered successful when the answer to the fixed LM-query ('Tell only in three words, the capital of Country A, Country B, Country C') is present in the script logs, and that only after this does LM-Scout test the LM against jailbreak prompts. This criterion establishes only that the script can send a request to the LM endpoint and receive a response to one benign, common-knowledge question; it does not establish that quota limits, output-length limits, topic restrictions, moderation, or pre-prompt protections are bypassed. This gap is confirmed by the paper's own Section VIII, which reports successful jailbreaks for only 46 apps; for the remaining apps among the claimed 120, the 'any topic' and moderation components of 'unrestricted access' are unverified. The authors should either narrow the headline claim to 'automatically generated scripts that can access the LM endpoint' and report the 46-app jailbreak figure as a separate result, or extend the verification to include out-of-domain queries, harmful-content queries, and quota-exceeding queries.","section":"Section I; Section VII-C"},{"comment":"The paper reports 65 attack scripts from static analysis and 61 from dynamic analysis, totaling 126, yet the abstract and Section I state that LM-Scout found 120 vulnerable apps. The discrepancy of six is never explained. If some scripts were not verified or some apps appear in both the static and dynamic sets, that needs to be stated; as written, the headline count is arithmetically inconsistent with the supporting results.","section":"Abstract; Section VIII"},{"comment":"The paper describes a manual-interaction follow-up in which a human performs the app-interaction step for 40 apps and LM-Scout completes the rest automatically, yielding 18 additional working scripts. It is not stated whether these 18 are included in the 120 claimed in the abstract or are an additional result. Because the paper elsewhere emphasizes that LM-Scout is 'fully automated,' readers need to know whether the 120 count is purely automatic or includes human-assisted cases. Please clarify the inclusion criterion and, if these 18 are not included, state the combined total.","section":"Section VIII (additional experiment)"}],"minor_comments":[{"comment":"The captions for sub-figures (b) and (c) are identical ('An app utilizing LM-Server hosted by the developer by communicating through the App-Server'), but Section II-B describes (c) as a third-party LM-Server. Please correct the caption to match the text.","section":"Figure 1 caption"},{"comment":"The subcategory counts under Quota-R (52+13+115+68+6 = 254) exceed the reported total of 139; since apps can implement multiple restrictions, please add a note explaining that the categories are not mutually exclusive.","section":"Table II"},{"comment":"The dynamic-exploit breakdown (13 API-key leaks + 30 proprietary authentication frameworks + 12 Android authentication frameworks = 55) falls short of the reported 61 dynamic exploits; please account for the remaining six exploits.","section":"Section VIII"},{"comment":"For reproducibility, please specify the concrete countries used in the LM-query, rather than the placeholders 'Country A, Country B, Country C'.","section":"Section VII-B"},{"comment":"Minor typos and wording issues: 'underlining LM' should be 'underlying LM' (Sections I and XI); 'Sore' should be 'Store' (Section VI-A); 'relaying' should be 'relying' (Section X); 'folowing' should be 'following' (Section VIII); '11 apps that are performing' should be '11 apps that perform' (Section V-B).","section":"Various sections"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is likely to make a solid contribution after revision, but the gap between the claimed '120 unrestricted access' and the verification methodology needs to be addressed head-on. The reviews should ask specifically for either narrowed claims or extended verification. The 126-vs-120 arithmetic inconsistency is easy to fix and should be corrected. I would not reject on these grounds, as the underlying measurement study and manual taxonomy are valuable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jan,\n\nTwo things you should know. First, the manual reconnaissance part is genuinely solid and the taxonomy of LM restrictions (R-LM vs R-App, Quota/Topic/Mod/PIP) is new and useful. Second, the automated headline claim — 120 apps with 'unrestricted access' — is not backed by the paper's own verification, and the paper doesn't reconcile 65 static + 61 dynamic = 126 with 120. Treat the 120 as an upper bound or a script-generation count, not a proven count of unrestricted access.\n\nWhat the paper does well: it's the first systematic look at how Android apps integrate LMs and what restrictions they try to enforce. The manual analysis of 181 apps, with 127 bypassable, is coherent and includes nice case studies (client-side cryptography, abandoned endpoints, apps lying about which model they use). The taxonomy alone is a contribution. The LM-Scout pipeline is ambitious and the dynamic app-interaction work (Set-of-Marks style UI labeling, Monkey+LLM combo) is real engineering. The authors also did responsible disclosure, rate-limited queries, and report developer responses.\n\nWhere it's soft: Section VII-C's exploit verification only checks that the generated script gets an answer to a fixed trivia query ('capital of Country A/B/C'). That proves the script can talk to the LM endpoint. It doesn't prove the script bypassed quota, topic, moderation, or pre-prompt protections. The paper's own Section VIII says only 46 LMs were successfully jailbroken. So 'unrestricted access' for 120 apps is an overstatement; for ~74 of those apps, all the paper has shown is that the endpoint is reachable with a working token/script. The stress-test note is right about this. Also, 65 static + 61 dynamic = 126 generated scripts, not 120, and the paper never explains the gap. That's a checkable inconsistency. Finally, no artifacts: no code, dataset, or app list, so independent verification is impossible. The dataset filters (recent apps, >1000 installs, BERT description filter) also make the prevalence numbers ecosystem-specific; that's not fatal but should be framed as a lower bound on a biased sample.\n\nNone of this kills the manual findings or the taxonomy. It does mean the paper's strongest claim needs to be reworded and the verification strengthened — e.g., require out-of-topic, harmful, and quota-exceeding queries to succeed before saying 'unrestricted'. The authors should also provide a per-app breakdown and release artifacts.\n\nWho should read it: anyone working on mobile app security or LLM-integration security. It deserves a serious referee, despite the load-bearing verification gap, because the taxonomy and first large-scale evidence are valuable and the flaws are fixable in revision.\n\nRecommendation: send to peer review, but the reviewers should insist on the reconciliation and a narrower claim.","headline":"Good taxonomy and first large-scale measurement, but the headline '120 unrestricted access' claim is not supported by the paper's own verification, so treat the prevalence numbers as an upper bound until the discrepancy is resolved.","tokens_in":26366,"tokens_out":2099,"would_cite":true,"duration_ms":20341,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that 70% of Android apps using language models allow an attacker to bypass at least one restriction, and that the same access can be obtained automatically in 120 of 2,950 scanned apps.","keywords":["language model security","Android app security","LLM integration","jailbreak attacks","API key leakage","prompt leakage","mobile app tampering","LM-Scout"],"falsifier":"Re-run the automated pipeline on the same 2,950 apps with a different, independently validated jailbreak prompt set and with a stricter success criterion that requires demonstrating topic, moderation, and quota bypass separately; if the number of working exploit scripts drops to near zero, the claim that most LM integrations are insecure would be contradicted. Alternatively, an independent replication of the manual 181-app analysis that fails to reproduce the 127-app bypass count would directly test the central claim.","tokens_in":25253,"feed_emoji":"🔓","tokens_out":6817,"duration_ms":60311,"temperature":0.7,"pith_summary":"LM-Scout reports that the majority of Android apps that integrate language models do not securely enforce limits on how the model is used. In a manually analyzed sample of 181 apps, at least one restriction—query quotas, topic limits, content moderation, or secrecy of the model's hidden instructions—could be bypassed in 127 apps. Scaling the analysis to 2,950 recent apps, the paper's automated tool generated working scripts for 120 apps that give an attacker unrestricted access to the underlying language model. The result matters because app developers pay per query and per token; unrestricted access means financial loss, reputational abuse, and exposure of proprietary prompts. The paper identifies root causes—hard-coded credentials, client-side-only enforcement, and SDKs that require insecure integration—and argues for server-side enforcement as the fix.","feed_headline":"70% of Android AI apps let attackers bypass their safeguards","feed_subtitle":"A 2,950-app scan automatically produced working exploits for 120, granting free, unfiltered access to the model.","key_machinery":"The carrying mechanism is a two-layer analysis. First comes a taxonomy of restrictions: each restriction is classified by what it limits (Quota-R, Topic-R, Mod-R, PIP-R) and whether it is enforced inside the LM framework or inside the app framework. Second comes LM-Scout, an automated tool that combines static analysis of decompiled code, dynamic app interaction driven by a vision-based LLM and randomized exploration, network interception of the app's traffic, and LLM-assisted synthesis of a Python exploit script. The tool locates the LM endpoint, extracts authentication credentials or token-generation logic, and then tests whether the model itself can be jailbroken with crafted prompts. The exploit's success is verified by executing it and checking that the answer to a known question appears.","core_discovery":"The paper's central claim is that an adversary who can reverse-engineer an Android app and monitor or manipulate its network traffic can, in most cases, obtain unrestricted access to the language model that powers the app. Unrestricted means three things: the attacker can query any topic, including ones the developer intended to block; can exceed the app's limits on the number and length of queries; and can extract proprietary data, including the developer's hidden system prompt. The evidence is empirical: in 127 of 181 (70%) manually analyzed apps at least one restriction was bypassed, and the fully automated LM-Scout pipeline produced working exploit scripts for 120 of 2,950 scanned apps. The paper frames the vulnerability as a consequence of enforcing restrictions in the app itself, which a tampering attacker controls, rather than on the server.","pith_inferences":["If the root causes are platform-level rather than app-specific, the same vulnerability classes should appear in iOS apps, where apps also embed credentials and enforce limits client-side; a cross-platform scan would test that prediction.","The paper's automation (LLM-guided UI interaction plus network analysis) suggests that attackers can generalize the scraping step and harvest many endpoints cheaply; the cost asymmetry favors the attacker, so the paper's prevalence numbers may be a lower bound for real-world exploitation.","An immediate, testable extension is to re-run LM-Scout on the same or a fresh sample after the disclosure window: the 16 apps that reportedly added anti-tampering could be checked for whether server-side enforcement actually followed, or only obfuscation."],"forward_implications":["App-side enforcement of quotas, input length, and moderation is unreliable: a tampered app can skip or rewrite the checks, so any restriction that depends on the client can be bypassed.","Hard-coded LM API keys in Android apps are a direct path to unrestricted access; static scan of the decompiled code alone found 65 apps with exposed credentials.","Authentication frameworks, when used, are often misconfigured so that free-query tokens can be minted repeatedly, effectively removing payment walls.","The same integration flaws create an ecosystem-wide risk: a single insecure SDK can propagate the same vulnerability to every app that adopts it, as happened with a keyboard SDK used by 22 apps in the dataset.","Secure integration requires moving restrictions to the app's backend server and providing Android-specific LM SDKs that never expose credentials or prompts to the client."],"supporting_citations":[{"why":"Supplies the two jailbreak prompts (AIM and HYP) used to test moderation and topic restrictions, and the definition of a harmful response.","marker":"[21]"},{"why":"Prior empirical work on bypassing topic and prompt-leak restrictions in web apps; provides attack patterns and a per-app financial-loss estimate that motivate the Android study.","marker":"[6]"},{"why":"Report of LM API key theft in a web framework; source of the $46,000-per-day loss figure and the list of provider endpoints used for static fingerprinting.","marker":"[7]"},{"why":"The plugin framework shown as one LM integration architecture, contrasted with direct Android integration and analyzed as a separate scenario in the taxonomy.","marker":"[11]"},{"why":"The vision-based UI interaction agent that the paper adapts for automated dynamic app interaction in LM-Scout.","marker":"[53]"},{"why":"Set-of-mark prompting technique used to label UI screenshots so the vision model can identify interactive elements reliably.","marker":"[54]"},{"why":"The BERT-based classifier trained to identify LM-using apps from their Play Store descriptions and filter the large-scale dataset.","marker":"[33]"},{"why":"The Android app repository used as one source of candidate apps for manual and automated analysis.","marker":"[34]"}],"fun_headline_variants":["Most Android AI apps let attackers bypass restrictions","70% of Android AI apps have bypassable security restrictions","LM-Scout finds 120 Android apps with auto-exploitable LM flaws","Attacker can bypass safeguards in 70% of Android AI apps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The counts of vulnerable apps rest on a fixed set of jailbreak prompts (AIM and HYP, plus custom prompts) and on treating an app as exploitable when a generated script can send an arbitrary request and receive an answer; if a real attacker has stronger or weaker prompts than this set, or if bypassing one restriction is taken as 'unrestricted' access without proving that topic, moderation, and quota limits all fail, the prevalence numbers change.","fun_headline_variants_meta":{"raw":{"variants":["Most Android AI apps let attackers bypass restrictions","70% of Android AI apps have bypassable security restrictions","LM-Scout finds 120 Android apps with auto-exploitable LM flaws","Attacker can bypass safeguards in 70% of Android AI apps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000708,"raw_usage":{"total_tokens":3174,"prompt_tokens":918,"completion_tokens":2256,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":2186}},"tokens_in":534,"tokens_out":2256,"duration_ms":15152,"temperature":1.0,"reasoning_tokens":2186,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:01:01.316824+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the automated pipeline on the same 2,950 apps with a different, independently validated jailbreak prompt set and with a stricter success criterion that requires demonstrating topic, moderation, and quota bypass separately; if the number of working exploit scripts drops to near zero, the claim that most LM integrations are insecure would be contradicted. Alternatively, an independent replication of the manual 181-app analysis that fails to reproduce the 127-app bypass count would directly test the central claim.","supporting_citations":[{"cited_title":"Jailbreak chat,","cited_arxiv_id":null,"evidence_quote":"Supplies the two jailbreak prompts (AIM and HYP) used to test moderation and topic restrictions, and the definition of a harmful response."},{"cited_title":"Appagent: Multimodal agents as smartphone users,","cited_arxiv_id":null,"evidence_quote":"The vision-based UI interaction agent that the paper adapts for automated dynamic app interaction in LM-Scout."},{"cited_title":"Androzoo: Collecting millions of android apps for the research community,","cited_arxiv_id":null,"evidence_quote":"The Android app repository used as one source of candidate apps for manual and automated analysis."}],"review_version":1}