{"id":"ea78664a-72cf-44c8-b481-b2ad3f51a001","arxiv_id":"2606.30373","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"First systematic security analysis of AI-Apps on pre-trained model hubs identifies five threat categories, ten attack vectors, three novel architectural flaws, and real-world prevalence of credential leaks and injection risks across 970k+ apps.","lead":"The paper performs the first systematic security analysis of AI-powered apps on model hubs like Hugging Face, mapping their lifecycle to OWASP categories and scanning over 970,000 apps for vulnerabilities such as credential leaks and code injection. Smart generalists should read it to grasp how platform design choices create exploitable attack surfaces in the expanding AI ecosystem.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Insightor accuracy unvalidated at scale; no reported false-positive rates or ground-truth checks","rationale":"The reader's weakest assumption directly identifies the empirical bottleneck. Because the full text is referenced but the scanner methodology and validation details remain the unverified link between the 970k corpus and the reported vulnerability counts, the concern is unchanged. No other internal inconsistency in the threat taxonomy or disclosure process rises to the same load-bearing level.","tokens_in":1761,"tokens_out":289,"duration_ms":14611,"concrete_test":"Select 200 apps flagged by Insightor (50 per vulnerability class) plus 200 unflagged controls; have two independent reviewers manually inspect for the claimed issues using the paper's definitions; compute precision and recall. If precision falls below 70% on any class, the scale claims are unreliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline quantitative claims (thousands of credential leaks, hundreds of input-injection cases, tens of backdoors across 970k apps) rest entirely on Insightor. The paper maps threats to OWASP categories and describes three novel architectural issues, but the scale findings require the scanner to have low false positives and high recall on real AI-App code. No section supplies manual validation of a sample, precision/recall numbers, or comparison against a labeled subset, so the reported counts cannot be distinguished from scanner artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents the first systematic security analysis of AI-powered applications (AI-Apps) on pre-trained model hubs such as Hugging Face. It maps the AI-App lifecycle to OWASP risk taxonomies, identifying five threat categories and ten attack vectors. The authors develop the Insightor analysis framework, apply it to over 970,000 public AI-Apps, and report thousands of credential leaks, hundreds of input-injection vulnerabilities allowing arbitrary code execution, and tens of embedded backdoors. They also uncover three novel architectural vulnerabilities inherent to platform design and note amplification of traditional issues (e.g., world-readable logs). Responsible disclosure to affected platforms and developers is stated.","tokens_in":1880,"tokens_out":475,"duration_ms":28512,"significance":"If the scanner-based findings hold after validation, the work would highlight substantial real-world security risks in the emerging AI-App ecosystem on model hubs, including novel platform-inherent issues and scaled traditional flaws. The large-scale empirical scan (970k apps) combined with OWASP structuring and responsible disclosure could inform platform hardening and developer practices. The absence of self-referential derivations or fitted parameters is a strength for direct observational claims.","major_comments":[{"comment":"Section describing the Insightor framework and its application to the 970k-app corpus: the headline quantitative claims (thousands of credential leaks, hundreds of input-injection cases, tens of backdoors) rest entirely on Insightor detections, yet no false-positive rates, precision/recall figures, ground-truth sample validation, or manual confirmation of any detected issues are reported. This directly undermines the ability to distinguish reported counts from scanner artifacts.","section":"Insightor description and results sections"},{"comment":"Results reporting the scale findings: without any described validation procedure (e.g., manual review of a random subset or comparison against labeled AI-App code), the concrete counts cannot be assessed for reliability and are load-bearing for the central claim of 'active exploitation' and 'critical failures'.","section":"Results section on empirical findings"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":"The soundness concern is isolated to scanner validation; if addressed with concrete metrics and a validation subsection, the manuscript could be suitable for a security venue. No other internal inconsistencies noted."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the validation of our empirical results. We agree that explicit validation is necessary to support the quantitative claims and will revise the manuscript accordingly.","responses":[{"response":"We acknowledge that the submitted manuscript did not report false-positive rates or a formal validation procedure for Insightor. The framework employs rule-based and static analysis techniques (credential regex patterns, taint tracking for injection sinks, and signature matching for backdoors). In the revision we will add a dedicated validation subsection describing a manual review process on a random sample of detections across categories, including observed precision and any false positives encountered. This will be cross-referenced from the Insightor description.","revision_made":"yes","referee_comment":"Section describing the Insightor framework and its application to the 970k-app corpus: the headline quantitative claims (thousands of credential leaks, hundreds of input-injection cases, tens of backdoors) rest entirely on Insightor detections, yet no false-positive rates, precision/recall figures, ground-truth sample validation, or manual confirmation of any detected issues are reported. This directly undermines the ability to distinguish reported counts from scanner artifacts."},{"response":"We agree and will incorporate the validation results directly into the results section. The revised text will qualify the reported counts with the sampling-based precision estimates and describe the validation methodology so that reliability can be evaluated.","revision_made":"yes","referee_comment":"Results reporting the scale findings: without any described validation procedure (e.g., manual review of a random subset or comparison against labeled AI-App code), the concrete counts cannot be assessed for reliability and are load-bearing for the central claim of 'active exploitation' and 'critical failures'."}],"tokens_in":1464,"tokens_out":381,"duration_ms":28242,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main things to know are that the paper maps AI app risks on model hubs to OWASP categories and flags three platform-specific architectural issues that seem worth examining, while the large-scale prevalence numbers depend on a tool whose accuracy is not shown.\n\nThey structure the lifecycle of these apps, identify five threat categories and ten vectors, and point out how broken access control, insecure reuse, and data exposure show up in this setting. The claim that some problems are amplified by the way hubs share models and allow inference without strong isolation is the part that feels new. Responsible disclosure is the right call.\n\nThe soft spot is the measurement. The abstract reports thousands of credential leaks, hundreds of injection cases, and tens of backdoors from Insightor, but supplies no false-positive rates, no manual validation on samples, and no ground-truth checks. Without that, the concrete counts cannot be separated from possible scanner errors. The architectural analysis does not need the same scale data to stand, but the overall impact claim does.\n\nThis paper is for people who work on securing shared ML platforms or third-party model services. A reader looking for an initial taxonomy and examples of hub-level design choices will get something from it. It deserves a serious referee because it is the first systematic attempt at this scope, even if the empirical section needs more supporting detail on how the scanner was tested.","headline":"The architectural vulnerabilities look like a real first look at platform risks, but the 970k-app counts rest on an unvalidated scanner with no reported checks.","tokens_in":2410,"tokens_out":352,"would_cite":false,"duration_ms":21441,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"AI-Apps on model hubs exhibit broken access control, input injection, and credential leaks that enable code execution and backdoors.","keywords":["AI security","model hubs","AI-Apps","access control","input injection","credential leaks","backdoors","platform vulnerabilities"],"falsifier":"A manual audit of a random sample of the apps flagged by the framework to verify the reported credential leaks, injection points, or backdoors, or confirmation that the platforms have not addressed the disclosed issues.","tokens_in":2670,"feed_emoji":"🔐","tokens_out":642,"duration_ms":25132,"temperature":0.7,"pith_summary":"The paper performs the first systematic security analysis of AI-powered Applications hosted on pre-trained model platforms. It maps the AI-App lifecycle to risk taxonomies to identify five threat categories and ten attack vectors. The work reveals critical failures such as broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure, along with three novel architectural vulnerabilities tied to platform design. Applying a custom scanner to over 970,000 public AI-Apps shows thousands leaking credentials, hundreds permitting arbitrary code execution via input injection, and tens containing embedded backdoors.","feed_headline":"Thousands of AI-Apps on hubs leak credentials and allow code execution","feed_subtitle":"Analysis of over 970k public apps reveals broken controls and three novel architectural flaws enabling backdoors and arbitrary execution.","key_machinery":"The AI-App lifecycle mapped to OWASP-style risk taxonomies that surfaces five threat categories and ten attack vectors, including three novel architectural vulnerabilities.","core_discovery":"The central claim is that AI-Apps on leading platforms suffer from broken access control, insecure resource reuse, insufficient input validation, sensitive data exposure, and three novel architectural vulnerabilities inherent to the platform design; these issues are amplified from traditional web problems and appear at scale, with thousands of apps leaking credentials, hundreds enabling code execution, and tens harboring backdoors.","pith_inferences":["The amplification of generic web flaws in this ecosystem may require platform-specific security models rather than standard web hardening alone.","Similar vulnerabilities could affect other AI hosting services that allow public custom apps with shared resources.","Extending detection to private or fine-tuned apps on the same platforms would test whether the issues are limited to public listings."],"forward_implications":["Platforms must strengthen isolation and access controls for user-developed AI-Apps.","Traditional issues such as world-readable logs become high-impact when combined with AI-App execution environments.","Responsible disclosure of the identified issues can lead to platform-level fixes for the affected apps.","The scale of findings indicates that untrusted third-party AI-Apps pose immediate risks to users performing inference or fine-tuning."],"fun_headline_variants":["AI-Apps on Hubs Leak Credentials Allow Code Execution","Thousands of AI-Apps Expose Credentials via Broken Controls","Novel Flaws Allow Backdoors in Pre-Trained Model AI-Apps","Input Injection Hits Hundreds of Public AI Applications","Sensitive Data Exposed in AI Apps on Leading Platforms"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The custom analysis framework accurately detects credential leaks, input injection vulnerabilities, and embedded backdoors across 970,000 public AI-Apps at scale without significant false positives or missed cases.","fun_headline_variants_meta":{"raw":{"variants":["AI-Apps on Hubs Leak Credentials Allow Code Execution","Thousands of AI-Apps Expose Credentials via Broken Controls","Novel Flaws Allow Backdoors in Pre-Trained Model AI-Apps","Input Injection Hits Hundreds of Public AI Applications","Sensitive Data Exposed in AI Apps on Leading Platforms"]},"model":"grok-4.3","cost_usd":0.004349,"raw_usage":{"total_tokens":2192,"prompt_tokens":690,"num_sources_used":0,"completion_tokens":77,"cost_in_usd_ticks":43487000,"prompt_tokens_details":{"text_tokens":690,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1425,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":690,"tokens_out":77,"duration_ms":14852,"temperature":1.0,"reasoning_tokens":1425,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T05:19:24.677499+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A manual audit of a random sample of the apps flagged by the framework to verify the reported credential leaks, injection points, or backdoors, or confirmation that the platforms have not addressed the disclosed issues.","supporting_citations":[],"review_version":1}