Pith. sign in

REVIEW 4 cited by

Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2506.12430 v2 pith:TNC2AVNF submitted 2025-06-14 cs.CR cs.CV

Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025

Zonghao Ying , Siyang Wu , Run Hao , Peng Ying , Shixuan Sun , Pengyu Chen , Junze Chen , Hao Du
show 39 more authors
Kaiwen Shen Shangkun Wu Jiwei Wei Shiyuan He Yang Yang Xiaohai Xu Ke Ma Qianqian Xu Qingming Huang Shi Lin Xun Wang Changting Lin Meng Han Yilei Jiang Siqi Lai Yaozhi Zheng Yifei Song Xiangyu Yue Zonglei Jing Tianyuan Zhang Zhilei Zhu Aishan Liu Jiakai Wang Siyuan Liang Xianglong Kong Hainan Li Junjie Mu Haotong Qin Yue Yu Lei Chen Felix Juefei-Xu Qing Guo Xinyun Chen Yew Soon Ong Xianglong Liu Dawn Song Alan Yuille Philip Torr Dacheng Tao
This is my paper
classification cs.CR cs.CV
keywords challengesafetyatlasadversarialattackscompetitionmllmmllms
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) have enabled transformative advancements across diverse applications but remain susceptible to safety threats, especially jailbreak attacks that induce harmful outputs. To systematically evaluate and improve their safety, we organized the Adversarial Testing & Large-model Alignment Safety Grand Challenge (ATLAS) 2025}. This technical report presents findings from the competition, which involved 86 teams testing MLLM vulnerabilities via adversarial image-text attacks in two phases: white-box and black-box evaluations. The competition results highlight ongoing challenges in securing MLLMs and provide valuable guidance for developing stronger defense mechanisms. The challenge establishes new benchmarks for MLLM safety evaluation and lays groundwork for advancing safer multimodal AI systems. The code and data for this challenge are openly available at https://github.com/NY1024/ATLAS_Challenge_2025.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents

    cs.CR 2025-10 unverdicted novelty 7.0

    SecureWebArena is a new benchmark suite for holistic security evaluation of LVLM-based web agents using diverse simulated environments, attack taxonomies, and multi-layered failure analysis across reasoning, behavior,...

  2. Before Agents Speak: Pre-hoc Failure Risk Inference in Multi-Agent Systems

    cs.CR 2026-07 conditional novelty 6.0

    HalluProp infers per-agent and system-level hallucination risk in multi-agent LLMs before interaction via role–query misalignment, topology-aware propagation, and differentiable Noisy-OR aggregation.

  3. A Patch-based Cross-view Regularized Framework for Backdoor Defense in Multimodal Large Language Models

    cs.CV 2026-04 unverdicted novelty 5.0

    A patch-augmented cross-view regularization method reduces backdoor attack success rates in multimodal LLMs by enforcing output differences between original and perturbed views while using entropy constraints to prese...

  4. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    cs.CV 2025-09 conditional novelty 5.0

    MSEA+ARC, a multi-scale and ranking-based residualization method, claims consistent F1-IoU gains over TAM for token-level MLLM visual attribution.