REVIEW 6 cited by
ShieldGemma 2: Robust and Tractable Image Content Moderation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce ShieldGemma 2, a 4B parameter image content moderation model built on Gemma 3. This model provides robust safety risk predictions across the following key harm categories: Sexually Explicit, Violence \& Gore, and Dangerous Content for synthetic images (e.g. output of any image generation model) and natural images (e.g. any image input to a Vision-Language Model). We evaluated on both internal and external benchmarks to demonstrate state-of-the-art performance compared to LlavaGuard \citep{helff2024llavaguard}, GPT-4o mini \citep{hurst2024gpt}, and the base Gemma 3 model \citep{gemma_2025} based on our policies. Additionally, we present a novel adversarial data generation pipeline which enables a controlled, diverse, and robust image generation. ShieldGemma 2 provides an open image moderation tool to advance multimodal safety and responsible AI development.
Forward citations
Cited by 6 Pith papers
-
SenBen: Sensitive Scene Graphs for Explainable Content Moderation
SenBen is the first large-scale scene graph benchmark for sensitive content, paired with a 241M distilled model that outperforms most VLMs and safety APIs on grounded detection while running much faster.
-
Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models
A demographically diverse annotation dataset shows that safety perceptions for text-to-image outputs vary by rater identity and that conventional safety classifiers under-detect bias harms flagged by minority-group raters.
-
Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation
Seven model-agnostic image transformations bypass OpenAI, Amazon, and Google image-moderation APIs, including under non-trivial perceptual-similarity constraints.
-
Every Sample Counts: Supervised Fine-Tuning of Language Models with Pointwise Constraints
Pointwise constrained fine-tuning via sample-wise augmented Lagrangians and learned relaxations reduces tail constraint violations across safety, tool-calling, and re-ranking while preserving average task performance.
-
VLMs Can Aggregate Scattered Training Patches
Open-source VLMs can infer image IDs or safety labels after training only on scattered patches of those images, a capability that can be abused to bypass image moderation.
-
Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Values
An external 'superego' module that filters agentic AI plans against user-selected 'constitutions' plus a universal safety floor is reported to cut harmful outputs by up to 98% on safety benchmarks.
Discussion (0). Continue with ORCID to comment.