REVIEW 21 cited by
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs), including GPT-3.5, LLaMA, and PaLM, seem to be knowledgeable and able to adapt to many tasks. However, we still cannot completely trust their answers, since LLMs suffer from \textbf{hallucination}\textemdash fabricating non-existent facts, deceiving users with or without their awareness. However, the reasons for their existence and pervasiveness remain unclear. In this paper, we demonstrate that nonsensical prompts composed of random tokens can also elicit the LLMs to respond with hallucinations. Moreover, we provide both theoretical and experimental evidence that transformers can be manipulated to produce specific pre-define tokens by perturbing its input sequence. This phenomenon forces us to revisit that \emph{hallucination may be another view of adversarial examples}, and it shares similar characteristics with conventional adversarial examples as a basic property of LLMs. Therefore, we formalize an automatic hallucination triggering method as the \textit{hallucination attack} in an adversarial way. Finally, we explore the basic properties of attacked adversarial prompts and propose a simple yet effective defense strategy. Our code is released on GitHub\footnote{https://github.com/PKU-YuanGroup/Hallucination-Attack}.
Forward citations
Cited by 21 Pith papers
-
Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit
Security practitioners use LLMs independently for low-risk productivity tasks while showing interest in enterprise platforms, but reliability, verification needs, and security risks limit broader autonomy.
-
DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models
Monitoring attention entropy and image-output correlation during decoding, then applying targeted contrastive corrections, reduces hallucination in multimodal LLMs without retraining.
-
Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming
A dynamic red-teaming audit reports that 94% of MedQA-correct answers fail under adversarial mutation, with 86% privacy leak rates, 81% bias shift rates, and 66-74% hallucination rates across 15 medical LLMs.
-
Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems
LLM tool agents fail at parameter filling in five recurring ways; perturbing tool documents and user queries drives most failures, and invented parameter names are tied to the model rather than the input.
-
EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora
EraRAG uses hyperplane-based locality-sensitive hashing to build a hierarchical retrieval graph whose affected regions only are re-summarized when new documents arrive, cutting update cost by up to an order of magnitude.
-
Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
PhantomCircuit traces knowledge overshadowing to attention circuits during training and prunes circuit edges to recover the overshadowed answer.
-
Thinking beyond the anthropomorphic paradigm benefits LLM research
Anthropomorphic language and assumptions are common and growing in LLM research, and the authors propose a framework for moving beyond them while keeping what is useful.
-
Unleashing the Power of Large Language Model for Denoising Recommendation
LLaRD uses LLM-generated preference and relation knowledge plus an information-bottleneck objective to denoise implicit feedback and improve recommendation accuracy on Steam, Yelp, and Amazon-Book.
-
SeeMe: Mitigating Hallucinations in Large Vision-Language Models through Effective Visual Token Engineering
Restructuring visual tokens via cross-modal prune–merge–refine consistently lowers hallucination rates on MME, POPE and AMBER across four 7B LVLMs without any training.
-
How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributions
Attacks that break LLMs best are not the ones that improve safety most; a Shapley- and greedy-based framework that selects attack subsets by downstream defender utility outperforms attacker-centric and attribution-onl...
-
Investigating Symbolic Triggers of Hallucination in Gemma Models Across HaluEval and TruthfulQA
Symbolic triggers like modifiers and named entities keep Gemma hallucination rates at 64-79% across model scales, suggesting larger models do not eliminate this failure mode.
-
QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting
A contextual bandit that chooses among five query-rewrite strategies, conditioned on 17 linguistic features, reduces LLM hallucination on QA benchmarks and beats static prompting and no-rewrite baselines.
-
Large Language Model Agent for Structural Drawing Generation Using ReAct Prompt Engineering and Retrieval Augmented Generation
A six-stage LLM agent chain converts natural-language descriptions of three beam types into Python code for AutoCAD drawings, with per-step success rates between 77% and 100% over 100 runs.
-
Concept Incongruence: An Exploration of Time and Death in Role Playing
LLMs asked to role-play dead historical figures rarely abstain from answering post-death questions, and their factual accuracy drops due to poorly encoded death states and role-playing-induced shifts in temporal repre...
-
Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert-Like Systems Engineering Artifacts and a Characterization of Failure Modes
Closed-source LLMs can produce systems engineering artifact text that scores nearly identically to a human expert benchmark on MAUVE text similarity, but expert review of the best-scoring outputs reveals three failure...
-
Bridging the Gap: Leveraging Retrieval-Augmented Generation to Better Understand Public Concerns about Vaccines
A RAG-based tool that answers complex queries about vaccine concerns from social media, evaluated on 35,103 Shingrix posts with automated faithfulness and relevance scores.
-
LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation
A RAG pipeline with OCR, table and image to text conversion, and a RAFT-tuned reranker reports high QA scores, but its 50-question evaluation overlaps with its training manuals and its baseline comparison uses only 5 ...
-
Ensemble based approach to quantifying uncertainty of LLM based classifications
LLM classification uncertainty is estimated from vote distributions across paraphrased questions, but the method is not validated against baselines or held-out data.
-
Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs
A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.
-
From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Models
The paper argues hallucinations and jailbreaks share the same optimization dynamics and shows that defenses for one also reduce the other, but the theoretical support is largely circular.
-
Charting the Future of Scholarly Knowledge with AI: A Community Perspective
A community perspective on how AI can support scholarly knowledge extraction, organization, and communication, with a proposed classification and ethical considerations.
Discussion (0). Continue with ORCID to comment.