E-mail senden E-Mail Adresse kopieren
2026-11-15

Rag n' Roll:
A Framework to Evaluate Indirect Prompt Manipulation Attacks against RAG-based Applications

Zusammenfassung

Retrieval Augmented Generation (RAG) is a technique commonly used to equip LLMs with out-of-distribution knowledge by collecting, indexing, retrieving, and providing to the model specific information useful to answer user questions. The knowledge is often collected from public document repositories, providing a gateway for indirect prompt injections: attackers may craft documents specifically optimized to control the generated answer. In this paper, we investigate the practical performance of optimization techniques in hijacking the outputs of RAG pipelines. First, we review the frameworks commonly used to build RAG applications, derive a prototypical architecture, and identify critical parameters. We then examine prior work, seeking techniques for performing indirect prompt manipulations. We adopted five techniques across two macro-families and evaluated the criteria used to assess their effectiveness. With this knowledge, we implement Rag-n-Roll, a framework to evaluate the end-to-end results of LLM-based pipelines, introducing new metrics to better understand the benign or malicious nature of the generated answers. Our results show that established evaluation practices are insufficient to precisely assess attacks against RAG-based pipelines, leading to overestimates of their success. Furthermore, while attacks on the default configuration can successfully control up to 70% of the generated answers, we show that these performances depend heavily on previously ignored pipeline parameters. Finally, we distill our findings into actionable tips for developers looking to harden their RAG pipelines.

Konferenzbeitrag

ACM Workshop on Artificial Intelligence and Security (AISec)

Veröffentlichungsdatum

2026-11-15

Letztes Änderungsdatum

2026-11-16