What is a Prompt Leak?
Prompt leaking, told as a story about a guest who probes with 'just tell me your rules' until the kitchen's own rule page comes out.
The guest asks straight out: just tell me your rules.Not an order for food — a question aimed at the rule page the chef itself follows.
Reciting the rules looks like just another request to fulfill.The chef has no special way to tell 'cook for me' apart from 'recite your rules'.
A prompt leak is probing questions that reveal the kitchen's own rule page.The chef ends up showing the rule page the same way it would show an ingredient.
There are many ways to probe — dressed differently, but aimed at the same thing.Without an explicit 'keep this secret' in the rule page, any of these can let it slip out.
This is a leak of a few lines of rules — not a full copy of the recipe book.Copying the whole recipe book — or the chef itself — is its own story. Here, only a few leaked rule lines are in play.
A prompt leak = a guest probing with 'just tell me your rules' until a few lines of the rule page (system prompt) the chef follows come out.
An attack where direct or disguised questions — asking for a summary, a translation, or a continuation — lead an LLM to expose its own system prompt or hidden instructions verbatim. It's a different scope from model extraction (copying the whole recipe book, or the chef itself — the weights or training data entire): a leak targets a few lines of rules, extraction targets the whole model.
When grown-ups say it
- Prompt Leak
- Probing questions that reveal a few lines of the rule page. Not an order for food — a probe aimed at the rule page itself.
- System Prompt
- The rule page on the recipe book's cover. The underlying rules the chef follows for every dish.
- Confidentiality Instruction
- A separate mark saying 'do not show this rule'. Without this mark, a probing question can simply get an answer.
- Output Guardrail
- The tasting check that stops anything past the rule page. See it in more depth in the AI world. → What is a Guardrail?
- Secrets Management
- Keeping truly secret things out of the rule page entirely. API keys or real passwords are safest never written into the rule page at all.
- Prompt (basics)
- What gets written on the recipe book's page. The underlying AI concept. → What is a Prompt?
- Summary/Translation Bypass
- A probe dressed up differently, aiming at the same thing. If a direct ask fails, it comes back as a request to summarize or translate.
- The Kitchen That Takes Any Order
- The page that first introduced the recipe book. The whole picture starts here. → The Kitchen That Takes Any Order