System Prompt Extraction: Risks and Practical Defenses

System prompt extraction is an attempt to make an AI application reveal instructions that its operator did not intend to show. Those instructions can explain the assistant’s role and behavior, but they should never be the only thing protecting private data or powerful actions.

An exposed instruction and a broken permission check are different findings. Tools like fenbs can retain reviewed rules and remediation decisions so your next assistant knows what was agreed; the application still needs to enforce access itself. Start by asking what a disclosure would actually let someone read or do.

What does system prompt extraction actually reveal?

A system prompt is application-supplied guidance for the model, the software producing an answer. It might describe a support assistant’s scope, writing style, or response format. A successful extraction could reveal some of that text, summaries of it, or details inferred from the assistant’s behavior.

A response labeled “my system prompt” is not proof that the application disclosed its real configuration. The model might invent a plausible answer, repeat text from the conversation, or combine several sources. If you own the application, compare the response with the version of its instructions used for that session.

The phrase “system prompt extractor” describes a tool or technique attempting this task; it does not guarantee a complete or authentic result. Treat its output as a claim to investigate. Our system prompt examples show ordinary instruction content, while system prompt leaks covers disclosure reports and their interpretation.

Why can instructions become visible?

The application supplies instructions so the model can use them. That makes their wording part of information the model processes, rather than a password vault. A refusal instruction may reduce unwanted disclosure, but it cannot turn all text available to a model into a reliably protected secret.

OWASP’s system prompt leakage guidance↗ (opens in a new tab) says system prompts should not be treated as secrets or security controls. It specifically warns against placing credentials in them and delegating authorization to the model. Authorization means checking whether this particular person is allowed to perform this particular action.

Instructions can also be exposed through your own support records, debugging screens, or shared transcripts. Review those routes alongside model responses. An external page containing hostile directions creates a related problem called indirect prompt injection: the assistant encounters instructions inside material it was supposed to read as data.

Which protections must live outside the prompt?

Design sensitive operations to remain protected even when someone knows your instruction text. A helpful review question is: “If we published this paragraph, would the application still refuse an unauthorized request?” If the answer depends on the model keeping a secret or obeying a sentence, move that responsibility into the application.

• Credentials: keep passwords, connection keys, and sign-in tokens in a dedicated secret store. Let the tool use them without placing them in the conversation.
• Access: check the signed-in person’s permission whenever a tool reads or changes a record. Do not accept a role that the model supplies as evidence of authority.
• Inputs: validate proposed tool arguments, the details a tool receives, before executing them. Check the record, destination, and allowed operation. Validation means checking a value against explicit requirements.
• Consequential actions: require the appropriate approval in the actual workflow. A model saying “approved” must not create the approval it needs.

Keep behavioral guidance in the prompt: explain the task, name the boundaries, and tell the assistant how to report a refusal. Keep the refusal mechanism in the service performing the action. The wider distinction between selecting useful context and writing instructions is covered in context engineering versus prompt engineering.

How should you assess a suspected extraction?

Review the evidence in a controlled environment you own or are authorized to test. Preserve the time, application version, relevant conversation, and observed response in the approved incident system. Share a limited summary with colleagues rather than copying a possible secret into every task or chat.

1. Check authenticity. Compare the response with the instructions actually supplied to that run, marking exact matches separately from guesses and ordinary public information.
2. Classify exposure. Distinguish writing guidance from private business information, personal information, and usable credentials. Record uncertainty instead of calling every matching sentence a breach.
3. Inspect consequences. Check whether an unauthorized read or change was possible and whether the service rejected it. Do not infer a successful action from the assistant claiming it happened.
4. Assign remediation. Remove unnecessary sensitive content, replace exposed credentials through your normal process, and repair any access or validation failure that made the disclosure consequential.

Keep the approved response and its rationale in fenbs Decisions and rules, with tasks for the outstanding fixes. Store the sensitive evidence in the system your organization has approved for that evidence.

How can you test defenses without using real secrets?

Build a small test version with invented records and a harmless marker that is unique to the test. Check ordinary work as well as attempts to obtain internal instructions. The useful result includes whether the assistant still completes its legitimate job, whether any marker appears, and whether unauthorized tool requests are refused.

Anthropic’s prompt leak guidance↗ (opens in a new tab) recommends reducing unnecessary proprietary details, reviewing outputs, and testing the effect of leak-prevention measures on performance. It also states that no method is foolproof. An extra page of secrecy instructions can complicate the assistant’s task without fixing a permissive tool.

Repeat relevant checks after changing the model, instruction text, available tools, or connected data. Record the expected refusal and the observed service result. A clean test run gives evidence about those cases; it does not prove that every possible future conversation is safe.

Give each finding a clear stopping condition. For example, a test account must receive a refusal when requesting an invented record owned by another account, even if the model produces a convincing explanation for access. That checks the boundary directly. Asking whether the assistant sounds careful measures a different quality and can miss the actual failure.

How to do this with fenbs

1. Create a free account and add a project for the application review. Keep its description free of secrets and sensitive incident evidence.
2. Record the person-approved rules in Decisions and rules, such as keeping credentials outside prompts and checking permission in every sensitive tool. Include the reason and scope of each rule.
3. Add tasks for the specific findings. Describe the affected component, the change needed, and the safe check that will show the fix works.
4. Connect a supported assistant through Integrations. Ask it to read the project context before work and save the actual check results and remaining steps before it stops.

fenbs is not a prompt firewall, a secret vault, or a security testing service. It fits the shared record of decisions and follow-up work; specialist controls belong in the application. Use the AI governance hub to plan the surrounding review responsibilities.