Once you add an AI feature such as an internal help bot or a customer reply drafting tool, sooner or later you hear things like "Yesterday it answered briefly, today it writes an essay" or "It switches between formal and casual speech." Before you consider changing the model, there is one place to open first: the system prompt.
What goes in a system prompt?
Anthropic's Claude developer docs describe the system prompt as the place to set the role, tone and behavior to keep consistent. They note that defining the role in a single sentence is enough to focus the answers and tone on that use. Google's Gemini API docs also cover setting the model's behavior in advance under the name "system instructions."
To avoid confusion, split the two places like this.
| Where | What it holds | Example |
|---|---|---|
| System prompt | Rules that don't change | Role, tone, restrictions, answer format |
| User message | Work that changes each time | Today's question, pasted material |
When you open a feature with inconsistent answers, the rules are often scattered across user messages, or the system prompt contains sentences that contradict each other.
Organize it in four parts: role, must do, must not do, answer format
Rather than writing a long prompt from the start, begin by filling in these four parts.
- Role: who is answering? Example: "You handle order inquiries for an online store."
- Must do: actions it always takes. Example: "If there's no order number, ask for it first."
- Must not do: draw the line. Example: "Don't confirm whether a refund is possible."
- Answer format: length and shape. Example: "Polite speech, 3 sentences or fewer, one next step on the last line."
Here's what a filled-in example looks like.
[Role] You handle order inquiries for an online store.
[Must do] If there's no order number, ask for it first. Only state confirmed shipping information.
[Must not do] Don't confirm refund or exchange eligibility. Don't make up prices or stock.
[Answer format] Polite speech, 3 sentences or fewer, one next step on the last line.3 common mistakes
- Contradicting sentences: if "explain in detail" and "answer briefly" sit side by side, the AI picks differently every time. Write down which one comes first.
- "Appropriately" or "use your judgment": words without a standard give unstable results. Write numbers or shapes instead, such as "3 sentences" or "1 table."
- No exceptions: always include what to do when an unknown question comes in, for example "If you don't know, say so and offer to connect a staff member."
Compare with the same questions every time you change it
A system prompt is never finished in one go. Pick 10 questions you always use and compare the answers before and after each change side by side. Mix questions real users ask often, questions it must not answer and ambiguous questions.
When comparing, look at just three things.
- Did it keep the format? (length, tone, structure)
- Did it say something it shouldn't?
- Did it say "I don't know" when it didn't know?
If you skip this check, you can fix one thing, break another and deploy without noticing. Leave one line with the date and reason for each change, and when answers go wrong later you can quickly find where things started to differ.
References
- Anthropic, Claude developer docs "System prompts": https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/system-prompts
- Google, Gemini API docs "System instructions": https://ai.google.dev/gemini-api/docs/system-instructions
In a one-hour AX consultation we map AI candidates and similar cases for you.
We build web apps, mobile apps and business systems, then add and run the AI they need. We build and operate our own meeting notes, contract and company analysis AI.
See our projects