Most AI demos shown in a meeting run smoothly. The questions are prepared, the data is tidy, and the person using it knows the system. But once real users arrive, you get questions full of typos, unexpected files, and conversations far longer than you planned for. That's why the same feature can feel seamless in a demo and shaky in production.
This post is a checklist of five things the development team and product planners should agree on before putting an AI feature into a live service, based on Anthropic's and OpenAI's official evaluation guides.
1. Write down what "it works" means first
Everyone who watched the demo has a different idea of what "it works" means. Before launch, put into words what this feature must achieve to count as a success. OpenAI's evaluation guide also makes defining objectives and success criteria the first step.
- Does the answer rely only on what's in the internal policy documents?
- Does the summary avoid inventing numbers that aren't in the source?
- Does it follow the required format (a table, a three-line summary, etc.)?
2. Test with many questions — including the strange ones
Anthropic's documentation stresses building tests that mirror the real distribution of your work and not skipping edge cases (rare but inevitable exceptions). It also says that many questions you can grade automatically beat a handful of carefully hand-picked ones.
| Questions to include | Example |
|---|---|
| Ordinary questions | Your most common inquiries, as they come in |
| Typos and abbreviations | "whats the refund polcy" |
| Long questions | Inquiries with a long background story attached |
| Questions that should have no answer | Asking about something not in the source material |
3. Design the failure screen first
An AI response can be slow, come back in a broken format, or fail entirely. Decide in advance what the user will see when that happens.
- If no response arrives within a set time, show a message and a retry button
- Don't display a malformed response as-is; request it once more
- If it still fails, offer a path to a human agent or the existing feature
4. Decide what to keep: logs and feedback
To improve quality after launch, you need to see which answers went out for which questions. OpenAI's guide also recommends continuous evaluation, adding real usage data to your test set over time. Since personal information can be mixed in, decide on the storage scope and retention period at the same time.
- Question, answer, model used, response time
- Results of users' thumbs up / thumbs down buttons
- Mask personal information or don't store it
5. Cost caps and rules for input data
AI APIs charge for what you use. Setting caps on how many times one user can use the feature per day and how long a document can be sent at once helps prevent unexpected bills. We cover how to estimate costs separately in the next post (estimating LLM API costs).
Set rules for input data, too. If you decide upfront which documents can be sent to the AI and how customer information will be masked, you won't have to reopen the discussion every time the feature expands.
A demo shows what's possible; these five things turn that possibility into a feature people use every day. The tighter your launch schedule, the more worthwhile it is to run through this list first.
References
- Anthropic, Create strong empirical evaluations: https://platform.claude.com/docs/en/test-and-evaluate/develop-tests
- OpenAI, Evaluation best practices: https://developers.openai.com/api/docs/guides/evaluation-best-practices
We review your current system and data, then outline the migration order and scope.
We build web apps, mobile apps and business systems, then add and run the AI they need. We build and operate our own meeting notes, contract and company analysis AI.
See our projects