Skip to main content
A simulation describes an interaction you want to assess against an agent configuration. Save the definition separately from the result: accepting a test case does not mean a conversation ran or passed.

​
Prepare a useful case

1

Choose one scenario

Name a concrete situation, such as a customer asking about an unsupported action. State the expected response and the boundary the agent must preserve.
2

Supply representative context

Prepare the chat history, dynamic variables, environment, and channel needed by the case. Use synthetic data and avoid copying private transcripts into fixtures.
3

Define the outcome

Specify response conditions, examples, or expected tool behavior supported by the test schema. Include a failure case as well as a normal interaction.
4

Save and read back

Use Create test, then retrieve the saved definition. Keep the case ID and agent version you intend to assess.

​
Execution boundary

The current implementation stores test definitions and exposes invocation reads, but automated execution and resubmission remain gated. Do not report a success rate from saved definitions or empty invocation results. When execution is released, require the exact configuration version, input scenario, result status, and review evidence for each run. See Testing.

​
Frequently asked questions

A text scenario cannot establish microphone behavior, speech recognition, audio quality, or channel reliability. Those need separate supported runtime tests.
Prefer meaningful behavior and constraints when wording can vary. Exact text is appropriate only when that text is itself a requirement.