Skip to main content
An experiment compares a proposed configuration with a baseline using the same scenarios and scoring rules. Planning that comparison is useful before automated execution becomes available.
Threetone does not currently provide an experiment resource or an A/B experiment runner. Traffic allocation and automated tests remain subject to execution gates. This page describes a review plan, not a dashboard workflow.
Start with an existing agent and a narrow hypothesis, such as whether a shorter instruction improves a specific handoff decision.
1

Record the baseline

Keep the agent, branch, and immutable version IDs in your review notes. Use versions and drafts to distinguish a saved proposal from a published snapshot.
2

Define a controlled change

Change one instruction or model choice. Prepare the same test definitions, synthetic inputs, and scoring rubric for both candidates.
3

Set the decision rule

Record the required outcomes, unacceptable failures, sample size, and cost or latency measurements needed. Leave result cells unmeasured until comparable execution evidence exists.
Before runtime comparisons are released, review configuration diffs and scenario coverage. Neither successful publication nor a manual prompt review establishes a measured improvement.
No. The current deployment execution gate applies, and there is no experiment assignment or statistical reporting workflow.
Voice and custom-model publication are unavailable. Preserve eligible changes as drafts and record the blocked stage.