Skip to main content
POST
Create Disposition Test Run

Overview

Dispositions are enabled per organization. If your organization does not have access, these endpoints return 404.
Starts an asynchronous test run of the draft or the published definition against up to 250 of the agent’s completed production calls, or replays the frozen cohort of an earlier run. The response is the queued run; poll Get Disposition Test Run for progress and read results with Get Test Run Rows. Test runs bill inference like production runs. An organization may start at most 10 test runs per hour and hold at most 3 in queued or running at once. Requires an admin, owner, operator, or prompter role.

Headers

string
required
Your API key for authentication.

Path Parameters

string
required
The agent’s unique identifier. Must be a UUID; otherwise returns 400 with the message agentId must be a valid UUID. Returns 404 Agent not found if the agent is not in your organization.
string
required
The disposition’s unique identifier. Must be a UUID; otherwise returns 400 with the message dispositionId must be a valid UUID. Returns 404 Disposition not found if it does not belong to this agent.

Body Parameters

The body takes one of two shapes: a current-definition run (target plus population, with optional executionSla) or a replay (replayRunId alone). Mixing fields from both shapes, or adding unknown fields, returns 400 BAD_REQUEST with the message Invalid request body.
string
Required for a current-definition run. draft runs the live draft, which is first validated as strictly as publish (every value must pin a runnable judge or extractor version, though an editable pin is allowed); failures return 400 with the message Definition failed validation. published runs the current published version and returns 409 CONFLICT if the disposition has never been published. A definition with no values returns 400 with the message Add at least one value before running a test.
string
realtime or queued. Defaults from the definition’s timing: immediate gives realtime, within_24h gives queued.
object
Required for a current-definition run. Which completed, non-simulated production calls of this agent to run against. Two shapes:Recent sample: { "environment": "production", "requestedSize": 25 }, optionally with "mode": "recent". requestedSize is an integer from 1 to 250; the newest eligible calls are sampled. Returns 400 if the agent has no eligible completed calls to sample.Explicit calls: { "mode": "calls", "callIds": ["..."] }, optionally with "environment": "production". callIds is 1 to 250 unique call UUIDs. Every call must belong to this agent and be completed with non-zero duration, otherwise 400 with the message One or more selected calls are not eligible completed production calls for this agent. Calls that cannot enter dispositions for compliance reasons return 400 listing the offending ids.
string
Alternative body: { "replayRunId": "..." } with no other fields. Re-executes the frozen call cohort and definition snapshot of a finished run (complete, partial, failed, or cancelled) on this disposition, with configurationSource: "frozen_run". Returns 404 with the message Replay source test run not found if no such run exists, 400 with the message Replay source test run has no frozen call cohort if it recorded no calls, and 409 with the message Replay source frozen inputs are incomplete; run a new test instead if any call lacks a frozen input snapshot.

Response

Returns 201 on success with the run in queued status. Returns 503 SERVICE_UNAVAILABLE if the run could not be handed to the execution queue; the run is recorded as failed and the request can be retried.
object
The new test run. Same shape as the entries of List Disposition Test Runs; the fields that matter on creation are listed here.
null | array
null on success, or a list of error objects if the request failed.

Docs for agents: llms.txt