Create Eval Run
Experiments
Create Eval Run
Start a new eval run over a batch of calls.
POST
Create Eval Run
Headers
string
required
Your API key for authentication.
Body Parameters
array of string
required
The call IDs to score. Between 1 and 5000 calls.
array of objects
The eval agents to score with. Maximum 50 agents. Each object:
string
The scoring modality. One of
text, audio, or full. Defaults to text.number
Percentage of evaluations that must pass for the run to be marked passing overall, 0-100. Omit to run without a threshold.
string
UUID of a workbench setup to associate with this run. Must be provided together with
workbench_setup_version_id.string
UUID of the pinned workbench setup version. The version’s agent roster is used when
attached_agents is omitted. Must be provided together with workbench_setup_id.string
How this run was initiated. One of
manual, auto, or backfill. Defaults to manual.object
Key-value metadata to attach to the run. Keys and values must be strings.
You must supply either a non-empty
attached_agents array or a workbench_setup_version_id (whose pinned roster supplies the agents). If you provide workbench_setup_id or workbench_setup_version_id, both fields are required together. A valid billing record must be on file before a run can start.Response
string
Unique identifier for the newly created eval run.
string
The organization that owns this run.
string
Initial status of the run. Will be
PENDING immediately after creation.string
How the run was triggered.
string
The scoring modality for this run.
array of string
The call IDs submitted for scoring.
array of objects
The agent attachments submitted with the run. Each object contains
eval_agent_id, eval_agent_version_id, weight, and target_level_keys.string or null
The associated workbench setup, if provided.
string or null
The pinned workbench setup version, if provided.
integer
Number of calls resolved for the run.
integer
Number of eval agents resolved for the run.
integer
Total call-by-agent evaluations to perform.
integer
Number of evaluations completed so far.
null
Always
null at creation time.null
Always
null at creation time.string or null
Error code if the run failed, otherwise
null.string or null
Human-readable error message if the run failed, otherwise
null.string or null
Internal workflow identifier, if applicable.
object
Key-value metadata attached to the run.
string
ISO 8601 timestamp when the run was created.
null
Always
null at creation time.null
Always
null at creation time.