Skip to main content
PUT
Set Checks

Overview

Creates or fully replaces the check config for one environment: the test scenarios to drive conversations from, the judges that score them, and how many conversations to generate per scenario. The whole config is validated up front, so an unrunnable config fails here with 400 BAD_REQUEST instead of erroring mid-run. Runs already in flight keep the config they started with. Read it back with Get Checks and run it with Start Check Run. Requires an admin, owner, operator, or prompter role.

Headers

string
required
Your API key for authentication.

Path Parameters

string
required
The agent’s unique identifier.
string
required
staging or production. Checks do not attach to dev. Any other value returns 400 BAD_REQUEST.

Body Parameters

array
required
IDs of this agent’s test scenarios, 1 to 5 after duplicates are removed. Every ID must belong to this agent and must target the agent rather than a pathway. Returns 400 BAD_REQUEST otherwise.
array
required
The judges to attach. At least one, and each judge may appear once. Returns 400 BAD_REQUEST when a judge or judge version is not found in your organization, the version does not belong to the judge, or the version is not runnable.
string
required
The judge’s unique identifier.
string
required
The judge version to pin. Must belong to eval_agent_id. Runs keep using this version until you replace the config.
array
default:"[]"
For graded judges, the level keys counted as passing. At least one is required, and each must be a level defined on the pinned version. For pass/fail judges, omit it or pass an empty array; any level key returns 400 BAD_REQUEST.
boolean
default:"true"
true makes the judge blocking: a run fails when this judge fails. false makes it informational; its verdict is reported but never fails the run.
boolean
default:"true"
Whether check runs can be started for this environment. Because this endpoint replaces the whole config, omitting it re-enables a disabled config.
integer
default:"5"
Conversations to generate per scenario on each run. Fractional values are floored and the result is clamped to 1 to 50, then rejected with 400 BAD_REQUEST if it exceeds your organization’s per-scenario limit (10 unless raised for your account).

Response

object
The saved check config.
array
One entry per judge, in the order submitted.
null | array
null on success, or a list of error objects if the request failed.

Docs for agents: llms.txt