Skip to main content
POST
/
v1
/
node_test
/
invoke
{
  "new_prompt": "You are the best financial advisor...",
  "judge_prompt": "Evaluate if the response is helpful and accurate.",
  "loop_prompt": "Continue if the user has more questions.",
  "extract_variables": [
    {
      "name": "user_name",
      "type": "string",
      "description": "The name of the user",
      "spelling_precision": true
    }
  ],
  "conversations": [
    {
      "type": "call",
      "id": "330d8a20-27bc-4d00-b67a-8474c9a6d4e1"
    },
    {
      "type": "call",
      "id": "906f16de-2719-4b7e-864b-a4b002586e7c"
    }
  ],
  "node_id": "a7bbd409-504b-4ba3-a9d1-12f6bc270f58",
  "pathway_id": "05f4b269-e79a-4825-b4cd-7778f782bfad",
  "pathway_version": "2",
  "n_permutations": 5
}

{
  "data": {
    "run_id": "727a85f1-1959-4ec1-95a4-248a2eecf1ae"
  },
  "errors": null
}

​
Headers

​
authorization
string
required
Your API key for authentication.

​
Body Parameters

​
new_prompt
string
The new prompt to test for this node. This will be used to simulate the agent’s responses against the pinned and auto-selected conversations. Either new_prompt or static_text can be provided, but not both.
​
static_text
string
Static text that the agent will always reply with. Either static_text or new_prompt can be provided, but not both. This is useful to test variable extractions on static text nodes.
​
judge_prompt
string
The judge prompt to evaluate the test results. If not provided, the default judge prompt will be used. The default judge prompt will evaluate the agent responses against the node prompt.
​
loop_prompt
string
The prompt for the node loop condition.
​
extract_variables
array
Array of variables to extract from the conversation.
​
extract_variables[].name
string
required
The name of the variable to extract.
​
extract_variables[].type
string
required
The type of the variable. Must be one of: "string", "integer", or "boolean".
​
extract_variables[].description
string
required
A description of what the variable represents.
​
extract_variables[].spelling_precision
boolean
required
Whether spelling precision should be enforced for this variable.
​
conversations
array
required
Array of conversation references to include in the test. The system may automatically add up to 4 additional recent completed calls for the same node and pathway.
​
conversations[].id
string
required
Unique identifier of a call to include in the test run.
​
conversations[].type
string
required
Type of the conversation. We only support “call” for now.
​
node_id
string
required
Identifier of the node to test within the specified pathway.
​
pathway_id
string
required
Identifier of the pathway that the node and conversations belong to.
​
pathway_version
string
The version of the pathway to test against.
​
n_permutations
number
Optional number of permutations to generate for each conversation. If omitted, this defaults to 5. The total generations per conversation will be n_permutations + 1 (one for the original user messages plus one per permutation).

​
Response

​
data.run_id
string
Identifier of the created node test run. Use this ID with GET /v1/node_test/run/:id to retrieve the full results.
{
  "new_prompt": "You are the best financial advisor...",
  "judge_prompt": "Evaluate if the response is helpful and accurate.",
  "loop_prompt": "Continue if the user has more questions.",
  "extract_variables": [
    {
      "name": "user_name",
      "type": "string",
      "description": "The name of the user",
      "spelling_precision": true
    }
  ],
  "conversations": [
    {
      "type": "call",
      "id": "330d8a20-27bc-4d00-b67a-8474c9a6d4e1"
    },
    {
      "type": "call",
      "id": "906f16de-2719-4b7e-864b-a4b002586e7c"
    }
  ],
  "node_id": "a7bbd409-504b-4ba3-a9d1-12f6bc270f58",
  "pathway_id": "05f4b269-e79a-4825-b4cd-7778f782bfad",
  "pathway_version": "2",
  "n_permutations": 5
}

{
  "data": {
    "run_id": "727a85f1-1959-4ec1-95a4-248a2eecf1ae"
  },
  "errors": null
}