Invoke Node Test Run

{
  "new_prompt": "You are the best financial advisor...",
  "judge_prompt": "Evaluate if the response is helpful and accurate.",
  "loop_prompt": "Continue if the user has more questions.",
  "extract_variables": [
    {
      "name": "user_name",
      "type": "string",
      "description": "The name of the user",
      "spelling_precision": true
    }
  ],
  "conversations": [
    {
      "type": "call",
      "id": "330d8a20-27bc-4d00-b67a-8474c9a6d4e1"
    },
    {
      "type": "call",
      "id": "906f16de-2719-4b7e-864b-a4b002586e7c"
    }
  ],
  "node_id": "a7bbd409-504b-4ba3-a9d1-12f6bc270f58",
  "pathway_id": "05f4b269-e79a-4825-b4cd-7778f782bfad",
  "pathway_version": "2",
  "n_permutations": 5
}

{
  "data": {
    "run_id": "727a85f1-1959-4ec1-95a4-248a2eecf1ae"
  },
  "errors": null
}

POST

node_tests

invoke

{
  "new_prompt": "You are the best financial advisor...",
  "judge_prompt": "Evaluate if the response is helpful and accurate.",
  "loop_prompt": "Continue if the user has more questions.",
  "extract_variables": [
    {
      "name": "user_name",
      "type": "string",
      "description": "The name of the user",
      "spelling_precision": true
    }
  ],
  "conversations": [
    {
      "type": "call",
      "id": "330d8a20-27bc-4d00-b67a-8474c9a6d4e1"
    },
    {
      "type": "call",
      "id": "906f16de-2719-4b7e-864b-a4b002586e7c"
    }
  ],
  "node_id": "a7bbd409-504b-4ba3-a9d1-12f6bc270f58",
  "pathway_id": "05f4b269-e79a-4825-b4cd-7778f782bfad",
  "pathway_version": "2",
  "n_permutations": 5
}

{
  "data": {
    "run_id": "727a85f1-1959-4ec1-95a4-248a2eecf1ae"
  },
  "errors": null
}

Headers

string

required

Your API key for authentication.

Body Parameters

string

The new prompt to test for this node. This will be used to simulate the agent’s responses against the pinned and auto-selected conversations. Either new_prompt or static_text can be provided, but not both.

string

Static text that the agent will always reply with. Either static_text or new_prompt can be provided, but not both. This is useful to test variable extractions on static text nodes.

string

The judge prompt to evaluate the test results. If not provided, the default judge prompt will be used. The default judge prompt will evaluate the agent responses against the node prompt.

string

The prompt for the node loop condition.

array

Array of variables to extract from the conversation.

string

required

The name of the variable to extract.

string

required

The type of the variable. Must be one of: "string", "integer", or "boolean".

string

required

A description of what the variable represents.

boolean

required

Whether spelling precision should be enforced for this variable.

array

required

Array of conversation references to include in the test. The system may automatically add up to 4 additional recent completed calls for the same node and pathway.

string

required

Unique identifier of a call to include in the test run.

string

required

Type of the conversation. We only support “call” for now.

string

required

Identifier of the node to test within the specified pathway.

string

required

Identifier of the pathway that the node and conversations belong to.

string

The version of the pathway to test against.

number

Optional number of permutations to generate for each conversation. If omitted, this defaults to 5. The total generations per conversation will be n_permutations + 1 (one for the original user messages plus one per permutation).

Response

string

Identifier of the created node test run. Use this ID with GET /v1/node_tests/run/:id to retrieve the full results.

{
  "new_prompt": "You are the best financial advisor...",
  "judge_prompt": "Evaluate if the response is helpful and accurate.",
  "loop_prompt": "Continue if the user has more questions.",
  "extract_variables": [
    {
      "name": "user_name",
      "type": "string",
      "description": "The name of the user",
      "spelling_precision": true
    }
  ],
  "conversations": [
    {
      "type": "call",
      "id": "330d8a20-27bc-4d00-b67a-8474c9a6d4e1"
    },
    {
      "type": "call",
      "id": "906f16de-2719-4b7e-864b-a4b002586e7c"
    }
  ],
  "node_id": "a7bbd409-504b-4ba3-a9d1-12f6bc270f58",
  "pathway_id": "05f4b269-e79a-4825-b4cd-7778f782bfad",
  "pathway_version": "2",
  "n_permutations": 5
}

{
  "data": {
    "run_id": "727a85f1-1959-4ec1-95a4-248a2eecf1ae"
  },
  "errors": null
}

Docs for agents: llms.txt

⌘I

Basic Tutorials

Calls

Conversational Pathways

Knowledge Bases

Numbers

Blocked Numbers

Widgets

Tools

Contacts

Memory

Node tests

Agent Testing

Evals

Guard Rails

Web Agents

Live Translation

Custom Twilio Accounts

Batches

Prompts

Account

Organizations

Messaging

SIP Trunks

Custom Dialing Pools

Personas

Citation Schemas

Alarms

Triage

Invoke Node Test Run

Headers

Body Parameters

Response

​Headers

​Body Parameters

​Response

Headers

Body Parameters

Response