Interact with the real world mid-call using custom tools.
speak
, wait
, and button press
tool. When you create a custom tool, you add it to the existing ‘toolbox’ for the phone agent to pick from.
A few natural questions arise:
name
- the agent will see this in the list of toolsdescription
- a short explanation of what the tool doesinput_schema
- a JSON schema describing the input dataspeech
(optional) - a string that will be spoken to the agent while your tool waits for a responseresponse_data
- An array of objects that describe how to extract data from the response. Within the response data, you can create variables that the phone agent can reference in its prompt.BookAppointment
and the description to Books an appointment for the customer
.
input_schema
is converted into the variable "{{input}}"
that you can use in the request body/query/headers"{{input.property.subproperty}}"
"{{input.service}}"
to have whatever type of appointment that the customer wantsinterruption_threshold
parameter to about 200
so that the AI doesn’t interject so quickly.
Scroll down to see the full example.
speech
property. The phone agent will say the speech
while it makes the request.
An example speech might look like: Perfect, I'll schedule that right now, give me just a second.
For the restaurant ordering example, the speech could be Thank you, placing that order now.
data
field determines how you extract the data while the name
field determines the variable name for reference in the prompt.
Here’s an example response data:
How does the phone agent determine what it sends an API request?
example
within it. Both of those pieces of information provide context about what data to pass.Then the phone agent extracts the information from the transcript and passes it to the request body.You can improve the accuracy of the input data by creating a very clear input_schema
. That includes providing a detailed example
within.How does the phone agent decide when to use the custom tool?
How do I access the info the custom tool responds with?
response_data
property.Once you’ve given the variable a name
, you can reference it in the prompt using double brackets ({{}}
).Note, with the current setup, you might reference variables that have null values. Here’s the restaurant example prompt:You are taking a drive thru order from a customer. Find out everything that they want like a drive thru cashier. Continue until they say they’re done. Repeat the full order back to them after that, and ask if that’s correct. If they confirm that it’s correct, then and only then will you place their order using the PlaceOrder tool. After you place it, tell them their order total and to pull forward. Their order price is {{order_price}}
In the above prompt, the order_price
will be null until the data comes back. That’s okay though. The prompt is structured to first take the order, then use the PlaceOrder tool, and then finally respond with the order price.By the time the phone agent is asked for the order price, it will have the information.Custom tools will continue getting more robust, to further prevent scenarios where variables without value from being referenced in prompts.