​
Introduction

Custom tools allow your agent to interact with any web API mid-call. Do things like:

Send a message

Dispatch SMS or emails using the person’s contact info.

Schedule an appointment

Set appointments using live calendar availability.

Create a support ticket

Generate support tickets in your issue tracker.

Update your CRM

Update your CRM with relevant details during the call.

​
Background

To understand how custom tools work, let’s take a peek under the hood of the Bland AI phone agent.

During the conversation, the phone agent is constantly listening to figure out when it’s supposed to respond. When the phone agent realizes it’s time to respond, it reviews the tools in its toolbox and picks between them.

Those tools include a speak, wait, and button press tool. When you create a custom tool, you add it to the existing ‘toolbox’ for the phone agent to pick from.

A few natural questions arise:

  1. How do I define my custom tool?
  2. How do I make sure my tool gets picked at the right time?
  3. How does information from the call get passed to my custom tool’s API request?
  4. How do I fill the silence (when my custom tool is running)?
  5. How does the response from my custom tool get added to the call?

Keep reading to find out.

​
Creating your custom tool

​
Custom Tool Example

{
    "name": "BookAppointment",
    "description": "Books an appointment for the customer",
    "url": "https://your-api.com/book-appointment",
    "method": "POST",
    "headers": {
        "Authorization": "Bearer YOUR_API_KEY"
    },
    "body": {
        "date": "{{input.date}}",
        "time": "{{input.time}}",
        "service": "{{input.service}}"
    },
    "input_schema": {
        "example": {
            "speech": "Got it - one second while I book your appointment for tomorrow at 10 AM.",
            "date": "2024-04-20",
            "time": "10:00 AM",
            "service": "Haircut"
        },
        "type": "object",
        "properties": {
            "speech": "string",
            "date": "YYYY-MM-DD",
            "time": "HH:MM AM/PM",
            "service": "Haircut, Coloring, Trim, or Other"
        }
    },
    "response": {
        "succesfully_booked_slot": "$.success",
        "stylist_name": "$.stylist_name"
    }
}

​
From API request to custom tool

The next step is to convert the API request into a custom tool. Custom tools have the following properties:

  • name - the agent will see this in the list of tools
  • description - a short explanation of what the tool does
  • input_schema - a JSON schema describing the input data
  • speech (optional) - a string that will be spoken to the agent while your tool waits for a response
  • response_data - An array of objects that describe how to extract data from the response. Within the response data, you can create variables that the phone agent can reference in its prompt.

​
Name & Description

The agent will see the name in the list of tools. The name, plus the description, help the AI phone agent when it decides which tool to use.

For this example we’ll set the name to BookAppointment and the description to Books an appointment for the customer.

​
Input Schema

The input schema is critical. It defines the shape of the API request, the different inputs the request can take, and also includes an example (which helps our system when creating requests).

Here’s what the input schema could look like:

    "input_schema": {
        "example": { // "example" is a special property that shows an example of what the input object the agent creates should look like
            "speech": "Got it - one second while I book your appointment for tomorrow at 10 AM.",
            "date": "2024-04-20",
            "time": "10:00 AM",
            "service": "Haircut"
        },
        "type": "object",
        "properties": {
            "speech": "string",
            "date": "YYYY-MM-DD",
            "time": "HH:MM AM/PM",
            "service": "Haircut, Coloring, Trim, or Other"
        }
    }

Two important notes about input schema:

  1. input_schema is converted into the variable "{{input}}" that you can use in the request body/query/headers
  2. To access nested properties, use dot notation: "{{input.property.subproperty}}"
    • For example, later on you could use "{{input.service}}" to have whatever type of appointment that the customer wants

What you’re doing here is describing the structure of the variables that the agent will create.

Special Note: If you need to gather detailed letter-by-letter information from the user, raise your interruption_threshold parameter to about 200 so that the AI doesn’t interject so quickly.

Scroll down to see the full example.

​
Speech

Because requesting external APIs might take a while, we enable you to define a speech property. The phone agent will say the speech while it makes the request.

An example speech might look like: Perfect, I'll schedule that right now, give me just a second.

For the restaurant ordering example, the speech could be Thank you, placing that order now.

​
Response data

Once your API request comes back, you need to extract the response data, and then make the phone agent aware of the new information.

The data field determines how you extract the data while the name field determines the variable name for reference in the prompt.

Here’s an example response data:

"response": {
    "succesfully_booked_slot": "$.success",
    "stylist_name": "$.stylist_name", // if your API returns a JSON object with a key "stylist_name"
}

​
Full example

Below is the entire API request for sending a phone call using the outlined custom tool:

{
    "phone_number": "+12223334444",
    "prompt": "...",
    "tools": [
        {
            "name": "BookAppointment",
            "description": "Books an appointment for the customer",
            "url": "https://your-api.com/book-appointment",
            "method": "POST",
            "headers": {
                "Authorization": "Bearer YOUR_API_KEY"
            },
            "body": {
                "date": "{{input.date}}",
                "time": "{{input.time}}",
                "service": "{{input.service}}"
            },
            "input_schema": {
                "example": {
                    "speech": "Got it - one second while I book your appointment for tomorrow at 10 AM.",
                    "date": "2024-04-20",
                    "time": "10:00 AM",
                    "service": "Haircut"
                },
                "type": "object",
                "properties": {
                    "speech": "string",
                    "date": "YYYY-MM-DD",
                    "time": "HH:MM AM/PM",
                    "service": "Haircut, Coloring, Trim, or Other"
                }
            },
            "response": {
                "succesfully_booked_slot": "$.success",
                "stylist_name": "$.stylist_name"
            }
        }
    ]
}

​
Frequently asked questions

If you have any additional questions, reach out at hello@bland.ai and one of our engineers will help.