Skip to main content
POST
Analyzes a website to discover available URLs from its sitemap. This is useful for finding all the pages available on a website before creating a web scraping knowledge base.

Headers

string
required
Your API key for authentication.
string
required
Must be application/json.

Body Parameters

string
required
The base URL of the website to discover sitemap URLs from.

Response

object
Discovered URLs and sitemap information.
string[]
Array of discovered URLs from the website’s sitemap.
string
The sitemap URL that was found and processed (if any).
null
Will be null on successful discovery.

Docs for agents: llms.txt