Map a site

6.25 credits / call

Crawls a website and returns the list of URLs it contains, without downloading any page content. Use this to discover what pages exist on a domain before deciding which ones to read or extract data from.

Use when

You want to discover what pages a domain has (all docs, blog, or pricing URLs) before deciding what to scrape.

Not for

Reading the content of those pages (use Read a page on each), or finding sites across the open web (use Web Search).

Cost

6.25 credits / call

What it accepts

Set these inside the intent when you run it.

urlrequired

The base URL of the site to map.

filteroptional

A keyword to narrow the returned URLs to a specific section or topic.

What you get back

A list of URLs found on the site, optionally filtered by keyword.

Run it

Run this sub-skill directly: pin it with operation and pass its inputs in the intent. (Omit operation and the Generic Scrapers skill will route from your intent instead.)

curl -X POST "https://skill.askfaro.com/skills/web-scraping/run" \
  -H "Authorization: Bearer $FARO_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"intent":{"operation":"map_site","url":"https://example.com","filter":"pricing"}}'

Example requests

›List all the URLs on this documentation site
›Find all the blog post URLs on this domain
›What pages exist under the /pricing section of this site?

← Back to Generic Scrapers