Sign In Register
0 searches
+ 0 extra credits

Extract API

Release notes
API uptime
75.638%

Our Extract API is a router that gets you the very best Markdown for any URL. Depending on the URL, it will use one of our existing engines to extract the perfect Markdown, or in the unfortunate event we don't cover it yet (our goal is to cover 99% of the Internet), through our best educated guess what a Markdown for this page should be.


The API endpoint is https://serpapi.com/search?engine=extract

Head to the playground for a live and interactive demo.

API parameters

Search query

url

Required

Parameter defines the URL of the page to extract. It must be an absolute http or https URL. Search pages that have a dedicated SerpApi engine, such as Google or Bing, are redirected to that engine.

Mode

mode

Optional

Parameter defines the extraction mode. It can be set to:
scrape (default) - Extract a single page.
map - List the website's URLs from the page links and sitemaps.
crawl - Follow links on the same website and extract every page visited.

Crawl mode consumes one search credit per successfully extracted page.

Crawl & map settings

max_pages

Optional

Used with mode=crawl. Parameter defines the maximum number of pages to extract, including the starting page. Default: 10. Maximum: 100. Each successfully extracted page consumes one search credit.

max_depth

Optional

Used with mode=crawl. Parameter defines how many links deep the crawl follows from the starting page. 0 extracts only the starting page. Default: 2. Maximum: 3.

include_subdomains

Optional

Used with mode=map and mode=crawl. Parameter defines whether to include other subdomains of the same website, e.g. blog.example.com when starting from www.example.com. Default: false (same host only).

include_sitemap

Optional

Used with mode=map. Parameter defines whether to also read the website's sitemaps, from /sitemap.xml and /robots.txt. Default: true.

Fetch & content settings

prefetch_delay

Optional

Parameter defines a delay in milliseconds before the first page is fetched (0–10,000). Default: 0. The Extract API doesn't run JavaScript, so this is not a rendering delay.

timeout

Optional

Parameter defines the HTTP request timeout in milliseconds (1,000–120,000). Default: 30,000 (30 seconds).

include_tags

Optional

Parameter defines a comma-separated list of CSS selectors to keep, e.g. article, main, .post-content. Everything else inside <body> is removed before parsing.

exclude_tags

Optional

Parameter defines a comma-separated list of CSS selectors to remove before parsing, e.g. nav, footer, .ads, #cookie-banner.

include_markdown

Optional

Parameter defines whether to include the page's main content as Markdown in markdown. Default: true.

SerpApi parameters

engine

Required

Set parameter to extract to use the Extract API engine.

device

Optional

Parameter defines the device to use to get the results. It can be set to desktop (default) to use a regular browser, tablet to use a tablet browser (currently using iPads), or mobile to use a mobile browser.

no_cache

Optional

Parameter will force SerpApi to fetch the Extract results even if a cached version is already present. A cache is served only if the query and all parameters are exactly the same. Cache expires after 1h. Cached searches are free, and are not counted towards your searches per month. It can be set to false (default) to allow results from the cache, or true to disallow results from the cache. no_cache and async parameters should not be used together.

async

Optional

Parameter defines the way you want to submit your search to SerpApi. It can be set to false (default) to open an HTTP connection and keep it open until you got your search results, or true to just submit your search to SerpApi and retrieve them later. In this case, you'll need to use our Searches Archive API to retrieve your results. async and no_cache parameters should not be used together. async should not be used on accounts with Ludicrous Speed enabled.

zero_trace

Optional

Enterprise only. Parameter enables ZeroTrace mode. It can be set to false (default) or true. Enable this mode to skip storing search parameters, search files, and search metadata on our servers. This may make debugging more difficult.

api_key

Required

Parameter defines the SerpApi private key to use.

output

Optional

Parameter defines the format of the result. It can be set to json (default) to get a structured JSON of the results, html to get the raw HTML retrieved, or md to get a Markdown-formatted version optimized for LLMs and AI agents.

json_restrictor

Optional

Parameter defines the fields you want to include in the output, reducing payload size for faster response. See the JSON Restrictor documentation for more details.

API results

JSON results

JSON output begins with search_metadata and search_parameters. In scrape mode (default), it continues with page_information, meta_tags, open_graph, twitter_card, dublin_core, json_ld, microdata, markdown, inline_links, inline_images, inline_videos, inline_audios and feeds. Empty sections are omitted. Map mode returns a urls array, crawl mode returns a pages array.

Entries with a URL, such as inline_links, urls and pages, include a serpapi_extract_link for extracting that URL next.

A search status is accessible through search_metadata.status. It flows this way: Processing -> Success || Error. If a search has failed, error will contain an error message. search_metadata.id is the search ID inside SerpApi.

HTML results

HTML output returns the HTML of the requested page, up to 5 MB. You can access the HTML page by using the search.html endpoint instead of search or search.json, or by adding output=html to your request. For previous searches, you can check the /searches/SEARCH_ID.html?api_key=YOUR_API_KEY endpoint.

Markdown results

Markdown output returns the page's main content as Markdown. Map mode returns the list of URLs, and crawl mode returns the Markdown of each page under its URL. See Markdown Output.

To access, use the output=md parameter, the /search.md endpoint, or request header Accept: text/markdown.

Note on Engine Redirects

Search pages that have a dedicated SerpApi engine, such as Google, Bing, eBay or Walmart, are redirected to that engine and return its results.

Following robots.txt

The Extract API follows each website's robots.txt. It uses the rules for the serpapi user agent, or the rules for * if there are none for serpapi.

For example, adding these lines to a website's robots.txt would fully block the Extract API, while still allowing other crawlers:

User-agent: serpapi
Disallow: /

If the URL, or a URL it redirects to, is disallowed, the search returns an error. Map mode leaves out disallowed URLs and sitemaps, and crawl mode doesn't follow disallowed links.

Crawl Credit Usage

Crawl mode consumes one search credit per successfully extracted page, including the starting page, so max_pages=10 can cost up to 10 credits. Failed pages and cached responses are not charged. search_metadata.credits_consumed shows the cost.

API examples

Page Markdown output: md


Markdown Example

# Example Domain

This domain is for use in documentation examples without needing permission. Avoid use in operations.

[Learn more](https://iana.org/domains/example)

Scrape a single page url: https://example.com/


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123456",
    "status": "Success",
    "credits_consumed": 1,
    "json_endpoint": "https://serpapi.com/searches/abcd1234ef567890/6a1234abc56789def0123456.json",
    "created_at": "2026-06-02 14:21:08 UTC",
    "processed_at": "2026-06-02 14:21:08 UTC",
    "extract_url": "https://example.com/",
    "raw_html_file": "https://serpapi.com/searches/abcd1234ef567890/6a1234abc56789def0123456.html",
    "total_time_taken": 0.58
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop"
  },
  "page_information": {
    "title": "Example Domain",
    "final_url": "https://example.com/",
    "language": "en"
  },
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)",
  "inline_links": [
    {
      "title": "Learn more",
      "link": "https://iana.org/domains/example",
      "internal": false,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fiana.org%2Fdomains%2Fexample"
    }
  ]
}

Map a site mode: map


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123457",
    "status": "Success",
    "credits_consumed": 1,
    "extract_url": "https://books.toscrape.com/",
    "total_time_taken": 0.62
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://books.toscrape.com/",
    "mode": "map",
    "device": "desktop"
  },
  "urls": [
    {
      "url": "https://books.toscrape.com/index.html",
      "title": "Books to Scrape",
      "source": "page",
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fbooks.toscrape.com%2Findex.html"
    },
    {
      "url": "https://books.toscrape.com/catalogue/category/books_1/index.html",
      "title": "Books",
      "source": "page",
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&url=https%3A%2F%2Fbooks.toscrape.com%2Fcatalogue%2Fcategory%2Fbooks_1%2Findex.html"
    }
  ]
}

Crawl a site mode: crawl


JSON Example

{
  "search_metadata": {
    "id": "6a1234abc56789def0123458",
    "status": "Success",
    "credits_consumed": 3,
    "extract_url": "https://books.toscrape.com/",
    "total_time_taken": 0.85
  },
  "search_parameters": {
    "engine": "extract",
    "url": "https://books.toscrape.com/",
    "mode": "crawl",
    "max_pages": "3",
    "max_depth": "1",
    "device": "desktop"
  },
  "pages": [
    {
      "url": "https://books.toscrape.com/",
      "depth": 0,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2F",
      "page_information": { "title": "All products | Books to Scrape - Sandbox" },
      "markdown": "[Books to Scrape](https://books.toscrape.com/index.html) We love being scraped!\n\n- [Home](https://books.toscrape.com/index.html)\n- All products\n\n..."
    },
    {
      "url": "https://books.toscrape.com/index.html",
      "depth": 1,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2Findex.html",
      "page_information": { "title": "All products | Books to Scrape - Sandbox" }
    },
    {
      "url": "https://books.toscrape.com/catalogue/category/books_1/index.html",
      "depth": 1,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&max_depth=1&max_pages=3&url=https%3A%2F%2Fbooks.toscrape.com%2Fcatalogue%2Fcategory%2Fbooks_1%2Findex.html",
      "page_information": { "title": "Books | Books to Scrape - Sandbox" }
    }
  ]
}

Additional Parameters

Filter output to specific HTML tags include_tags


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "include_tags": "h1,p,a"
  },
  "markdown": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)",
  "inline_links": [
    {
      "title": "Learn more",
      "link": "https://iana.org/domains/example",
      "internal": false,
      "serpapi_extract_link": "https://serpapi.com/search.json?device=desktop&engine=extract&include_tags=h1%2Cp%2Ca&url=https%3A%2F%2Fiana.org%2Fdomains%2Fexample"
    }
  ]
}

Exclude specific HTML tags exclude_tags


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "exclude_tags": "nav,footer,aside,script"
  }
}

Delay before fetching prefetch_delay (milliseconds)


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "prefetch_delay": "2000"
  }
}

Custom fetch timeout timeout (milliseconds)


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "desktop",
    "timeout": "30000"
  }
}

Mobile device device: mobile


JSON Example

{
  ...
  "search_parameters": {
    "engine": "extract",
    "url": "https://example.com/",
    "device": "mobile"
  }
}

Block Your Website

Don't want the Extract API to access your website? Send us its address and we will block it. This blocks your whole domain, including its subdomains, so you don't also need a robots.txt rule.