> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/products/web-api/scrape.md).

# /scrape

Learn how to use Oxylabs Web API /scrape endpoint to fetch and parse any web page in the format you want.

The `/v1/scrape` endpoint fetches web pages by URL, automatically handling headless rendering, proxy rotation, access challenges, and localization. It returns structured page data in Markdown, HTML, JSON, or as PNG screenshots.

## Request sample

The endpoint accepts a JSON payload defining the target URL, requested output formats, rendering options, and geo-location context.

```bash
curl https://webapi.oxylabs.io/v1/scrape \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
      "url": "https://artificialintelligenceact.eu/implementation-timeline/",
      "output": ["markdown"]
  }'
```

### Request parameters

<table><thead><tr><th width="98">Parameter</th><th width="566">Description</th><th width="93">Type</th></tr></thead><tbody><tr><td><mark style="background-color:green;"><code>url</code></mark></td><td>Absolute <code>http</code> / <code>https</code> target URL.</td><td>string</td></tr><tr><td><code>output</code></td><td>Response format: <code>markdown</code>, <code>html</code>, <code>json</code>, <code>screenshot</code>. Supports multiple values. Default: <code>html</code></td><td>array of strings</td></tr><tr><td><code>json</code></td><td>When <code>output: ["json"]</code>, can be used for custom AI parsing through <code>prompt</code> or <code>schema</code>. See <a href="/products/web-api/scrape/ai-parsing.md">AI Parsing</a>.</td><td>object</td></tr><tr><td><code>location</code></td><td>Two-letter ISO 3166-1 alpha-2 country code (e.g., <code>"US"</code>, <code>"DE"</code>, <code>"JP"</code>).</td><td>string</td></tr><tr><td><code>device</code></td><td>User-agent and viewport emulation: <code>"desktop"</code> (default) or <code>"mobile"</code>. </td><td>string</td></tr><tr><td><code>run_js</code></td><td>Enables JavaScript rendering when set to <code>true</code>. Default: <code>false</code>.</td><td>boolean</td></tr></tbody></table>

&#x20;    – mandatory parameter.

<details>

<summary><strong><code>output</code></strong></summary>

The `output` array allows fetching multiple output formats in a single request execution. Below is an example of combined output request:

```json
{
  "url": "https://example.com/product",
  "output": ["markdown", "json"],
  "json": {
    "prompt": "Extract the product name, stock status, and price as a number."
  }
}
```

<table data-header-hidden><thead><tr><th width="119"></th><th></th></tr></thead><tbody><tr><td><code>markdown</code></td><td>Converts the DOM tree into clean, structured Markdown, stripping navigation scripts and styling formatting. Optimized for ingestion into LLM context windows to minimize token usage.</td></tr><tr><td><code>html</code></td><td>The raw page markup. Use it when you need the markup itself (attributes, embedded JSON-LD, inline data). Not recommended for page content collection.</td></tr><tr><td><code>screenshot</code></td><td>Base64-encoded PNG image of the fully rendered page. <strong><code>run_js: true</code> is required</strong>, otherwise the request is rejected.</td></tr><tr><td><code>json</code></td><td>Structured extraction in the same request as the retrieval (no selectors to write or maintain). Describe the fields in a <code>prompt</code>, or pin their exact shape with a <code>schema</code>. See <a href="/products/web-api/scrape/ai-parsing.md">AI parsing</a>.</td></tr></tbody></table>

{% hint style="info" %}
**Exception:** `video` output format is available for `/scrape/media` [dedicated scrapers](/products/web-api/scrape/dedicated-scrapers.md).
{% endhint %}

</details>

<details>

<summary><strong><code>run_js</code></strong></summary>

Executes the page's own JavaScript before the content is captured.

Dropping `run_js` defaults to the optimal execution path for the given source path. Explicitly setting `"run_js": true` uses a headless browser instance to execute client-side scripts, dynamic XHR requests, and Single Page Application (SPA) rendering.

```json
{
  "url": "https://example.com/spa-dashboard",
  "output": ["markdown"],
  "run_js": true
}
```

* **Latency impact:** Headless browser rendering increases request duration significantly. Set client HTTP timeouts to at least `150` seconds. When high volume is required, [async delivery](/products/web-api/scrape/delivery-modes.md) is recommended.
* **Resource optimization:** Always try fetching content without it. For static pages or RSS/atom feeds, pass `"run_js": false` to maximize request throughput and minimize latency. Choose rendering only when necessary.

</details>

<details>

<summary><strong><code>location</code></strong></summary>

Geo-targeting routes requests through localized proxy exit nodes. This ensures localized pricing, language variations, and region-exclusive content are served correctly.

```json
{
  "url": "https://example.com/pricing",
  "output": ["html"],
  "location": "DE",
  "check_empty_geo": true
}
```

{% hint style="info" %}
When using [Dedicated Scrapers](/products/web-api/scrape/dedicated-scrapers.md), some localization parameters may be named differently.
{% endhint %}

</details>

## Output sample

A successful request returns an HTTP `200 OK` status with a standard envelope. Fields inside `results[]` with unrequested output formats return `null`.

```json
{
  "status": 200,
  "results": [
    {
      "html": null,
      "markdown": "# Implementation Timeline\n\nProviders need to be compliant...",
      "screenshot": null,
      "json": null,
      "metadata": {
        "page": 1,
        "run_js": false,
        "url": "https://artificialintelligenceact.eu/implementation-timeline/"
      }
    }
  ],
  "params": {
    "url": "https://artificialintelligenceact.eu/implementation-timeline/",
    "output": ["markdown"],
    "json": { "schema": null, "prompt": null },
    "location": null,
    "device": "desktop",
    "run_js": null
  },
  "metadata": {
    "timestamp": 1787431568,
    "request_id": "7503078851572948993"
  }
}
```

{% hint style="info" %}
**`results[]` field with unrequested output formats return `null`.** \
For example, asking for `["markdown"]` populates `markdown` and leaves `html`, `screenshot` and `json` as null. Any [dedicated scraper](/products/web-api/scrape/dedicated-scrapers.md) puts its structured fields under `json`.
{% endhint %}

### Output dictionary

<table><thead><tr><th width="208">Field</th><th>Description</th><th width="92">Type</th></tr></thead><tbody><tr><td><code>status</code></td><td>HTTP status code, <code>200</code> on success. See <a href="/products/web-api/troubleshooting.md">Error codes</a>.</td><td>integer</td></tr><tr><td><code>results</code></td><td>One entry for the requested URL.</td><td>array</td></tr><tr><td><code>results[].markdown</code></td><td>Page as Markdown, if requested. <code>null</code> otherwise.</td><td>string</td></tr><tr><td><code>results[].html</code></td><td>Raw result markup, if requested. <code>null</code> otherwise.</td><td>string</td></tr><tr><td><code>results[].json</code></td><td>Parsed fields from a <a href="/products/web-api/scrape/dedicated-scrapers.md">dedicated scraper</a> or <a href="/products/web-api/scrape/ai-parsing.md">AI parser</a>, if requested <code>null</code> otherwise.</td><td>object</td></tr><tr><td><code>results[].screenshot</code></td><td>Base64-encoded image string, if requested. <code>null</code> otherwise.</td><td>string</td></tr><tr><td><code>results[].metadata.url</code></td><td>Fetched URL for scraping after redirects.</td><td>string</td></tr><tr><td><code>params</code></td><td>All parameters use for the requests with applied defaults.</td><td>object</td></tr><tr><td><code>metadata.timestamp</code></td><td>Unix timestamp in seconds.</td><td>integer</td></tr><tr><td><code>metadata.request_id</code></td><td>Identifier for this request. <strong>Include it in support tickets.</strong></td><td>string</td></tr></tbody></table>

## Latency and concurrency

A scrape renders a real page with budget in **seconds, not milliseconds**. We recommend a couple of of safe practices to build around that:

<table data-header-hidden><thead><tr><th width="123"></th><th></th></tr></thead><tbody><tr><td><strong>Client timeout</strong></td><td>150 seconds is a safe default, while a 10-second timeout may fail on pages that would have succeeded. For higher volume (e.g. media, large SERP runs, etc.) with a risk of client timeout, run the same scrapers with async delivery instead of blocking the request. See <a href="/products/web-api/scrape/delivery-modes.md">Delivery Modes</a>.</td></tr><tr><td><strong>Concurrency limit</strong></td><td>Scraping ten search results in parallel multiplies cost and latency for pages you won't read. Scrape the two or three that plausibly answer the question and only increase concurrency gradually until <code>429</code> error is reached.</td></tr></tbody></table>

## Dedicated scrapers

Beyond generic URL scraping, target-specific scrapers can be used with `POST /v1/scrape/{target}/{collection}`. They accept target-specific parameters (e.g., ASINs, browse nodes, store zip codes) and return structured fields under `results[].json`:

```bash
curl https://webapi.oxylabs.io/v1/scrape/amazon/product \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "B0935DN1BN",
    "output": ["json"],
    "location": "10115",
    "domain": "de",
    "currency": "EUR"
  }'
```

See [Dedicated scrapers](/products/web-api/scrape/dedicated-scrapers.md) for target overviews (Google, Amazon, YouTube, Bing, AI & LLMs, Commerce, etc.).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://developers.oxylabs.io/products/web-api/scrape.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.