> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/products/cn/web-api.md).

# Web API

了解如何使用和集成 Oxylabs Web API——用于网页搜索和页面数据采集的一体化 REST API。

Oxylabs Web API 通过统一的 REST 接口，为应用程序和 AI 代理提供对开放网络的实时访问。它通过不同的端点处理 JavaScript 渲染、机器人流量挑战、代理轮换和地理定位，满足网页数据获取各阶段的需求：

<table><thead><tr><th width="106">端点</th><th width="504">用途</th><th width="148">输入</th></tr></thead><tbody><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/search.md">/search</a></td><td>发现并排序网页自然搜索结果。</td><td>关键字 / 查询</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/scrape.md">/scrape</a></td><td>将页面渲染并提取为 Markdown、HTML、JSON 或截图。</td><td>页面 URL</td></tr></tbody></table>

{% hint style="success" %}
Web API 端点旨在配合使用： **用 search 查找，用 scrape 阅读。** `/search` 的描述只是摘要，因此任何需要准确掌握的信息都应来自 `/scrape`.
{% endhint %}

## 使用方法

### 获取 API 密钥

在 [Oxylabs 控制面板](https://dashboard.oxylabs.io)中创建一个 **Web API** 实例，从产品实例中获取 API 密钥，并将其导出到项目环境中（例如操作系统终端，或项目工作区中的 CLI 终端）：

```bash
export OXYLABS_WEB_API_KEY=your_api_key_here
```

有关密钥管理和轮换，请参阅 [身份验证](https://developers.oxylabs.io/products/cn/web-api/shen-fen-yan-zheng.md) 。

### 选择集成方式

* **要将其接入 AI 代理（Claude Code、Cursor 或你自己的代理）？**\
  查看 [AI 代理集成](#ai-agent-integration) ，只需一条命令即可开始使用。
* **要构建自己的集成？**\
  按照以下步骤直接使用 REST 端点进行集成。

### 手动集成

{% stepper %}
{% step %}

#### 搜索

{% tabs %}
{% tab title="cURL" %}

```bash
curl https://webapi.oxylabs.io/v1/search \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"query": "eu ai act compliance deadlines", "max_results": 3}'
```

{% endtab %}

{% tab title="Python" %}

```python
import os, requests

resp = requests.post(
    "https://webapi.oxylabs.io/v1/search",
    headers={"Authorization": f"Bearer {os.environ['OXYLABS_WEB_API_KEY']}"},
    json={"query": "eu ai act compliance deadlines", "max_results": 3},
    timeout=120,
)
resp.raise_for_status()

for r in resp.json()["results"]:
    print(r["metadata"]["position"], r["title"], r["url"], sep=" | ")
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const res = await fetch("https://webapi.oxylabs.io/v1/search", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OXYLABS_WEB_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ query: "eu ai act compliance deadlines", max_results: 3 }),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);

const { results } = await res.json();
results.forEach((r) => console.log(r.metadata.position, r.title, r.url));
```

{% endtab %}
{% endtabs %}

**输出示例**

```json
{
  "state": "done",
  "results": [
    {
      "title": "Implementation Timeline | EU Artificial Intelligence Act",
      "outline": [
        "Date 2 August 2025 Providers: need to be compliant with the AI Act by 2 August 2027..."
      ],
      "url": "https://artificialintelligenceact.eu/implementation-timeline/",
      "metadata": { "position": 1 }
    }
  ],
  "params": { "query": "eu ai act compliance deadlines", "location": null, "max_results": 3 },
  "metadata": { "timestamp": 1787431568, "request_id": "1788875675880097469" }
}
```

请参阅 [/search](https://developers.oxylabs.io/products/cn/web-api/search.md) 了解请求参数和输出字典。
{% endstep %}

{% step %}

#### 抓取

读取由 `/search` 收集的页面，或读取您的 URL 列表：

{% tabs %}
{% tab title="cURL" %}

```bash
curl https://webapi.oxylabs.io/v1/scrape \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
      "url": "https://artificialintelligenceact.eu/implementation-timeline/",
      "output": ["markdown"]
  }'
```

{% endtab %}

{% tab title="Python" %}

```python
page = requests.post(
    "https://webapi.oxylabs.io/v1/scrape",
    headers={"Authorization": f"Bearer {os.environ['OXYLABS_WEB_API_KEY']}"},
    json={
        "url": "https://artificialintelligenceact.eu/implementation-timeline/",
        "output": ["markdown"],
    },
    timeout=120,
)
page.raise_for_status()
print(page.json()["results"][0]["markdown"][:2000])
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
const page = await fetch("https://webapi.oxylabs.io/v1/scrape", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OXYLABS_WEB_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://artificialintelligenceact.eu/implementation-timeline/",
    output: ["markdown"],
  }),
});
if (!page.ok) throw new Error(`${page.status} ${await page.text()}`);

const { results } = await page.json();
console.log(results[0].markdown.slice(0, 2000));
```

{% endtab %}
{% endtabs %}

请参阅 [/scrape](https://developers.oxylabs.io/products/cn/web-api/scrape.md) 了解请求参数、输出字典和交付模式。
{% endstep %}

{% step %}

#### 组合使用：从搜索到抓取

搜索摘要（`outline`）经过截断，不足以支持深入分析或 LLM 流程。标准集成模式会查询 `/v1/search` 以发现权威链接，然后将这些 URL 发送到 `/v1/scrape` ，请求 Markdown 输出以去除不必要的 HTML 内容。

下面的 Python 示例在一个工作流中使用了这两个端点：

{% code expandable="true" %}

```python
import os, requests

API = "https://webapi.oxylabs.io"
HEADERS = {"Authorization": f"Bearer {os.environ['OXYLABS_WEB_API_KEY']}"}


def research(question: str, read_top: int = 2) -> list[dict]:
    # 1. Discover target URLs
    hits = requests.post(
        f"{API}/v1/search",
        headers=HEADERS,
        json={"query": question, "max_results": 10},
        timeout=120,
    )
    hits.raise_for_status()
    # 2. Extract full page content as Markdown
    pages = []
    for hit in hits.json()["results"][:read_top]:
        page = requests.post(
            f"{API}/v1/scrape",
            headers=HEADERS,
            json={"url": hit["url"], "output": ["markdown"]},
            timeout=120,
        )
        if not page.ok:
            print(f"skipped {hit['url']}: {page.status_code}")
            continue
        pages.append(
            {
                "url": hit["url"],
                "title": hit["title"],
                "content": page.json()["results"][0]["markdown"],
            }
        )
    return pages


for page in research("eu ai act compliance deadlines"):
    print(f"--- {page['title']} ({page['url']})")
    print(page["content"][:500])
```

{% endcode %}
{% endstep %}
{% endstepper %}

### AI 智能体集成

将 Web API 直接连接到 AI 框架、编程助手（Cursor、Claude Code）或自定义智能体。

#### 选项 1：集成提示词

将以下完整提示词粘贴到 AI 编程智能体（Claude Code、Cursor、Copilot）中，指示其按照我们的技术规范构建自定义集成：

{% code expandable="true" %}

```
Integrate the Oxylabs Web API into this project for live web search and page reading.

Read https://developers.oxylabs.io/products/web-api/for-agents.md and follow
it. It has the base URL, auth, both endpoint schemas, reference code, the error and retry
rules, and the checklist to satisfy. If the page does not load, stop and tell me; do not
reconstruct the API from memory.

Decide the path and tell me before writing anything:
- Will this project's own code call the API at runtime? -> write the client the page
  specifies, matching how this codebase already makes outbound HTTP calls.
- Is the web access for you during this session rather than for the project's code?
  -> install the MCP server and the skills, the way the page describes.

The key is in OXYLABS_WEB_API_KEY or a .env here. If it is in neither, ask me. Never
print the key: not in code, logs, tests, or your replies to me.

Prove it works before writing tests: make one real search, scrape one URL from the
results, and show me both request ids. If a call returns 401, stop and tell me.

When a scrape comes back empty or skeletal, retry it once with run_js: true. If it is still
empty and the site is on a country TLD (.lt, .es, .co.uk), retry once more with run_js and
将位置设为该国家，然后报告页面无法读取。

然后针对模拟传输层测试重试和错误映射，并运行测试。
如果你无法在此环境中运行命令，请说明一次，并提供准确的
替代运行命令。

请汇报：你采取的步骤、页面上的每项检查清单内容是否已完成、
我还需要自行完成的事项，以及你未能完成的内容。不要询问我
是否继续；直接开始工作。
```

{% endcode %}

该提示会引导代理查看一个页面，其中包含完整的产品权威信息：基础 URL、授权方式、两个端点的架构、参考代码、错误和重试规则，以及代理检查清单。

完整的代理规范参考，请参阅 [代理指南](https://developers.oxylabs.io/products/cn/web-api/for-agents.md) 页面。

{% hint style="info" %}
**如果代理无法获取 URL**，请直接提供完整页面。 `for-agents.md` 是纯 Markdown 文档，内容完整自足，因此可以保存该文件或将其粘贴到提示前面，并删除提及 URL 的段落。不要将规范本身放进提示中：其中重复列出的两条重试规则，是因为代理会在任务执行过程中应用它们；其余内容则应放在页面上，以免出现偏差。
{% endhint %}

#### 选项 2：代理技能

安装官方代理技能，向 AI 代理传授研究方法、来源引用规则以及从搜索到抓取的执行流程：

```bash
npx skills add oxylabs/web-api-skills
```

如需了解其他安装方式，请参阅 [Oxylabs 代理技能](https://developers.oxylabs.io/ai-workflows/agent-skills).

#### 选项 3：MCP 服务器

连接官方 MCP 服务器或自行托管该服务器，即可将 Web API 搜索、抓取和专用爬虫工具直接提供给 Claude Code、Cursor 或自定义 AI 代理。只需将代理指向由 Oxylabs 托管的服务器： `https://mcp.oxylabs.io/web-api-mcp`:

<details>

<summary><i class="fa-claude">:claude:</i>  <strong>Claude Code</strong></summary>

```bash
claude mcp add --transport http --scope user oxylabs-web-api \
  https://mcp.oxylabs.io/web-api-mcp \
  --header "Authorization: Bearer ${OXYLABS_WEB_API_KEY}"
```

</details>

<details>

<summary><i class="fa-openai">:openai:</i>  <strong>ChatGPT/Codex</strong></summary>

```bash
codex mcp add oxylabs-web-api \
  --url https://mcp.oxylabs.io/web-api-mcp \
  --bearer-token-env-var OXYLABS_WEB_API_KEY
```

</details>

<details>

<summary><i class="fa-cursor">:cursor:</i>  <strong>Cursor</strong> （或其他支持自定义标头的客户端）</summary>

对于支持带自定义标头的远程 MCP 服务器的客户端，请将服务器 URL 和一个 `Authorization` 标头添加到客户端的 MCP 配置中。例如，在 Cursor 的 `mcp.json`:

```json
{
  "mcpServers": {
    "oxylabs-web-api": {
      "url": "https://mcp.oxylabs.io/web-api-mcp",
      "headers": {
        "Authorization": "Bearer ${env:OXYLABS_WEB_API_KEY}"
      }
    }
  }
}
```

不同客户端的字段名称可能不同；请查阅客户端文档，了解如何添加带标头的远程 MCP 服务器。

</details>

如需了解有关 Web API MCP 或自行托管官方 Oxylabs 服务器的更多信息，请参阅 [MCP 服务器](https://developers.oxylabs.io/ai-workflows/mcp).

#### 技能还是 MCP？

<table data-header-hidden><thead><tr><th width="210"></th><th></th><th></th></tr></thead><tbody><tr><td></td><td><a href="https://developers.oxylabs.io/ai-workflows/agent-skills"><strong>技能</strong></a></td><td><a href="https://developers.oxylabs.io/ai-workflows/mcp"><strong>MCP 服务器</strong></a></td></tr><tr><td>为代理提供</td><td>判断力：何时搜索、信任哪些来源以及如何引用</td><td>可直接调用的类型化工具</td></tr><tr><td>需要运行中的进程</td><td>否</td><td>是</td></tr><tr><td>适用于</td><td>加载技能目录的智能体</td><td>任何 MCP 客户端</td></tr></tbody></table>

也可以同时安装两者。服务器会打包 `oxylabs-web-api` 并将其作为 `oxylabs://skill/web-api` 资源提供，因此只连接服务器就能让智能体同时获得两部分功能。如果希望不依赖运行中的服务器使用这些技能，或者希望将代码库迁移到 Oxylabs，请单独安装这些技能。

## 了解更多

<table data-header-hidden><thead><tr><th width="170"></th><th></th></tr></thead><tbody><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/shen-fen-yan-zheng.md"><strong>身份验证</strong></a></td><td>密钥处理、请求头要求和身份验证故障排除。</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/search.md"><strong>搜索</strong></a></td><td>的完整请求参数和响应架构 <code>/v1/search</code>.</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/search/company.md"><strong>公司搜索</strong></a></td><td>的请求参数和输出字典 <code>/v1/search/companies</code>.</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/scrape.md"><strong>抓取</strong></a></td><td>的请求参数和输出字典 <code>/v1/scrape</code>.</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/scrape/zhuan-yong-zhua-qu-qi.md"><strong>专用爬虫</strong></a></td><td>针对特定目标解析的端点（Google、Amazon、YouTube、Walmart 等）。</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/scrape/jiao-fu-mo-shi.md"><strong>交付模式</strong></a></td><td>Realtime 调用、异步单个任务、批量队列和云存储。</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/scrape/ai-jie-xi.md"><strong>AI 解析</strong></a></td><td>使用自然语言提示或 JSON Schema 提取结构化 JSON。</td></tr><tr><td><a href="https://developers.oxylabs.io/products/cn/web-api/gu-zhang-pai-cha.md"><strong>故障排除</strong></a></td><td>HTTP 状态码、重试逻辑和故障排除提示。</td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://developers.oxylabs.io/products/cn/web-api.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
