> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/products/cn/web-scraper-api.md).

# 网页爬虫API

[**网页爬虫API**](https://oxylabs.io/products/scraper-api/web) 是一个 **一站式网页数据采集解决方案** 旨在从任何公开网站大规模提取实时数据。它涵盖网页抓取的每个阶段，从抓取 URL、提高访问成功率，到数据解析并交付到你首选的存储位置，因此你无需管理代理、请求管理或基础设施。

该工具旨在满足企业级安全标准，包括 SOC 2 Type II 合规，并提供可快速适应的基础设施，能够动态调整以适配目标网站，确保在搜索引擎、电商网站、旅游平台等场景下都具有高成功率和可靠的数据提取。

## 开始使用

**创建你的 API 用户凭据**: 在 [**Oxylabs 控制台**](https://dashboard.oxylabs.io/en/registration) 中注册免费试用或购买产品，以创建你的 API 用户凭据（`USERNAME` 和 `PASSWORD`).

### 请求示例

该 API 会自动管理代理轮换、请求重试，并处理机器人监测系统，作为其集成基础设施的一部分，因此只需一次请求即可检索所有结构化数据。

下面你会找到示例 cURL 请求。其他编程语言的示例，请参考相关章节： [**亚马逊**](/api-targets/cn/dian-zi-shang-wu/amazon.md), [**谷歌**](/api-targets/cn/sou-suo-yin-qing/google.md), [**其他网站**](/api-targets/cn/ren-yi-yu-ming.md).

{% tabs %}
{% tab title="亚马逊" %}

```bash
curl 'https://realtime.oxylabs.io/v1/queries' \
--user "USERNAME:PASSWORD" \
-H "Content-Type: application/json" \
-d '{
        "source": "amazon_product",
        "query": "B07FZ8S74R",
        "geo_location": "90210",
        "parse": true
    }'
```

{% endtab %}

{% tab title="谷歌" %}

```bash
curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
        "source": "google_search",
        "query": "adidas",
        "geo_location": "California,United States",
        "parse": true
    }'
```

{% endtab %}

{% tab title="其他" %}

```bash
curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
        "source": "universal",
        "url": "https://sandbox.oxylabs.io/"
    }'
```

{% endtab %}
{% endtabs %}

我们在示例中使用同步 [**Realtime**](/products/cn/web-scraper-api/integration-methods/realtime.md) 集成方式。如果你想使用 [**Proxy Endpoint**](/products/cn/web-scraper-api/integration-methods/proxy-endpoint.md) 或异步 [**Push-Pull**](/products/cn/web-scraper-api/integration-methods/push-pull.md) 集成，请参阅 [**集成方法**](/products/cn/web-scraper-api/integration-methods.md) 部分。

{% file src="/files/8365b1362a7a5614a3b24348eef8a863a5a2f564" %}

{% file src="/files/b0ad6ed9e7d782f0549a12e6f6ad11e8af390529" %}

### 请求参数值

1. <mark style="background-color:green;">**`来源`**</mark> - 此参数设置将用于处理你的请求的 scraper。
2. <mark style="background-color:green;">**`URL`**</mark> 或 <mark style="background-color:green;">**`query`**</mark> - 请提供 `URL` 或 `query` 对应你想抓取的页面类型。请参考下面的表格及相应的目标子页面，了解何时使用每个参数。
3. 另外，你还可以包含其他参数，例如 `geo_location`, `user_agent_type`, `parse` （我们的解析器列表见 [**这里**](/products/cn/web-scraper-api/features/result-processing-and-storage/dedicated-parsers.md)), `render` 等，以定制你的抓取请求。了解更多： [**功能**](/products/cn/web-scraper-api/features.md).

&#x20;    \- 必填参数

### 使用 URL 或参数化输入进行抓取

Oxylabs 支持两大类输入——URL 和参数化输入，如查询、产品或视频 ID。 [通用目标](/api-targets/cn/ren-yi-yu-ming.md) 没有专用 source 的目标可使用 `universal` source。

<table><thead><tr><th width="217">目标</th><th width="246">来源（抓取 URL）</th><th>来源（使用 Query、Product 或 Video ID）</th></tr></thead><tbody><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/eee0a29a7f9647fa3b2e30da2e23384bf6c62355"><strong>亚马逊</strong></a></td><td><code>亚马逊</code></td><td><p><code>amazon_product</code>,</p><p><code>amazon_search</code>,</p><p><code>amazon_pricing</code>,</p><p><code>amazon_sellers</code>,</p><p><code>amazon_bestsellers</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/55be38a1fe24f7b9e3a53af4997ef9dffa756d02"><strong>谷歌</strong></a></td><td><code>谷歌</code></td><td><p><code>google_search</code>,</p><p><code>google_ads</code>,</p><p><code>google_ai_mode</code>,</p><p><code>google_lens</code>,</p><p><code>google_maps</code>,</p><p><code>google_travel_hotels</code>,</p><p><code>google_trends_explore</code>,</p><p><code>google_shopping_product</code>,</p><p><code>google_shopping_search</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/bdac41b6ea4b822f9c8d579d8cf45fc05a3903d5"><strong>必应</strong></a></td><td><code>必应</code></td><td><code>bing_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/8ba7b9866933c07dbd89a241f943b688c3c9135e"><strong>YouTube</strong></a></td><td><code>universal</code></td><td><p><code>youtube_search</code>,</p><p><code>youtube_search_max</code>,</p><p><code>youtube_video_trainability</code>,</p><p><code>youtube_download</code>,</p><p><code>youtube_subtitles</code>,</p><p><code>youtube_metadata</code>,</p><p><code>youtube_channel</code>,</p><p><code>youtube_autocomplete</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/cfc9663bf38b26995722336a338807c0cc77d1f0"><strong>ChatGPT</strong></a></td><td><code>universal</code></td><td><code>chatgpt</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/715c7b8615cf3035160a3156a3f54bc4b9eaad46"><strong>Perplexity</strong></a></td><td><code>universal</code></td><td><code>perplexity</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/dbc15ce8b27c64b8369dd2a58dc55942642fc42d"><strong>沃尔玛</strong></a></td><td><code>walmart</code></td><td><p><code>walmart_search</code>,</p><p><code>walmart_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/95c60d9ddc962379d85c88888bd1860fb9a0ad7c"><strong>TikTok</strong></a></td><td><code>universal</code></td><td><p><code>tiktok_shop_search</code>,</p><p><code>tiktok_shop_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/92f309e72cd90cf70793edd7ca9afbe2160fba8d"><strong>eBay</strong></a></td><td><code>ebay</code></td><td><p><code>ebay_search</code>,</p><p><code>ebay_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/44db4ff0d01056030840f7333540ddb32428f6fb"><strong>Etsy</strong></a></td><td><code>etsy</code></td><td><p><code>etsy_search</code>,</p><p><code>etsy_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/2d97aa37287d0fceb22cc16a625446a94e94cf42"><strong>Best Buy</strong></a></td><td><code>universal</code></td><td><p><code>bestbuy_search</code>,</p><p><code>bestbuy_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/47f41a2c1b4376833f9b82647133aa5b0853ebf2"><strong>Bed Bath &#x26; Beyond</strong></a></td><td><code>bedbathandbeyond</code></td><td><code>bedbathandbeyond_search</code>,<br><code>bedbathandbeyond_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/ee652f91a33555aaf54b4b77bc12a08fb38dfc9a"><strong>Bodega Aurrerá</strong></a></td><td><code>bodegaaurrera</code></td><td><code>bodegaaurrera_搜索</code>,<br><code>bodegaaurrera_商品</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/6520b2bf9f3ea1eb3065924242bed2a053d4f9df"><strong>Instacart</strong></a></td><td><code>instacart</code></td><td><code>instacart_search</code>,<br><code>instacart_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/5fe1697afe015de72b64a9ea507137b3a756c960"><strong>克罗格</strong></a></td><td><code>kroger</code></td><td><p><code>kroger_search</code>,</p><p><code>kroger_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/10dfba67b248a3c083ec74a128dccda3362a8bee"><strong>Lowe's</strong></a></td><td><code>lowes</code></td><td><p><code>lowes_search</code>,</p><p><code>lowes_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/903682985f543bd9727e3dfd24758f278efd33e8"><strong>Publix</strong></a></td><td><code>publix</code></td><td><code>publix_search</code>,<br><code>publix_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/3a24e533d53eaa117e6aef1037199e33e4d963cf"><strong>目标</strong></a></td><td><code>Target</code></td><td><p><code>target_search</code>,</p><p><code>target_product</code>,</p><p><code>target_category</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/96bcff64cd288a05b6b9ba93a62f48cee6172a6d"><strong>Grainger</strong></a></td><td><code>grainger</code></td><td><code>grainger_search</code>,<br><code>grainger_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/487872e7cd619b0c8519eef8cfb174a05759cb06"><strong>Costco</strong></a></td><td><code>costco</code></td><td><p><code>costco_search</code>,</p><p><code>costco_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/018fbcdf634cc44b1c4a997e2176668b3c8f20ae"><strong>Menards</strong></a></td><td><code>menards</code></td><td><code>menards_search</code>,<br><code>menards_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/5b0a11fbb30a35e4cd602c7280262f326e717dc5"><strong>Petco</strong></a></td><td><code>universal</code></td><td><code>petco_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/e81abdfdb3edb9d30dd1f8efdfe8c93d8c148fc7"><strong>Staples</strong></a></td><td><code>universal</code></td><td><code>staples_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/44b70b31868823d71972f521f7122071b87a90f6"><strong>Allegro</strong></a></td><td><code>universal</code></td><td><p><code>allegro_search</code>,</p><p><code>allegro_product</code></p></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/caab1a6fba523b8ef61c06ece512a5f8df34f7cc"><strong>Idealo</strong></a></td><td><code>universal</code></td><td><code>idealo_搜索</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/cf08b80367fc5609a83065706285de471b8f6c9e"><strong>MediaMarkt</strong></a></td><td><code>mediamarkt</code></td><td><code>mediamarkt_search</code>,<br><code>mediamarkt_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/5cd043e8a4f046b8a544db29833ce0d21fb083df"><strong>Cdiscount</strong></a></td><td><code>cdiscount</code></td><td><code>cdiscount_search</code>,<br><code>cdiscount_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/ba34d3ceaeb626e107ab882fe771a37f47c073eb"><strong>Alibaba</strong></a></td><td><code>alibaba</code></td><td><code>alibaba_search</code>,<br><code>alibaba_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/3d7263b39a751500e707601572fc89ea9f58eb5e"><strong>AliExpress</strong></a></td><td><code>aliexpress</code></td><td><code>aliexpress_search</code>,<br><code>aliexpress_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/cbf39a624daaf6ffebb6b85e3fea50e8ce8b0a7b"><strong>IndiaMART</strong></a></td><td><code>indiamart</code></td><td><code>indiamart_search</code>,<br><code>indiamart_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/c296be50720e2ee7ed7e9e918e5b9fe10aa3e7df"><strong>Avnet</strong></a></td><td><code>universal</code></td><td><code>avnet_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/8c84141a2fee5187d2cd99a0b9a3dfd014199357"><strong>Lazada</strong></a></td><td><code>lazada</code></td><td><code>lazada_search</code>,<br><code>lazada_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/68384ded09d2629cf8ce41fe371a6c75d670a103"><strong>Rakuten</strong></a></td><td><code>universal</code></td><td><code>rakuten_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/d1429c0e2aa8cdb2123529fed5e7e191fd2c53c1"><strong>Tokopedia</strong></a></td><td><code>universal</code></td><td><code>tokopedia_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/d9904a47e7a0d595dd00f03824407e3d2feec0b9"><strong>Flipkart</strong></a></td><td><code>flipkart</code></td><td><code>flipkart_search</code>,<br><code>flipkart_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/6665a02b368baff2e5b7df07eca4cd00badc418f"><strong>MercadoLibre</strong></a></td><td><code>universal</code></td><td><code>mercadolibre_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/fa97af3838411695dfbc5e7b814f0d6e9296ac06"><strong>Mercado Livre</strong></a></td><td><code>universal</code></td><td><code>mercadolivre_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/c496aae19de353eeaba3beef7bc3be5f33d70798"><strong>Magazine Luiza</strong></a></td><td><code>magazineluiza</code></td><td><code>magazineluiza_search</code>,<br><code>magazineluiza_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/2fd7a5e7ed9ef01db53a251f7dd14092974d3106"><strong>Falabella</strong></a></td><td><code>falabella</code></td><td><code>falabella_search</code>,<br><code>falabella_product</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/dc51d804ea6966ace76ea06d7c6e24cfd8212bc8"><strong>Dcard</strong></a></td><td><code>universal</code></td><td><code>dcard_search</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/42d0e27c13b743dbf881f5df9e74a732feaa0113"><strong>Airbnb</strong></a></td><td><code>airbnb</code></td><td><code>airbnb_商品</code></td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/71ea357cc9cfa1716fd53a85811095056e26594b"><strong>Zillow</strong></a></td><td><code>zillow</code></td><td>使用 <code>query</code> 不支持该参数</td></tr><tr><td><a href="/spaces/9AaN7B8HNpFdIYXcFlbw/pages/735154d31df06c6a6bd78f40b3fd6cde4fb2d362"><strong>其他网站</strong></a></td><td><code>universal</code></td><td>使用 <code>query</code> 不支持该参数</td></tr></tbody></table>

{% hint style="info" %}
如果你需要帮助完成第一次请求或优化你的设置，我们的 24/7 专家支持团队可通过在线聊天提供服务。
{% endhint %}

## 通过 Web Scraper API Playground 进行测试

试用 [**网页爬虫API**](https://oxylabs.io/products/scraper-api/web) 和 [**OxyCopilot**](https://oxylabs.io/products/scraper-api/ai-web-scraper-copilot) 在 [**Web Scraper API Playground**](https://dashboard.oxylabs.io/?route=/api-playground).

{% embed url="<https://www.youtube.com/watch?v=kDhknxrod6U>" %}

{% embed url="<https://youtu.be/9JoF8_5r5HY?si=61c3Zkx6FrH06PVa>" %}

## 通过 Postman 进行测试

使用 Postman 开始使用我们的 API，Postman 是一个用于发起 HTTP 请求的便捷工具。下载我们的 [**网页爬虫API Postman 集合**](https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FzrXw45naRpCZ0Ku9AjY1%2Fuploads%2FMeGA0TZQMcAFHoVhRSQi%2FWeb%20Scraper%20API.new_postman_collection.json?alt=media\&token=9f51d41b-6604-4eef-b6c1-5024cf52c5bf) 并导入它。该集合包含演示爬虫功能的示例。可根据需要自定义这些示例，或立即开始抓取。

如需逐步说明，请观看下面的视频教程。如果您是 Postman 新手，请查看这个简短的 [**指南**](/integrations/cn/wang-ye-pa-chong-api-ji-cheng/postman.md).

{% embed url="<https://www.youtube.com/watch?v=WOD0mZnu-j0>" %}

{% hint style="info" %}
*此处提供的所有信息均按“现状”基础提供，仅供参考。对于您使用本页所含任何信息，我们不作任何陈述，并且不承担全部责任。 在进行任何类型的抓取活动之前，您应咨询您的法律顾问，并仔细阅读相关网站的服务条款，或获得抓取许可。*
{% endhint %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://developers.oxylabs.io/products/cn/web-scraper-api.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.