# 网页解锁器

[**网页解锁器**](https://oxylabs.io/products/web-unblocker) 是一个 **由 AI 驱动的代理解决方案** 它会管理解锁流程，即使面对最难访问的网站，也能提取公开数据。 

{% hint style="info" %}
你可以试用 网页解锁器 **可免费使用 1 周**。注册 Oxylabs [**仪表板**](https://dashboard.oxylabs.io/en/registration) 即可开始。
{% endhint %}

该产品并不是普通代理，而是一个更高级的系统，可管理多种抓取流程，例如：

* 自动代理类型管理
* 浏览器指纹生成
* 自动重试
* 维持会话
* JavaScript 渲染

[**网页解锁器**](https://github.com/oxylabs/web-unblocker) 与普通代理相比，可显著提高成功率。它还支持 **自定义请求头** 由用户定义的，以及 **IP 粘性** 以及 **可复用的 Cookie** 和 **POST 请求**.

### 入门指南

集成 [**网页解锁器**](https://oxylabs.io/products/web-unblocker) 很容易，尤其是如果你之前使用过普通 [**代理**](https://oxylabs.io/products/residential-proxy-pool) 进行网页抓取。唯一的区别是，我们要求你使用 `-k` 或 `--insecure` cURL 标志（或你所选语言中的等效表达式）忽略 SSL 证书。

要使用 网页解锁器 发起请求，你需要使用 `unblock.oxylabs.io:60000` 代理端点。请看下面的 cURL 示例。你可以在其他语言中找到代码示例 [**这里**](/products/cn/web-unblocker/making-requests.md) 或在我们的 [**GitHub**](https://github.com/oxylabs/product-integrations/tree/master/web-unblocker).

{% hint style="info" %}
使用 [**ip.oxylabs.io/location**](https://ip.oxylabs.io/location) 来检查你的 IP 参数——该域名提供来自四个地理位置数据库的信息：MaxMind、IP2Location、DB-IP 和 IPinfo.io。这些参数包括 IP 地址、提供商、国家、城市、邮政编码、ASN、组织名称、时区以及元数据（当数据库披露时）。
{% endhint %}

{% tabs %}
{% tab title="cURL" %}

```bash
curl -k -x https://unblock.oxylabs.io:60000 \\
-U 'USERNAME:PASSWORD' \
'https://ip.oxylabs.io/location'
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

# 在这里使用你的网页解锁器凭据。
USERNAME, PASSWORD = 'YOUR_USERNAME', 'YOUR_PASSWORD'

# 定义代理字典。
代理 = {
  'http': f'http://{USERNAME}:{PASSWORD}@unblock.oxylabs.io:60000',
  'https': f'https://{USERNAME}:{PASSWORD}@unblock.oxylabs.io:60000',
}

response = requests.request(
    'GET',
    'https://ip.oxylabs.io/location',
    verify=False，# 忽略 SSL 证书
    代理=代理,
)

# 将结果页面打印到标准输出
print(response.text)

# 将返回的 HTML 保存到 result.html 文件
with open('result.html', 'w') as f:
    f.write(response.text)
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
import fetch from 'node-fetch';
import { HttpsProxyAgent } from 'https-proxy-agent';

const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';

const agent = new HttpsProxyAgent(
  `https://${username}:${password}@unblock.oxylabs.io:60000`
);

// 忽略证书
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;

const response = await fetch('https://ip.oxylabs.io/location', {
  method: 'get',
  代理：agent,
});

console.log(await response.text());
```

{% endtab %}

{% tab title="PHP" %}

```php
<?php
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://ip.Oxylabs.io/location');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, 'https://unblock.oxylabs.io:60000');
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'YOUR_USERNAME' . ':' . 'YOUR_PASSWORD');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

$result = curl_exec($ch);
echo $result;

if (curl_errno($ch)) {
    echo 'Error:' . curl_error($ch);
}
curl_close($ch);
```

{% endtab %}

{% tab title="Golang" %}

```go
package main

import (
	"crypto/tls"
	"fmt"
	"io/ioutil"
	"net/http"
	"net/url"
)

func main() {
	const Username = "YOUR_USERNAME"
	const Password = "YOUR_PASSWORD"

	proxyUrl, _ := url.Parse(
		fmt.Sprintf(
			"https://%s:%s@unblock.oxylabs.io:60000",
			Username,
			Password,
		),
	)
	customTransport := &http.Transport{Proxy: http.ProxyURL(proxyUrl)}

	// 忽略证书
	customTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true}

	client := &http.Client{Transport: customTransport}
	request, _ := http.NewRequest("GET",
		"https://ip.oxylabs.io/location",
		nil,
	)

	response, _ := client.Do(request)

	responseText, _ := ioutil.ReadAll(response.Body)
	fmt.Println(string(responseText))
}

```

{% endtab %}

{% tab title="C#" %}

```csharp
using System;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

namespace OxyApi
{
    class Program
    {
        static async Task Main(string[] args)
        {
            var webProxy = new WebProxy
            {
                Address = new Uri("https://unblock.oxylabs.io:60000"),
                BypassProxyOnLocal = false,
                UseDefaultCredentials = false,

                Credentials = new NetworkCredential(
                userName: "YOUR_USERNAME",
                password: "YOUR_PASSWORD"
                )
            };

            var httpClientHandler = new HttpClientHandler
            {
                Proxy = webProxy,
            };

            // 忽略证书
            httpClientHandler.ClientCertificateOptions = ClientCertificateOption.Manual;
            httpClientHandler.ServerCertificateCustomValidationCallback =
                (httpRequestMessage, cert, cetChain, policyErrors) =>
                {
                    return true;
                };


            var client = new HttpClient(handler: httpClientHandler, disposeHandler: true);

            Uri baseUri = new Uri("https://ip.oxylabs.io/location");
            client.BaseAddress = baseUri;

            var requestMessage = new HttpRequestMessage(HttpMethod.Get, "");

            var response = await client.SendAsync(requestMessage);
            var contents = await response.Content.ReadAsStringAsync();

            Console.WriteLine(contents);
        }
    }
}
```

{% endtab %}

{% tab title="Java" %}

```java
package org.example;

import org.apache.hc.client5.http.auth.AuthScope;
import org.apache.hc.client5.http.auth.CredentialsProvider;
import org.apache.hc.client5.http.classic.methods.HttpGet;
import org.apache.hc.client5.http.config.RequestConfig;
import org.apache.hc.client5.http.impl.auth.CredentialsProviderBuilder;
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClients;
import org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManagerBuilder;
import org.apache.hc.client5.http.ssl.NoopHostnameVerifier;
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactoryBuilder;
import org.apache.hc.client5.http.ssl.TrustAllStrategy;
import org.apache.hc.core5.http.HttpHost;
import org.apache.hc.core5.http.io.entity.EntityUtils;
import org.apache.hc.core5.http.message.StatusLine;
import org.apache.hc.core5.ssl.SSLContextBuilder;

import java.util.Arrays;
import java.util.Properties;


public class Main {

    public static void main(final String[] args)throws Exception {
        final CredentialsProvider credsProvider = CredentialsProviderBuilder.create()
                .add(new AuthScope("unblock.oxylabs.io", 60000), "USERNAME", "PASSWORD".toCharArray())
                .build();
        final HttpHost target = new HttpHost("https", "ip.oxylabs.io", 443);
        final HttpHost proxy = new HttpHost("https", "unblock.oxylabs.io", 60000);
        try (final CloseableHttpClient httpclient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setProxy(proxy)
                // 我们建议接受我们的证书，而不是允许不安全（http）流量
                .setConnectionManager(PoolingHttpClientConnectionManagerBuilder.create()
                        .setSSLSocketFactory(SSLConnectionSocketFactoryBuilder.create()
                                .setSslContext(SSLContextBuilder.create()
                                        .loadTrustMaterial(TrustAllStrategy.INSTANCE)
                                        .build())
                                .setHostnameVerifier(NoopHostnameVerifier.INSTANCE)
                                .build())
                        .build())
                .build()) {

            final RequestConfig config = RequestConfig.custom()
                    .build();
            final HttpGet request = new HttpGet("/location");
            request.setConfig(config);

            System.out.println("正在执行请求 " + request.getMethod() + " " + request.getUri() +
                    " 通过 " + proxy + " 请求头: " + Arrays.toString(request.getHeaders()));

            httpclient.execute(target, request, response -> {
                System.out.println("----------------------------------------");
                System.out.println(request + "->" + new StatusLine(response));
                EntityUtils.consume(response.getEntity());
                返回 null;
            });
        }
    }
}
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
如果您观察到成功率较低或获取到空内容，请尝试添加额外的 `"x-oxylabs-render: html"` 请求中添加 header。
{% endhint %}

{% hint style="warning" %}
如果 网页解锁器 用于抓取依赖通过 JavaScript 加载数据的网站，请参阅 [**JavaScript 渲染**](/products/cn/web-unblocker/custom-browser-instructions/javascript-rendering.md) 部分。该产品并不适合直接与无头浏览器（例如 Chromium、PhantomJS、Splash 等）及其驱动程序（例如 Playwright、Selenium、Puppeteer 等）一起使用。
{% endhint %}

观看下面的视频，了解一个在不被阻止的情况下抓取难以访问目标的示例：

{% embed url="<https://www.youtube.com/watch?v=KGOsWPF4Wfs>" %}

#### 课程

如果你想进一步了解如何使用 [**网页解锁器**](https://github.com/oxylabs/web-unblocker) 大规模获取数据——我们建议观看这节 Scraping Experts 课程：

{% embed url="<https://experts.oxylabs.io/pages/bypassing-sophisticated-anti-bot-systems>" %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://developers.oxylabs.io/products/cn/web-unblocker.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.