自定义解析器

查看 Oxylabs 自定义解析器快速入门的步骤。

自定义解析器是网页爬虫API的一项免费功能，可让你 创建解析和数据处理逻辑 该逻辑会在原始 HTML 结果上执行。你可以使用 AI 自动生成解析器，或在高级场景中手动编写。

有关详细说明和示例，请参阅以下页面：

快速入门 通过 API 生成解析器 解析器预设

手动编写说明 解析函数列表

---

快速开始

1. 生成解析器

我们建议从我们的 AI 驱动 OxyCopilot 工具开始，它可以让你无需编写任何代码即可生成爬虫和解析器。

要访问 OxyCopilot，请登录 Oxylabs 控制面板 并选择 Web Scraper API Playground 左侧菜单中的。

按照视频中显示的步骤 生成解析器:

视频中展示的是相同步骤：

1. 输入 URL(s) 你要抓取并解析的
2. 指定任何参数 例如 JavaScript 渲染
3. 编写提示词 描述你想解析的内容
4. 运行 OxyCopilot

当你对生成的解析器满意后，加载指令。

2. 将解析器保存为预设

您可以通过 OxyCopilot 轻松保存生成的解析器，以便日后使用。请参见以下步骤：

1. 将预设分配给 特定 API 用户
2. 点击 保存
3. 输入预设名称 和描述（可选）

保存预设后，您可以在 API 请求中使用它。

新创建的预设最多可能需要约一分钟，才会对 Realtime 和 Push-Pull 上的抓取任务可用。在此之前，引用它的请求会被拒绝，并返回 400 - 提供的 parser_preset 不存在。

3. 使用解析器进行 API 请求

要在网页爬虫API中使用你的预设，请发送一个包含以下内容的有效载荷： parser_preset 参数设置为你的预设名称。在下面的代码示例中，我们复用的是 example_parser 前面步骤中创建的预设。

cURL

curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
        "source": "universal",
        "url": "https://example.com/",
        "parse": true,
        "parser_preset": "example_parser"
    }'

Python

import requests
from pprint import pprint


# Set the parser preset to use.
payload = {
    'source': 'universal',
    'url': 'https://example.com/',
    'parse': True,
    'parser_preset': 'example_parser'
}

# Get a response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('USERNAME', 'PASSWORD'),
    json=payload
)

# Print prettified response to stdout.
pprint(response.json())

Node.js

const https = require("https");

const username = "USERNAME";
const password = "PASSWORD";
const body = {
    source: "universal",
    url: "https://example.com/",
    parse: true,
    parser_preset: "example_parser"
};

const options = {
    hostname: "realtime.oxylabs.io",
    path: "/v1/queries",
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        Authorization:
            "Basic " + Buffer.from(`${username}:${password}`).toString("base64"),
    },
};

const request = https.request(options, (response) => {
    let data = "";

    response.on("data", (chunk) => {
        data += chunk;
    });

    response.on("end", () => {
        const responseData = JSON.parse(data);
        console.log(JSON.stringify(responseData, null, 2));
    });
});

request.on("error", (error) => {
    console.error("Error:", error);
});

request.write(JSON.stringify(body));
request.end();

HTTP

# 你提交的整个字符串必须进行 URL 编码。

https://realtime.oxylabs.io/v1/queries?source=universal&url=https%3A%2F%2Fexample.com%2F&parse=true&parser_preset=example_parser&access_token=12345abcde

PHP

<?php

$params = array(
    'source' => 'universal',
    'url' => 'https://example.com/',
    'parse' => true,
    'parser_preset' => 'example_parser'
);

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "USERNAME" . ":" . "PASSWORD");

$headers = array();
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);
echo $result;

if (curl_errno($ch)) {
    echo 'Error:' . curl_error($ch);
}
curl_close($ch);

Golang

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	const Username = "USERNAME"
	const Password = "PASSWORD"

	payload := map[string]interface{}{
		"source": "universal",
		"url": "https://example.com/",
		"parse": true,
		"parser_preset": "example_parser",
	}

	jsonValue, _ := json.Marshal(payload)

	client := &http.Client{}
	request, _ := http.NewRequest("POST",
		"https://realtime.oxylabs.io/v1/queries",
		bytes.NewBuffer(jsonValue),
	)

	request.SetBasicAuth(Username, Password)
	response, _ := client.Do(request)

	responseText, _ := ioutil.ReadAll(response.Body)
	fmt.Println(string(responseText))
}

C#

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Json;
using System.Threading.Tasks;

namespace OxyApi
{
    class Program
    {
        static async Task Main()
        {
            const string Username = "USERNAME";
            const string Password = "PASSWORD";

            var parameters = new {
                source = "universal",
                url = "https://example.com/",
                parse = true,
                parser_preset = "example_parser"
            };

            var client = new HttpClient();

            Uri baseUri = new Uri("https://realtime.oxylabs.io");
            client.BaseAddress = baseUri;

            var requestMessage = new HttpRequestMessage(HttpMethod.Post, "/v1/queries");
            requestMessage.Content = JsonContent.Create(parameters);

            var authenticationString = $"{Username}:{Password}";
            var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.ASCIIEncoding.UTF8.GetBytes(authenticationString));
            requestMessage.Headers.Add("Authorization", "Basic " + base64EncodedAuthenticationString);

            var response = await client.SendAsync(requestMessage);
            var contents = await response.Content.ReadAsStringAsync();

            Console.WriteLine(contents);
        }
    }
}

Java

package org.example;

import okhttp3.*;
import org.json.JSONObject;
import java.util.concurrent.TimeUnit;

public class Main implements Runnable {
    private static final String AUTHORIZATION_HEADER = "Authorization";
    public static final String USERNAME = "USERNAME";
    public static final String PASSWORD = "PASSWORD";

    public void run() {
        JSONObject jsonObject = new JSONObject();
        jsonObject.put("source", "universal");
        jsonObject.put("url", "https://example.com/");
        jsonObject.put("parse", true);
        jsonObject.put("parser_preset", "example_parser");

        Authenticator authenticator = (route, response) -> {
            String credential = Credentials.basic(USERNAME, PASSWORD);
            return response
                    .request()
                    .newBuilder()
                    .header(AUTHORIZATION_HEADER, credential)
                    .build();
        };

        var client = new OkHttpClient.Builder()
                .authenticator(authenticator)
                .readTimeout(180, TimeUnit.SECONDS)
                .build();

        var mediaType = MediaType.parse("application/json; charset=utf-8");
        var body = RequestBody.create(jsonObject.toString(), mediaType);
        var request = new Request.Builder()
                .url("https://realtime.oxylabs.io/v1/queries")
                .post(body)
                .build();

        try (var response = client.newCall(request).execute()) {
            if (response.body() != null) {
                try (var responseBody = response.body()) {
                    System.out.println(responseBody.string());
                }
            }
        } catch (Exception exception) {
            System.out.println("Error: " + exception.getMessage());
        }

        System.exit(0);
    }

    public static void main(String[] args) {
        new Thread(new Main()).start();
    }
}

JSON

{
    "source": "universal",
    "url": "https://example.com/",
    "parse": true,
    "parser_preset": "example_parser"
}

输出示例

{
  "results": [
    {
      "_request": {...},
      "_response": {...},
      "content": {
        "parse_status_code": 12000,
        "title": "Example Domain"
      },
      "created_at": "2026-09-08 08:26:26",
      "is_render_forced": false,
      "job_id": "7503005818992731137",
      "page": 1,
      "parser_preset": "example_parser",
      "parser_type": "preset",
      "session_info": {...},
      "status_code": 200,
      "type": "parsed",
      "updated_at": "2026-09-08 08:26:26",
      "url": "https://example.com/"
    }
  ]
}

获取已解析任务的 HTML 内容

已解析任务还可以返回其解析所依据的原始 HTML：添加 type 查询参数并使用 raw （或 parsed,raw 同时使用）到 Realtime 端点或 Push-Pull 结果 URL。参见 多格式输出.