Getting started

Learn to use Oxylabs Custom Parser. On this page, you'll find comprehensive examples, tips, and details on what happens if parsing fails.

How to use Custom Parser

Scenario example

Parse the price of the first product listed on an e-commerce site, https://sandbox.oxylabs.io/products?q=super+mario:

There are three ways to create a parser. This page walks through the quickest one, OxyCopilot, and the other two are covered on their own pages:

• Generate parsers with OxyCopilot
• Generate parsers via API
• Write parsing instructions manually

Generate parsers with OxyCopilot

OxyCopilot lets you describe your needs in plain English to automatically create scrapers and parsers for a website. Learn the basics by following the steps outlined below and check out OxyCopilot documentation for more information.

Open the Web Scraper API Playground on our dashboard to access OxyCopilot.

1

Enter the URL(s)

Click the OxyCopilot button at the top-left side and enter up to 3 URLs of the same page type and click Next. Let's use this sandbox URL: https://sandbox.oxylabs.io/products?q=super+mario.

You can also configure the scraper manually by filling in the Website, Scraper, and URL fields at the top, and adjusting additional parameters like JavaScript rendering in the left-side menu.

2

Set up scraper parameters

Next, specify scraper parameters, browser instructions, and enable JavaScript rendering if your target website requires that.

The sandbox page does not need JavaScript rendering, so keep the defaults and click Next.

Rendering matters when the data you want is not in the HTML the server sends but is added by JavaScript after the page loads, as is common for prices, product listings and reviews. Without rendering, Custom Parser only sees the initial HTML, so instructions targeting that data return nothing. Read more about JavaScript rendering.

3

Write the prompt

Explain the data you want to extract from a page. Make sure to be descriptive and provide the most important information. You can find prompt examples for popular websites in our OxyCopilot prompts library.

Paste the following prompt to extract the price of the first product:

Parse the price of the first product on the page.

Click the Generate instructions button to send your prompt.

4

Review parsed data and instructions

Once OxyCopilot finishes, you'll see the following window where parsed data is on the right-side:

If you want to make any adjustments, you can do so here. Modify the URL(s), refine the prompt, enable JavaScript rendering, or edit the parsing schema to suit your needs. When you update any fields in this window, you can rerun the request by selecting Start new request.

You may also view and directly edit the parsing instructions here:

Once you're happy with the result, Load instructions to continue.

5

Save the parser as a preset

You can easily save your parsing instructions as a parser preset. This lets you reuse the preset in OxyCopilot and with your API requests.

In the Web Scraper API Playground, you can optionally choose the user for which to save the preset. Once you're all set, simply click Save:

A pop-up will appear prompting you to name the preset and add an optional description:

6

Use the preset with API requests

To use a preset with your Web Scraper API requests, set parse to true and specify the preset name with the parser_preset parameter.

Endpoint: POST https://realtime.oxylabs.io/v1/queries

{
    "source": "universal",
    "url": "https://sandbox.oxylabs.io/products",
    "parse": true,
    "parser_preset": "sandbox_first_product_price"
}

Running the request will provide the following JSON output:

{
    "results": [
        {
            "_request": {...},
            "_response": {...},
            "content": {
                "parse_status_code": 12000,
                "price": 91.99
            },
            "created_at": "2026-09-08 11:21:03",
            "is_render_forced": false,
            "job_id": "7503049765014236161",
            "page": 1,
            "parser_preset": "sandbox_first_product_price",
            "parser_type": "preset",
            "session_info": {...},
            "status_code": 200,
            "type": "parsed",
            "updated_at": "2026-09-08 11:21:09",
            "url": "https://sandbox.oxylabs.io/products"
        }
    ]
}

Advanced usage

OxyCopilot is the quickest way to create a parser, but every step of it is also available directly through the API:

• Generate parsing instructions via API – send a prompt or a JSON schema together with a few example URLs and receive ready-to-use instructions. See Generating parsing instructions via API.
• Save and manage presets via API – create, update, list and delete parser presets, and enable self-healing. See Parser Presets.
• Write parsing instructions manually – build your own extraction pipeline with XPath and CSS selectors. Start with How to write parsing instructions, then browse the Parsing instruction examples.

What happens if parsing fails when using Custom Parser

If Custom Parser cannot apply one of your instructions, the job still succeeds: the result carries parse_status_code 12005 (parsed with warnings) and a _warnings list describing which instruction failed and why. You are charged for such results.

For example, this request asks for a field whose XPath matches nothing on the page:

{
    "source": "universal",
    "url": "https://sandbox.oxylabs.io/products",
    "parse": true,
    "parsing_instructions": {
        "first_product_price": {
            "_fns": [
                {
                    "_fn": "xpath_one",
                    "_args": [
                        "//div[contains(@class, 'price-wrapper')]/text()"
                    ]
                },
                {
                    "_fn": "amount_from_string"
                }
            ]
        },
        "number_of_reviews": {
            "_fns": [
                {
                    "_fn": "xpath",
                    "_args": [
                        "//this-will-not-match-anything"
                    ]
                },
                {
                    "_fn": "length"
                }
            ]
        }
    }
}

The price is parsed normally, the failing field is returned as null, with a warning:

{
    "results": [
        {
            "_request": {...},
            "_response": {...},
            "content": {
                "_warnings": [
                    {
                        "_fn": "xpath",
                        "_fn_idx": 0,
                        "_msg": "XPath expressions did not match any data.",
                        "_path": ".number_of_reviews"
                    }
                ],
                "first_product_price": 91.99,
                "number_of_reviews": null,
                "parse_status_code": 12005
            },
            "created_at": "2026-09-08 09:00:40",
            "is_render_forced": false,
            "job_id": "7503014433245923329",
            "page": 1,
            "parser_preset": null,
            "parser_type": "custom",
            "session_info": {...},
            "status_code": 200,
            "type": "parsed",
            "updated_at": "2026-09-08 09:00:44",
            "url": "https://sandbox.oxylabs.io/products"
        }
    ]
}

To ignore a failing field instead of receiving a warning, see Parsing instruction examples › Error handling.

If Custom Parser encounters an exception and breaks during the parsing operation, it can return status codes 12002, 12006, and 12007. You will not be charged for these unexpected errors.

Instructions referencing a non-existent function are rejected upon submission and no job is created. The API returns an HTTP 400 error naming the missing function and its path: function not_a_real_fn is not defined.

Status codes

See our status codes outlined here.