Global Parameter Values

Below is the list of global parameter values which work the same way in all data sources. On the right side menu you can see the full list of parameters and click on any of them to jump to its section.

Callback_url

A callback is a POST request that we send to your machine, informing you that the data extraction task is completed and the result is ready to be downloaded. A callback URL, in turn, is the URL we will send the POST request to.

Parameter values

Use the callback_url parameter to specify the URL where we should POST our notification to.

Render

The render parameter lets you enable JavaScript execution. Use it when the target website requires JavaScript to load the necessary content or when the site uses an anti-bot tool that utilizes JavaScript to fingerprint their visitors.

JavaScript rendering takes more time to scrape the page. Please set timeout on the client side to 180 seconds if using Realtime or Proxy Endpoint integration methods.

Parameter values

There are two available values for this parameter: html (get raw output) and png (get a Base64-encoded screenshot).

ValueDescription

html

The output will include an HTML result.

png

The output will include a PNG screenshot of the result.

Source

source parameter is one of the most important parameters. It sets the scraper that we will use to process your request (e.g. google_search, baidu, etc.).

Parameter values

You can find the exact source parameter value on the documentation page of the target of your choice.

The section below outlines the product-target-source relationships.

E-Commerce Scraper API

TargetSources

amazon, amazon_bestsellers, amazon_pricing, amazon_product, amazon_questions, amazon_reviews, amazon_search, amazon_sellers

google_shopping, google_shopping_search, google_shopping_product, google_shopping_pricing

wayfair, wayfair_search

universal_ecommerce

SERP Scraper API

TargetSources

google, google_search, google_ads, google_images, google_lens, google_maps, google_travel_hotels, google_suggest, google_trends_explore

yandex, yandex_search

bing, bing_search

baidu, baidu_search

Web Scraper API

TargetSources

universal

Parse

For some sources of SERP Scraper API and E-Commerce Scraper API , we have developed dedicated parsers. If you want to get structured data, add the parse parameter. Find more details in the respective target pages.

TargetSourcesPage types

google, google_search, google_ads, google_images

Web search, images search, news search, reverse image search pages.

bing, bing_search

Search pages.

google_shopping, google_shopping_search, google_shopping_product, google_shopping_pricing

Search, product and offer listing pages.

amazon, amazon_search, amazon_product, amazon_pricing, amazon_reviews, amazon_questions, amazon_bestsellers, amazon_sellers

Search, product, offer listing, reviews, questions and answers, best sellers and sellers pages.

universal_ecommerce

Product pages.

universal_ecommerce

Product pages.

universal_ecommerce

Search, product and vendor pages.

universal_ecommerce

Search and product pages

User_agent_type

The user_agent_type parameter lets you specify what kind of a User-Agent header value we should use to fulfil your request.

You can find a complete list of available values in the attachment or in the table below.

Parameter values

ValueDescription

desktop

A User-Agent of a desktop browser.

desktop_chrome

A User-Agent of one of the latest versions of a desktop Chrome browser.

desktop_edge

A User-Agent of one of the latest versions of a desktop Edge browser.

desktop_firefox

A User-Agent of one of the latest versions of a desktop Firefox browser.

desktop_opera

A User-Agent of one of the latest versions of a desktop Opera browser.

desktop_safari

A User-Agent of one of the latest versions of a desktop Safari browser.

mobile

A User-Agent of a mobile browser.

mobile_android

A User-Agent of one of the latest versions of an Android mobile browser.

mobile_ios

A User-Agent of one of the latest versions of an iOS browser.

tablet

A User-Agent of a tablet browser.

tablet_android

A User-Agent of one of the latest versions of an Android tablet browser.

tablet_ios

A User-Agent of one of the latest versions of an iPad tablet browser.

Client notes

You can add client_notes parameter when submitting a job via v1/queries endpoint. These client notes are then saved to database and to s3 storage along other job information.

Payload example

{
    "source": "universal",
    "url": "https://sandbox.oxylabs.io",
    "client_notes": "123456789"
}

Response example

{
    "_links": [
        {
            "href": "http://data.oxylabs.io/v1/queries/7099386176976459777",
            "method": "GET",
            "rel": "self"
        },
        {
            "href": "http://data.oxylabs.io/v1/queries/7099386176976459777/results",
            "method": "GET",
            "rel": "results"
        }
    ],
    "callback_url": null,
    "client_id": 1234,
    "client_notes": "123456789",
....

Last updated