Global Parameter Values
Below is the list of global parameter values which work the same way in all data sources. On the right side menu you can see the full list of parameters and click on any of them to jump to its section.
A callback is a
POST request that we send to your machine, informing you that the data extraction task is completed and the result is ready to be downloaded. A callback URL, in turn, is the URL we will send the POST request to.Use the
callback_url parameter to specify the URL where we should POST our notification to.The
render parameter lets you enable JavaScript execution. Use it when the target website requires JavaScript to load the necessary content or when the site uses an anti-bot tool that utilizes JavaScript to fingerprint their visitors. JavaScript rendering takes more time to scrape the page. Please set timeout on the client side to 180 seconds if using Realtime or Proxy Endpoint integration methods.
There are two available values for this parameter: html (get raw output) and png (get a Base64-encoded screenshot).
Value | Description |
|---|---|
html | The output will include an HTML result. |
png | The output will include a PNG screenshot of the result. |
source parameter is one of the most important parameters. It sets the scraper that we will use to process your request (e.g. google_search, baidu, etc.). You can find the exact
source parameter value on the documentation page of the domain of your choice.The section below outlines the product-domain-source relationships.
E-Commerce Scraper API
Domain | Sources |
|---|---|
amazon, amazon_bestsellers, amazon_pricing, amazon_product, amazon_questions, amazon_reviews, amazon_search, amazon_sellers | |
google, google_shopping_search, google_shopping_product, google_shopping_pricing | |
wayfair, wayfair_search | |
universal_ecommerce |
SERP Scraper API
Real Estate Scraper API
Web Scraper API
Domain | Sources |
|---|---|
universal |
The
user_agent_type parameter lets you specify what kind of a User-Agent header value we should use to fulfil your request.You can find a complete list of available values in the attachment or in the table below.
user_agent_type.json
1KB
Code
user_agent_type parameter values
Value | Description |
|---|---|
desktop | A User-Agent of a desktop browser. |
desktop_chrome | A User-Agent of one of the latest versions of a desktop Chrome browser. |
desktop_edge | A User-Agent of one of the latest versions of a desktop Edge browser. |
desktop_firefox | A User-Agent of one of the latest versions of a desktop Firefox browser. |
desktop_opera | A User-Agent of one of the latest versions of a desktop Opera browser. |
desktop_safari | A User-Agent of one of the latest versions of a desktop Safari browser. |
mobile | A User-Agent of a mobile browser. |
mobile_android | A User-Agent of one of the latest versions of an Android mobile browser. |
mobile_ios | A User-Agent of one of the latest versions of an iOS browser. |
tablet | A User-Agent of a tablet browser. |
tablet_android | A User-Agent of one of the latest versions of an Android tablet browser. |
tablet_ios | A User-Agent of one of the latest versions of an iPad tablet browser. |
You can add
client_notes parameter when submitting a job via v1/queries endpoint. These client notes are then saved to database and to s3 storage along other job information.{
"source": "universal",
"url": "https://sandbox.oxylabs.io",
"client_notes": "123456789"
}
{
"_links": [
{
"href": "http://data.oxylabs.io/v1/queries/7099386176976459777",
"method": "GET",
"rel": "self"
},
{
"href": "http://data.oxylabs.io/v1/queries/7099386176976459777/results",
"method": "GET",
"rel": "results"
}
],
"callback_url": null,
"client_id": 1234,
"client_notes": "123456789",
....
Last modified 10d ago