Global Parameter Values
Below is the list of global parameter values which work the same way in all data sources. On the right side menu you can see the full list of parameters and click on any of them to jump to its section.
Callback_url
A callback is a POST
request that we send to your machine, informing you that the data extraction task is completed and the result is ready to be downloaded. A callback URL, in turn, is the URL we will send the POST request to.
Parameter values
Use the callback_url
parameter to specify the URL where we should POST
our notification to.
Render
The render
parameter lets you enable JavaScript execution. Use it when the target website requires JavaScript to load the necessary content or when the site uses an anti-bot tool that utilizes JavaScript to fingerprint their visitors.
JavaScript rendering takes more time to scrape the page. Please set timeout on the client side to 180 seconds if using Realtime or Proxy Endpoint integration methods.
Parameter values
There are two available values for this parameter: html (get raw output) and png (get a Base64-encoded screenshot).
Value | Description |
---|---|
| The output will include an HTML result. |
| The output will include a PNG screenshot of the result. |
Source
source
parameter is one of the most important parameters. It sets the scraper that we will use to process your request (e.g. google_search
, baidu
, etc.).
Parameter values
You can find the exact source
parameter value on the documentation page of the target of your choice.
The section below outlines the product-target-source relationships.
E-Commerce Scraper API
Target | Sources |
---|---|
| |
| |
| |
|
SERP Scraper API
Target | Sources |
---|---|
| |
| |
| |
|
Web Scraper API
Target | Sources |
---|---|
All targets (Including: Booking.com, Expedia, Hotels.com, Priceline, Trivago, Zillow, Redfin, Zoopla, Reddit and many more) |
|
Parse
For some sources of SERP Scraper API and E-Commerce Scraper API , we have developed dedicated parsers. If you want to get structured data, add the parse
parameter. Find more details in the respective target pages.
Target | Sources |
---|---|
| |
| |
| |
| |
| |
| |
|
User_agent_type
The user_agent_type
parameter lets you specify what kind of a User-Agent header value we should use to fulfil your request.
You can find a complete list of available values in the attachment or in the table below.
Parameter values
Value | Description |
---|---|
| A User-Agent of a desktop browser. |
| A User-Agent of one of the latest versions of a desktop Chrome browser. |
| A User-Agent of one of the latest versions of a desktop Edge browser. |
| A User-Agent of one of the latest versions of a desktop Firefox browser. |
| A User-Agent of one of the latest versions of a desktop Opera browser. |
| A User-Agent of one of the latest versions of a desktop Safari browser. |
| A User-Agent of a mobile browser. |
| A User-Agent of one of the latest versions of an Android mobile browser. |
| A User-Agent of one of the latest versions of an iOS browser. |
| A User-Agent of a tablet browser. |
| A User-Agent of one of the latest versions of an Android tablet browser. |
| A User-Agent of one of the latest versions of an iPad tablet browser. |
Client notes
You can add client_notes
parameter when submitting a job via v1/queries
endpoint. These client notes are then saved to database and to s3 storage along other job information.
Payload example
Response example
Last updated