SERP Scraper API can render JavaScript when scraping. This is necessary in some Google services, such as Travel.
Below is a quick overview of all the available data
source values we support with Google.Source | Description | Structured data |
|---|---|---|
google | Submit any Google URL you like. | Depends on the URL. |
google_search | SERPs. | Yes. |
google_ads | SERPs, optimized for maximum ad rate. num=10 only. | Yes. |
google_hotels | Legacy Hotels service. | No. |
google_travel_hotels | Travel: Hotels service. | No. |
google_images | Reverse Image Search. | Yes. |
google_suggest | Autocomplete search term suggestions. | Yes. |
google_trends_explore | Trends. | Yes. |
Although we do not have dedicated parsers for each Google source, you can write your own parsing instructions with Custom Parser feature and get structured data.
You can jump to your preferred Google page type by selecting the page link below. Each page contains the parameter table as well as code examples to help you get started with your query.
Using the
locale parameter will allow you to change the interface language (not the results) of the Google Search web page. For example, if you use domain
com and locale parameter de-DE, the results will still be American, but Accept-Language header value will be set to de-DE,de;q=0.8. This would imitate a person from US searching in com domain, who has the UI of his browser set to German language. If you don't use this parameter, we will set 'Accept-Language' parameter to match the domain (i.e.
en-US for com). [
{
"locale":{
"en-ai":{
"description":"Anguilla - English",
"domain":"com.ai"
},
"es-pr":{
"description":"Puerto Rico - Spanish",
"domain":"com.pr"
},
...
"en-by":{
"description":"Belarus - English",
"domain":"by"
},
"en-in":{
"description":"India - English",
"domain":"co.in"
}
}
}
]
[
{
"results_language": "af",
"language": "Afrikaans"
},
{
"results_language": "ar",
"language": "Arabic"
},
...
{
"results_language": "vi",
"language": "Vietnamese"
}
]
There are a few ways you can use the
geo_location parameter to get correctly-localized Google results.- Using Google's Canonical Location Name. It is very straightforward. Just pass us one of the values found on the CSV download here. Example:
"geo_location": "New York,New York,United States". - Using a state name. Strip the first part of Google's Canonical Location Name and pass a
geo_locationvalue in a"State,Country"format. It works with the United States, Australia, India, and other countries with federated states. Example:"geo_location": "California,United States". - Using a country name. To get results localized for the geographical center point of a country, pass an official country name. Example:
"geo_location": "United Kingdom". - Using coordinates and radius. To get hyperlocal search results (beneficial for searches such as "restaurants near me"), you can pass latitude, longitude, and radius values. The following example passes the coordinates of Space Needle in Seattle, WA:
"geo_location": "lat: 47.6205, lng: -122.3493, rad: 25000".
If you pass a misspelled
geo_location parameter, the chances are that either us or Google will interpret and correct it for you. Nonetheless, we recommend combining locale and domain parameters with parameter structures outlined above to get the most accurate results.google_travel_hotels can accept a limited number of geo_location values - please check the attached file below to see geo_location values that don't yield accurate results. google_travel_hotels-bad_geo_locations.csv
24KB
Text
The
google_suggest source accepts only two-letter ISO-3166 country codes, e.g. US, DE, LT, etc. Don't use any other input format with this source.Use
domain parameter to specify which Google top-level domain (TLD) you would like to scrape. The full list of available domains for Google can be found here.The
context:limit_per_page parameter lets you specify that you would like to scrape a few Google Search pages with the same session (IP address and cookie set). This lets you minimize the chance of seeing overlapping organic results (e.g., the last organic result on the first page being the same as the first organic result on the second page).To use this feature, include a JSON array with JSON objects containing the following data:
Parameter | Description | Example |
|---|---|---|
page | The number of the page you would like to scrape. Any integer value greater than 0 will work | 1 |
limit | The number of results on the page in question. Any integer value between 1 and 100 (inclusive) will work. Note: Our API supports continuous scroll. | 90 |
The example below shows a part of the JSON payload that pertains to the
context:limit_per_page parameter:"context": [{
"key": "limit_per_page",
"value": [
{"page": 1, "limit": 10},
{"page": 2, "limit": 90}
]
}]
The example below shows the whole JSON payload:
{
"source": "google_search",
"query": "adidas",
"parse": true,
"context": [
{
"key": "limit_per_page",
"value": [
{"page": 1, "limit": 10},
{"page": 2, "limit": 90}
]
}]
}
Last modified 3mo ago