Oxylabs Documentation
English
Search
K

URL

The google source is designed to retrieve content from various Google URLs. This means that instead of sending multiple parameters, you can provide us with a direct URL to the required Google page. We do not strip any parameters or alter your URLs in any other way.
This data source also supports parsed data (structured data in JSON format), as long as the URL submitted is for Google (SERP page). If we cannot confirm this is a SERP page request, we will return a failure message.

Query parameters

Parameter
Description
Default Value
source
Data source. More info.
google
url
Direct URL (link) to Google page
-
user_agent_type
Device type and browser. The full list can be found here.
desktop
render
Enables JavaScript rendering. More info.
callback_url
URL to your callback endpoint. More info.
-
geo_location
The results will be adapted for geographical location. Using this parameter correctly is extremely important to get accurate data. For more information, read about our suggested geo_location parameter structures here.
-
parse
true will return parsed data, as long as the URL submitted is for Google.
-
- required parameter

Code examples

In the example below, the API will retrieve a Google Scholar search page.
JSON
cURL
Python
PHP
HTTP
{
"source": "google",
"url": "https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp="
}
curl --user "user:pass1" 'https://realtime.oxylabs.io/v1/queries' -H "Content-Type: application/json" -d '{"source": "google", "url": "https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp="}'
import requests
from pprint import pprint
# Structure payload.
payload = {
'source': 'google',
'url': 'https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp='
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('user', 'pass1'),
json=payload,
)
# Instead of response with job status and results url, this will return the
# JSON response with results.
pprint(response.json())
<?php
$params = [
'source' => 'google',
'url' => 'https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp='
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "user" . ":" . "pass1");
$headers = [];
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
echo $result;
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close ($ch);
# URL has to be encoded to escape `&` and `=` characters:
# URL: https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp=
# Encoded URL: https%3A%2F%2Fscholar.google.com%2Fscholar%3Fhl%3Den%26q%3Dnewton%26btnG%3D%26as_sdt%3D1%252C5%26as_sdtp%3D
https://realtime.oxylabs.io/v1/queries?source=google&url=https%3A%2F%2Fscholar.google.com%2Fscholar%3Fhl%3Den%26q%3Dnewton%26btnG%3D%26as_sdt%3D1%252C5%26as_sdtp%3D&access_token=12345abcde
The example above uses the Realtime integration method. If you would like to use some other integration method in your query (e.g. Push-Pull or Proxy Endpoint), refer to the integration methods section.

HTML example

Last modified 28d ago