URL
The
google source is designed to retrieve content from various Google URLs. This means that instead of sending multiple parameters, you can provide us with a direct URL to the required Google page. We do not strip any parameters or alter your URLs in any other way.This data source also supports parsed data (structured data in JSON format), as long as the URL submitted is for Google (SERP page). If we cannot confirm this is a SERP page request, we will return a failure message.
Parameter | Description | Default Value |
|---|---|---|
source | google | |
url | Direct URL (link) to Google page | - |
user_agent_type | desktop | |
render | | |
callback_url | - | |
geo_location | The results will be adapted for geographical location. Using this parameter correctly is extremely important to get accurate data. For more information, read about our suggested geo_location parameter structures here. | - |
parse | true will return parsed data, as long as the URL submitted is for Google. | - |
- required parameter
In the example below, the API will retrieve a Google Scholar search page.
JSON
cURL
Python
PHP
HTTP
{
"source": "google",
"url": "https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp="
}
curl --user "user:pass1" 'https://realtime.oxylabs.io/v1/queries' -H "Content-Type: application/json" -d '{"source": "google", "url": "https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp="}'
import requests
from pprint import pprint
# Structure payload.
payload = {
'source': 'google',
'url': 'https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp='
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('user', 'pass1'),
json=payload,
)
# Instead of response with job status and results url, this will return the
# JSON response with results.
pprint(response.json())
<?php
$params = [
'source' => 'google',
'url' => 'https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp='
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "user" . ":" . "pass1");
$headers = [];
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
echo $result;
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close ($ch);
# URL has to be encoded to escape `&` and `=` characters:
# URL: https://scholar.google.com/scholar?hl=en&q=newton&btnG=&as_sdt=1%2C5&as_sdtp=
# Encoded URL: https%3A%2F%2Fscholar.google.com%2Fscholar%3Fhl%3Den%26q%3Dnewton%26btnG%3D%26as_sdt%3D1%252C5%26as_sdtp%3D
https://realtime.oxylabs.io/v1/queries?source=google&url=https%3A%2F%2Fscholar.google.com%2Fscholar%3Fhl%3Den%26q%3Dnewton%26btnG%3D%26as_sdt%3D1%252C5%26as_sdtp%3D&access_token=12345abcde
The example above uses the Realtime integration method. If you would like to use some other integration method in your query (e.g. Push-Pull or Proxy Endpoint), refer to the integration methods section.
Last modified 28d ago