AI Overviews
The google_search source is designed to retrieve AI Overview (formerly known as Search Generative Experience (SGE)) results.
Important update: The schema structure for AI Overview results has changed from a single ai_overview object to an array of ai_overviews to accommodate multiple AI Overview blocks in a single response. Additional AI overview sometimes appears in the “People also ask for” or “Things to know” sections. Please update your parsing logic accordingly.
Request samples
curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
"source": "google_search",
"query": "Emporio Armani EA3192"
}'We use synchronous Realtime integration method in our examples. If you would like to use Proxy Endpoint or asynchronous Push-Pull integration, refer to the integration methods section.
Request parameter values
Generic
Basic setup and customization options for scraping Google Web search results.
source
Sets the scraper.
google_search
query
UTF-encoded keyword.
-
user_agent_type
Device type and browser. The full list can be found here.
desktop
- mandatory parameter
Google Advanced Search Operators
When scraping, you might find it useful to combine Google advanced search operators with your query. It enables you to customize the scope of the search, ensuring that the results are more relevant and focused. Explore these special commands here and here. See an example below.
{
"source": "google_search",
"query": "iphone 15 launch inurl:apple",
}Localization
Adapt search results to specific geographical locations, domains, and languages.
geo_location
The geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggested geo_location parameter structures here.
-
locale
Accept-Language header value which changes your Google search page web interface language. More info.
-
Pagination
Controls for managing the pagination and retrieval of search results.
start_page
Starting page number.
1
pages
Number of pages to retrieve.
1
limit
Number of results to retrieve in each page.
10
context:limit_per_page
Scrape multiple pages using the same IP address and session (cookie set). By specifying the page numbers in a JSON array with the page key and indicating the number of organic results per page using the limit key, you can minimize the chance of seeing overlapping organic results across pages (e.g., the last organic result on the first page being the same as the first organic result on the second page). See example.
-
Continuous scroll support
The SERP Scraper API fully supports Google Search continuous scroll. It automatically detects continuous scrolling layouts, efficiently loading the requested organic results without any extra parameters required.
Limit per page
To use this feature, include a JSON array with JSON objects containing the following data:
page
The number of the page you would like to scrape. Any integer value greater than 0 will work
1
limit
The number of results on the page in question. Any integer value between 1 and 100 (inclusive) will work.
90
Request sample
{
"source": "google_search",
"query": "adidas",
"parse": true,
"context": [
{
"key": "limit_per_page",
"value": [
{"page": 1, "limit": 10},
{"page": 2, "limit": 90}
]
}]
}Filtering
Options to filter and refine search results based on various criteria. Learn how to use context parameters here.
context:
filter
Setting the value of this parameter to 0 lets you see results that would otherwise be excluded due to similarity to other results.
1
context:
safe_search
Safe search. Set to true to enable it.
false
context:
udm
udm parameter allows switching between different search tabs, such as images, places, or videos, to customize the type of results displayed. Find the accepted values here.
-
context:
tbm
To-be-matched or tbm parameter. Accepted values are: app, blg, bks, dsc, isch, nws, pts, plcs, rcp, lcl
-
context:
tbs
This parameter is like a container for more obscure google parameters, like limiting/sorting results by date as well as other filters some of which depend on the tbm parameter (e.g. tbs=app_os:1 is only available with tbm value app). More info here.
-
udm and tbm context parameters cannot be used together in a single scraping request; please select only one. Using both simultaneously may lead to conflicts or unexpected behavior.
Other
Additional advanced settings and controls for specialized requirements.
context:
fpstate
Setting the fpstate value to aig will make Google load more apps. This parameter is only useful if used together with the render parameter.
-
context:
nfpr
true will turn off spelling auto-correction
false
Context parameters
All context parameters should be added to the context array as objects with key and value pairs, e.g.:
...
"context": [
{
"key": "filter",
"value": "0"
}
]
...Data structure and processing
Data dictionary
All of the available keys within AI Overview SERP feature are defined in the table below:
Note: the examples below reflect the new schema structure with ai_overviews as an array.
answer_text
General information about the Google Search query term.
array
bullet_list
Indicates information on items rendered in a bulleted list within the AI Overview SERP feature.
array
products
A list of products with their respective details.
object
source_panel
A list of sources used to generate AI Overview result in Google SERP feature.
object
additional_questions
A list of additional questions with their respective details.
object
recipes
A list of recipes with their respective details.
object
info_list
A list of information most frequently used to represent steps of a process relevant to the search query.
array
pos_overall
An indication of the position of the AI Overview SERP feature result among all available Google SERP results.
Working with Multiple AI Overviews
Since AI Overviews are now returned as an array, you'll need to iterate through them:
// Processing all AI Overviews in a response
response.results.ai_overviews.forEach((aiOverview, index) => {
console.log(`Processing AI Overview #${index + 1}`);
// Process individual components
if (aiOverview.answer_text) {
// Handle answer text
}
if (aiOverview.bullet_list) {
// Handle bullet lists
}
// And so on...
});In most cases, there will be only one AI Overview, but your code should be prepared to handle multiple entries.
Explore examples of AI Overview results below, categorized by different search queries: Item, Recipes, Instruction.
Item
In the following example, "Emporio Armani EA3192" query was used.
...
"ai_overviews": [
{
"answer_text": [
{
"text": [
"Emporio Armani is a luxury brand known for its timeless designs and commitment to Italian craftsmanship.",
"The Emporio Armani EA3192 is a pair of prescription glasses that are designed for men.",
"They have a rectangular frame that is made from hypoallergenic, lightweight plastic.",
"The lenses are UV and anti-reflective.",
"Here are some insights from reviews:"
],
"pos": 1
}
],
"bullet_list": [
{
"list_title": "Pros",
"points": [
"Comfortable",
"Minimal branding"
],
"pos": 1
}
],
"products": {
"items": [
{
"product_name": "Emporio Armani Men Eyeglasses Shiny Black Size: 53",
"price": "$163.00",
"previous_price": null,
"seller": "Lenscrafters",
"shipping": "Free delivery",
"url": "https://www.lenscrafters.com/lc-us/emporio-armani/8056597975810?srsltid=AfmBOoqWO55KIJevhQp_c5OAp0JOM_8Uza3_gCvn9S4x4Vz342AUSQewzJI",
"pos": 1
}
]
},
"source_panel": {
"items": [
{
"url": "https://www.italist.com/magazine/what-is-emporio-armani/#:~:text=Emporio%20Armani%20is%20a%20luxury,reinforce%20the%20perception%20of%20luxury.",
"source": "italist.com",
"date": "May 5, 2023",
"title": "What is Emporio Armani and is Emporio Armani a luxury brand?",
"pos": 1
},
...
]
},
"additional_questions": {
"items": [
{
"question": "Is Emporio Armani the same as Giorgio Armani?",
"pos": 1
}
]
},
"pos_overall": 1
}
],
...Answer text
text
A short answer on the Google Search query.
array
pos
An indicator denoting the position of a given item among all AI Overview answer results.
integer
Bullet list
list_title
The title of the bullet list.
string
points
The items rendered in the bullet list. All items are comma-separated.
array
pos
An indicator denoting the position of a given item among all AI Overview bullet list results.
integer
Products
items
A list of products with their respective details.
array
items.product_name
The name of the product.
string
items.price
The current price of the product.
string
items.previous_price
The initial price of the product (if applicable).
string
items.seller
The seller of the product.
string
items.shipping
The type of shipping available for the product.
string
items.url
The URL of the product.
string
items.pos
An indicator denoting the position of a given item among all AI Overview Product results.
integer
Source panel
items
A list of sources used to generate AI Overview result in Google SERP feature.
object
items.url
The URL of the website.
string
items.source
The name of the website.
string
items.date
The date when the article has been published.
string
items.title
The title of the article.
string
items.pos
An indicator denoting the position of a given item among all AI Overview source panel results.
integer
Additional questions
items.question
A list of comma-separated Additional questions relevant to the original search query.
array
items.pos
An indicator denoting the position of a given item among all AI Overview additional questions results.
integer
Recipe
In the following example, "recipe for an apple pie" query was used.
...
"ai_overviews": [
{
"recipes": {
"items": [
{
"description": "This simple recipe features a homemade crust, Granny Smith apples, and warm spices.",
"image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/...",
"source": "Allrecipes",
"rating": "4.8",
"rating_count": "13K",
"duration": "90 min",
"url": "https://www.allrecipes.com/recipe/12682/apple-pie-by-grandma-ople/",
"title": "Apple Pie by Grandma Ople"
},
...
]
}
}
],
...Recipes
items
A list of recipes with their respective details.
array
items.description
A short description of the recipe.
string
items.image
A base64-encoded thumbnail image of the recipe.
string
items.source
The name of the site where the recipe is published.
string
items.rating
The rating of the recipe.
string
items.rating_count
The number of ratings.
string
items.duration
The time that is needed to prepare the dish.
string
items.url
The URL to the full recipe.
string
items.title
The title of the recipe.
string
In the following example, "recipe of a juicy succulent steak" query was used.
Info list
list_title
The title of the list.
string
list_items
Contains a list of items with their respective details.
array
list_items.title
The title of the section within the list.
string
list_items.content
A short description of the section.
string
list_item.pos
An indicator denoting the position of a given item among all AI Overview Info list results.
integer
Instruction
In the following example, "docker exec commands explained" query was used.
...
"ai_overviews": [
{
"answer_text": [
{
"text": [
"The docker execExecIn computing, exec is a functionality of an operating system that runs an executable file in the context of an already existing process, replacing the previous executable...",
"It is a powerful tool that can be used for a variety of tasks, such as troubleshooting a container, running a script, or installing software."
],
"pos": 1
},
{
"text": [
"The syntax for the docker exec command is as follows:"
],
"command": [
"docker exec [OPTIONS] CONTAINER COMMAND [ARG...]"
],
"pos": 2
},
...
}
]
}
],
...Answer text command
command
An instruction given to a computer or software to perform a specific task.
string
Last updated
Was this helpful?