# AI Overviews The `google_search` and `google_ads` sources can return AI Overviews from Google Search results in all supported global regions. `ai_overviews` result array supports multiple AI blocks in a single response, including sections like “People also ask” and “Things to know.” To maximize the chance of receiving fully rendered AI Overviews content, you must include the following parameters: | Parameter | Value | | ----------------- | ------------------------------- | | `source` | `google_search` or `google_ads` | | `render` | `html` | | `user_agent_type` | `desktop` or `mobile` | Using these parameters also allows you to receive both the general response and AI Overviews content with a single request. {% hint style="warning" %} Google dynamically generates this content or loads a cached version, so AI-generated answers may vary over time, even with identical parameters. {% endhint %} {% hint style="info" %} Explore[ **data dictionary**](#data-dictionary) for some of the AI Overviews SERP features. {% endhint %} ## AI Overviews regional availability Google AI Overviews are available in most countries apart from a few exceptions. Currently biggest excluded countries are: * France * China * Iran * North Korea {% hint style="warning" %} Google AI Overviews feature is continuously rolling out with more countries included over time. {% endhint %} ## Request samples {% tabs %} {% tab title="cURL" %} ```shell curl 'https://realtime.oxylabs.io/v1/queries' \ --user 'USERNAME:PASSWORD' \ -H 'Content-Type: application/json' \ -d '{ "source": "google_search", "query": "how to sell on amazon", "render": "html", "user_agent_type": "desktop", "parse": true }' ``` {% endtab %} {% tab title="Python" %} ```python import requests from pprint import pprint # Updated payload. payload = { 'source': 'google_search', 'query': 'how to sell on amazon', 'render': 'html', 'user_agent_type': 'desktop', } # Get response. response = requests.request( 'POST', 'https://realtime.oxylabs.io/v1/queries', auth=('user', 'pass1'), json=payload, ) # Print prettified response to stdout. pprint(response.json()) ``` {% endtab %} {% tab title="PHP" %} ```php 'google_search', 'query' => 'how to sell on amazon', 'render' => 'html', 'user_agent_type' => 'desktop', ]; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params)); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_USERPWD, "user" . ":" . "pass1"); $headers = []; $headers[] = "Content-Type: application/json"; curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); $result = curl_exec($ch); echo $result; if (curl_errno($ch)) { echo 'Error:' . curl_error($ch); } curl_close($ch); ``` {% endtab %} {% tab title="HTTP" %} ```http https://realtime.oxylabs.io/v1/queries?source=google_search&query=Emporio%20Armani%20EA3192&render=html&user_agent_type=desktop&access_token=12345abcde ``` {% endtab %} {% tab title="JSON" %} ```json { "source": "google_search", "query": "how to sell on amazon", "render": "html", "user_agent_type": "desktop" } ``` {% endtab %} {% endtabs %} We use synchronous [**Realtime**](https://developers.oxylabs.io/~/changes/1231/scraping-solutions/web-scraper-api/integration-methods/realtime) integration method in our examples. If you would like to use [**Proxy Endpoint**](https://developers.oxylabs.io/~/changes/1231/scraping-solutions/web-scraper-api/integration-methods/proxy-endpoint) or asynchronous [**Push-Pull**](https://developers.oxylabs.io/~/changes/1231/scraping-solutions/web-scraper-api/integration-methods/push-pull) integration, refer to the [**integration methods**](https://developers.oxylabs.io/~/changes/1231/scraping-solutions/web-scraper-api/integration-methods) section. ## Request parameter values ### Generic Basic setup and customization options for scraping Google Web search results with AI Overviews.
ParameterDescriptionDefault Value
sourceSets the scraper. Use google_search or google_ads.google_search
queryThe keyword or phrase to search for.-
renderEnables JavaScript rendering when set to html. More info.-
parseReturns parsed data when set to true.false
callback_urlURL to your callback endpoint. More info.-
user_agent_typeDevice type and browser. Use desktop or mobile.desktop
\- mandatory parameter #### Google Advanced Search Operators When scraping, you might find it useful to combine Google advanced search operators with your query. It enables you to customize the scope of the search, ensuring that the results are more relevant and focused. Explore these special commands [**here**](https://ahrefs.com/blog/google-advanced-search-operators/) and [**here**](https://www.semrush.com/kb/831-how-to-use-google-advanced-search-operators). See the example query below. ```json { "source": "google_search", "query": "iphone 15 launch inurl:apple", "render": "html", "user_agent_type": "desktop" } ``` ### Localization Adapt search results to specific geographical locations, domains, and languages.
ParameterDescriptionDefault Value
geo_locationThe geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggested geo_location parameter structures here.-
domainDomain localization for Google. The full list of available domains can be found here.com
localeAccept-Language header value which changes your Google search page web interface language. More info.-
context:
results_language		Results language. List of supported Google languages can be found here.
Learn how to use context parameters here.		-
### Pagination Controls for managing the pagination and retrieval of search results.
ParameterDescriptionDefault Value
start_pageStarting page number.1
pagesNumber of pages to retrieve.1
limitNumber of results to retrieve in each page.10
context:limit_per_pageScrape multiple pages using the same IP address and session (cookie set). By specifying the page numbers in a JSON array with the page key and indicating the number of organic results per page using the limit key, you can minimize the chance of seeing overlapping organic results across pages (e.g., the last organic result on the first page being the same as the first organic result on the second page). See example.-
#### Continuous scroll support The Web Scraper API fully supports Google Search continuous scroll. It automatically detects continuous scrolling layouts, efficiently loading the requested organic results without any extra parameters required. #### Limit per page {% hint style="warning" %} Due to recent Google limit changes, we've adjusted Web Scraper API's behavior. Maximum results per page will match Google's organic output, which is typically 10 results. {% endhint %} To use this feature, include a JSON array with JSON objects containing the following data:
ParameterDescriptionExample
pageThe number of the page you would like to scrape. Any integer value greater than 0 will work1
limitThe number of results on the page in question. Any integer value between 1 and 100 (inclusive) will work.90
#### Request sample ```json { "source": "google_search", "query": "how to sell on amazon", "render": "html", "user_agent_type": "dekstop", "parse": true, "context": [ { "key": "limit_per_page", "value": [ {"page": 1, "limit": 10}, {"page": 2, "limit": 90} ] }] } ``` ### Filtering Options to filter and refine search results based on various criteria. Learn how to use context parameters [**here**](#context-parameters).
ParameterDescriptionDefault Value
context:
filter		Setting the value of this parameter to 0 lets you see results that would otherwise be excluded due to similarity to other results.1
context:
safe_search		Safe search. Set to true to enable it.false
context:
udm		udm parameter allows switching between different search tabs, such as images, places, or videos, to customize the type of results displayed. Find the accepted values here.-
context:
tbm		To-be-matched or tbm parameter. Accepted values are: app, blg, bks, dsc, isch, nws, pts, plcs, rcp, lcl-
context:
tbs		This parameter is like a container for more obscure Google parameters, like limiting/sorting results by date as well as other filters some of which depend on the tbm parameter (e.g. tbs=app_os:1 is only available with tbm value app). More info here.-
{% hint style="warning" %} `udm` and `tbm` context parameters cannot be used together in a single scraping request; please select only one. Using both simultaneously may lead to conflicts or unexpected behavior. {% endhint %} ### Other Additional advanced settings and controls for specialized requirements.
ParameterDescriptionDefault Value
context:
fpstate		Setting the fpstate value to aig will make Google load more apps. This parameter is only useful if used together with the render parameter.-
context:
nfpr		true will turn off spelling auto-correctionfalse
### Context parameters All context parameters should be added to the `context` array as objects with `key` and `value` pairs, e.g.: ```json ... "context": [ { "key": "filter", "value": "0" } ] ... ``` ## Data structure and processing ### Data dictionary All of the available keys within AI Overviews SERP feature are defined in the table below:
Key (results.ai_overview)DescriptionType
answer_textGeneral information about the Google Search query term.array
bullet_listIndicates information on items rendered in a bulleted list within the AI Overviews SERP feature.array
productsA list of products with their respective details.object
source_panelA list of sources used to generate AI Overviews result in Google SERP feature.object
additional_questionsA list of additional questions with their respective details.object
recipesA list of recipes with their respective details.object
info_listA list of information most frequently used to represent steps of a process relevant to the search query.array
pos_overallAn indication of the position of the AI Overviews SERP feature result among all available Google SERP results.integer
### Working with Multiple AI Overviews Since AI Overviews are now returned as an array, you have to iterate through them: {% tabs %} {% tab title="JavaScript" %} ```javascript // Processing all AI Overviews in a response response.results.ai_overviews.forEach((aiOverview, index) => { console.log(`Processing AI Overview #${index + 1}`); // Process individual components if (aiOverview.answer_text) { // Handle answer text } if (aiOverview.bullet_list) { // Handle bullet lists } // And so on... }); ``` {% endtab %} {% tab title="Python" %} ```python # Processing all AI Overviews in a response for index, ai_overview in enumerate(response['results']['ai_overviews']): print(f"Processing AI Overview #{index + 1}") # Process individual components if 'answer_text' in ai_overview: answer_texts = ai_overview['answer_text'] for answer in answer_texts: # Process each answer text if 'text' in answer: for text_item in answer['text']: print(f"Answer text: {text_item}") if 'bullet_list' in ai_overview: bullet_lists = ai_overview['bullet_list'] for bullet_list in bullet_lists: if 'list_title' in bullet_list: print(f"List title: {bullet_list['list_title']}") if 'points' in bullet_list: for point in bullet_list['points']: print(f"- {point}") # Continue processing other elements... ``` {% endtab %} {% tab title="PHP" %} ```php $aiOverview) { $indexNumber = $index + 1; echo "Processing AI Overview #{$indexNumber}\n"; // Process answer text if (isset($aiOverview['answer_text']) && is_array($aiOverview['answer_text'])) { foreach ($aiOverview['answer_text'] as $answer) { if (isset($answer['text']) && is_array($answer['text'])) { foreach ($answer['text'] as $textItem) { echo "Answer text: {$textItem}\n"; } } } } // Process bullet lists if (isset($aiOverview['bullet_list']) && is_array($aiOverview['bullet_list'])) { foreach ($aiOverview['bullet_list'] as $bulletList) { if (isset($bulletList['list_title'])) { echo "List title: {$bulletList['list_title']}\n"; } if (isset($bulletList['points']) && is_array($bulletList['points'])) { foreach ($bulletList['points'] as $point) { echo "- {$point}\n"; } } } } // Continue processing other elements... } } ?> ``` {% endtab %} {% endtabs %} In most cases, there will be only one AI Overview, but your code should be prepared to handle multiple entries. Explore examples of AI Overviews results below, categorized by different search queries: [**Item**](#item), [**Recipes**](#recipe), [**Instruction**](#instruction). ### Item In the following example, "Emporio Armani EA3192" query was used.
```json ... "ai_overviews": [ { "answer_text": [ { "text": [ "Emporio Armani is a luxury brand known for its timeless designs and commitment to Italian craftsmanship.", "The Emporio Armani EA3192 is a pair of prescription glasses that are designed for men.", "They have a rectangular frame that is made from hypoallergenic, lightweight plastic.", "The lenses are UV and anti-reflective.", "Here are some insights from reviews:" ], "pos": 1 } ], "bullet_list": [ { "list_title": "Pros", "points": [ "Comfortable", "Minimal branding" ], "pos": 1 } ], "products": { "items": [ { "product_name": "Emporio Armani Men Eyeglasses Shiny Black Size: 53", "price": "$163.00", "previous_price": null, "seller": "Lenscrafters", "shipping": "Free delivery", "url": "https://www.lenscrafters.com/lc-us/emporio-armani/8056597975810?srsltid=AfmBOoqWO55KIJevhQp_c5OAp0JOM_8Uza3_gCvn9S4x4Vz342AUSQewzJI", "pos": 1 } ] }, "source_panel": { "items": [ { "url": "https://www.italist.com/magazine/what-is-emporio-armani/#:~:text=Emporio%20Armani%20is%20a%20luxury,reinforce%20the%20perception%20of%20luxury.", "source": "italist.com", "date": "May 5, 2023", "title": "What is Emporio Armani and is Emporio Armani a luxury brand?", "pos": 1 }, ... ] }, "additional_questions": { "items": [ { "question": "Is Emporio Armani the same as Giorgio Armani?", "pos": 1 } ] }, "pos_overall": 1 } ], ... ``` #### Answer text
Key (results.ai_overviews.answer_text)DescriptionType
textA short answer to the Google Search query.array
posAn indicator denoting the position of a given item among all AI Overviews answer results.integer
**Bullet list**
Key (results.ai_overviews.bullet_list)DescriptionType
list_titleThe title of the bullet list.string
pointsThe items rendered in the bullet list. All items are comma-separated.array
posAn indicator denoting the position of a given item among all AI Overviews bullet list results.integer
**Products**
Key (results.ai_overviews.products)DescriptionType
itemsA list of products with their respective details.array
items.product_nameThe name of the product.string
items.priceThe current price of the product.string
items.previous_priceThe initial price of the product (if applicable).string
items.sellerThe seller of the product.string
items.shippingThe type of shipping available for the product.string
items.urlThe URL of the product.string
items.posAn indicator denoting the position of a given item among all AI Overviews Product results.integer
#### Source panel
Key (results.ai_overviews.source_panel)DescriptionType
itemsA list of sources used to generate AI Overviews result in Google SERP feature.object
items.urlThe URL of the website.string
items.sourceThe name of the website.string
items.dateThe date when the article was published.string
items.titleThe title of the article.string
items.posAn indicator denoting the position of a given item among all AI Overviews source panel results.integer
#### Additional questions
Key (results.ai_overviews.additional_questions)DescriptionType
items.questionA list of comma-separated Additional questions relevant to the original search query.array
items.posAn indicator denoting the position of a given item among all AI Overviews additional questions results.integer
### Recipe In the following example, "recipe for an apple pie" query was used.
```json ... "ai_overviews": [ { "recipes": { "items": [ { "description": "This simple recipe features a homemade crust, Granny Smith apples, and warm spices.", "image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/...", "source": "Allrecipes", "rating": "4.8", "rating_count": "13K", "duration": "90 min", "url": "https://www.allrecipes.com/recipe/12682/apple-pie-by-grandma-ople/", "title": "Apple Pie by Grandma Ople" }, ... ] } } ], ... ``` **Recipes**
Key (results.ai_overviews.recipes)DescriptionType
itemsA list of recipes with their respective details.array
items.descriptionA short description of the recipe.string
items.imageA base64-encoded thumbnail image of the recipe.string
items.sourceThe name of the site where the recipe is published.string
items.ratingThe rating of the recipe.string
items.rating_countThe number of ratings.string
items.durationThe time that is needed to prepare the dish.string
items.urlThe URL to the full recipe.string
items.titleThe title of the recipe.string
In the following example, "recipe of a juicy succulent steak" query was used.
#### Info list
Key (results.ai_overviews.info_list)DescriptionType
list_titleThe title of the list.string
list_itemsContains a list of items with their respective details.array
list_items.titleThe title of the section within the list.string
list_items.contentA short description of the section.array
list_item.posAn indicator denoting the position of a given item among all AI Overviews Info list results.integer
### Instruction In the following example, "docker exec commands explained" query was used.
```json ... "ai_overviews": [ { "answer_text": [ { "text": [ "The docker execExecIn computing, exec is a functionality of an operating system that runs an executable file in the context of an already existing process, replacing the previous executable...", "It is a powerful tool that can be used for a variety of tasks, such as troubleshooting a container, running a script, or installing software." ], "pos": 1 }, { "text": [ "The syntax for the docker exec command is as follows:" ], "command": [ "docker exec [OPTIONS] CONTAINER COMMAND [ARG...]" ], "pos": 2 }, ... } ] } ], ... ``` **Answer text command** {% hint style="info" %} Note: details are rendered in the same array as for [**item related search query**](#answer-text), however, one new type of key (command) appears for instruction related type of query {% endhint %}
Key (results.ai_overviews.answer_text)DescriptionType
commandAn instruction given to a computer or software to perform a specific task.string