Web Search

The google_search source is designed to retrieve Google Search results (SERPs). This sub-page specifically presents information related to Google Web Search. To explore other result types, read Image Search or News Search.

Request samples

In the samples below, we make a request to get 2 results pages, from number 11 to number 12 , for search term adidas on google.nl domain. The SERP will be filtered to contain French-language results only. API will return parsed results.

curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
        "source": "google_search",
        "domain": "nl",
        "query": "adidas",
        "start_page": 11,
        "pages": 2,
        "parse": true,
        "context": [
            {
                "key": "results_language",
                "value": "fr"
            }
        ]
    }'

We use synchronous Realtime integration method in our examples. If you would like to use Proxy Endpoint or asynchronous Push-Pull integration, refer to the integration methods section.

Request parameter values

Generic

Basic setup and customization options for scraping Google Web search results.

ParameterDescriptionDefault Value

source

Sets the scraper.

google_search

query

UTF-encoded keyword.

-

render

Enables JavaScript rendering when set to html. More info.

-

parse

Returns parsed data when set to true. Explore output data dictionary.

false

callback_url

URL to your callback endpoint. More info.

-

user_agent_type

Device type and browser. The full list can be found here.

desktop

- mandatory parameter

Google Advanced Search Operators

When scraping, you might find it useful to combine Google advanced search operators with your query. It enables you to customize the scope of the search, ensuring that the results are more relevant and focused. Explore these special commands here and here. See an example below.

{
    "source": "google_search",
    "query": "iphone 15 launch inurl:apple",
}

Localization

Adapt search results to specific geographical locations, domains, and languages.

ParameterDescriptionDefault Value

geo_location

The geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data. For more information, read about our suggested geo_location parameter structures here.

-

domain

Domain localization for Google. The full list of available domains can be found here.

com

locale

Accept-Language header value which changes your Google search page web interface language. More info.

-

context: results_language

Results language. List of supported Google languages can be found here.

-

Pagination

Controls for managing the pagination and retrieval of search results.

ParameterDescriptionDefault Value

start_page

Starting page number.

1

pages

Number of pages to retrieve.

1

limit

Number of results to retrieve in each page.

10

context:limit_per_page

Scrape multiple pages using the same IP address and session (cookie set). By specifying the page numbers in a JSON array with the page key and indicating the number of organic results per page using the limit key, you can minimize the chance of seeing overlapping organic results across pages (e.g., the last organic result on the first page being the same as the first organic result on the second page). See example.

-

Continuous scroll support

The SERP Scraper API fully supports Google Search continuous scroll. It automatically detects continuous scrolling layouts, efficiently loading the requested organic results without any extra parameters required.

Limit per page

To use this feature, include a JSON array with JSON objects containing the following data:

ParameterDescriptionExample

page

The number of the page you would like to scrape. Any integer value greater than 0 will work

1

limit

The number of results on the page in question. Any integer value between 1 and 100 (inclusive) will work.

90

Request sample

{
    "source": "google_search",
    "query": "adidas",
    "parse": true,
    "context": [
        {
            "key": "limit_per_page",
            "value": [
                {"page": 1, "limit": 10},
                {"page": 2, "limit": 90}
                    ]
        }]
}

Filtering

Options to filter and refine search results based on various criteria.

ParameterDescriptionDefault Value

context:filter

Setting the value of this parameter to 0 lets you see results that would otherwise be excluded due to similarity to other results.

1

context:safe_search

Safe search. Set to true to enable it.

false

context: tbm

To-be-matched or tbm parameter. Accepted values are: app, blg, bks, dsc, isch, nws, pts, plcs, rcp, lcl

-

context: tbs

tbs parameter. This parameter is like a container for more obscure google parameters, like limiting/sorting results by date as well as other filters some of which depend on the tbm parameter (e.g. tbs=app_os:1 is only available with tbm value app). More info here.

-

Other

Additional advanced settings and controls for specialized requirements.

ParameterDescriptionDefault Value

context: fpstate

Setting the fpstate value to aig will make Google load more apps. This parameter is only useful if used together with the render parameter.

-

context: nfpr

true will turn off spelling auto-correction

false

Structured data

SERP Scraper API is capable of extracting either an HTML or JSON object that contains Google search results, offering structured data on various elements of the results page.

google_search structured output
{
    "results": [
        {
            "content": {
                "url": "https://www.google.com/search?q=adidas&uule=w+CAIQICINdW5pdGVkIHN0YXRlcw&gl=us&hl=en",
                "page": 1,
                "results": {
                    "pla": {
                        "items": [
                            {
                                "pos": 1,
                                "url": "https://www.adidas.com/us/nmd_r1-primeblue-shoes/GZ9257.html?dfw_tracker=24819-GZ9257-0010",
                                "price": "$150.00",
                                "title": "NMD_R1 Primeblue Shoes Black M 8.5 / W 9.5 - Mens Originals Shoes",
                                "seller": "adidas",
                                "url_image": "https://encrypted-tbn1.gstatic.com/shopping?q=tbn:ANd9GcSZALnENHKXERktsOCnUSscRd4v0cSUw9E0XPaZwgyruM0Wuw-MhWcchAVzZRBdS-OP3_167R3jCg&usqp=CAc",
                                "image_data": "iVBORw0KGgoAAAANSUhE...JRU5ErkJggg=="
                            },
                            ...
                            {
                                "pos": 22,
                                "url": "https://www.adidas.com/us/ultraboost-5.0-dna-shoes/GY6452.html?dfw_tracker=24819-GY6452-0006",
                                "price": "$90.00",
                                "title": "Ultraboost 5.0 DNA Shoes Magic Grey 13K - Kids Originals Shoes",
                                "seller": "adidas",
                                "url_image": "https://encrypted-tbn3.gstatic.com/shopping?q=tbn:ANd9GcSTCgTZ1im_6ZX9YSCrjQfjVEnTmpmiqqYAHAhXzA9VhJdrokOeOnWsbEOxqA8zPkaos192xPF5ogA&usqp=CAc",
                                "image_data": "iVBORw0KGgoAAAANSUhEU...kJggg=="
                            }
                        ],
                        "pos_overall": 1
                    },
                    "paid": [],
                    "images": {
                        "items": [
                            {
                                "alt": "Image result for adidas",
                                "pos": 1,
                                "url": "/search?q=adidas&gl=us&h...E#imgrc=Dw-CBW17i8BqsM",
                                "source": "https://www.britannica.com/topic/Adidas-AG"
                            },
                            ...
                            {
                                "alt": "Image result for adidas",
                                "pos": 10,
                                "url": "/search?q=adidas&gl=us&hl=en&tbm=isch&sourXRM%252Cvz-MIh...QF6BAhLEAE#imgrc=_oOH_rR4W_-X-M",
                                "source": "https://www.shutterstock.com/search/adidas"
                            }
                        ],
                        "pos_overall": 11
                    },
                    "organic": [
                        {
                            "pos": 1,
                            "url": "https://www.adidas.com/us",
                            "desc": "adidas is about more than sportswear and workout clothes. We partner with the best in the industry to co-create. This way we offer our fans the sporting goods, ...",
                            "title": "adidas Official Website | adidas US",
                            "sitelinks": {
                                "expanded": [
                                    {
                                        "url": "https://www.adidas.com/us/women",
                                        "title": "Women"
                                    },
                                    ...
                                    {
                                        "url": "https://www.adidas.com/us/sale",
                                        "title": "Sale"
                                    }
                                ]
                            },
                            "url_shown": "https://www.adidas.com› ...",
                            "pos_overall": 2
                        },
                        ...
                        {
                            "pos": 5,
                            "url": "https://play.google.com/store/apps/details?id=com.adidas.app&hl=en_US&gl=US",
                            "desc": "YOUR ADIDAS - MORE THAN A SHOPPING APP The home of sport and sneakers, the adidas app puts you closer to the action with access to sneaker drops, ...Size: 115MUpdated: May 3, 2022 Rating: 4.8 · ‎322,391 votes · ‎Free · ‎Android · ‎Sports",
                            "title": "adidas - Apps on Google Play",
                            "url_shown": "https://play.google.com› store › apps › details › id=com....",
                            "pos_overall": 12
                        }
                    ],
                    "twitter": {
                        "pos": 1,
                        "url": "https://twitter.com/adidas",
                        "items": [
                            {
                                "pos": 1,
                                "url": "https://twitter.com/adidas/status/1521901157064531968?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet",
                                "content": "What do you like to listen to most during workouts?",
                                "timeframe": "5 days ago"
                            },
                            ...
                            {
                                "pos": 3,
                                "url": "https://twitter.com/adidas/status/1519345069366652928?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet",
                                "content": "How would you describe your favorite sport to an alien?",
                                "timeframe": "Apr 27, 2022"
                            }
                        ],
                        "title": "",
                        "pos_overall": 7
                    },
                    "knowledge": {
                        "title": "Adidas",
                        "factoids": [
                            {
                                "title": "Customer service chat",
                                "content": "Online Chat"
                            },
                            ...
                            {
                                "title": "Website",
                                "content": "http://www.adidas-group.com/"
                            }
                        ],
                        "profiles": [
                            {
                                "url": "https://www.instagram.com/adidas",
                                "title": "Instagram"
                            },
                            ...
                            {
                                "url": "https://www.youtube.com/user/adidas",
                                "title": "YouTube"
                            }
                        ],
                        "subtitle": "Design company",
                        "description": "Adidas AG is a German multinational corporation, founded and headquartered in Herzogenaurach, Bavaria, that designs and manufactures shoes, clothing and accessories. It is the largest sportswear manufacturer in Europe, and the second largest in the world, after Nike.",
                        "related_searches": [
                            {
                                "url": "/search?gl=us&hl=en&q=Nike&si=AC1wQDCwN61-ebmuwbQCO5QCrgOvEq5bkWeIzJ5JczItzAKNdbIxoHfFJ8UFY3e1CorwNJXI6gTWniM6aVu3nEoseblyAI-5N39_1F3DTWWABELjIdJa9ZgI_g5n-c9ZUZdzEZeV2VGOkDNPl5DwiBeyifKPveoCnBsiT3wVpKdr52FPqK5rzQSzspA-sjztjVcZy2WfOwFITo5EfSXyZ5AsknO1X-dzXUCOlIIGCvFWqi9OKgK5PaWp7EcXls7n8Ag_L7HC3Pgg2k5FWR631X7Hvi1268shAg%3D%3D&sa=X&ved=2ahUKEwiLlsjE8dH3AhUQkGoFHT2IBUQQxA16BAhdEAU",
                                "title": "Nike"
                            },
                            ...
                            {
                                "url": "/search?gl=us&hl=en&q=Asics&si=AC1wQDDagiMg03ncxeOQZbwVe-CJxRCchC-jr2hCPTxjc9wbgNZ1pCnWqLo_0N5RPhArBCY1qCgzESLe6Y9hB2HKnzquQNjJW9iLV6gQvDXzTwkl6LZHjq2aOuZII0W5uCWjpc0oOVU5JzyOjRUNEYoe_N_KsbL4xn4A0Fl2S4JMYAeaBZmu8jPX8EFVl_C3gerS4CRTyrfcL2bL-VhUNLEXVUlkFxrLlAyeZfUMVYSDo4WqV1c8-Bgp-rBwBzqW73Q7s1kOZHBtCQ8aBNfbfulPCljSG9vXBrcj9RqQRA8pD8TlH1rz8qE%3D&sa=X&ved=2ahUKEwiLlsjE8dH3AhUQkGoFHT2IBUQQxA16BAhdEAs",
                                "title": "Asics"
                            }
                        ]
                    },
                    "top_stories": {
                        "items": [
                            {
                                "pos": 1,
                                "url": "https://www.adidas.com/us/adilette-comfort-slides/GZ5898.html",
                                "title": "adidas Adilette Comfort Slides - Beige | women swim | adidas US",
                                "source": "",
                                "timeframe": "8 hours ago"
                            },
                            ...
                            },
                            {
                                "pos": 3,
                                "url": "https://www.adidas.com/us/women-clothing-sale",
                                "title": "Women's Clothing Sale Up to 50% Off | adidas US",
                                "source": "",
                                "timeframe": "Sep 30, 2021"
                            }
                        ],
                        "pos_overall": 3
                    },
                    "instant_answers": [
                        {
                            "type": "unknown",
                            "_parsed": false,
                            "pos_overall": 5
                        },
                    ],
                    "popular_products": {
                        "items": [
                            {
                                "pos": 1,
                                "title": "Adidas Yeezy Kids Foam Runner",
                                "image_data": "UklGRhwJAABXRUJQVlA...rAZNVIAAA"
                            },
                            ...
                            {
                                "pos": 10,
                                "title": "Adidas Men's Kaptir 2.0",
                                "image_data": "UklGRhQNAAB...+wTh1cLB/uQAAAAAAAAA="
                            }
                        ],
                        "pos_overall": 4
                    },
                    "related_searches": {
                        "pos_overall": 14,
                        "related_searches": [
                            "adidas shoes",
                            "adidas yeezy",
                            "adidas outlet",
                            "adidas pants",
                            "adidas sneakers",
                            "adidas superstar",
                            "adidas wikipedia"
                        ]
                    },
                    "search_information": {
                        "query": "adidas",
                        "showing_results_for": "adidas",
                        "total_results_count": 1440000000
                    },
                    "total_results_count": 1440000000,
                    "related_searches_categorized": [
                        {
                            "category": {
                                "name": "Adidas logo",
                                "type": "collapsed"
                            },
                            "pos_overall": 13
                        }
                    ]
                },
                "last_visible_page": 10,
                "parse_status_code": 12000
            },
            "created_at": "2022-05-09 07:24:16",
            "updated_at": "2022-05-09 07:24:20",
            "page": 1,
            "url": "https://www.google.com/search?q=adidas&uule=w+CAIQICINdW5pdGVkIHN0YXRlcw&gl=us&hl=en",
            "job_id": "6929330182268395521",
            "status_code": 200,
            "parser_type": "v2"
        }
    ]
}

The composition of elements may differ based on whether it was a desktop or mobile search.

To help you identify certain elements, refer to the data dictionaries for each type of Google search result, whether obtained from desktop or mobile searches.

Output data dictionary

Navigate through the details using the right-side navigation or scrolling down the page.

HTML example

JSON structure

The Google Web Search structured output includes fields like URL, page, results, and others. The table below presents a detailed list of each SERP feature we parse, along with its description and data type. The table also includes some metadata.

The number of items and fields for a specific result type may vary depending on the search query.

Key NameDescriptionType

url

The URL of the Google search page.

string

page

Page number relative to the Google SERP pagination.

integer

results

A dictionary containing the results of the search.

object

results.pla

A list of product listing ads with their respective details.

object

results.paid

A list of sponsored results with their respective details.

array

results.images

A list of image results with their respective details.

object

results.flights

A list of flights with their respective details.

object

results.organic

A list of unpaid listings with their respective details.

array

results.organic_videos

A list of organic videos with their respective details.

array

results.top_sights

A list of notable landmarks or attractions with their respective details.

array

results.jobs

A list of jobs listings with their respective details.

object

results.local_service_ads

A list of sponsored local service providers.

object

results.video_box

A single video item with its respective details.

object

results.recipes

A list of recipes with their respective details.

object

results.twitter

A list of Twitter (X) results with their respective details.

array

results.knowledge

A list of relevant information retrieved from the knowledge panel.

object

results.local_pack

A list containing local business listings relevant to the search.

object

results.item_carousel

A list of illustrative items for informational queries with their respective details.

object

results.videos

A list of videos with their respective details.

object

results.related_searches_categorized

A list of related searches that other people have done.

array

results.hotels

A list of hotels relevant to the location used in the search query.

object

results.apps

A list of applications with their respective details.

array

results.finance

A summary of financial data for companies, including stock prices, market cap, and other key metrics.

object

results.sports_games

Information on recent sports games, detailing scores, teams, game types, and highlights.

object

results.discussions_and_forums

A list of discussion threads and forum posts with URLs, titles, sources, and comment counts.

object

results.featured_snippet

A list of specific results retrieved from the organic part of SERPs.

array

results.top_stories

A list of articles with their respective details when a search query is identified as news-oriented.

object

results.popular_products

A list of unpaid and organic displays of Google Shopping product listings with their respective details.

object

results.related_searches

A list of related searches presented at the bottom of the search page.

object

results.related_questions

A list of related interrogative search queries with their respective details.

object

results.search_information

A list of details for the submitted search query.

object

total_results_count

The total number of results found for the search query.

string

last_visible_page

Value identifying the maximum page number visible in the search query results page. (-1 when loading of more results is initiated by scrolling).

integer

parse_status_code

The status code of the parsing job. You can see the parser status codes described here.

integer

created_at

The timestamp when the scraping job was created.

timestamp

updated_at

The timestamp when the scraping job was finished.

timestamp

job_id

The ID of the job associated with the scraping job.

string

status_code

The status code of the scraping job. You can see the scraper status codes described here.

integer

parser_type

The type of the parser used for breaking down the HTML content.

string

In the following sections, parsed JSON code snippets are shortened where more than one item for the result type is available.

Product Listing Ads (PLA)

The pla section of search results refers to Product Listing Ads, also known as Google Shopping Ads, prominently displayed at the top of the search page in a carousel format.

...
"pla": {
    "items": [
        {
            "pos": 1,
            "url": "https://www.amazon.com/Switch-Controllers-Wildcat-Touchscreen-Bluetooth/dp/B09N4S9TWV?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1GYW88KAZG2FO",
            "price": "$675.49",
            "title": "Switch PURTCH Newest w/Yellow & Blue controllers Wildcat Bundle(2000 V-Bucks and Code for Wildcat Bundle Included), 6.2\" Touchscreen LCD Display, 802.11",
            "seller": "Amazon.com",
            "url_image": "https://encrypted-tbn0.gstatic.com/shopping?q=tbn:ANd9GcSdvbTlG8SxMmPAm6cjeiFXvrNCIF0ZIBUUqz4B4Uww2fY7sHhaEQxbDv_JP78tu-YO3kWFbTESJPcAx5h2zezUK3bpZJnVriMGOi7Tx7nJNA8&usqp=CAc",
            "image_data": "/9j/4AAQSkZJRgABAQAAAQABAAD/..."
        },
        ...
    ],
    "pos_overall": 1
},
...
Key (results.pla)DescriptionTypeLayout

items

All PLAs available within the page.

array

Desktop and Mobile

items.pos

An indicator denoting the position of a given item among PLA results.

integer

Desktop and Mobile

items.url

The URL of the product.

string

Desktop and Mobile

items.price

The price of the product in the listing ad.

string

Desktop and Mobile

items.title

The title of the product in the listing ad.

string

Desktop and Mobile

items.seller

The seller of the product in the listing ad.

string

Desktop and Mobile

items.url_image

The URL of the product image.

string

Desktop

items.image_data

The base64-encoded thumbnail image of the product.

string

Desktop and Mobile

pos_overall

An indication of the position of the PLA SERP feature within the SERP.

integer

Desktop and Mobile

The paid section refers to advertisements powered by Google Ads, usually appearing above organic (non-paid) results.

...