> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/api-targets/cn/search-engines/google/search/news-search.md). # 新闻搜索 大规模抓取 Google 新闻结果，并获取完全解析的数据。提取包含标题、来源和发布日期的文章。 该 `google_search` 该数据源旨在检索 Google 搜索结果（SERPs）。此子页面专门展示与 Google 新闻搜索相关的数据。要查看其他结果类型，请阅读此处： [**网页搜索**](https://github.com/oxylabs/gitbook-public-english/blob/master/scraping-solutions/web-scraper-api/targets/google/search/broken-reference/README.md), [**图片搜索**](https://github.com/oxylabs/gitbook-public-english/blob/master/scraping-solutions/web-scraper-api/targets/google/search/broken-reference/README.md). {% hint style="warning" %} 要抓取 Google News 搜索，请包含 `context:udm` 参数，并将值设置为 `12` 或 `context:tbm` 参数，并将值设置为 `nws`. {% endhint %} {% hint style="info" %} 查看输出 [**数据字典**](#data-dictionary) 可查看每个 News SERP 功能的数据字典，其中提供简要说明、截图、解析后的 JSON 代码片段，以及定义每个解析字段的表格。可使用右侧导航或向下滚动页面浏览详细信息。 {% endhint %} ## 请求示例 在下面的示例中，我们发送请求以获取搜索词为 `adidas`. ### udm {% tabs %} {% tab title="cURL" %} ```shell curl 'https://realtime.oxylabs.io/v1/queries' \\ --user 'USERNAME:PASSWORD' \\ -H 'Content-Type: application/json' \\ -d '{ "source": "google_search", "query": "adidas", "parse": true, "context": [ { "key": "udm", "value": 12 } ] }' ``` {% endtab %} {% tab title="Python" %} ```python import requests from pprint import pprint # Structure payload. payload = { 'source': 'google_search', 'query': 'adidas', 'parse': True, 'context': [ {'key': 'udm', 'value': 12}, ], } # Get response. response = requests.post( 'https://realtime.oxylabs.io/v1/queries', auth=('USERNAME', 'PASSWORD'), json=payload, ) # Print prettified response to stdout. pprint(response.json()) ``` {% endtab %} {% tab title="Node.js" %} ```javascript const https = require("https"); const username = "USERNAME"; const password = "PASSWORD"; const body = { source: "google_search", query: "adidas", parse: true, context: [ { key: "udm", value: 12 }, ], }; const options = { hostname: "realtime.oxylabs.io", path: "/v1/queries", method: "POST", headers: { "Content-Type": "application/json", Authorization: "Basic " + Buffer.from(`${username}:${password}`).toString("base64"), }, }; const request = https.request(options, (response) => { let data = ""; response.on("data", (chunk) => { data += chunk; }); response.on("end", () => { const responseData = JSON.parse(data); console.log(JSON.stringify(responseData, null, 2)); }); }); request.on("error", (error) => { console.error("Error:", error); }); request.write(JSON.stringify(body)); request.end(); ``` {% endtab %} {% tab title="HTTP" %} ```http source=google_search&query=adidas&parse=true&context[0][key]=udm&context[0][value]=12&access_token=12345abcde ``` {% endtab %} {% tab title="PHP" %} ```php 'google_search', 'query' => 'adidas', 'parse' => true, 'context' => [ [ 'key' => 'udm', 'value' => 12, ] ] ); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params)); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_USERPWD, "USERNAME" . ":" . "PASSWORD"); $headers = array(); $headers[] = "Content-Type: application/json"; curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); $result = curl_exec($ch); echo $result; if (curl_errno($ch)) { echo 'Error:' . curl_error($ch); } curl_close($ch); ``` {% endtab %} {% tab title="Golang" %} ```go package main import ( "bytes" "encoding/json" "fmt" "io/ioutil" "net/http" ) func main() { const Username = "USERNAME" const Password = "PASSWORD" payload := map[string]interface{}{ "source": "google_search", "query": "adidas", "parse": true, "context": []map[string]interface{}{ {"key": "udm", "value": 12}, }, } jsonValue, _ := json.Marshal(payload) client := &http.Client{} request, _ := http.NewRequest("POST", "https://realtime.oxylabs.io/v1/queries", bytes.NewBuffer(jsonValue), ) request.SetBasicAuth(Username, Password) response, _ := client.Do(request) responseText, _ := ioutil.ReadAll(response.Body) fmt.Println(string(responseText)) } ``` {% endtab %} {% tab title="C#" %} ```csharp using System; using System.Collections.Generic; using System.Net.Http; using System.Net.Http.Json; using System.Threading.Tasks; namespace OxyApi { class Program { static async Task Main() { const string Username = "USERNAME"; const string Password = "PASSWORD"; var parameters = new { source = "google_search", query = "adidas", parse = true, context = new dynamic [] { new { key = "udm", value = 12 }, } }; var client = new HttpClient(); Uri baseUri = new Uri("https://realtime.oxylabs.io"); client.BaseAddress = baseUri; var requestMessage = new HttpRequestMessage(HttpMethod.Post, "/v1/queries"); requestMessage.Content = JsonContent.Create(parameters); var authenticationString = $"{Username}:{Password}"; var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.ASCIIEncoding.UTF8.GetBytes(authenticationString)); requestMessage.Headers.Add("Authorization", "Basic " + base64EncodedAuthenticationString); var response = await client.SendAsync(requestMessage); var contents = await response.Content.ReadAsStringAsync(); Console.WriteLine(contents); } } } ``` {% endtab %} {% tab title="Java" %} ```java package org.example; import okhttp3.*; import org.json.JSONArray; import org.json.JSONObject; import java.util.concurrent.TimeUnit; public class Main implements Runnable { private static final String AUTHORIZATION_HEADER = "Authorization"; public static final String USERNAME = "USERNAME"; public static final String PASSWORD = "PASSWORD"; public void run() { JSONObject jsonObject = new JSONObject(); jsonObject.put("source", "google_search"); jsonObject.put("query", "adidas"); jsonObject.put("parse", true); jsonObject.put("context", new JSONArray() .put(new JSONObject() .put("key", "udm") .put("value", 12)) ); Authenticator authenticator = (route, response) -> { String credential = Credentials.basic(USERNAME, PASSWORD); return response .request() .newBuilder() .header(AUTHORIZATION_HEADER, credential) .build(); }; var client = new OkHttpClient.Builder() .authenticator(authenticator) .readTimeout(180, TimeUnit.SECONDS) .build(); var mediaType = MediaType.parse("application/json; charset=utf-8"); var body = RequestBody.create(jsonObject.toString(), mediaType); var request = new Request.Builder() .url("https://realtime.oxylabs.io/v1/queries") .post(body) .build(); try (var response = client.newCall(request).execute()) { if (response.body() != null) { try (var responseBody = response.body()) { System.out.println(responseBody.string()); } } } catch (Exception exception) { System.out.println("Error: " + exception.getMessage()); } System.exit(0); } public static void main(String[] args) { new Thread(new Main()).start(); } } ``` {% endtab %} {% tab title="JSON" %} ```json { "source": "google_search", "query": "adidas", "parse": true, "context": [ { "key": "udm", "value": 12 } ] } ``` {% endtab %} {% endtabs %} ### tbm {% tabs %} {% tab title="cURL" %} ```shell curl 'https://realtime.oxylabs.io/v1/queries' \\ --user 'USERNAME:PASSWORD' \\ -H 'Content-Type: application/json' \\ -d '{ "source": "google_search", "query": "adidas", "parse": true, "context": [ { "key": "tbm", "value": "nws" } ] }' ``` {% endtab %} {% tab title="Python" %} ```python import requests from pprint import pprint # Structure payload. payload = { 'source': 'google_search', 'query': 'adidas', 'parse': True, 'context': [ {'key': 'tbm', 'value': 'nws'}, ], } # Get response. response = requests.post( 'https://realtime.oxylabs.io/v1/queries', auth=('USERNAME', 'PASSWORD'), json=payload, ) # Print prettified response to stdout. pprint(response.json()) ``` {% endtab %} {% tab title="Node.js" %} ```javascript const https = require("https"); const username = "USERNAME"; const password = "PASSWORD"; const body = { source: "google_search", query: "adidas", parse: true, context: [ { key: "tbm", value: "nws" }, ], }; const options = { hostname: "realtime.oxylabs.io", path: "/v1/queries", method: "POST", headers: { "Content-Type": "application/json", Authorization: "Basic " + Buffer.from(`${username}:${password}`).toString("base64"), }, }; const request = https.request(options, (response) => { let data = ""; response.on("data", (chunk) => { data += chunk; }); response.on("end", () => { const responseData = JSON.parse(data); console.log(JSON.stringify(responseData, null, 2)); }); }); request.on("error", (error) => { console.error("Error:", error); }); request.write(JSON.stringify(body)); request.end(); ``` {% endtab %} {% tab title="HTTP" %} ```http source=google_search&query=adidas&parse=true&context[0][key]=tbm&context[0][value]=nws&access_token=12345abcde ``` {% endtab %} {% tab title="PHP" %} ```php 'google_search', 'query' => 'adidas', 'parse' => true, 'context' => [ [ 'key' => 'tbm', 'value' => 'nws', ] ] ); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params)); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_USERPWD, "USERNAME" . ":" . "PASSWORD"); $headers = array(); $headers[] = "Content-Type: application/json"; curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); $result = curl_exec($ch); echo $result; if (curl_errno($ch)) { echo 'Error:' . curl_error($ch); } curl_close($ch); ``` {% endtab %} {% tab title="Golang" %} ```go package main import ( "bytes" "encoding/json" "fmt" "io/ioutil" "net/http" ) func main() { const Username = "USERNAME" const Password = "PASSWORD" payload := map[string]interface{}{ "source": "google_search", "query": "adidas", "parse": true, "context": []map[string]interface{}{ {"key": "tbm", "value": "nws"}, }, } jsonValue, _ := json.Marshal(payload) client := &http.Client{} request, _ := http.NewRequest("POST", "https://realtime.oxylabs.io/v1/queries", bytes.NewBuffer(jsonValue), ) request.SetBasicAuth(Username, Password) response, _ := client.Do(request) responseText, _ := ioutil.ReadAll(response.Body) fmt.Println(string(responseText)) } ``` {% endtab %} {% tab title="C#" %} ```csharp using System; using System.Collections.Generic; using System.Net.Http; using System.Net.Http.Json; using System.Threading.Tasks; namespace OxyApi { class Program { static async Task Main() { const string Username = "USERNAME"; const string Password = "PASSWORD"; var parameters = new { source = "google_search", query = "adidas", parse = true, context = new dynamic [] { new { key = "tbm", value = "nws" }, } }; var client = new HttpClient(); Uri baseUri = new Uri("https://realtime.oxylabs.io"); client.BaseAddress = baseUri; var requestMessage = new HttpRequestMessage(HttpMethod.Post, "/v1/queries"); requestMessage.Content = JsonContent.Create(parameters); var authenticationString = $"{Username}:{Password}"; var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.ASCIIEncoding.UTF8.GetBytes(authenticationString)); requestMessage.Headers.Add("Authorization", "Basic " + base64EncodedAuthenticationString); var response = await client.SendAsync(requestMessage); var contents = await response.Content.ReadAsStringAsync(); Console.WriteLine(contents); } } } ``` {% endtab %} {% tab title="Java" %} ```java package org.example; import okhttp3.*; import org.json.JSONArray; import org.json.JSONObject; import java.util.concurrent.TimeUnit; public class Main implements Runnable { private static final String AUTHORIZATION_HEADER = "Authorization"; public static final String USERNAME = "USERNAME"; public static final String PASSWORD = "PASSWORD"; public void run() { JSONObject jsonObject = new JSONObject(); jsonObject.put("source", "google_search"); jsonObject.put("query", "adidas"); jsonObject.put("parse", true); jsonObject.put("context", new JSONArray() .put(new JSONObject() .put("key", "tbm") .put("value", "nws")) ); Authenticator authenticator = (route, response) -> { String credential = Credentials.basic(USERNAME, PASSWORD); return response .request() .newBuilder() .header(AUTHORIZATION_HEADER, credential) .build(); }; var client = new OkHttpClient.Builder() .authenticator(authenticator) .readTimeout(180, TimeUnit.SECONDS) .build(); var mediaType = MediaType.parse("application/json; charset=utf-8"); var body = RequestBody.create(jsonObject.toString(), mediaType); var request = new Request.Builder() .url("https://realtime.oxylabs.io/v1/queries") .post(body) .build(); try (var response = client.newCall(request).execute()) { if (response.body() != null) { try (var responseBody = response.body()) { System.out.println(responseBody.string()); } } } catch (Exception exception) { System.out.println("Error: " + exception.getMessage()); } System.exit(0); } public static void main(String[] args) { new Thread(new Main()).start(); } } ``` {% endtab %} {% tab title="JSON" %} ```json { "source": "google_search", "query": "adidas", "parse": true, "context": [ { "key": "tbm", "value": "nws" } ] } ``` {% endtab %} {% endtabs %} 我们的示例使用同步 [**Realtime**](/products/cn/web-scraper-api/integration-methods/realtime.md) 集成方式。如果你想使用 [**Proxy Endpoint**](/products/cn/web-scraper-api/integration-methods/proxy-endpoint.md) 或异步 [**Push-Pull**](/products/cn/web-scraper-api/integration-methods/push-pull.md) 集成，请参阅 [**集成方式**](/products/cn/web-scraper-api/integration-methods.md) 部分。 ## 请求参数值 ### 通用 抓取 Google News 搜索结果的基本设置和自定义选项。
参数说明默认值
source设置爬虫。使用 google_search.–
query要搜索的关键词或短语。–
context:
udm		要获取 News 搜索结果，请将值设置为 12。查找其他可接受的值 这里.–
context:
tbm		要获取 News 搜索结果，请将值设置为 nws。其他可接受的值有： app, blg, bks, dsc, isch, pts, plcs, rcp, lcl–
render设置为 html 是此 source 的必需项。 更多信息.–
parse设置为时返回解析数据 true。查看输出 数据字典.false
callback_url你的回调端点 URL。 更多信息–
user_agent_type设备类型和浏览器。完整列表可在 这里.desktop
\- 必填参数 {% hint style="warning" %} **注意：** `udm` 和 `tbm` context 参数不能在单个抓取请求中一起使用；请选择其中一个。同时使用两者可能会导致冲突或意外行为。 {% endhint %} #### Google 高级搜索运算符 在抓取时，你可能会发现将 Google 高级搜索运算符与你的查询结合使用很有用。这样可以自定义搜索范围，确保结果更相关、更聚焦。查看这些特殊命令 [**这里**](https://ahrefs.com/blog/google-advanced-search-operators/) 和 [**这里**](https://www.semrush.com/kb/831-how-to-use-google-advanced-search-operators)。请参见下方示例。 ```json { "source": "google_search", "query": "iphone 15 launch inurl:apple" } ``` ### 本地化 将搜索结果适配到特定地理位置和语言。
参数说明默认值
geo_location结果应适配的地理位置。正确使用此参数对获取正确数据极其重要。有关更多信息，请阅读我们建议的 geo_location 参数结构 这里.-
localeAccept-Language 该请求头值会更改你的 Google 搜索页面 Web 界面语言。 更多信息.-
### 分页 用于管理分页和检索搜索结果。
参数说明默认值
start_page起始页码。1
pages要检索的页数。1
limit每页要检索的结果数量。10

context:

limit_per_page

如果你想使用同一 IP 地址抓取多个页面，请包含一个 JSON 数组，并使用 page 键指定页码。你还必须通过添加一个 limit 键来指明每页的自然结果数量。 查看示例.-
#### 每页限制 要使用此功能，请包含一个带有以下数据的 JSON 对象数组：
参数说明示例
page你想抓取的页码。任何大于 0 的整数值均可1
limit当前页上的结果数量。任何介于 1 和 100 （含）之间的整数值均可。90
#### 请求示例 ```json { "source": "google_search", "query": "adidas", "parse": true, "context": [ { "key": "limit_per_page", "value": [ {"page": 1, "limit": 10}, {"page": 2, "limit": 90} ] }] } ``` ### 过滤 可根据各种条件筛选并细化搜索结果。
参数说明默认值
context:safe_search安全搜索。设为 true 以启用。false
context:
tbs		tbs 参数。此参数类似于一个容器，用于放置更隐晦的 Google 参数，例如按日期限制/排序结果以及其他筛选条件，其中一些取决于 tbm 参数（例如 tbs=app_os:1 仅适用于 tbm 值 app）。更多信息 这里.-
### 其他 用于特殊需求的附加高级设置和控制。
参数说明默认值
context:
nfpr		true 将关闭拼写自动更正false
### 上下文参数 所有上下文参数都应添加到 `context` 数组中，作为具有 `键` 和 `值` 对的对象，例如： ```json ... "context": [ { "key": "filter", "value": "0" } ] ... ``` ## 结构化数据 SERP 爬虫 API 能够提取包含 Google 搜索结果的 HTML 或 JSON 对象，并在结果页面的各个元素上提供结构化数据。
google_search 新闻结构化输出 ```json { "results": [ { "content": { "url": "https://www.google.com/search?q=adidas&tbm=nws&uule=w+CAIQICINdW5pdGVkIHN0YXRlcw&gl=us&hl=en", "page": 1, "results": { "main": [ { "url": "https://www.cnn.com/2022/05/06/business/under-armour-stock-adidas-nike/index.html", "desc": "Tripped-up supply chains and a coronavirus surge in China are causing \nheadaches for top athletic brands.", "title": "Wall Street is fed up with Under Armour, Nike and Adidas", "source": "CNN", "pos_overall": 1, "relative_publish_date": "2 days ago" }, ... { "url": "https://www.cnbc.com/2022/05/06/dsw-tests-layout-to-spotlight-brands-like-adidas-crocs-birkenstock.html", "desc": "DSW is trying out a new store look and layout at a location opening this \nweekend in Houston, in an attempt to focus customers' attention on...", "title": "DSW is testing a store layout that puts the spotlight on brands like \nAdidas, Crocs and Birkenstock", "source": "CNBC", "pos_overall": 10, "relative_publish_date": "2 days ago" } ], "total_results_count": 57300000 }, "parse_status_code": 12000 }, "created_at": "2022-05-09 07:25:03", "updated_at": "2022-05-09 07:25:07", "page": 1, "url": "https://www.google.com/search?q=adidas&tbm=nws&uule=w+CAIQICINdW5pdGVkIHN0YXRlcw&gl=us&hl=en", "job_id": "6929330379711060993", "status_code": 200, "parser_type": "v2" } ] } ```
{% hint style="info" %} 我们仅解析以下新闻搜索结果： **desktop** 搜索。 {% endhint %} ## 输出数据字典 #### HTML 示例
#### JSON 结构 Google 新闻搜索的结构化输出包含以下字段： `URL`, `page`, `results`以及其他字段。下表列出了我们解析的每个 SERP 功能的详细信息，包括其描述和数据类型。表中还包含一些元数据。 {% hint style="info" %} 特定结果类型的项目和字段数量可能会因搜索查询而异。 {% endhint %}
键说明类型
urlGoogle 搜索页面的 URL。字符串
results包含搜索结果的字典。数组
results.main未付费新闻结果列表及其各自的详细信息。数组
results.additional趋势文章列表及其各自的详细信息。对象
results.total_results_count搜索查询找到的结果总数。数组
parse_status_code解析任务的状态码。你可以查看已说明的解析器状态码 这里.整数
created_at抓取任务创建时的时间戳。时间戳
updated_at抓取任务完成时的时间戳。时间戳
page相对于 Google SERP 分页的页码。整数
job_id与抓取任务关联的任务 ID。字符串
status_code抓取任务的状态码。你可以查看已说明的网页爬虫API状态码 这里.整数
{% hint style="info" %} 在以下部分中，当某个结果类型有多个项目时，解析后的 JSON 代码片段会被缩短。 {% endhint %} ### 主要 显示未付费新闻结果列表，并提供每篇文章的相关详细信息。
```json ... "main": [ { "url": "https://www.yahoo.com/lifestyle/tiger-woods-nikes-epic-partnership-015311819.html", "desc": "That there could ever be a world in which Tiger Woods wasn't sponsored by \nNike seemed...", "title": "How Tiger Woods and Nike's Epic Partnership Fell Apart", "source": "Yahoo", "pos_overall": 1, "relative_publish_date": "1 day ago" }, ... }, ... ```
键（results.main）说明类型
url完整文章的 URL。字符串
desc完整文章的简短摘录。字符串
title文章标题。字符串
source文章发布的网站名称。字符串
pos_overall表示该结果在 News SERP 主要结果中的总体位置。整数
relative_publish_date描述文章发布于多久之前。字符串
### 其他 展示趋势文章列表及其相关详细信息。
```json ... "additional": [ { "items": [ { "pos": 1, "url": "https://www.complex.com/sneakers/a/brendan-dunne/nike-book-1-colorways-haven-hike-rattlesnake", "title": "Nike Book 1 Colorways Haven Hike Rattlesnake", "source": "Complex", "relative_publish_date": "1 day ago" }, ... ], "pos_overall": 2, "section_title": "Devin Booker confirms issues with Nike Book 1 launch" } ... ```
键（results.additional）说明类型
items文章列表及其各自详细信息。数组
items.pos用于标示文章在列表中的唯一标识。整数
items.url完整文章的 URL。字符串
items.title文章标题。字符串
items.source文章发布的网站名称。字符串
items.relative_publish_date描述文章发布于多久之前。字符串
pos_overall表示该结果在 News SERP 附加结果中的总体位置。整数
section_title附加部分的名称。字符串
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://developers.oxylabs.io/api-targets/cn/search-engines/google/search/news-search.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.