> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/products/cn/web-scraper-api/features/result-processing-and-storage/result-aggregator.md).

# 结果聚合器

学习如何使用结果聚合器将多个网页爬虫API响应合并为单个更大的输出。

该 **结果聚合器** 可让你将来自不同抓取或解析任务的多个小结果收集到一个聚合文件中。当你运行大量任务并返回许多小文件、这些文件可以合并为更大的输出集合，或者需要以批文件（JSON、JSONL 或 Gzip）处理结果时，这最为有用。

聚合后的响应可以交付到你的 [云存储](/products/cn/web-scraper-api/features/result-processing-and-storage/cloud-storage.md) （Google Cloud Storage、Amazon S3 或其他 S3 兼容服务）。

## 如何使用

{% stepper %}
{% step %}

### 创建聚合器

首先，定义一个包含交付存储目标和交付触发器的聚合器实例。

#### 请求示例

以下请求创建一个聚合器，每 1 小时上传一个批文件（`cron 计划`）或当文件达到 500MB（`524288000` 字节），以先发生者为准。

```bash
curl -X POST https://data.oxylabs.io/v1/aggregators \
-u "USERNAME:PASSWORD" \
-H "Content-Type: application/json" \
-d '{
  "name": "amazon_hourly",
  "storage_type": "s3",
  "storage_url": "s3://my_bucket/batches",
  "max_result_count": 10000,
  "max_size_bytes": 524288000,
  "schedule": "0 */1 * * *"
}'
```

#### 请求参数

| 参数                                                          | 说明                                                                                                             | 类型    |
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ----- |
| <mark style="background-color:green;">`name`</mark>         | 唯一的聚合器标识符。                                                                                                     | `字符串` |
| <mark style="background-color:green;">`storage_type`</mark> | 存储提供商（ `s3`, `gcs`，或 `s3_compatible`).                                                                         | `字符串` |
| <mark style="background-color:green;">`storage_url`</mark>  | 目标 bucket/container 路径。                                                                                        | `字符串` |
| `file_output_type`                                          | 输出格式（`json`, `jsonl`, `gzip_json`，或 `gzip_jsonl`)                                                              | `字符串` |
| `max_size_bytes`                                            | <p>最大批大小限制（字节）。</p><p>最大： <strong>1GB</strong>.</p>                                                            | `整数`  |
| `schedule`                                                  | <p>聚合频率，采用 <strong>cron 表达式</strong>。（例如， <code>0 \*/1 \* \* \*</code> 每小时一次）。<br>最大： <strong>1h</strong>.</p> | `字符串` |
| `max_result_count`                                          | 当结果数量达到上限时触发交付。                                                                                                | `整数`  |
| `callback_url`                                              | 你的回调端点 URL。 [**更多信息**](/products/cn/web-scraper-api/integration-methods/push-pull.md#callback)                 | `字符串` |

– 必填参数。
{% endstep %}

{% step %}

### 向聚合器发送请求

创建聚合器后，你可以使用 `aggregate_name` 参数将抓取任务路由到它。你无需在这些请求中指定存储详情，聚合器会处理交付。

#### 请求示例

```bash
curl --user "USERNAME:PASSWORD" \
'https://data.oxylabs.io/v1/queries' \
-H "Content-Type: application/json" \
-d '{
    "source": "universal",
    "url": "https://www.example.com",
    "aggregate_name": "amazon_hourly"
}'
```

{% endstep %}

{% step %}

### 检索聚合器信息

你可以随时查看聚合器的配置和使用统计信息。

#### 请求示例

```bash
GET https://data.oxylabs.io/v1/aggregators/{name}
```

#### 响应示例

```json
{
    "name": "amazon_hourly",
    "callback_url": "",
    "storage_type": "s3",
    "storage_url": "s3://my_bucket/path_for_aggregates",
    "max_result_count": 1048576,
    "max_size_bytes": 524288000,
    "schedule": "0 */1 * * *",
    "file_output_type": "jsonl",
    "filename_prefix": "",
    "filename_suffix": "",
    "created_at": "2025-12-05T13:30:32Z",
    "usage_statistics": {
        "total_result_count": 0,
        "total_bytes_delivered": 0,
        "total_files_delivered": 0
    }
}
```

{% endstep %}
{% endstepper %}

### 其他操作

* **列出所有聚合器**: `GET https://data.oxylabs.io/v1/aggregators` — 返回 `{"aggregators": [...]}`.
* **删除聚合器**: `DELETE https://data.oxylabs.io/v1/aggregators/{name}`

## 交付与输出

### 自动交付

当以下任一情况发生时，批文件将关闭并上传：

* 该 `schedule` 达到时间限制（最大值：1 小时）。
* 该 `max_size_bytes` 达到大小限制（最大值：1GB）。
* 该 `max_result_count` 达到结果上限。

### 手动交付

你可以在达到限制之前，使用 `POST https://data.oxylabs.io/v1/aggregators/{name}/trigger` 端点强制立即交付当前批次，如下面示例所示：

```bash
curl -X POST https://data.oxylabs.io/v1/aggregators/amazon_hourly/trigger -u "USERNAME:PASSWORD"
```

### 输出结构

输出批文件会以唯一时间戳保存到你的存储中：

```
my_bucket/
├── batches/
│   ├── 2024-08-08T01:00:00.000-00:00-amazon_hourly.jsonl
│   ├── 2024-08-08T02:00:00.000-00:00-amazon_hourly.jsonl
│   └── 2024-08-08T03:00:00.000-00:00-amazon_hourly.jsonl
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://developers.oxylabs.io/products/cn/web-scraper-api/features/result-processing-and-storage/result-aggregator.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.