> For the complete documentation index, see [llms.txt](https://developers.oxylabs.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developers.oxylabs.io/products/cn/web-scraper-api/features/result-processing-and-storage/result-aggregator.md).

# 结果聚合器

该 **结果聚合器** 可让您将来自单独抓取或解析任务的多个小结果收集到一个聚合文件中。当您运行大量任务并返回许多小文件、这些文件可以合并为更大的输出集合，或者需要以批处理文件（JSON、JSONL 或 Gzip）处理结果时，这最有用。

聚合后的响应可以交付到您的 [云存储](/products/cn/web-scraper-api/features/result-processing-and-storage/cloud-storage.md) （Google Cloud Storage、Amazon S3 或其他兼容 S3 的服务）。

## 如何使用

{% stepper %}
{% step %}

### 创建聚合器

首先，定义一个聚合器实例，并指定交付存储目标和交付触发条件。

#### 请求示例

以下请求创建一个聚合器，每 1 小时上传一个批处理文件（`cron 调度`）或者当文件达到 500MB（`524288000` 字节），以先到者为准。

```bash
curl -X POST https://data.oxylabs.io/v1/aggregators \

-u "USERNAME:PASSWORD" \

-H "Content-Type: application/json" \\
-d '{
  "name": "amazon_hourly",
  "storage_type": "s3",
  "storage_url": "s3://my_bucket/batches",
  "max_result_count": 10000,
  "max_size_bytes": 524288000,
  "schedule": "0 */1 * * *"
}'
```

#### 请求参数

| 参数                                                          | 描述                                                                                                              | 类型    |
| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ----- |
| <mark style="background-color:green;">`name`</mark>         | 唯一的聚合器标识符。                                                                                                      | `字符串` |
| <mark style="background-color:green;">`storage_type`</mark> | 存储提供商（ `s3`, `gcs`，或 `s3_compatible`).                                                                          | `字符串` |
| <mark style="background-color:green;">`storage_url`</mark>  | 目标 bucket/container 路径。                                                                                         | `字符串` |
| `file_output_type`                                          | 输出格式（`json`, `jsonl`, `gzip_json`，或 `gzip_jsonl`)                                                               | `字符串` |
| `max_size_bytes`                                            | <p>批处理大小上限（字节）。</p><p>最大值： <strong>1GB</strong>.</p>                                                            | `整数`  |
| `schedule`                                                  | <p>聚合频率，使用 <strong>cron 表达式</strong>。（例如， <code>0 \*/1 \* \* \*</code> 每小时一次）。<br>最大值： <strong>1h</strong>.</p> | `字符串` |
| `max_result_count`                                          | 当结果数量达到上限时触发交付。                                                                                                 | `整数`  |
| `callback_url`                                              | 您的回调端点 URL。 [**更多信息**](/products/cn/web-scraper-api/integration-methods/push-pull.md#callback)                  | `字符串` |

– 必填参数。
{% endstep %}

{% step %}

### 向聚合器发送请求

聚合器创建后，您可以使用 `aggregate_name` 参数将抓取任务路由到它。您无需在这些请求中指定存储详情，聚合器会处理交付。

#### 请求示例

```bash
curl --user "USERNAME:PASSWORD" \
'https://data.oxylabs.io/v1/queries' \\
-H "Content-Type: application/json" \\
-d '{
    "source": "universal",
    "url": "https://www.example.com",
    "aggregate_name": "amazon_hourly"
}'
```

{% endstep %}

{% step %}

### 检索聚合器信息

您可以随时查看聚合器的配置和使用统计信息。

#### 请求示例

```bash
GET https://data.oxylabs.io/v1/aggregators/{name}
```

#### 响应示例

```json
{
    "name": "amazon_hourly",
    "callback_url": "",
    "storage_type": "s3",
    "storage_url": "s3://my_bucket/path_for_aggregates",
    "max_result_count": 1048576,
    "max_size_bytes": 524288000,
    "schedule": "0 */1 * * *",
    "file_output_type": "jsonl",
    "filename_prefix": "",
    "filename_suffix": "",
    "created_at": "2025-12-05T13:30:32Z",
    "usage_statistics": {
        "total_result_count": 0,
        "total_bytes_delivered": 0,
        "total_files_delivered": 0
    }
}
```

{% endstep %}
{% endstepper %}

## 交付与输出

### 自动交付

当发生以下任一情况时，批处理文件会关闭并上传：

* 该 `schedule` 达到时间限制（最大值：1 小时）。
* 该 `max_size_bytes` 达到大小限制（最大值：1GB）。
* 该 `max_result_count` 达到结果限制。

### 手动交付

您可以在达到限制前，使用以下方式强制立即交付当前批次： `POST https://data.oxylabs.io/v1/aggregators/{name}/trigger` 端点，如下例所示：

```bash
curl -X POST https://data.oxylabs.io/v1/aggregators/amazon_hourly/trigger -u "USERNAME:PASSWORD"
```

### 输出结构

输出批处理文件会以唯一时间戳保存到您的存储中：

```
my_bucket/
├── batches/
│   ├── 2024-08-08T01:00:00.000-00:00-amazon_hourly.jsonl
│   ├── 2024-08-08T02:00:00.000-00:00-amazon_hourly.jsonl
│   └── 2024-08-08T03:00:00.000-00:00-amazon_hourly.jsonl
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://developers.oxylabs.io/products/cn/web-scraper-api/features/result-processing-and-storage/result-aggregator.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.