Traffic and Billing

Learn about Web Scraper API's traffic and billing, how traffic usage is calculated, and find out which results are billable.

Traffic usage

Web Scraper API count the number of results. Results refer to distinct entities of content (e.g., web page HTML) that were successfully retrieved from the web.

• Successful results (billed): All results from the target site with 2xx or 4xx status codes are counted as successful and are billable.

• Unsuccessful results (not billed): Other results are counted as unsuccessful and are not billed.

Status code 429 (Too many requests) is not billed. This occurs when you exceed the rate limit. If you encounter this frequently, contact your account manager to increase your limits.

For more information about response codes, please refer to the Response Codes table.

If the response contains 2xx or 4xx status codes, and the result doesn’t contain expected information, we still count it as a successful result.

Billing

You will be billed for the number of results with successfully extracted data.

Failed attempts to scrape due to our system errors (5xx and 6xx status codes) won’t be included, and you won’t be billed for them.

If a request results in a failed attempt due to a fault or error on your side, you will still be billed.

Please find a few billing case examples below:

Case example #1 - Successful scraping attempt

You create a scraping job, and it takes three attempts for Scraper API to fetch good content from the website successfully. You will be billed only for the result of the last attempt, which was successful.

Case example #2 - Scraping failed due to our error

You create a scraping job that has failed due to our error, e.g., we have reached our internal retry limit. In this case, you will not be billed for the received result.

Case example #3 - Scraping failed due to the client's error

You create a scraping job that has failed due to an error on your side. You will be billed for this result.