Quick Start: Web Scraper API

Oxylabs Web Scraper API is an all-in-one solution that handles proxy rotation, JavaScript rendering, and anti-bot systems automatically. This guide will take you from an empty terminal to real-time web data in just a few minutes.

Follow the detailed steps below or watch a quick video walk-through:

1

Setup & credentials

Before sending your first request, you need to authorize your access:

1. Create an account: Sign up at the Oxylabs Dashboard.

2. Choose a plan: Pick a free trial for testing or a subscription right away.

3. Create an API User: Go to "Web Scraper API" section on the left and select "Users" tab.

4. Store your credentials: These specific API credentials (not your dashboard login) are required for all requests.

2

Your first request

The universal source works for any public website. Open your terminal and run this command to scrape our sandbox site:

curl --user 'USERNAME:PASSWORD' \
'https://realtime.oxylabs.io/v1/queries' \
-H 'Content-Type: application/json' \
-d '{
    "source": "universal",
    "url": "https://sandbox.oxylabs.io/products"
}'

3

Getting structured data

If you are scraping a supported domain (like Amazon or Google), the API can return structured JSON instead of raw HTML. Just set the source to a dedicated target and use "parse": true. For example, let's scrape an Amazon product:

curl 'https://realtime.oxylabs.io/v1/queries' \
--user "USERNAME:PASSWORD" \
-H "Content-Type: application/json" \
-d '{
    "source": "amazon_product", 
    "query": "B07FZ8S74R", 
    "parse": true
}'

Essential Parameters

These are the most common parameters used to customize your request:

Parameter	Type	Description
source	string	Required. The scraping engine (e.g., universal, amazon, Google Search etc.).
url / query	string	The target link or the keyword/ID for dedicated targets.
parse	boolean	Set to true for structured JSON results on supported domains.
render	string	Set to html to enable a headless browser for JavaScript-heavy sites.
geo_location	string	Localize your results (e.g., United States or 90210).

Common response codes

Code	Status	Description
200	OK	Success. Your data is in the results object.
401	Unauthorized	Check your API User credentials in the dashboard.
429	Too Many Requests	You have exceeded your concurrency limit.

You can find the full list of response codes in the Web Scraper API product section.

Next Steps

Now that you've sent your first query, explore the rest of our documentation based on your needs:

• Looking for specific domains? Head to the API Targets section for dedicated guides on Amazon, Google, eBay, and many more popular websites.

• Need advanced features? Visit the Products space to learn about Custom Parsing, Scheduler, Cloud Storage delivery, and more.

• Scale with AI? Check out solutions for AI workflows or see how OxyCopilot can automate your parsing logic using plain English.

• Different Integrations? We use synchronous Realtime integration in our examples. You can find Proxy Endpoint or asynchronous Push-Pull integration in the integration methods page.