(LEGACY) E-Commerce Scraper API

E-Commerce Scraper API is now a part of Web Scraper API.

E-Commerce Scraper API is designed to collect e-commerce data from any online marketplace.

Getting started

Create your API user credentials: Sign up for a free trial or purchase the product in the Oxylabs dashboard to create your API user credentials (USERNAME and PASSWORD).

If you need more than one API user for your account, please contact our customer support or message our 24/7 live chat support.

Request sample

cURL

curl 'https://realtime.oxylabs.io/v1/queries' \
--user "USERNAME:PASSWORD" \
-H "Content-Type: application/json" \
-d '{
        "source": "amazon_product",
        "query": "B07FZ8S74R",
        "geo_location": "90210",
        "parse": true
    }'

Python

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon_product',
    'query': 'B07FZ8S74R',
    'geo_location': '90210',
    'parse': True
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('USERNAME', 'PASSWORD'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())

Node.js

const https = require("https");

const username = "USERNAME";
const password = "PASSWORD";
const body = {
    source: "amazon_product",
    query: "B07FZ8S74R",
    geo_location: "90210",
    parse: true,
};

const options = {
    hostname: "realtime.oxylabs.io",
    path: "/v1/queries",
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        Authorization:
            "Basic " + Buffer.from(`${username}:${password}`).toString("base64"),
    },
};

const request = https.request(options, (response) => {
    let data = "";

    response.on("data", (chunk) => {
        data += chunk;
    });

    response.on("end", () => {
        const responseData = JSON.parse(data);
        console.log(JSON.stringify(responseData, null, 2));
    });
});

request.on("error", (error) => {
    console.error("Error:", error);
});

request.write(JSON.stringify(body));
request.end();

PHP

<?php

$params = array(
    'source' => 'amazon_product',
    'query' => 'B07FZ8S74R',
    'geo_location' => '90210',
    'parse' => true,
);

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "USERNAME" . ":" . "PASSWORD");

$headers = array();
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);
echo $result;

if (curl_errno($ch)) {
    echo 'Error:' . curl_error($ch);
}
curl_close($ch);
?>

C#

using System;
using System.Net.Http;
using System.Net.Http.Json;
using System.Threading.Tasks;

namespace OxyApi
{
    class Program
    {
        static async Task Main()
        {
            const string Username = "USERNAME";
            const string Password = "PASSWORD";

            var parameters = new {
                source = "amazon_product",
                query = "B07FZ8S74R",
                geo_location = "90210",
                parse = true
            };

            var client = new HttpClient();

            Uri baseUri = new Uri("https://realtime.oxylabs.io");
            client.BaseAddress = baseUri;

            var requestMessage = new HttpRequestMessage(HttpMethod.Post, "/v1/queries");
            requestMessage.Content = JsonContent.Create(parameters);

            var authenticationString = $"{Username}:{Password}";
            var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes(authenticationString));
            requestMessage.Headers.Add("Authorization", "Basic " + base64EncodedAuthenticationString);

            var response = await client.SendAsync(requestMessage);
            var contents = await response.Content.ReadAsStringAsync();

            Console.WriteLine(contents);
        }
    }
}

Golang

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	const Username = "USERNAME"
	const Password = "PASSWORD"

	payload := map[string]interface{}{
		"source":       "amazon_product",
		"query":        "B07FZ8S74R",
		"geo_location": "90210",
		"parse":        true,
	}

	jsonValue, _ := json.Marshal(payload)

	client := &http.Client{}
	request, _ := http.NewRequest("POST",
		"https://realtime.oxylabs.io/v1/queries",
		bytes.NewBuffer(jsonValue),
	)

	request.SetBasicAuth(Username, Password)
	request.Header.Set("Content-Type", "application/json")
	response, _ := client.Do(request)

	responseText, _ := ioutil.ReadAll(response.Body)
	fmt.Println(string(responseText))
}

HTTP

https://realtime.oxylabs.io/v1/queries?source=amazon_product&query=B07FZ8S74R&geo_location=90210&parse=true&access_token=12345abcde

Java

package org.example;

import okhttp3.*;
import org.json.JSONObject;
import java.util.concurrent.TimeUnit;

public class Main implements Runnable {
    private static final String AUTHORIZATION_HEADER = "Authorization";
    public static final String USERNAME = "USERNAME";
    public static final String PASSWORD = "PASSWORD";

    public void run() {
        JSONObject jsonObject = new JSONObject();
        jsonObject.put("source", "amazon_product");
        jsonObject.put("query", "B07FZ8S74R");
        jsonObject.put("geo_location", "90210");
        jsonObject.put("parse", true);

        Authenticator authenticator = (route, response) -> {
            String credential = Credentials.basic(USERNAME, PASSWORD);
            return response
                    .request()
                    .newBuilder()
                    .header(AUTHORIZATION_HEADER, credential)
                    .build();
        };

        var client = new OkHttpClient.Builder()
                .authenticator(authenticator)
                .readTimeout(180, TimeUnit.SECONDS)
                .build();

        var mediaType = MediaType.parse("application/json; charset=utf-8");
        var body = RequestBody.create(jsonObject.toString(), mediaType);
        var request = new Request.Builder()
                .url("https://realtime.oxylabs.io/v1/queries")
                .post(body)
                .build();

        try (var response = client.newCall(request).execute()) {
            if (response.body() != null) {
                try (var responseBody = response.body()) {
                    System.out.println(responseBody.string());
                }
            }
        } catch (Exception exception) {
            System.out.println("Error: " + exception.getMessage());
        }

        System.exit(0);
    }

    public static void main(String[] args) {
        new Thread(new Main()).start();
    }
}

JSON

{
    "source": "amazon_product",
    "query": "B07FZ8S74R",
    "geo_location": "90210",
    "parse": true
}

Output example

{
    "results": [
        {
            "content": {
                "ads": [
                    {
                        "pos": 1,
                        "asin": "B07KRY43KN",
                        "type": "organic_also_viewed",
                        "price": 29.99,
                        "title": "Echo Glow - Multicolor smart lamp, Works with Alexa",
                        "images": [
                            "https://images-na.ssl-images-amazon.com/images/I/61kaGoho9HL._AC_UL160_SR160,160_.jpg"
                        ],
                        "rating": 4.5,
                        "location": "carousel",
                        "price_upper": 29.99,
                        "reviews_count": 48742,
                        "is_prime_eligible": false
                    },
                    {...}
                ],
                "url": "https://www.amazon.com/dp/B07FZ8S74R",
                "asin": "B07FZ8S74R",
                "page": 1,
                "brand": "Amazon",
                "price": 0,
                "stock": "Currently unavailable. We don't know when or if this item will be back in stock.",
                "title": "Echo Dot (3rd Gen, 2018 release) - Smart speaker with Alexa - Charcoal",
                "coupon": "",
                "images": [
                    "https://m.media-amazon.com/images/I/61MZfowYoaL._AC_SL1000_.jpg",
                    "..."
                ],
                "rating": 4.7,
                "category": [],
                "currency": "USD",
                "delivery": [],
                "page_type": "Product",
                "price_sns": 0,
                "variation": [],
                "has_videos": false,
                "sales_rank": [],
                "top_review": "I recently added the Echo Dot (3rd Gen, 2018 release) to my smart home setup, and I'm thoroughly impressed with its performance and capabilities. This compact smart speaker packs a punch in terms of functionality and convenience.Setup was a breeze \u2013 within minutes, I had Alexa up and running, ready to assist. The voice recognition is remarkably accurate, and Alexa's responses are both quick and helpful. Whether it's checking the weather, setting timers, or playing my favorite music, the Echo Dot effortlessly integrates into my daily routine.The sound quality is surprisingly robust for such a compact device. The clarity and depth of the audio are impressive, making it perfect for streaming music or even serving as a secondary speaker for a larger sound system.I appreciate the sleek and unobtrusive design of the Echo Dot, seamlessly blending into any room's decor. The Charcoal color adds a touch of elegance, making it a visually pleasing addition to my living space.One of the standout features is the smart home control. The Echo Dot effortlessly connects with various smart devices, allowing me to control lights, thermostats, and more with simple voice commands. It truly enhances the overall convenience of my smart home setup.In conclusion, the Echo Dot (3rd Gen, 2018 release) has proven to be an invaluable addition to my home. Its combination of voice recognition accuracy, impressive sound quality, and seamless smart home integration make it a standout product in the smart speaker market. If you're looking for a reliable and feature-rich smart speaker, the Echo Dot is an excellent choice \u2013 highly recommended!\n  \nRead more",
                "asin_in_url": "B07FZ8S74R",
                "description": "",
                "parent_asin": "B07H65KP63",
                "price_upper": 0,
                "pricing_str": "",
                "pricing_url": "https://www.amazon.com/gp/offer-listing/B07FZ8S74R?startIndex=0",
                "manufacturer": "Amazon",
                "price_buybox": -1,
                "product_name": "Echo Dot (3rd Gen, 2018 release) - Smart speaker with Alexa - Charcoal",
                "bullet_points": "MEET ECHO DOT - Our most compact smart speaker that fits perfectly into small spaces.\nRICH AND LOUD SOUND - Better speaker quality than Echo Dot Gen 2 for richer and louder sound. Pair with a second Echo Dot for stereo sound.\nALEXA HELPS YOU DO MORE WITH PRIME - Listen to millions of songs with Amazon Music, use your voice to for 2-day shipping, listen to audiobooks on Audible, and much more.\nMAKE YOUR LIFE EASIER - Alexa can set timers, check the weather, read the news, adjust thermostats, answer questions, and more to help with daily tasks.\nDESIGNED TO PROTECT YOUR PRIVACY \u2013 Built with multiple layers of privacy controls including the ability to delete your recordings, mute your mic, and more in-app privacy controls.\nCONTROL MUSIC WITH A SIMPLE PHRASE - Echo Dot allows you to stream songs from Amazon Music, Apple Music, Spotify, SiriusXM, and others, as well as via Bluetooth. Voice control with Alexa makes it easy to skip to the next song, adjust the volume, and pause without getting up.\nCONNECT WITH OTHERS HANDS-FREE - Call friends and family who have the Alexa app or an Echo device. Instantly drop in on other rooms or announce to the whole house that dinner's ready.\nALEXA HAS SKILLS - With tens of thousands of skills and counting, Alexa is always getting smarter and adding new skills like tracking fitness, playing games, and more.",
                "price_initial": 0,
                "pricing_count": 1,
                "reviews_count": 985157,
                "sns_discounts": [],
                "developer_info": [],
                "price_shipping": 0,
                "featured_merchant": [],
                "is_prime_eligible": false,
                "parse_status_code": 12005,
                "answered_questions_count": 0,
                "rating_stars_distribution": [
                    {
                        "rating": 5,
                        "percentage": 82
                    },
                    {...}
                ]
            },
            "created_at": "2024-07-01 09:05:45",
            "updated_at": "2024-07-01 09:05:55",
            "page": 1,
            "url": "https://www.amazon.com/dp/B07FZ8S74R",
            "job_id": "7213467808577910785",
            "status_code": 200,
            "parser_type": ""
        }
    ]
}

We use synchronous Realtime integration method in our examples. If you would like to use Proxy Endpoint or asynchronous Push-Pull integration, refer to the integration methods section.

Request parameter values

1. source - This parameter sets the scraper that will be used to process your request.

2. URL or query - Provide the URL or query for the type of page you want to scrape. Refer to the table below and the corresponding target sub-pages for detailed guidance on when to use each parameter.

3. Optionally, you can include additional parameters such as geo_location, user_agent_type, parse, render and more to customize your scraping request. Read more: Features.

- mandatory parameter

Target	Source (Scraping URL)	Source (Using Query)
Amazon	amazon	amazon_product, amazon_search, amazon_pricing, amazon_sellers, amazon_bestsellers, amazon_reviews, amazon_questions
Google Shopping	google_shopping	google_shopping_product, google_shopping_search, google_shopping_pricing
Walmart	universal_ecommerce	Not available
Best Buy	universal_ecommerce	Not available
Etsy	universal_ecommerce	Not available
Target	universal_ecommerce	Not available
Other Targets	universal_ecommerce	Not available

If you need any assistance in making your first request, feel free to contact us via the 24/7 available live chat.

Testing via Scraper APIs Playground

Login to Oxyabs dashboard and try E-Commerce Scraper API in the Scraper APIs Playground.

All information herein is provided on an “as is” basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on this page. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website’s terms of service or receive a scraping license.