Proxy Endpoint

If you have ever used regular proxies for data scraping, integrating the Proxy Endpoint delivery method will be a breeze. All you need to do is use our entry node as a proxy, authorize with Scraper API credentials, and ignore certificates. In cURL, it's -k or --insecure. Your data will reach you on an open connection.

Proxy Endpoint only works with the URL-based data sources, where full URL is provided. Therefore, it only accepts a handful of additional job parameters, which should be sent as headers.

The product is not designed to be used with headless browsers (e.g., Chromium, PhantomJS, Splash, etc.) and their drivers (e.g., Playwright, Selenium, Puppeteer, etc.) directly.

Endpoint

GET realtime.oxylabs.io:60000

Input

Please see a request example below.

curl -k -x https://realtime.oxylabs.io:60000 \
-U 'USERNAME:PASSWORD' \
-H 'x-oxylabs-user-agent-type: desktop_chrome' \
-H 'x-oxylabs-geo-location: Germany' \
'https://www.example.com'

Output

Below you will find a sample response from https://example.com:

Sample response
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
        
    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 2em;
        background-color: #fdfdff;
        border-radius: 0.5em;
        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        div {
            margin: 0 auto;
            width: auto;
        }
    }
    </style>    
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents. You may use this
    domain in literature without prior coordination or asking for permission.</p>
    <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>

Accepted parameters

When making your request, along with the URL, you can send us some job parameters that we will use while executing your job. The job parameters should be sent in your request headers - see an example here.

Here is the list of job parameters that you can send with Proxy Endpoint requests:

ParameterDescription

x-oxylabs-user-agent-type

There is no way to indicate a specific User-Agent, but you can let us know which user-agent type you would like us to use. A list of supported User-Agent types can be found here.

x-oxylabs-geo-location

In some cases, you may need to indicate the geographical location that the result should be adapted for. This parameter corresponds to the geo_location parameter, described separately in source-level documentation. Accepted values depend on the URL you would like us to scrape. Read more here.

x-oxylabs-render

JavaScript execution. Read more here.

x-oxylabs-parse

Setting it to 1 will return parsed data for the targets for which we have dedicated parsers. Read more here.

Last updated