JavaScript Rendering

If the page you wish to scrape requires JavaScript to dynamically load all necessary data into the DOM, you can include a render parameter in your requests instead of setting up and using a headless browser manually. Requests with this parameter will be fully rendered, and the data will be stored in either an HTML file or a PNG screenshot, depending on the specified parameter.

HTML

Set the render parameter to html to get the raw output of the rendered page.

PNG (Screenshot)

Set the render parameter to png to get a Base64-encoded screenshot of the rendered page.

If you want to scrape an image and download it, please refer to this section.

Request sample

curl --user "user:pass" \
'https://realtime.oxylabs.io/v1/queries' \
-H "Content-Type: application/json" \
-d '{"source": "universal_ecommerce", "url": "https://www.example.com", "render": "html"}'

JavaScript rendering takes more time to scrape the page. Please set timeout on the client side to 180 seconds if using Realtime or Proxy Endpoint integration methods.

To ensure lowest traffic consumption, our system does not load unnecessary assets during page rendering.

Forcing rendering on specific pages

For successful scraping, some page types of specific domains require rendering due to their dynamic content. Our system automatically enforces rendering for these pages, even if not explicitly set by the user.

Please note that rendered jobs consume more traffic compared to non-rendered jobs.

We want our users to be fully aware of this when scraping the following pages:

This approach provides the best possible scraping experience, ensuring data accuracy and reliability from these challenging pages.

If you wish to disable rendering, you can do so by adding the following parameter to your requests:

"render": ""

Browser instructions

With our Headless Browser you can also execute various browser instructions such as clicking, scrolling, inputting, waiting and more. Read more:

Browser instructions

Last updated