Web Scraper API features
Our Web Scraper API comes with freely available features that you can use to scale, speed up, and improve your public data-gathering efforts. Refer to the following list of features and visit their documentation pages for in-depth configuration steps.
OxyCopilot
Develop web scrapers and parsers with an AI-powered feature, OxyCopilot, via Scraper API Playground by simply providing target URLs and writing your needs in plain English. To learn more about how OxyCopilot works and explore ready-to-use prompts, visit the OxyCopilot prompts and code samples library available on our website.
Cloud integration
The cloud integration feature enables you to automatically retrieve job results directly to your Amazon S3 or Google Cloud Storage. This way, you don’t have to make additional requests to get the data from us.
Batch queries
For efficient scraping operations, Web Scraper API allows you to submit up to 1000 query or URL parameters per batch. Head to our documentation to learn more.
Headless Browser
With the Headless Browser feature, you can render JavaScript on web pages, manipulate DOM, and execute browser actions like entering text, clicking elements, scrolling, and more.
Custom Parser
When you want to parse the HTML of a web page, you can do so with Custom Parser by crafting your own parsing and data processing logic. This feature is especially valuable in scenarios where you want to retrieve parsed data, but we don’t have a dedicated parser for a specific target website.
Scheduler
For automatic execution of recurring scraping and parsing jobs, you can leverage the Scheduler feature to create schedules. We recommend using this feature together with cloud integration to retrieve data at specified intervals.
Browser instructions
Instead of coding browser instructions manually, you can easily define them through an intuitive step-by-step interface in the Playground. The system automatically generates the necessary code, and you can download the instructions in a structured JSON format for easy integration into API requests.
Last updated
Was this helpful?