To start scraping with one of our Scraper APIs, follow the simple steps below:
- 3.Put together a query and send it to our API.
- Under the chosen page type, you will find code examples in different programming languages. Use them to build your query and make sure always to include the following elements:
Content-type. When submitting jobs, always send the
- Payload. It's a collection of query parameters that specify in detail the job you would like our service to do. Pay attention to mandatory parameters (
URL). They are marked in green in the query parameter tables. You can make a very basic request using those mandatory parameters or add various additional ones (e.g.,
password. You must provide your API user credentials. Otherwise, your query won't work. Our Scraper APIs use basic HTTP authentication.
If you want to try our API manually before using it at scale, we recommend using Postman. Every scraper API, has its own Postman collection, which you can import to Postman and start scraping right away. The collections contain request templates for various sources and integration methods. More information can be found here:
- At any point you can check out your all-time usage statistics by querying the following endpoint:
GET https://data.oxylabs.io/v2/stats. It's also possible to return your monthly or daily numbers. Visit the below section for more information.
- Try out Web Crawler - a Scraper APIs feature that lets you crawl any site, select useful content and have it delivered to you in bulk.
- Check out our Scraper APIs Scheduler functionality. It can be used for recurring scraping and parsing jobs.
- Discover the Custom Parser feature, which empowers you to define your own parsing and data processing logic to be executed on raw scraping results.
- Check out Oxylabs GitHub for tutorials on how to scrape websites, use our tools, implement products or integrate them using the most popular programming languages (e.g., C#, Java, NodeJs, PHP, Python, etc.).
All information herein is provided on an “as is” basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on this page. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website’s terms of service or receive a scraping license.