WebHarvy
Integrate WebHarvy web scraper with Oxylabs Proxies. Step-by-step setup guide for proxy configuration and authentication.
WebHarvy is a web scraping tool that extracts text, HTML, and images from web pages. The tool handles logins, form submissions, navigation, pagination, scheduled scraping and supports proxies.
Follow the steps below to integrate the following Oxylabs proxies with WebHarvy:
Step 1
Download and install the WebHarvy app.
Step 2
Once set up, navigate to Settings.
Step 3
Click on Proxy Settings. Select to mark Enable network connection via Proxy Server and choose HTTP, HTTPS, or SOCKS5 as your proxy protocol Type.
Step 4
Fill in the connection details for your chosen proxy type:
Residential & Mobile Proxies
Type: HTTP, HTTPS, or SOCKS5
Address: pr.oxylabs.io
Port: 7777
You may also use additional parameters with residential and mobile proxies to set the geo-location (country, state, city, and etc.) and control sessions. Check out the Residential Proxies documentation to learn more.
Datacenter Proxies
Type: HTTP, HTTPS, or SOCKS5
Address: dc.oxylabs.io
Port: 8001
The port is the sequential number assigned to an IP address from the proxy list. Thus, port 8001 uses the first IP address from your list. You may also enable proxy IP rotation by setting the port to 8000.
Furthermore, you can specify the geo-location, for example, the US, by setting the Proxy username to user-USERNAME-country-US.
The port 8001 picks a random IP address but stays consistent during a session. Alternatively, you can rotate proxy IPs with each request by using port 8000.
If you want to use a specific geo-location, for example, the US, set the Proxy username to user-USERNAME-country-US.
Dedicated Datacenter Proxies (Enterprise)
Type: HTTP or SOCKS5
Address: 1.2.3.4 (IP address from your proxy list)
Port: 60000 (HTTP), 1180 (SOCKS5)
For Enterprise Dedicated Datacenter Proxies (purchased via sales), you’ll have to choose an IP address from the acquired proxy list.
Dedicated Datacenter Proxies (Self-service)
Proxy type: HTTP, HTTPS, or SOCKS5
Address: ddc.oxylabs.io
Port: 8001
For Self-Service Dedicated Datacenter Proxies (purchased via Oxylabs dashboard), the port indicates the sequential number of an IP address from the acquired proxy list. If you want to get a different IP with each request, change the Port to 8000 to enable proxy rotation.
ISP Proxies
Type: HTTP, HTTPS, or SOCKS5
Address: isp.oxylabs.io
Port: 8001
ISP Proxies assign each port to a specific IP address and location. Port 8001 always uses the first IP from your proxy list. Use port 8000 to enable proxy rotation.
To get rotating IPs from a specific country's proxy pool, set the username using this format: user-USERNAME-country-XX, where XX is the country code (e.g., US for United States).
Dedicated ISP Proxies
Type: HTTP, HTTPS, or SOCKS5
Address: disp.oxylabs.io
Port: 8001
You will need to choose the port assigned to an IP address from the purchased proxy list, which you can find on the Oxylabs dashboard. The 8001 port will pick the first IP address from your list for all requests. Use port 8000 for automatic proxy IP rotation.
Step 5
Select Requires Authentication to enter your Oxylabs proxy Username and Password:
Residential & Mobile proxies
Username: customer-USERNAME
Password: PASSWORD
Datacenter proxies
Username: user-USERNAME
Password: PASSWORD
Dedicated Datacenter proxies (Enterprise)
Username: USERNAME
Password: PASSWORD
Dedicated Datacenter proxies (Self-service)
Username: user-USERNAME
Password: PASSWORD
ISP proxies
Username: user-USERNAME
Password: PASSWORD
Dedicated ISP proxies
Username: user-USERNAME
Password: PASSWORD
Make sure to add the customer- or user- prefix to your username if your proxy type requires it.
Click on the + button to add your new proxy to the list.
Step 6
Press Apply to finish your WebHarvy proxy integration.
That's all. Now you can browse the internet and mark the specific rows to scrape. By clicking Start, you can begin selecting your target data.
Find the original WebHarvy integration blog post here.
Last updated
Was this helpful?