Selenium

Search…
Selenium
​Selenium is a tool that helps automate web browser interactions for website testing and more. It's useful when you need to interact with a browser to perform a number of tasks, such as clicking on buttons, scrolling, etc. Even if Selenium is primarily used for website testing, you can also use it for web scraping because it helps locate the required public data on a website.
To integrate Selenium with Oxylabs proxies, follow the instructions below:
Firstly, you'll need to install Selenium Wire to extend Selenium's Python bindings because using the default Selenium module for implementing proxies that require authentication makes the whole process complicated. You can do it using the pip command: pip install selenium-wire
Another recommended package for this integration is webdriver-manager. It's a package that simplifies the management of binary drivers for different browsers. In this case, there's no need to manually download a new version of a web driver after each update.
You can install the webdriver-manager using the pip command as well: pip install webdriver-manager
Proxy authentication
Once you set up everything, you can move on to the next part – proxy authentication. You'll be prompted to specify your account credentials for proxies to work.
USERNAME = "your_username"
PASSWORD = "your_password"
ENDPOINT = "pr.oxylabs.io:7777"

You'll need to adjustyour_username and your_password fields with the username and password of your proxy user.
Testing proxy server connection
If you need to check if the proxy is working, you should visit ip.oxylabs.io. If everything is working correctly, it will return an IP address of a proxy that you're using.
Checking if a proxy is working:
try:
    driver.get("https://ip.oxylabs.io/")
    return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
finally:
    driver.quit()
Full code for Oxylabs' Residential Proxies integration with Selenium:
from selenium.webdriver.common.by import By
from seleniumwire import webdriver
# A package to have a chromedriver always up-to-date.
from webdriver_manager.chrome import ChromeDriverManager
​
USERNAME = "your_username"
PASSWORD = "your_password"
ENDPOINT = "pr.oxylabs.io:7777"
​
def chrome_proxy(user: str, password: str, endpoint: str) -> dict:
    wire_options = {
        "proxy": {
            "http": f"http://{user}:{password}@{endpoint}",
            "https": f"http://{user}:{password}@{endpoint}",
        }
    }
​
    return wire_options
​
def execute_driver():
    options = webdriver.ChromeOptions()
    options.headless = True
    proxies = chrome_proxy(USERNAME, PASSWORD, ENDPOINT)
    driver = webdriver.Chrome(
        ChromeDriverManager().install(), options=options, seleniumwire_options=proxies
    )
    try:
        driver.get("https://ip.oxylabs.io/")
        return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
    finally:
        driver.quit()
​
if __name__ == "__main__":
    print(execute_driver())
Incogniton
Next - PROXIES
Next-Gen Residential Proxies
Last modified 2mo ago
Copy link