List of instructions

Explore available browsing instructions to automate dynamic browsing with Web Scraper API.

General arguments

All the instructions defined below have a consistent set of arguments. The arguments are as follows.

type

• Type: Enum["click", "input", "scroll", "scroll_to_bottom", "wait", "wait_for_element", "fetch_resource"]

• Description: Browser instruction type.

timeout_s

• Type: int

• Description: How long until action is skipped if not completed in time.

• Restrictions: 0 < timeout_s <= 60

• Default value: 5

wait_time_s

• Type: int

• Description: How long to wait before executing next action.

• Restrictions: 0 < wait_time_s <= 60

• Default value: 0

on_error

• Type: Enum["error", "skip"]

• Description: Indicator what to do with instructions in case this instruction fails:

  • "error": Stops the execution of browser instructions.

  • "skip": Continues with the next instruction.

• Default value: "error"

Example with general arguments

{
    "type": "wait_for_element",
    "selector": {
        "type": "text",
        "value": "Load More Items"
    },
    "timeout_s": 5,
    "wait_time_s": 2,
    "on_error": "skip"

}

Instructions

click

• Description: Clicks an element and wait a set count of seconds.

• Args:

  • type: str = "click"

  • selector: dict

    • type: Enum["xpath", "css", "text"]

    • value: str

• Example:

{
    "type": "click",
    "selector": {
        "type": "xpath",
        "value": "//button"
    }
}

input

• Description: Enters a text into a selected element.

• Args:

  • type: str = "input"

  • selector: dict

    • type: Enum["xpath", "css", "text"]

    • value: str

  • value: str

  • Example:

{
    "type": "input",
    "selector": {
        "type": "xpath",
        "value": "//input"
    },
    "value": "pizza boxes"
}

scroll

• Description: Scrolls a set count of pixels.

• Args:

  • type: str = "scroll"

  • x: int

  • y: int

• Example:

{
    "type": "scroll",
    "x": 0,
    "y": 100
}

scroll_to_bottom

• Description: Scrolls to bottom for a set count of seconds.

• Args:

  • type: str = "scroll_to_bottom"

• Example:

{
    "type": "scroll_to_bottom",
    "timeout_s": 10
}

wait

• Description: Waits a set count of seconds.

• Args:

  • type: str = "wait"

• Example:

{
    "type": "wait",
    "wait_time_s": 2
}

wait_for_element

• Description: Waits for element to load for a set count of seconds.

• Args:

  • type: str = "wait_for_element"

  • selector: dict

    • type: Enum["xpath", "css", "text"]

    • value: str

  • Example:

{
    "type": "wait_for_element",
    "selector": {
        "type": "text",
        "value": "Load More Items"
    },
    "timeout_s": 5
}

fetch_resource

The fetch_resource instruction must be the final instruction in the browser instructions list; any subsequent instructions will not be executed.

• Description: Fetches the first occurrence of a Fetch/XHR resource matching the set pattern.

• Args:

  • type: str = "fetch_resource"

  • filter: str(RegEx expression)

  • on_error: Enum["error", "skip"]

• Example:

{
    "type": "fetch_resource",
    "filter": "/graphql/item/"
}

Instructions' validation

Any inconsistency in regards to instruction format will result in a 400 status code and a corresponding error message.

For example, payload as such:

{
    "source": "universal",
    "url": "https://www.example.com/",
    "render": "html",
    "browser_instructions": [
        {
            "type": "unsupported-wait",
            "wait_time_s": 5
        }
    ]
}

Will result in:

{    
    "errors": {
        "message": "Unsupported action type `unsupported-wait`, choose from 'click,fetch_resource,input,scroll,scroll_to_bottom,wait,wait_for_element'"
    }
}