AI-Map

Learn how to map and explore domains intelligently to find relevant URLs.

Overview

AI-Map helps you discover relevant pages within a website. It uses AI to explore domains systematically based on your specific criteria and returns a list of URLs that match your criteria.

You can preview the tool here and integrate it into your workflows by our Python/JavaScript SDKs, MCP server, or one of our 3rd-party integrations.

How to use

1. Enter a URL or domain you want to map.

2. Describe your target pages with a natural language prompt.

3. Configure mapping parameters such as source limit, mapping depth, and geo-location.

4. Run the AI mapping task to collect URLs of your target pages from the domain.

Installation

To begin, make sure you have access to an API key (or get a free trial with 1000 credits) and Python 3.10 or above installed. You can install the oxylabs-ai-studio package using pip:

pip install oxylabs-ai-studio

Code example (Python)

The following example shows how to use an AI-Map search agent to perform a simple website mapping task.

from oxylabs_ai_studio.apps.ai_map import AiMap
import json

# Initialize authorization
ai_map = AiMap(api_key="<API_KEY>")

# Define the payload for request
payload = {
    "url": "https://career.oxylabs.io",
    "user_prompt": "job ad pages",
    "return_sources_limit": 10,
    "geo_location": None,
    "render_javascript": False,
}

# Start the mapping
result = ai_map.map(**payload)

# Output the results in JSON
print("URLs:")
print(json.dumps(result.data, indent=2))

Learn more about AI-Crawler and Oxylabs AI Studio Python SDK in our PyPI repository. You can also check out our AI Studio JavaScript SDK guide for JS users.

Request parameters

Parameter	Description	Default Value
url*	Starting URL or domain for mapping	–
user_prompt*	Natural language prompt for pages to find	–
output_format	Max number of sources to return	25
render_javascript	Enable JavaScript rendering for dynamic content	False
geo_location	Proxy location in ISO2 format	–

* – mandatory parameters

Output sample

URLs:
[
  "https://career.oxylabs.io/job/247ac098/head-of-marketing",
  "https://career.oxylabs.io/job/2d9ae321/product-owner",
  "https://career.oxylabs.io/job/496841ed/php-developer-golang",
  "https://career.oxylabs.io/job/736ef009/talent-sourcer",
  "https://career.oxylabs.io/job/7a4a4415/senior-python-engineer-webshare-product",
  "https://career.oxylabs.io/job/88f7340c/devops-squad-lead",
  "https://career.oxylabs.io/job/9946db47/junior-project-manager",
  "https://career.oxylabs.io/job/b2e1397e/account-executive-north-america",
  "https://career.oxylabs.io/job/d72db15b/senior-event-marketing-manager",
  "https://career.oxylabs.io/job/e1efecf0/seo-and-growth-marketing-lead"
]

Practical use cases

• Mapping product categories on e-commerce sites.

• Listing all blog posts from a company’s domain.

• Documenting API endpoints.

• Discovering all documentation pages or help articles.

• Finding job postings on career portals.

• And many more…