The Home Depot
Learn how to start scraping The Home Depot.
Home Depot has no dedicated source. You scrape its pages with the universal source by passing the page url. Product pages have a dedicated parser: add "parse": true to receive structured JSON instead of the page HTML.
Search
Request samples
curl 'https://realtime.oxylabs.io/v1/queries' \
--user 'USERNAME:PASSWORD' \
-H 'Content-Type: application/json' \
-d '{
"source": "universal",
"url": "https://www.homedepot.com/s/drill",
"render": "html"
}'import requests
from pprint import pprint
# Structure payload.
payload = {
'source': 'universal',
'url': 'https://www.homedepot.com/s/drill',
'render': 'html'
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('USERNAME', 'PASSWORD'),
json=payload
)
# Instead of response with job status and results url, this will return the
# JSON response with the result.
pprint(response.json())Request parameter values
source
Sets the scraper. Home Depot pages are scraped with the universal source.
universal
url
The URL of the search page, e.g. https://www.homedepot.com/s/drill.
-
render
Enables JavaScript rendering when set to html. Without rendering, most Home Depot search requests fail.
-
- mandatory parameter
Product
Request samples
Request parameter values
source
Sets the scraper. Home Depot pages are scraped with the universal source.
universal
url
The URL of the product page. The short form with the product ID only, e.g. https://www.homedepot.com/p/100650378, works too.
-
render
Enables JavaScript rendering when set to html. Without rendering, Home Depot returns the product page without its delivery details.
-
- mandatory parameter
Response example
A parsed product page for product ID 100650378 (lists shortened to one item, long values cut):
Output data dictionary
The table below presents a detailed list of each product page element we parse, along with its description and data type. The table also includes some metadata.
url
The URL of the product page.
string
product_id
The Home Depot product ID, the 9-digit number at the end of the product URL.
string
parent_id
The ID of the parent product the item belongs to.
string
group_id
The ID of the product group.
string
sku
The Home Depot store SKU.
string
model_number
The manufacturer model number.
string
mpn
The manufacturer part number.
string
upc
The UPC barcode number.
string
gtin
The GTIN of the product.
string
upcgtin13
The 13-digit GTIN form of the UPC.
string
product_name
The name of the product.
string
manufacturer
The brand or manufacturer of the product.
string
description
The product description.
string
features
List of the product's feature bullet points.
array
details
List of product specifications as name-value pairs.
array
details.name
The name of the specification.
string
details.value
The value of the specification.
string
manufacturer_description
Object repeating the manufacturer's details and features lists.
object
price
The price as shown on the page, with the currency symbol, e.g. $199.00.
string
final_price
The price as a number.
float
currency
The ISO 4217 three-letter code of the currency.
string
availability
The availability status, e.g. in_stock.
string
in_stock
Indicates whether the product is in stock.
boolean
discontinued
Indicates whether the product is discontinued.
boolean
add_to_cart
Indicates whether the product can be added to the cart.
boolean
delivery
Indicates whether delivery is offered. Accurate only when the page is rendered ("render": "html"); without rendering it comes back as false.
boolean
ship_to_store
Indicates whether shipping to a store is offered. Accurate only when the page is rendered ("render": "html"); without rendering it comes back as false.
boolean
shipping_cost (optional)
List of shipping costs, one object for each fulfillment method the product offers: ship_to_store, delivery or both. Returned when the page is rendered ("render": "html").
array
shipping_cost.ship_to_store
Ship-to-store pickup cost: shipping_price (number, 0 when free) and currency.
object
shipping_cost.delivery
Home delivery cost: shipping_price (number, 0 when free) and currency.
object
store_name
The name of the store the page was served for, e.g. Hutchinson.
string
zipcode (optional)
The ZIP code of that store, when shown.
string
store_country
The country of the store.
string
country_code
The country code of the site.
string
target_countries
List of countries the product is sold to.
array
rating
The average rating of the product.
float
star_rating
The rating rounded to whole stars.
integer
review_count
The number of reviews.
integer
reviews_count
The number of reviews (same value as review_count).
integer
reviews
List of reviews shown on the page.
array
reviews.review_id
The ID of the review.
string
reviews.title
The title of the review.
string
reviews.content
The text of the review.
string
reviews.rating
The rating given in the review.
integer
reviews.min_rating
The lowest possible rating.
integer
reviews.max_rating
The highest possible rating.
integer
top_reviews
List of the top reviews shown on the page.
array
top_reviews.name
The reviewer name.
string
top_reviews.title
The title of the review.
string
top_reviews.description
The text of the review.
string
top_reviews.rating
The rating given in the review.
integer
answered_questions
List of customer questions with answers.
array
answered_questions.question
The question.
string
answered_questions.answer
The answer.
string
answered_questions_count
The number of answered questions.
integer
main_image
The URL of the main product image.
string
image_urls
List of URLs of the product images.
array
additional_image_urls
List of URLs of customer photos.
array
video_count
The number of product videos.
integer
category
The product's own category.
object
category.name
The name of the category.
string
category.url
The URL of the category.
string
category_tree
List of the categories from the top level down to the product's category.
array
category_tree.name
The name of the category.
string
category_tree.url
The URL of the category.
string
category_urls
List of the category URLs in the category path.
array
root_category
The top-level category.
object
root_category.name
The name of the top-level category.
string
root_category.url
The URL of the top-level category.
string
product_category
The full category path, separated by >.
string
listing_has_variations
Indicates whether the product has variations.
boolean
warranty
Object with warranty details.
object
warranty.available
Indicates whether a warranty is offered.
boolean
warranty.type
The type of warranty.
string
warranty.length
The length of the warranty.
string
related_documents
List of documents for the product, e.g. manuals or guides.
array
related_documents.name
The name of the document.
string
related_documents.url
The URL of the document.
string
return_window
The return window in days.
integer
return_policy
The URL of the return policy.
string
seller_url
The URL of the seller.
string
seller_tos
The URL of the seller's terms of use.
string
seller_privacy_policy
The URL of the seller's privacy policy.
string
domain
The site the page was scraped from.
string
parse_status_code
The status code of the parsing job. You can see the parser status codes described here.
integer
created_at
The timestamp when the scraping job was created.
timestamp
updated_at
The timestamp when the scraping job was finished.
timestamp
page
The page number in the pagination of results.
integer
status_code
The HTTP status code of the scraped page.
integer
is_render_forced
Identifies whether rendering has been forced for this request.
boolean
parser_type
The type of parser used to extract the data, homedepot_product.
string
job_id
The ID of the job associated with the scraping job.
string
Category
Request samples
Request parameter values
source
Sets the scraper. Home Depot pages are scraped with the universal source.
universal
url
The URL of the category page, e.g. https://www.homedepot.com/b/Tools-Power-Tools-Drills/N-5yc1vZc27f.
-
- mandatory parameter
Localization and Filtering
context: store_id
Product pages
Home Depot store number as a 4-digit string. Find the store in the Home Depot store directory; its number is the last part of the store page URL, e.g. homedepot.com/l/21st-South/UT/Salt-Lake-City/84115/4403. Prefix 3-digit numbers with a zero, e.g. 121 → "0121". The parsed store_name shows the store the page was served for. Do not combine with render.
-
Last updated
Was this helpful?