Redfin
You can extract data from Redfin.com using Real Estate Scraper API. All you have to do is provide a Redfin.com URL while submitting your job.
To retrieve data from Redfin.com:
- 1.Make a
POSTrequest to thehttps://realtime.oxylabs.io/v1/queriesendpoint. - 2.The JSON payload that you post should contain the
"source": "universal"and"url": "https://some.redfin.url/"parameters.
When submitting a job, you have to send your job parameters inside a JSON payload. See the example below for more details:
Parameter | Description | Value examples |
|---|---|---|
source | universal | |
url | Link to a Redfin.com page. | https://zillow.com/NY/New-York/some_listing_url |
user_agent_type | desktop | |
geo_location | Geo-location of a proxy used to retrieve the data. The complete list of the supported locations can be found here. | United States |
locale | Locale, as expected in the Accept-Language header. | en-US |
render | html |
- required parameter
JSON
cURL
Python
PHP
C#
Golang
Java
Node.js
{
"source":"universal",
"url":"https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158"
}
curl --user "USERNAME:PASSWORD" \
'https://realtime.oxylabs.io/v1/queries' \
-H "Content-Type: application/json" \
-d '{"source": "universal", "url": "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158"}'
import requests
from pprint import pprint
# Structure payload.
payload = {
"source": "universal",
"url": "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158",
#"render": "html", # If page type requires
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('YOUR_USERNAME', 'YOUR_PASSWORD'), #Your credentials go here
json=payload,
)
pprint(response.json())
<?php
$params = array(
'source' => 'universal',
'url' => 'https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158',
//'render' => 'html', // If page type requires
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "YOUR_USERNAME" . ":" . "YOUR_PASSWORD"); //Your credentials go here
$headers = array();
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
echo $result;
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Net.Http.Json;
using System.Threading.Tasks;
namespace OxyApi
{
class Program
{
static async Task Main()
{
const string Username = "YOUR_USERNAME";
const string Password = "YOUR_PASSWORD";
var parameters = new Dictionary<string, string>()
{
{ "source", "universal" },
{ "url", "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158" },
//{ "render": "html" }, // If page type requires
};
var client = new HttpClient();
Uri baseUri = new Uri("https://realtime.oxylabs.io");
client.BaseAddress = baseUri;
var requestMessage = new HttpRequestMessage(HttpMethod.Post, "/v1/queries");
requestMessage.Content = JsonContent.Create(parameters);
var authenticationString = $"{Username}:{Password}";
var base64EncodedAuthenticationString = Convert.ToBase64String(System.Text.ASCIIEncoding.UTF8.GetBytes(authenticationString));
requestMessage.Headers.Add("Authorization", "Basic " + base64EncodedAuthenticationString);
var response = await client.SendAsync(requestMessage);
var contents = await response.Content.ReadAsStringAsync();
Console.WriteLine(contents);
}
}
}
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
func main() {
const Username = "YOUR_USERNAME"
const Password = "YOUR_PASSWORD"
payload := map[string]string{
"source": "universal",
"url": "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158",
//"render": "html", // If page type requires
}
jsonValue, _ := json.Marshal(payload)
client := &http.Client{}
request, _ := http.NewRequest("POST",
"https://realtime.oxylabs.io/v1/queries",
bytes.NewBuffer(jsonValue),
)
request.SetBasicAuth(Username, Password)
response, _ := client.Do(request)
responseText, _ := ioutil.ReadAll(response.Body)
fmt.Println(string(responseText))
}
package org.example;
import okhttp3.*;
import org.json.JSONObject;
public class Main implements Runnable {
private static final String AUTHORIZATION_HEADER = "Authorization";
public static final String USERNAME = "YOUR_USERNAME";
public static final String PASSWORD = "YOUR_PASSWORD";
public void run() {
JSONObject jsonObject = new JSONObject();
jsonObject.put("source", "universal");
jsonObject.put("url", "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158");
//jsonObject.put("render", "html"); // If page type requires
Authenticator authenticator = (route, response) -> {
String credential = Credentials.basic(USERNAME, PASSWORD);
return response
.request()
.newBuilder()
.header(AUTHORIZATION_HEADER, credential)
.build();
};
var client = new OkHttpClient.Builder()
.authenticator(authenticator)
.build();
var mediaType = MediaType.parse("application/json; charset=utf-8");
var body = RequestBody.create(jsonObject.toString(), mediaType);
var request = new Request.Builder()
.url("https://realtime.oxylabs.io/v1/queries")
.post(body)
.build();
try (var response = client.newCall(request).execute()) {
assert response.body() != null;
System.out.println(response.body().string());
} catch (Exception exception) {
System.out.println("Error: " + exception.getMessage());
}
System.exit(0);
}
public static void main(String[] args) {
new Thread(new Main()).start();
}
}
import fetch from 'node-fetch';
const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';
const body = {
'source': 'universal',
'url': 'https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158',
//'render': 'html', // If page type requires
};
const response = await fetch('https://realtime.oxylabs.io/v1/queries', {
method: 'post',
body: JSON.stringify(body),
headers: {
'Content-Type': 'application/json',
'Authorization': 'Basic ' + Buffer.from(`${username}:${password}`).toString('base64'),
}
});
console.log(await response.json());
The example above uses the Realtime integration method. If you would like to use some other integration method in your query (e.g., Push-Pull or Proxy Endpoint), refer to the integration methods section.
Although we do not have a dedicated parser for Redfin, you can write your own parsing instructions with Custom Parser feature and get structured data.
An example of a response that you can expect to get from the Realtime endpoint:
{
"results": [
{
"content": "<html>
REDFIN.COM PAGE CONTENT
</html>"
"created_at": "2019-10-01 00:00:01",
"updated_at": "2019-10-01 00:00:15",
"id": null,
"page": 1,
"url": "https://www.redfin.com/NY/New-York/464-W-44th-St-10036/home/180326158",
"job_id": "12345678900987654321",
"status_code": 200
}
]
}
Last modified 3mo ago