YouTube Downloader

The youtube_download source delivers the audio/video content of a YouTube video of your choice. Please note that due to the potentially large data volume this source is only available via the asynchronous Push-Pull integration and the Cloud Storage feature.

For a free trial of the YouTube Downloader, please reach out to our sales team.

How YouTube Downloader works

  1. Submit via API:

    • One or more YouTube video IDs;

    • Cloud storage details.

  2. Get the API response with the confirmation that your request to download a video has been successfully accepted.

  3. Get the videos in the specified cloud storage location as soon as they're processed by our system.

Input parameter values

Parameter
Description
Default value

source

Sets the scraper.

youtube_download

query

YouTube video ID.

-

storage_type

Storage type of the cloud service (gcs, s3 or s3_compatible)

-

storage_url

The bucket name (for AWS S3) or the URL (for other S3-compatible storage) of the cloud storage location you would like the result uploaded to.

-

callback_url

Your callback endpoint URL. More info

-

context: download_type

Specifies whether to download audio, video, or both - audio_video.

audio_video

context: video_quality

Sets video quality: best, worst, or a resolution - 480, 720, 1080, 1440, 2160(4K).

720

Submitting a single job

Endpoint

POST https://data.oxylabs.io/v1/queries

Input

Provide the job parameters in a JSON payload as shown in the example below:

curl --user "user:pass1" \
'https://data.oxylabs.io/v1/queries' \
-H "Content-Type: application/json" \
-d '{
    "source": "youtube_download",
    "query": "9cQBNYsCqQs",
    "storage_type": "s3",
    "storage_url": "s3://your-s3-bucket/your-folder/"
}'

Batch job

You can submit up to 5,000 video IDs within a single batch request.

Endpoint

POST https://data.oxylabs.io/v1/queries/batch

Input

curl --user "user:pass1" \
'https://data.oxylabs.io/v1/queries/batch' \
-H 'Content-Type: application/json' \
-d '{
    "source": "youtube_download",
    "query": ["9cQBNYsCqQs", "KGOsWPF4Wfs"],
        "context": [
        {
            "key": "download_type",
            "value": "video"
        },
        {
            "key": "video_quality",
            "value": "1080"
        }
    ],
    "storage_type": "s3",
    "storage_url": "your-s3-bucket/your-folder/"
}'

The API will respond with a JSON object, containing the job information for each job created. The job status will be pending until the download is completed.

Checking job status

If you provided a valid callback URL when submitting your job, we will notify you upon completion by sending a JSON payload to the specified callback URL.

However, if you submitted a job without using callback service, you can check the job status manually. Retrieve the URL from the href field in the rel:self section of the response message received after job submission. The URL for checking the job status will resemble the following: http://data.oxylabs.io/v1/queries/12345678900987654321. Querying this URL will return the job information, including its current status.

Endpoint

GET https://data.oxylabs.io/v1/queries/{id}

Cloud storage

You can get your YouTube videos delivered to Amazon S3 or other S3-compatible cloud storage.

Content files will appear in your cloud storage location and will be named according to the following naming schema: {video_id}_{job_id}.mp4 (for video) or {video_id}_{job_id}.m4a (for audio).

Storage parameters

Parameter
Description
Valid values

storage_type

Your cloud storage type.

s3 (AWS S3); s3_compatible (other S3-compatible storage solutions).

storage_url

Your cloud storage URL

Any AWS S3 bucket name. Any S3-compatible storage URL.

Amazon S3 setup

To get your job results uploaded to your Amazon S3 bucket, please set up access permissions for our service. To do that, go to https://s3.console.aws.amazon.com/ > S3 > Storage > Bucket Name (if don't have one, create a new one) > Permissions > Bucket Policy

Use the bucket policy below (don't forget to change the bucket name under YOUR_BUCKET_NAME):

{
    "Version": "2012-10-17",
    "Id": "Policy1577442634787",
    "Statement": [
        {
            "Sid": "Stmt1577442633719",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::324311890426:user/oxylabs.s3.uploader"
            },
            "Action": "s3:GetBucketLocation",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
        },
        {
            "Sid": "Stmt1577442633719",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::324311890426:user/oxylabs.s3.uploader"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*"
        }
    ]
}

This policy allows us to write to your bucket, give access to uploaded files to you, and know the location of the bucket.

S3-compatible storage setup

If you'd like to get your results delivered to an S3-compatible storage location, you will have to include your bucket's ACCESS_KEY:SECRET auth string in the storage_url value in the payload:

{
    "source": "youtube_transcript",
    "query": "jyOeeCtPu64",
    "storage_type": "s3_compatible",
    "storage_url": "https://ACCESS_KEY:SECRET@s3.oxylabs.io/my-videos"
}

Google Cloud Storage setup

CommentGive feedback on the editorTo get your job results uploaded to your Google Cloud Storage bucket, please set up special permissions for our service.

To do that, create a custom role with the storage.objects.create permission, and assign it to the Oxylabs service account email oxyserps-storage@oxyserps-storage.iam.gserviceaccount.com.

Last updated

Was this helpful?