Batch Processing

Process multiple lipsync generations efficiently

The Batch API allows you to process multiple lipsync generation requests asynchronously in a single operation. This is ideal for bulk video processing, personalized video campaigns, content localization, or any scenario requiring high-volume generation.

Overview

Batch processing enables you to submit upto 500 generations without having to handle queueing/concurrency, with a turn-around-time of 24 hrs. As of now, only generations with inputs upto 30s are supported in batch processing.

Batch API is available for Scale and Enterprise users only

Getting Started

1

Prepare Your Input File

Create a JSON Lines (.jsonl) file with your generation requests. Each line should contain a unique request_id, the endpoint (must be "/v2/generate"), and a payload with the standard generation request format (same as the Generate API). The file must be in JSON Lines (.jsonl) format with a minimum of 20 records, maximum file size of 5MB, and up to 500 requests per batch.

input.jsonl
1{"request_id": "request-1", "endpoint": "/v2/generate", "payload": {"model": "lipsync-2", "input": [{"type": "video", "url": "https://assets.sync.so/docs/example-video.mp4"}, {"type": "audio", "url": "https://assets.sync.so/docs/example-audio.wav"}]}}
2{"request_id": "request-2", "endpoint": "/v2/generate", "payload": {"model": "lipsync-2", "input": [{"type": "video", "url": "https://assets.sync.so/docs/example-video.mp4"}, {"type": "audio", "url": "https://assets.sync.so/docs/example-audio.wav"}]}}
2

Create a Batch

1from sync import Sync
2
3sync = Sync()
4
5batch = sync.batch.create(
6 input=open("input.jsonl", "rb")
7)
8
9print(f"Batch created with ID: {batch.id}")

Optional parameters:

  • webhook_url: Receive notifications when the batch completes
  • dry_run: Set to true to validate your input file without processing
3

Check Batch Status

Monitor your batch progress by polling the status:

1batch = sync.batch.get(batch_id)
2print(f"Status: {batch.status}")
3print(f"Progress: {batch.metrics}")

A batch can have one of the following status:

  1. PENDING: Batch created, waiting to start processing
  2. PROCESSING: Generations are being processed
  3. COMPLETED: All generations finished (successfully or with failures)
  4. FAILED: Batch processing failed entirely
4

Check Batch Results

When a batch completes, results are available as a JSON Lines file at the outputUrl of the get batch response. Each line contains:

output.jsonl
1{"request_id": "request-1", "endpoint": "/v2/generate", "payload": {...}, "status": "COMPLETED", "error": null, "response": {...}, "updated_at": "2024-01-15T10:35:00Z"}
2{"request_id": "request-2", "endpoint": "/v2/generate", "payload": {...}, "status": "FAILED", "error": {"code": "INVALID_INPUT", "message": "..."}, "response": null, "updated_at": "2024-01-15T10:35:00Z"}

The response field contains the same data as individual Generate API responses.

For a complete working example, see the batch processing example in our examples repository.

Webhook Notifications

When you provide a webhook_url, you’ll receive POST notifications when your batch completes:

webhook_payload.json
1{
2 "id": "batch_abc123",
3 "createdAt": "2024-01-15T10:30:00Z",
4 "status": "COMPLETED",
5 "webhookUrl": "https://your-webhook-url.com/batch-webhook",
6 "metrics": {
7 "totalGenerations": 5,
8 "successCount": 4,
9 "failedCount": 1,
10 "pendingCount": 0
11 },
12 "options": {},
13 "outputUrl": "https://output.sync.so/batch_abc123_results.jsonl"
14}