Multi-GPU Worker Scaling
For systems with multiple GPUs, OpenTranscribe supports parallel GPU workers to dramatically increase transcription throughput.
Overview
Standard Setup: 1 GPU = 1 worker, about 39x realtime end to end (measured, RTX A6000, large-v3-turbo)
Scaled Setup: 1 GPU = several workers in one container, transcribing several files simultaneously. Total throughput rises with the worker count but sub-linearly (about 51x at 4 workers in the same measurement).
When to Use
Multi-GPU scaling is ideal for:
- Batch processing large numbers of files
- High-throughput production systems
- Systems with dedicated transcription GPU
- Workflows with concurrent uploads
--gpu-scale vs. --with-gpu-split
These are two distinct multi-GPU features with different goals, enabled independently:
--gpu-scale (this page) | --with-gpu-split | |
|---|---|---|
| What it does | Runs N parallel Celery workers in one container against a dedicated GPU, for higher throughput (more files transcribed concurrently) | Runs transcription and diarization on separate GPUs for higher per-file performance |
| Enabled by | The --gpu-scale CLI flag on ./opentr.sh start dev / start prod -- not GPU_SCALE_ENABLED (see Step 1 below) | The --with-gpu-split CLI flag on ./opentr.sh (dev), or ENGINE_GPU_SPLIT=true in .env alone on the shipped ./opentranscribe.sh (no --with-* flags there) |
| Compose overlay | docker-compose.gpu-scale.yml (COMPOSE_PROFILES=gpu-scale) | docker-compose.gpu-split.yml (gpu-split profile) |
| Tuning | GPU_SCALE_WORKERS, GPU_SCALE_DEVICE_ID, GPU_SCALE_DEFAULT_WORKER | GPU_TRANSCRIBE_DEVICE_ID, GPU_DIARIZE_DEVICE_ID |
This page covers --gpu-scale only. For --with-gpu-split, see
Deployment Configuration.
The two flags combine on a 3+ GPU host if you want both behaviors at once.
Hardware Example
GPU 0: NVIDIA RTX A6000 (49GB) - Local LLM (vLLM/Ollama)
GPU 1: RTX 3080 Ti (12GB) - Default worker (disabled when scaling)
GPU 2: NVIDIA RTX A6000 (49GB) - 4 parallel workers (scaled)
Configuration
Step 1: Configure Environment
GPU_SCALE_ENABLED does not enable scalingScaling is turned on only by the --gpu-scale CLI flag in Step 2 -- no compose file or
startup script reads GPU_SCALE_ENABLED. It's consulted in exactly one unrelated place
(tasks/utility.py), to pick which GPU device IDs the system-stats task queries, so a stale
value misreports which GPU is in use without changing any scheduling. Don't set it expecting it
to turn scaling on or off.
Edit .env to configure which GPU the scaled workers use and how many run:
# Which GPU to use for scaled workers
GPU_SCALE_DEVICE_ID=2
# Number of parallel workers. If left unset it follows DIAR_NATIVE_MAX_INFLIGHT (default 2),
# so raise the sidecar's permits together with this value (see docker-compose.gpu-scale.yml)
GPU_SCALE_WORKERS=4
# Keep the default single-GPU worker running alongside the scaled workers (1) or disable it (0)
# (.env.example ships 1; the compose default when unset is 0)
GPU_SCALE_DEFAULT_WORKER=1
Step 2: Start with Scaling
# Development
./opentr.sh start dev --gpu-scale
# Production
./opentr.sh start prod --gpu-scale
# Reset with scaling
./opentr.sh reset dev --gpu-scale
Performance
| Workers | Measured throughput |
|---|---|
| 1 worker | 39.3x realtime |
| 2 workers | 47.4x realtime |
| 4 workers | 51.3x realtime |
Figures are full-pipeline throughput (transcription, speaker detection, speaker assignment) on one RTX A6000 with large-v3-turbo, from docs/BENCHMARK_RESULTS.md; a 4-worker run is not 4x faster. Adding workers mainly improves GPU utilisation; per-file time rises as workers compete.
VRAM Requirements
| Workers | Recommended VRAM | Supported Models |
|---|---|---|
| 2 | 12GB+ | large-v2 |
| 4 | 24GB+ | large-v2 |
| 6 | 48GB+ | large-v2 |
Monitoring
# Watch GPU usage
watch -n 1 nvidia-smi
# View scaled worker logs
./opentr.sh logs celery-worker-gpu-scaled
# Monitor task queue
# Open: http://localhost:5175/flower
Troubleshooting
Out of Memory Errors
Reduce worker count:
GPU_SCALE_WORKERS=2 # instead of 4
Poor GPU Utilization
Increase worker count (if VRAM available):
GPU_SCALE_WORKERS=6 # instead of 4
Next Steps
- GPU Setup
- Hardware Requirements
- Environment Variables
- Deployment Configuration -- GPU split mode and all other deployment types