The Question
Design
Distributed Cron Job Scheduler
Design a highly available and scalable distributed cron job scheduler capable of managing 100 million scheduled tasks. The system must support standard cron expressions for periodic execution and ensure at-least-once delivery with a peak throughput of 50,000 executions per second. Focus on the decoupling of job discovery and job execution, handling massive bursts of tasks scheduled at the same time (e.g., top of the hour), and providing a reliable mechanism to recover from system downtime without missing task windows.
PostgreSQL
Kafka
REST
JSONB
HPA
JWT
Exponential Backoff
Distributed Locking
March 16, 2026