The Question
Design
Scalable Strategy Backtesting Platform
Design an internal platform for quantitative researchers to perform large-scale historical backtesting. The system must allow users to submit custom Python-based strategy code, execute these jobs in parallel across a scalable compute fleet, and ensure strict resource isolation and security for untrusted code execution. Consider how to efficiently manage and serve terabytes of historical market data while providing comprehensive performance reporting and job lifecycle tracking.
S3
Parquet
Docker
K8s Jobs
PostgreSQL
SQS
Python
gVisor
March 7, 2026