The Question
Design

Scalable Video Sharing Platform

Design a high-scale video sharing platform similar to YouTube. The system must support millions of daily uploads, billions of daily views, and provide a seamless, low-latency playback experience globally. Key requirements include handling massive video files, asynchronous video processing for various devices/resolutions, and managing highly concurrent metadata reads and view count updates. Discuss the trade-offs between storage costs, processing latency, and delivery performance.
S3
CDN
Kafka
Cassandra
Redis
Elasticsearch
FFmpeg
HLS
DASH
DynamoDB
Kubernetes
gRPC
Questions & Insights

Clarifying Questions

What is the scale of the system? (Assumption: 2 Billion Monthly Active Users, 500 million Daily Active Users (DAU), and 5 million video uploads per day.)
What are the primary video constraints? (Assumption: Support up to 4K resolution, maximum video length of 120 minutes, and various formats like MP4, MOV.)
Is global low-latency playback a priority? (Assumption: Yes, we must use a CDN-heavy architecture to minimize buffering globally.)
What are the social features required for the MVP? (Assumption: Basic metadata, view counts, and search. Comments and likes are secondary but should be architected for.)
What is the consistency model for view counts? (Assumption: Eventual consistency is acceptable to prioritize high availability and low latency.)

Thinking Process

Core Bottleneck: Video files are massive; uploading and streaming them directly from a single server is impossible at scale.
Key Strategy: Decouple the Upload Path (Write-heavy, async transcoding) from the Viewing Path (Read-heavy, CDN-cached).
Progressive Logic:
How do we handle massive uploads without crashing the web server? (Use Presigned URLs for direct-to-S3 uploads).
How do we ensure the video plays on all devices (iPhone, Web, Android)? (Asynchronous Transcoding pipeline).
How do we prevent global buffering? (Edge Caching via CDN).
How do we scale metadata and view counts? (NoSQL storage for horizontal scaling).

Bonus Points

DASH/HLS Adaptive Bitrate Streaming: Implementing adaptive streaming to dynamically adjust video quality based on the user's network conditions.
Cost-Optimized Storage Tiering: Moving older, rarely-watched videos to "Cold Storage" (e.g., S3 Glacier) while keeping viral videos in SSD-backed caches.
Video Deduplication: Generating content-based hashes (perceptual hashing) to avoid storing and transcoding the same viral video uploaded by different users.
Quorum-based View Count Aggregation: Using a distributed counter with local buffering to handle millions of concurrent increments without locking the database.
Design Breakdown

Functional Requirements

Core Use Cases:
Users can upload videos.
Users can view videos with minimal buffering.
Users can search for videos by title.
Users can view video metadata (title, views, description).
Scope Control:
In-scope: Upload, Transcoding, Streaming, Metadata, Search.
Out-of-scope: Live streaming, Recommendations engine (ML), Social networking (Comments/Likes) beyond basic storage.

Non-Functional Requirements

Scale: Support 5M uploads/day and 5B views/day.
Latency: Under 200ms for metadata; "Instant start" for video playback (< 2 seconds to first frame).
Availability & Reliability: 99.99% availability; No data loss for uploaded original videos.
Consistency: Eventual consistency for view counts and search indexing.
Security & Privacy: Support for private videos and secure upload via signed URLs.

Estimation

Traffic Estimation:
Uploads: 5M videos / 86400s \approx 60 uploads/sec.
Views: 5B views / 86400s \approx 60,000 QPS (Average). Peak QPS \approx 120,000+.
Storage Estimation:
5M videos * 200MB (average compressed size) = 1 PB per day.
365 PB per year.
Bandwidth Estimation:
Outgoing: 60,000 views/sec * 5 Mbps (average bitrate) = 300 Gbps egress.

Blueprint

Concise Summary: A microservices-based architecture that separates video ingestion from delivery. It leverages cloud object storage and an asynchronous transcoding pipeline to transform raw uploads into streamable formats (HLS/DASH).
Major Components:
API Gateway: Handles authentication, rate limiting, and request routing.
Upload Service: Manages video metadata and issues presigned URLs for direct-to-cloud uploads.
Object Storage (S3): Stores raw and processed video files.
Transcoding Pipeline: An async worker pool that converts videos into multiple resolutions and chunks.
Metadata Store: A NoSQL database for fast retrieval of video info.
CDN: Distributes video chunks to edge locations for low-latency playback.
Simplicity Audit: This design avoids complex distributed file systems by using industry-standard Object Storage and CDNs, which are the most cost-effective and reliable for an MVP.
Architecture Decision Rationale:
Why this architecture?: Separating the upload and viewing paths prevents resource-intensive transcoding from affecting the playback experience.
Functional Satisfaction: Covers end-to-end from "Upload" to "Watch".
Non-functional Satisfaction: CDN ensures global scale; NoSQL ensures metadata scalability; S3 ensures 11 nines of durability.

High Level Architecture

Sub-system Deep Dive

Edge (Optional)

Content Delivery & Traffic Routing:
CDN: Mandatory for video content. We use Multi-CDN (e.g., Cloudflare, Akamai) to avoid vendor lock-in and optimize for regional performance. Static assets and video segments (.ts or .m4s files) are cached.
DNS: Latency-based routing to direct users to the nearest API Gateway or CDN POP.
Security & Perimeter:
API Gateway: Performs JWT validation and Rate Limiting (e.g., 10 uploads/hour/user) to prevent DoS.

Service

Topology & Scaling:
Stateless microservices deployed on Kubernetes (EKS/GKE).
Scaling based on Horizontal Pod Autoscaler (HPA) using CPU and Request count.
API Schema Design:
POST /v1/video/upload-init: Returns a Presigned URL and VideoID.
GET /v1/video/{id}/metadata: Returns title, view count, and manifest URL (m3u8).
Protocol: REST/JSON for metadata; HLS/DASH for streaming.
Resilience:
Transcoding: If a worker fails, the message remains in the MQ (Visibility Timeout) and is retried by another worker.

Storage

Access Pattern:
Metadata: Heavy read (99%), Low write (1%).
View Counts: Extremely heavy write/increment.
Database Table Design:
Videos Table: video_id (PK), user_id, title, description, status (Uploading/Processing/Ready), s3_path.
ViewCounts Table: video_id (PK), count.
Technical Selection:
Metadata: Cassandra or DynamoDB for horizontal scaling of millions of video records.
Search: Elasticsearch for fuzzy title search.
Videos: S3 for blob storage.
Distribution Logic: Shard by video_id.

Cache

Purpose: Reduce Metadata DB load for viral videos.
Key-Value Schema: video_meta:{video_id} -> JSON Blob.
TTL: 1 hour, with dynamic TTL for trending videos.
Technical Selection: Redis for sub-millisecond metadata lookups.

Messaging

Purpose: Decouple video upload from the long-running transcoding process.
Event Schema: video_id, raw_s3_location, user_id.
Failure Handling: Dead-letter queue (DLQ) for videos that fail transcoding after 3 retries (e.g., corrupt files).
Technical Selection: Kafka or AWS SQS for high durability and replayability.

Data Processing

Processing Model: Async Pipeline.
Processing DAG:
Inspection: Verify file integrity and metadata.
Transcoding: Parallel jobs for 1080p, 720p, 480p.
Segmentation: Split videos into 4-second chunks for HLS.
Thumbnail Generation: Extract frames at specific intervals.
Technical Selection: FFmpeg running on Spot Instances (cost-saving) controlled by a worker service.

Infrastructure (Optional)

Observability: Prometheus for metrics (Transcoding latency, 5xx rates); ELK for logs.
Platform Security: S3 buckets are private; CDN access via Signed Cookies/CloudFront Key Pairs.
Wrap Up

Advanced Topics

Trade-offs: We choose Availability (AP) over Consistency for view counts. A user might see 1.1M views while another sees 1.11M; this is acceptable.
Reliability: Original videos are stored with Cross-Region Replication (CRR) in S3 to survive a regional cloud failure.
Bottleneck Analysis:
Hot Shards: Viral videos create "hot" keys in Redis/DB. Mitigation: Cache-aside with high TTL and potentially local in-memory caching on the Metadata Service.
Transcoding Backlog: A surge in uploads can lag the pipeline. Mitigation: Auto-scaling workers based on MQ depth.
Distinguishing Insights:
Adaptive Bitrate Streaming (ABR): The system generates a Master Playlist (.m3u8) that lists different resolutions. The client-side player (like Video.js) automatically switches segments based on current download speed.
Pre-signed URLs: By having the client upload directly to S3, we remove the "Middleman" bottleneck, allowing the Upload Service to remain lightweight and handle millions of concurrent negotiations.