The Question
Design

Real-Time Community Chat Platform

Design a real-time messaging system similar to Discord, optimized for 100,000 Daily Active Users (DAU). The system must support persistent servers and channels, instant message delivery, and real-time user presence status. Focus on the core architectural trade-offs between stateful WebSocket management and stateless API services, while ensuring the message storage layer can handle high-frequency writes and time-ordered retrieval. Address how the system handles 'fan-out' when a message is sent to a channel with many active participants.
WebSocket
Redis
PostgreSQL
Cassandra
Pub/Sub
NoSQL
JWT
CDN
ScyllaDB
Questions & Insights

Clarifying Questions

What is the peak concurrency (PCU) during the day?
Assumption: For 100k DAU, we expect a peak of roughly 10% to 20%, which is 10k–20k concurrent users.
Is Voice/Video chat part of the MVP?
Assumption: No. We will focus on the core "Discord experience": Real-time text messaging, Server/Channel organization, and Presence (online/offline status).
What is the typical message volume per user?
Assumption: Average 20 messages per DAU, totaling 2 million messages per day.
Do we need to support massive servers (e.g., 500k members)?
Assumption: For a 100k DAU system, servers will likely be smaller (hundreds to low thousands). We will design for scalability but prioritize the MVP experience for typical communities.

Thinking Process

Core Bottleneck: The primary challenge is real-time message fan-out and presence synchronization. When one user sends a message, it must reach all active members of that channel instantly.
Step 1: How do we maintain persistent connections for real-time delivery? (WebSocket Gateway).
Step 2: How do we efficiently route messages from one user to many? (Pub/Sub mechanism).
Step 3: How do we store and retrieve massive amounts of small, time-ordered chat messages? (LSM-tree based NoSQL).
Step 4: How do we handle "Presence" without overwhelming the system with status updates? (Presence Service with localized caching).

Bonus Points

Presence Optimization: Implementing a "pull-based" presence or "lazy loading" for large servers to prevent the "Presence Storm" where a single user's status change triggers thousands of WebSocket writes.
Consistent Hashing for Gateways: Using consistent hashing to map users to specific WebSocket nodes, allowing for graceful restarts and session affinity.
Read-After-Write Consistency: Ensuring that when a user sends a message and refreshes, they see their own message immediately by using strong consistency for metadata but optimized eventual consistency for history.
Design Breakdown

Functional Requirements

Core Use Cases:
Users can create/join Servers and Channels.
Users can send and receive real-time text messages.
Users can see the online/offline status (Presence) of friends/server members.
Users can view message history in channels.
Scope Control:
In-Scope: Text chat, Server/Channel structure, Presence, Basic Auth.
Out-of-Scope: Voice/Video, File attachments (beyond basic links), Search, Rich Embeds, Roles/Permissions (keep it simple for MVP).

Non-Functional Requirements

Scale: Support 100k DAU and 20k peak concurrent connections.
Latency: End-to-end message delivery < 200ms.
Availability & Reliability: 99.9% availability; messages must not be lost once acknowledged by the server.
Consistency: High consistency for server/channel memberships; eventual consistency for presence and old message history.
Security: TLS for all traffic; standard JWT-based authentication.

Estimation

Traffic Estimation:
Write QPS: 2M messages / 86400s ≈ 23 msgs/sec (Average). Peak (5x) ≈ 115 msgs/sec.
Read QPS (History/Polling): ~10x writes ≈ 1.1k QPS.
WebSocket Events (Fan-out): If 100 users per channel, 115 msgs/sec * 100 = 11.5k outgoing events/sec.
Storage Estimation:
2M messages/day * 200 bytes/msg ≈ 400 MB/day.
146 GB per year. This fits easily on a single modern SSD, but we use NoSQL for horizontal scaling.
Bandwidth Estimation:
Incoming: 115 msgs/sec * 200 bytes ≈ 23 KB/s.
Outgoing: 11.5k events/sec * 200 bytes ≈ 2.3 MB/s.

Blueprint

Concise Summary: A WebSocket-based real-time architecture utilizing a Pub/Sub layer for message distribution and a NoSQL database for high-throughput message storage.
Major Components:
API Gateway: Handles authentication, rate limiting, and routes RESTful requests (Server/Channel management).
WebSocket Gateway: Maintains stateful connections with clients for real-time bidirectional communication.
Chat Service: Orchestrates message persistence and triggers the fan-out process.
Presence Service: Tracks user heartbeats and broadcasts status changes.
Redis (Pub/Sub & Cache): Acts as the glue for real-time message routing and stores ephemeral presence state.
NoSQL Database: Stores the primary chat history.
Simplicity Audit: This architecture avoids complex microservices like "Service Meshes" or "Stream Processing Engines" (Flink/Spark) which are not needed for 100k DAU. It relies on Redis for multiple roles (Cache + Pub/Sub) to reduce operational overhead.
Architecture Decision Rationale:
Why this architecture?: WebSockets are essential for the "Discord feel" of instant communication. Redis Pub/Sub is the simplest way to handle fan-out for 20k concurrent users.
Functional Satisfaction: All core flows (Chat, Presence, Servers) are handled via either REST (metadata) or WebSockets (events).
Non-functional Satisfaction: NoSQL ensures storage can grow indefinitely, and the stateless Chat Service allows for easy horizontal scaling.

High Level Architecture

Sub-system Deep Dive

Edge (Optional)

Content Delivery & Traffic Routing: Use a global CDN (e.g., Cloudflare) for the web assets and static content. DNS uses latency-based routing.
Security & Perimeter:
API Gateway: Handles JWT validation.
Rate Limiting: Applied at the Gateway (e.g., 5 messages per second per user) to prevent spam.
SSL/TLS: Terminated at the Load Balancer.

Service

Topology & Scaling:
API/Chat Services: Stateless, deployed in auto-scaling groups across 3 Availability Zones (AZs).
WebSocket Gateway: Stateful. Requires "Sticky Sessions" or client-side reconnection logic. Scaled based on open connection count (target 5k per small instance).
API Schema Design:
POST /v1/channels/{id}/messages (REST/JSON): Send message.
GET /v1/channels/{id}/messages (REST/JSON): Fetch history with cursor-based pagination.
WS /gateway: WebSocket upgrade endpoint.
Resilience & Reliability:
Exponential backoff for WebSocket reconnections.
Circuit breakers on the Chat Service to prevent Message DB overloads.

Storage

Access Pattern:
Writes: High volume (every message sent).
Reads: Frequent (fetching history on channel switch).
Patterns: "Get last 50 messages for Channel X", "Append message to Channel X".
Database Table Design:
Metadata DB (PostgreSQL):
Servers: id, name, owner_id.
Channels: id, server_id, name, type.
Memberships: user_id, server_id, joined_at.
Message DB (Cassandra / ScyllaDB):
Messages: (Partition Key: channel_id, Clustering Key: message_id DESC). This allows efficient retrieval of the latest messages for a channel.
Technical Selection:
PostgreSQL: For metadata where relational integrity (Foreign Keys) is vital for the server/member structure.
Cassandra/ScyllaDB: For messages. LSM-tree architecture is optimized for the high-write nature of chat and handles time-series data perfectly.

Cache

Purpose & Justification: Presence storage and Pub/Sub. Presence is too volatile for a persistent DB.
Key-Value Schema:
Key: presence:{user_id}, Value: online|idle|dnd, TTL: 60s (heartbeat based).
Technical Selection: Redis. It provides both the Sorted Sets needed for some ranking/presence features and the Pub/Sub mechanism for message fan-out.
Failure Handling: If Redis fails, presence shows all as "offline". Non-critical for the core "send message" flow if we use the DB as the primary message source.

Messaging

Purpose & Decoupling: Connects the Chat Service to the WebSocket Gateways.
Mechanism: Redis Pub/Sub.
Each Channel in Discord maps to a Redis Topic: channel:{channel_id}.
WebSocket Gateways subscribe to topics for all channels currently viewed by their connected users.
Failure Handling: Redis Pub/Sub is "fire and forget". If a client is offline, they miss the event but will fetch the history from the Message DB upon reconnecting.

Infrastructure (Optional)

Observability: Prometheus for metrics (active connections, msg/sec) and Grafana for visualization.
Distributed Coordination: Not needed for MVP; Redis handles the shared state of presence.
Wrap Up

Advanced Topics

Trade-offs (Consistency vs Availability): We choose Availability (AP) for the chat flow. If the Message DB is slightly out of sync across regions, it’s better than users not being able to chat.
Reliability: We use Idempotency Keys (generated by the client) for message sending to prevent duplicate messages during retries.
Bottleneck Analysis:
Hot Channels: A channel with 10k users. One message = 10k WebSocket writes.
Optimization: For the MVP (100k DAU), this is manageable. For 10x growth, we would implement "message coalescing" or "fan-out workers".
Security: All messages stored at rest using AES-256. Access control checks (is user in channel?) performed at the Chat Service before processing.