The Question
Design

Scalable Double-Entry Ledger System

Design a highly available and consistent ledger system for a financial platform that ensures data integrity through double-entry accounting. The system must handle high-volume transaction processing, guarantee idempotency for all requests, and provide a durable, immutable audit trail for all monetary movements across millions of accounts.
PostgreSQL
Redis
Kafka
CDC
ACID
Saga Pattern
Questions & Insights

Clarifying Questions

What is the expected peak throughput (Transactions Per Second) and total number of accounts?
Does the system require double-entry accounting (assets = liabilities + equity) to ensure mathematical integrity?
Are we supporting multi-currency transactions and real-time exchange rate conversions?
What are the consistency requirements? Is eventual consistency acceptable for balance reads, or must they be strictly consistent with the transaction log?
Are there specific regulatory requirements for data retention and auditability (e.g., 7 years of immutable history)?
Assumptions for MVP:
Scale: 5,000 TPS peak; ~10M active accounts.
Model: Strict double-entry accounting (every transaction has at least one debit and one credit).
Consistency: Strong consistency for writes and balance updates within the same shard.
Scope: Internal ledger for a fintech platform; multi-currency support included.

Thinking Process

Core Integrity: How do we guarantee that money is never created or destroyed? (Use Atomic Database Transactions for double-entry pairs).
Concurrency: How do we prevent race conditions on account balances? (Row-level locking or Optimistic Concurrency Control using versioning).
Scalability: How do we handle millions of accounts? (Horizontal sharding by account_id or ledger_id).
Auditability: How do we ensure the history is never tampered with? (Append-only immutable transaction tables and idempotent request keys).

Bonus Points

Idempotency Keys: Use client-generated UUIDs to ensure a retry of a "Move Funds" request never results in double-deduction.
Deterministic Processing: Implementing a state-machine approach where the ledger can be reconstructed entirely from the transaction log (Event Sourcing).
CDC for Downstream: Utilizing Change Data Capture (e.g., Debezium) to stream ledger updates to analytics without impacting the write path.
Hot-Partition Mitigation: Implementing "sharded counters" or "pending balance" buckets for high-traffic accounts (e.g., a corporate treasury account).
Design Breakdown

Functional Requirements

Create and manage accounts.
Record transactions (transfers) using double-entry principles.
Retrieve real-time account balances.
Provide an immutable audit trail of all movements.
Support idempotency for all financial operations.

Non-Functional Requirements

Atomicity: Transactions must be all-or-nothing.
Durability: Once a transaction is acknowledged, it must never be lost.
Availability: 99.99% (High availability for payment processing).
Latency: < 100ms for p99 transaction processing.

Estimation

Storage: 5,000 TPS * 86,400s/day \approx 432M transactions/day. At ~200 bytes per record (metadata + entries), that is ~86GB/day.
Read/Write Ratio: ~1:1 (Balance checks are as frequent as transactions in an MVP).
Throughput: 5,000 TPS is manageable by a well-tuned, sharded RDBMS cluster.

Blueprint

Concise Summary: A microservice-based system utilizing a sharded Relational Database to ensure ACID compliance for double-entry transactions.
Major Components:
Ledger Service: Handles business logic, validation, and orchestration of double-entry records.
Relational DB (PostgreSQL): Acts as the immutable source of truth for transactions and current balances.
Redis Cache: Provides low-latency access to current account balances for rapid validation.
Message Queue: Decouples the core ledger from downstream systems like notifications and reporting.
Simplicity Audit: This architecture avoids complex distributed transaction coordinators (like 2PC) by sharding accounts such that both sides of a simple internal transfer reside in the same database instance where possible, or using a "Saga" pattern for cross-shard moves.
Architecture Decision Rationale:
Why this architecture?: Financial systems prioritize correctness above all. RDBMS provides the strongest consistency model (ACID).
Functional Satisfaction: Double-entry is naturally modeled via relational tables with foreign key constraints.
Non-functional Satisfaction: Sharding solves scalability while replication ensures high availability.

High Level Architecture

Sub-system Deep Dive

Service

Topology & Scaling: Stateless instances deployed across 3 Availability Zones (AZs). Scaling triggered by CPU utilization and Request Latency.
API Schema Design:
POST /v1/transactions: REST protocol. Requires request_id (Idempotency Key), debit_account, credit_account, amount, currency. Returns 201 Created.
GET /v1/accounts/{id}/balance: REST. Returns current and available balance.
Resilience: Exponential backoff for DB connections; circuit breakers on downstream Event Bus.
Observability: Prometheus metrics for TPS and error rates. Structured JSON logs for transaction IDs to allow easy tracing.
Security: OAuth2/OIDC for API access; mTLS for service-to-service communication.

Storage

Access Pattern: Heavy writes (inserts into entries table) and point-reads (balance lookups).
Database Table Design:
accounts: id (PK), owner_id, currency, balance, version (for OCC), status.
transactions: id (PK), idempotency_key (Unique), description, created_at.
entries: id (PK), transaction_id (FK), account_id (FK), amount (Decimal), type (DEBIT/CREDIT).
Technical Selection: PostgreSQL. Rationale: Superior support for ACID, complex joins, and mature tooling for WAL (Write Ahead Log) archiving.
Distribution Logic: Sharding by account_id. For transfers between shards, the Ledger Service initiates a two-phase commit or a compensation-based Saga.
Reliability: Synchronous streaming replication to a hot standby in a different AZ. Daily snapshots to S3.

Cache

Purpose: Reducing read load on the DB for "check balance" operations before a transaction.
Key-Value Schema: balance:{account_id} -> Decimal Value. TTL: 1 hour with write-through or cache-invalidation on DB commit.
Technical Selection: Redis. Rationale: In-memory speed and support for atomic increments (INCRBYFLOAT).
Failure Handling: If Redis is down, the system falls back to querying the PostgreSQL read-replica.

Messaging

Purpose: Asynchronous propagation of ledger events (e.g., "TransferCompleted") to non-critical path services.
Event Schema: TransactionV1: {tx_id, timestamp, type, total_amount, metadata}.
Throughput: Partitioned by account_id to ensure ordered processing for a single account's history.
Technical Selection: Kafka. Rationale: Durability and replayability for downstream reconciliation.
Wrap Up

Advanced Topics

Monitoring: Key metrics include DB transaction lag, Idempotency hit rate, and Balance discrepancy (reconciliation) alerts.
Trade-offs: We choose Strong Consistency over Availability (CP in CAP) for the storage layer, as financial correctness is non-negotiable.
Bottlenecks: The primary bottleneck is the row-level lock on the accounts table for high-frequency accounts. Optimization: Use an "Entry-only" write path and aggregate balances asynchronously (Shadow Ledgers).
Failure Handling: Automated failover via Patroni/Etcd for the PostgreSQL cluster.