Design a Payment System

A payment system takes money from a customer and moves it to a merchant: card checkout on an online store, a ride fare, a subscription renewal. The volume is usually modest compared to social apps, but the correctness bar is extreme. Every payment happens exactly once, every cent is accounted for, and the system must stay correct when networks fail mid-request.

Requirements#

Functional

• Accept payments by card and other methods through external processors (Stripe, Adyen, banks).
• Pay out merchants on a schedule.
• Refunds and partial refunds.
• A full history of every money movement.

Non-functional

• No double charges and no lost payments, even with retries and crashes.
• Every transaction is auditable.
• Secure handling of card data (PCI DSS).

Payment flow#

1. Create a payment intent with status pending, stored before talking to anyone else.
2. Call the payment service provider (PSP) to authorise and capture the card.
3. Record the outcome: update the payment status and write ledger entries.
4. Handle webhooks: many PSPs confirm asynchronously, so a webhook can complete the payment later.

Idempotency#

Clients retry when they don't hear back. Without protection, a timeout during capture leads to a second charge. Every request carries an idempotency key:

POST /v1/payments
Idempotency-Key: order-7731-attempt-1
{ "amount": 4999, "currency": "USD", "source": "tok_visa" }

The server stores the key with the result. A repeat request with the same key returns the stored result instead of charging again. The same key is passed on to the PSP, so the processor de-duplicates too.

A state machine, not a boolean#

Payments move through explicit states, and only allowed transitions are accepted:

pending → authorised → captured → (partially) refunded
pending → failed
authorised → voided

If a crash happens between calling the PSP and saving the result, the payment stays pending. A recovery job asks the PSP for the true status and moves it forward. Nothing is guessed.

The ledger#

Balances are never stored as one mutable number. Each money movement is recorded as double-entry ledger lines that sum to zero:

Entries are append-only. Corrections are new entries, never edits. A balance is the sum of an account's entries, which makes the history fully auditable.

Reconciliation#

Your records, the PSP's records and the bank's settlement files can disagree because of webhooks lost in transit, timeouts or bugs. A daily job compares all three and flags mismatches for automatic fixes or human review. Reconciliation is how a payment system stays correct over months, not just per request.

Consistency choices#

• Use a relational database with transactions for payments and the ledger; correctness beats raw throughput here.
• Use the outbox pattern to publish events (receipts, merchant notifications): write the event in the same transaction as the state change, and a relay ships it to Kafka. No event is lost or sent for a rolled-back payment.

Security#

Never store raw card numbers. The client sends card details directly to the PSP, which returns a token; our system only sees tokens. This keeps most of the system out of PCI scope.