Wallet System
Backend wallet system covering idempotency, concurrency, and race-condition handling.
Problem & constraints
Financial-style backends have a class of bugs that only bite you in production — duplicated charges from retried requests, concurrent writes corrupting a balance, two simultaneous debits both reading the same snapshot before either commits. This project was built to work through those problems deliberately, with the constraints of a real system in mind.
No third-party payment processor to offload correctness to. Every operation needed to be idempotent, transactionally safe, and race-condition-free from first principles.
Key decisions
Idempotency keys on every write
Why: Client retries — from network failures, timeouts, or user double-taps — should never result in duplicate operations. Each write request carries a client-generated key; the server checks it before executing, and returns the cached result if the key has been seen.
Tradeoff: Extra storage and a lookup on every write request. Acceptable for correctness — a duplicated debit is far worse than a few milliseconds of overhead.
Row-level locking (SELECT FOR UPDATE) over optimistic concurrency
Why: Optimistic concurrency — read, modify, write-back, retry on conflict — adds retry logic and unpredictable latency under contention. For balance updates, pessimistic locking gives simpler, easier-to-reason-about correctness guarantees.
Tradeoff: More contention under very high write volume — requests queue behind the lock. For the scale this system targets, that is an acceptable tradeoff.
The hard part
The nastiest bug was a race condition where two concurrent debits could both read the same balance before either wrote back. Account balance: ₦1,000. Request A reads ₦1,000, deducts ₦600. Request B reads ₦1,000 (before A commits), deducts ₦600. Both succeed. Balance ends up at ₦400 — but ₦1,200 has been debited. The fix was wrapping the read-modify-write in a transaction with a row-level lock so only one operation can proceed at a time.
-- Debit operation (simplified)
BEGIN;
SELECT balance
FROM wallets
WHERE id = $1
FOR UPDATE; -- acquires row lock
-- application checks balance >= amount
UPDATE wallets
SET balance = balance - $2
WHERE id = $1;
INSERT INTO transactions (wallet_id, amount, type, idempotency_key)
VALUES ($1, $2, 'debit', $3);
COMMIT;Results & lessons
Shipped as a personal project to deeply understand the patterns behind financial backends. The core operations — credit, debit, transfer — are idempotent, transactionally safe, and handle concurrent writes correctly. Considering hosting it publicly as a reference implementation.
- →Idempotency is not an optimisation — it is a correctness requirement for any system that accepts retries.
- →Pessimistic locking is underrated. Optimistic concurrency is elegant until you are debugging a retry storm at 2 AM.
- →Writing the race condition out explicitly — in a test, in a comment, on paper — is the fastest way to understand it well enough to fix it.