In distributed systems, one truth stands above all:
👉 Failures will happen. Retries are inevitable.
But here’s the real challenge:
What happens when the same request is executed multiple times?
- Duplicate payments
- Multiple orders
- Inconsistent state
This is where idempotency becomes a critical concept in API design.
🧠 What is Idempotency?
An operation is idempotent if:
Performing it multiple times produces the same result as performing it once.
🧾 Simple Example
DELETE /users/123
- First call → deletes user
- Second call → user already deleted
👉 Result is the same: user does not exist
✔ This is idempotent
❌ Non-Idempotent Example
POST /orders
- First call → creates order #1001
- Second call → creates order #1002
👉 Duplicate orders = big problem
🔥 Why Idempotency Matters in Real Systems
In production, retries happen due to:
- Network timeouts
- Service failures
- Client retries
- Load balancer retries
Tools like NGINX or AWS Elastic Load Balancer may retry requests automatically.
👉 Without idempotency, retries can corrupt your data.
⚠️ Real-World Failures
💸 Duplicate Payments
A payment request is retried:
POST /payments
👉 Without idempotency:
- Customer gets charged twice
🛒 Duplicate Orders
User clicks “Buy” multiple times:
👉 System creates multiple orders
📉 Trading Systems
In a trading platform:
POST /execute-trade
👉 Duplicate execution = real financial loss
🧩 HTTP Methods and Idempotency
| Method | Idempotent? | Why |
|---|---|---|
| GET | ✅ Yes | Read-only |
| PUT | ✅ Yes | Replaces resource |
| DELETE | ✅ Yes | Removes resource |
| POST | ❌ No | Creates new resource |
| PATCH | ❌ Usually | Partial updates |
👉 The biggest challenge is making POST operations idempotent.
🏗️ How to Design Idempotent APIs
🔑 1. Idempotency Keys (Most Important Pattern)
Client sends a unique key:
POST /payments
Idempotency-Key: abc123
🔁 How It Works
- Server receives request with key
- Stores the result (DB or cache)
- If same key is reused:
- Return same response
- Do NOT reprocess
🧪 Example Flow
Client → POST /payments (key=abc123)
→ Payment processed
Client retries → same key
→ Server returns cached response
✔ No duplicate payment
✔ Safe retries
🧰 Implementation Strategies
1. Database Constraint
- Unique key per request
- Reject duplicates
✔ Simple
❌ Doesn’t return original response
2. Cache-Based (Best Practice)
Use Redis:
- Store key + response
- Add TTL (e.g., 24 hours)
✔ Fast
✔ Scalable
3. Hash-Based Requests
- Generate hash from request body
- Use as idempotency key
✔ Automatic
❌ Risk of collisions (rare but possible)
⚙️ Real Backend Example
💳 Payment API
POST /payments
Idempotency-Key: user123-transaction456
Server logic:
- Check if key exists
- If not:
- Process payment
- Store response
- If exists:
- Return stored response
📊 Internal Microservices
Even internal calls need idempotency:
Order Service → Payment Service
👉 If retry happens, system remains consistent.
⚠️ Common Mistakes
❌ Ignoring Idempotency in Critical Flows
- Payments
- Orders
- Financial transactions
👉 These MUST be idempotent.
❌ Storing Only the Key
👉 You must store:
- Request result
- Status
- Response
❌ Infinite Storage
👉 Always use TTL:
- Prevent memory leaks
- Clean old keys
🧠 Design Insight
Idempotency is not just a backend concern—it’s a system-wide guarantee.
👉 It works together with:
- Retries
- Message queues
- Event-driven systems
In tools like Apache Kafka, idempotency is essential to avoid duplicate event processing.
🚀 Advanced Pattern: Idempotent Consumers
In event-driven systems:
- Messages may be delivered multiple times
- Consumers must handle duplicates
👉 Solution:
- Track processed message IDs
- Ignore duplicates
💡 Final Thoughts
Idempotency is one of those concepts that separates mid-level engineers from senior engineers.
Without it:
- Systems break under retries
- Data becomes inconsistent
With it:
- Systems become safe, predictable, and resilient
👉 If your system handles money, orders, or critical data:
Idempotency is not optional—it’s mandatory.
