Idempotency

An operation is idempotent when applying it multiple times produces the same result as applying it once. In ecommerce, idempotency is the foundation of safe retry logic: network failures, timeouts, and at-least-once delivery guarantees mean the same operation is frequently submitted more than once — idempotency prevents those retries from creating duplicate charges, duplicate orders, or corrupt inventory state.

HTTP method idempotency

HTTP methods divide into idempotent and non-idempotent by definition (AWS Builders' Library, Malcolm Featonby; microservices.io, Chris Richardson):

Method	Idempotent by definition?	Notes
GET	✅	Read-only; safe to retry freely
PUT	✅	Replaces resource state entirely
DELETE	✅	Deleting a non-existent resource returns 404 but does not corrupt state
HEAD, TRACE	✅	Read-only
POST	❌	Mutating; creates side effects — requires explicit idempotency handling
PATCH	❌	Mutating; partial updates are not idempotent by default

Idempotency keys

The dominant implementation pattern across payment processors is the client-provided idempotency key — a unique token, generated by the caller, attached to every mutating request. The server stores the key alongside the operation's result; subsequent requests with the same key return the cached result without re-executing.

How payment processors implement idempotency keys (as-of 2026-08-19)

Provider	Header	Key format	Max length	Key validity	Error caching
Stripe	Idempotency-Key	UUID v4 recommended	255 chars	24 hours (purgeable)	Yes — including 500s
Adyen	idempotency-key	UUID v4 required	64 chars	Minimum 7 days	Yes (transient errors flagged separately)
Checkout.com	Cko-Idempotency-Key	UUID v4	Not stated	24 hours (extendable)	No — 4xx/5xx not cached
PayPal	PayPal-Request-Id	UUID	38 single-byte chars	Up to 45 days (API-dependent)	Differs — returns current state, not original cached response
AWS EC2	ClientToken	Auto-generated by SDK	—	Lifetime of resource + interval	Semantically equivalent response returned

(Sources: Stripe API docs 2026; Adyen API idempotency docs 2026; Checkout.com docs 2026-01-07; PayPal Developer docs 2026-08-11; AWS Builders' Library, Malcolm Featonby 2026-08-07)

Key format: UUID v4 vs ULID

Shopify Engineering (Bart de Water, 2022) documented a 50% decrease in INSERT statement duration by switching from UUID v4 to ULID (Universally Unique Lexicographically Sortable Identifier) for idempotency keys in a high-throughput internal system. (as-of 2022-07-28)

Rationale: ULIDs contain a 48-bit millisecond timestamp prefix followed by 80 bits of random data, making them time-sortable. UUID v4 is fully random. B-tree indices (used by most SQL databases) perform better with monotonically or near-monotonically increasing keys because inserts land at the end of the index rather than causing random page splits.

Gunnar Morling (Confluent/Debezium, 2025-11-25) extends this analysis:

• UUID v4: Consumer must retain the full history of processed keys for the retention window. If a duplicate arrives after expiry, it is re-processed silently — neither party is alerted.
• UUIDv7 / ULID: Timestamp prefix allows consumer to detect "too old" keys and flag them to the producer for manual reconciliation rather than silently re-processing.
• Monotonically increasing sequence: Consumer only needs to store the single latest processed key per partition. More space-efficient but operationally risky: in multi-threaded producers, thread A can fetch sequence 100, thread B fetches 101, B emits first — A's message with key 100 is incorrectly discarded.
• Log-derived keys (outbox + CDC): Combines benefits. A CDC process (e.g., Debezium embedded engine) tails the transaction log and assigns idempotency keys from log position ({Commit LSN, Event LSN} tuple, encoded into 128 bits). Monotonically increasing without serialising the producer. Requires database logical replication support.

(Source: Gunnar Morling, morling.dev, 2025-11-25)

Avoid using sensitive data as idempotency keys

Stripe explicitly states: "Avoid using sensitive data (for example, email addresses or personal identifiers) as idempotency keys." (Stripe API docs 2026). Thea (dev.to, April 2026) adds: "Always HTTPS, never log full keys in plaintext; hash or truncate in production logs."

Two-phase reservation: the correct implementation pattern

A naive check-then-act pattern for idempotency has a race condition: two concurrent requests can both check "key not found" and both proceed to process.

The production-grade implementation (Alok Ranjan Daftuar, Byteridge, dev.to March 2026; André Cytryn, March 2026; Shopify Engineering, Todd Jefferson, 2019) requires:

1. Atomically insert the key as IN_PROGRESS using INSERT ... ON CONFLICT DO NOTHING (or equivalent). If the insert fails (key already exists), return the cached response.
2. Execute the business operation.
3. Update the key record to COMPLETED with the full response payload stored.

Shopify Engineering (Todd Jefferson, 2019) documents a three-state lifecycle — IncomingRequest model with IN_PROGRESS → COMPLETED → cached response returned to client.

[!unverified] Using a compare-and-swap on completion is described by an André Cytryn comment (March 2026) to handle the case where a slow downstream call causes the IN_PROGRESS TTL to expire before completion: "The fix was decoupling IN_PROGRESS TTL (sized to p99 of your slowest call) from COMPLETED TTL (sized to your full retry window), with a compare-and-swap on completion so a late-arriving first response can't silently overwrite a retried completion." Not independently confirmed by a primary source.

Critical: The full response payload — not just a status flag — must be stored. If only an "executed" marker is stored without the response, the client cannot safely replay and receive the original result. (Alok Daftuar, dev.to, March 2026; confirmed by Paulo Victor Leite Lima Gomes, Sr Engineering Manager, Nubank, in comment thread)

Storage layer choices

Store	Speed	Durability	TTL support	Best for
PostgreSQL (INSERT ON CONFLICT DO NOTHING)	Moderate	High (survives restart)	Manual cleanup job	Transactional workloads, financial operations
Redis/Memcached	Fast	Semi-durable (configurable)	Native TTL	High-throughput webhook deduplication
In-memory (Map, Set)	Fastest	None (lost on restart)	Manual	Single-instance, non-critical operations only

(Source: Hookdeck, updated 2026-04-13)

[!unverified] André Cytryn (March 2026 comment) states that naive Redis approaches using SET NX + GET are not atomic — the atomicity only holds inside a single Lua execution context. Postgres INSERT ON CONFLICT DO NOTHING inside a serializable transaction provides cleaner correctness guarantees. Not independently confirmed by a primary source but widely repeated in practitioner threads.

"Fake idempotency" failure modes (practitioner-named):

1. Checking the key but not storing the response
2. Storing the key but not handling concurrency
3. Using in-memory storage in a load-balanced (multi-instance) environment
4. Generating a new key on each retry (treats the retry as a new operation)
5. No TTL on stored keys (table grows indefinitely)
6. Missing DB unique constraints (race conditions bypass application-level checks)

(Sources: Thea, dev.to, April 2026; KolachiTech/Masad Ashraf, May 2026; Alok Daftuar, dev.to, March 2026)

TTL and retry window alignment

The idempotency store TTL must exceed the provider's retry window, not match it. (Digital Applied, 2026-05-28; Hookdeck, updated 2026-04-13)

Provider retry windows and recommended dedup cache TTLs (as-of 2026-05-28):

Provider	Retry window	Recommended dedup TTL
Stripe	Up to 3 days (live mode)	≥ 3 days
Shopify (webhooks)	~48 hours (19 retries)	≥ 48 hours
Adyen	Minimum 7 days key validity	≥ 7 days
Checkout.com	24 hours default (extendable)	≥ 24 hours
Svix	~27 hours (~8 attempts)	≥ 24 hours

(Sources: Digital Applied 2026-05-28; Adyen docs 2026; Checkout.com docs 2026-01-07; KolachiTech 2026-05-05)

Ecommerce contexts

Payment API idempotency

All major PSPs implement idempotency keys on POST endpoints. The client generates a UUID, sends it in a header, and the PSP caches the result. On retry, the cached result is returned. (Stripe API docs 2026; Adyen docs 2026; Checkout.com docs 2026-01-07; PayPal docs 2026-08-11)

Shopify's centralized Payment Service tracks each payment attempt — which consists of one or more retried API requests — with an idempotency key unique per attempt. The service "looks up the steps the attempt completed (such as creating a local database record of the transaction) and makes sure we send only a single request to our financial partners." (Shopify Engineering, Bart de Water, 2022-07-28)

Checkout form submission — preventing duplicate orders

Sources of duplicate orders in ecommerce (KolachiTech, 2026-05-05):

• Double-click on "Place Order" button
• Network timeout causing client retry
• Shopify webhook re-delivery (at-least-once)
• Third-party app sync pushing same update
• Payment gateway re-sending confirmation

Key generation discipline for checkout flows (Thea, dev.to, April 2026):

"Generate once when checkout loads, persist in React state/ref or sessionStorage. The key should be tied to the intent, not the button click."

Spree Commerce Store API (fetched 2026-08-19) implements idempotency keys on the full checkout flow: cart creation, cart updates, order completion (POST /carts/:id/complete), payment session creation, and payment completion. The Idempotent-Replayed: true response header signals when a cached response is returned.

Shopify REST Admin API accepts Idempotency-Key on POST requests (24-hour TTL). Shopify GraphQL API has no native idempotency header support — teams must use upsert mutations or implement a client-side deduplication table. (KolachiTech, 2026-05-05)

Webhook idempotency

At-least-once delivery is guaranteed by Shopify, Stripe, Svix, and AWS SQS Standard queues by design — duplicate delivery is expected and not an error condition. (Digital Applied, 2026-05-28)

Webhook deduplication IDs (as-of 2026-05-28):

• Stripe: event id field
• Shopify: X-Shopify-Webhook-Id header
• Svix: webhook-id header (stable across retries)
• AWS SQS: MessageId

Most dangerous failure mode — timeout-induced duplicate: Handler completes the work but the HTTP response arrives after the provider's timeout. Provider concludes delivery failed and retries. Business logic executes twice. Correct fix: acknowledge with 200/202 immediately, process asynchronously from a durable queue. (Digital Applied, 2026-05-28; Hookdeck, 2026-04-13)

Store the idempotency key BEFORE processing, not after. If stored after processing and processing fails, the next retry is treated as a duplicate and skipped. (KolachiTech, 2026-05-05; Hookdeck, 2026-04-13)

Idempotent Consumer pattern (message queues)

In event-driven architectures with at-least-once delivery (Kafka, SQS Standard, RabbitMQ), message handlers must be designed as idempotent consumers. The canonical pattern (microservices.io, Chris Richardson):

1. On receiving a message, the handler inserts the message ID into a PROCESSED_MESSAGES table (composite primary key: (subscriberId, messageId)).
2. The INSERT fails if the message was already processed — handler rolls back and ignores the duplicate.
3. If the INSERT succeeds, the handler processes the message within the same transaction.

Natural vs. non-natural idempotency (microservices.io, 2020):

• Naturally idempotent: event contains currentBalance (set operation — applying it twice has the same effect)
• Non-idempotent: event contains only debitAmount (subtract operation — applying it twice doubles the deduction)

Idempotency vs. deduplication

Most practitioner guides use the terms interchangeably. A distinction drawn by Gunnar Morling (2025-11-25) and Alok Daftuar (March 2026):

• Idempotency: the operation produces the same result regardless of how many times it is applied — a property of the operation itself.
• Deduplication: detecting and discarding duplicate requests so the operation executes exactly once — a suppression mechanism applied when the operation is not naturally idempotent.

A naturally idempotent PUT can tolerate duplicates by design; deduplication is the guard that prevents a non-idempotent POST from running more than once. (Digital Applied, 2026-05-28)

Failure modes practitioners flag

Beyond the "fake idempotency" anti-patterns, named engineers identified additional failure classes (Alok Daftuar thread, dev.to, March 2026):

1. Completed-but-undelivered: operation succeeds, network drops before response reaches client. Retry arrives; server detects duplicate via idempotency key; returns cached response. This works only if the full response payload was stored, not just an "executed" flag.

2. Unknown result / timeout case: A timeout tells the caller nothing about whether the operation executed. Correct response: use the provider's "get by idempotency key" lookup endpoint (Stripe provides this; not all providers do) to check server state before retrying or abandoning. (Paulo Victor Leite Lima Gomes, Nubank Sr Engineering Manager, dev.to comment, March 2026)

3. Idempotency fragmentation: API-layer idempotency does not protect async boundaries downstream. Each queue, microservice hop, and webhook delivery constitutes a separate idempotency domain. (Mayckon Giovani, Principal Systems Engineer, dev.to comment, March 2026)

4. TOCTOU (Time-of-Check-to-Time-of-Use): Distinct from idempotency — a perfectly deduplicated commit of an invalid state is still an invalid commit. Fix: collapse validation and commit into a single atomic boundary — optimistic locking with a version field, or conditional write that rejects the commit if preconditions are no longer met. (Alok Daftuar thread commenter, dev.to, March 2026)

5. Schema evolution: new request schema + old idempotency key = fingerprint mismatch rejection. (Alok Daftuar, dev.to, March 2026)

Key terms

Term	Meaning
Idempotency key	A unique client-generated token sent with a mutating request; the server uses it to detect and deduplicate retries
Two-phase reservation	The production-grade idempotency implementation pattern: atomically insert key as IN_PROGRESS, execute, update to COMPLETED with full response
Idempotent consumer	A message handler designed to produce the same outcome when invoked multiple times for the same message
At-least-once delivery	The delivery guarantee provided by Kafka, SQS Standard, and all major webhook platforms — messages may be delivered more than once
ULID	Universally Unique Lexicographically Sortable Identifier — a time-prefixed alternative to UUID v4 that improves b-tree index insert performance
Idempotency fragmentation	The failure class where idempotency is implemented at the API layer but not at downstream async boundaries, creating false safety
Fake idempotency	An idempotency implementation that passes review but breaks in production due to shared state problems, missing unique constraints, or in-memory stores in distributed environments
TOCTOU	Time-of-Check-to-Time-of-Use — a race condition where state changes between validation and execution; not solved by idempotency keys

Benchmarks (as-of 2026-08-19)

Metric	Value	Source
Shopify INSERT duration reduction (UUIDv4 → ULID)	~50%	Shopify Engineering, Bart de Water, 2022-07-28
Stripe idempotency key max length	255 chars	Stripe API docs 2026
Adyen idempotency key max length	64 chars	Adyen docs 2026
Adyen key validity	Minimum 7 days	Adyen docs 2026
Stripe key TTL	24 hours	Stripe API docs 2026
Checkout.com key TTL	24 hours default	Checkout.com docs 2026-01-07
Checkout.com concurrent key conflict response	HTTP 409	Checkout.com docs 2026-01-07
Checkout.com recommended retry wait	≥ 30 seconds	Checkout.com docs 2026-01-07
Shopify webhook retries	19 times over 48 hours	KolachiTech 2026-05-05
Shopify webhook connection timeout	1 second	Digital Applied 2026-05-28
Shopify webhook full-request timeout	5 seconds	Digital Applied 2026-05-28
Stripe live-mode webhook retry window	Up to 3 days	Digital Applied 2026-05-28
Stripe replay-attack tolerance	5 minutes (embedded timestamp)	Digital Applied 2026-05-28

Next frontier

Two-Phase Commit (2PC) · Dead Letter Queue · Idempotent Producer (Kafka) · Optimistic Locking · Transactional Outbox Pattern