|4.51Mins Read

Idempotency and Related Concepts

Authors
Idempotent HTTP Methods in RESTful APIs

Idempotent HTTP Methods in RESTful APIs

In RESTful APIs, idempotency ensures that making the same request multiple times has the same effect as making it once. In other words, repeated calls do not further change the server’s state.

The MDN Web Docs define an idempotent HTTP method as one whose effect on the server of a single request is the same as for several identical requests. For example:

  • GET (read) is idempotent because fetching a resource repeatedly does not alter it.
  • DELETE is idempotent because deleting a resource twice leaves it in the same “deleted” state (the first call succeeds; subsequent calls find nothing to delete).

By contrast, POST usually creates new resources on each call, so it is not idempotent.

As noted by HackerNoon, the methods GET, PUT, DELETE, HEAD, OPTIONS, and TRACE are idempotent, whereas POST and PATCH are not. In practice, this means clients can safely retry idempotent calls if they time out, without worrying about unintended side effects.

Summary of HTTP Methods

  • GET / HEAD / OPTIONS / TRACE: Read-only requests that do not modify state. Repeating them returns (nearly) the same response and leaves the server unchanged.
  • PUT: Replaces or updates a resource. Issuing the same PUT repeatedly ends with the same final state (resource content).
  • DELETE: Removes a resource. After the first successful delete, subsequent DELETEs do nothing new (the resource is already gone).
  • POST / PATCH: Generally not idempotent. Each POST typically creates a new entity or has a distinct effect, so re-sending it will duplicate work.

All safe HTTP methods (those that only read data) are by definition idempotent. PUT and DELETE are also idempotent, while POST is not.

Understanding these differences helps design more reliable APIs and informs how clients should retry requests.


Using Idempotency Keys in APIs

For operations that are not naturally idempotent (e.g. financial transactions), APIs often use idempotency keys to enforce idempotency at the request level.

A client generates a unique ID (often a UUID) and sends it with the request, typically in an Idempotency-Key header.

How It Works

  • Server checks the key: If the key was seen before, the server returns the stored response and does not execute the operation again.
  • New key: If the key is new, the server processes the request, stores the result (and the key), and returns the result.
  • Key expiration: Keys are often given a time-to-live (e.g. 24 hours) to limit storage and stale entries.

Example

When charging a customer, the client includes:

Idempotency-Key: abc123

If a network error causes a timeout and the client retries with the same key, the server recognizes abc123 and returns the original charge response instead of creating a second charge.

As HackerNoon notes, RFC 8594 defines this header: the client is responsible for unique key generation, and the server can optionally store a fingerprint of the request to detect duplicates.

This pattern is widely used in payment APIs (e.g., Stripe) and helps ensure "exactly-once" behavior on retries.

Key Best Practices

  • Client-side: Generate a unique idempotency key per operation. On retry, reuse the same key.
  • Server-side: Check for existing key before processing. If found, return the earlier response; if not, perform the action and record the key with its outcome.
  • Design for idempotency: Ensure the underlying operation is safe to repeat or that duplicates can be detected and ignored.

Idempotent Database Operations

Idempotent Database Operations

In databases, an idempotent operation is one that, when repeated, produces the same result as a single execution.

A common idempotent pattern is the UPSERT (update-or-insert) operation. For example:

INSERTON CONFLICT UPDATE

Running this with the same values multiple times will not create duplicates.

Examples of Idempotent DB Operations

  • Upserts: In SQL and NoSQL, upsert commands ensure one-time insertion.
  • Fixed updates: SET status = 'active' is idempotent; repeating it doesn’t change anything beyond the first update.
  • Idempotent deletes: DELETE FROM table WHERE id = X is idempotent – once X is gone, repeating the delete does nothing.
  • Unique checks: Before inserting, check for an existing key or use unique constraints (e.g., using a transaction ID to detect a repeated payment request).
  • Retry-safe transactions: If a transaction fails and aborts, no partial changes are committed, so retrying does not cause double effects.

Cockroach Labs points out that some operations are “naturally idempotent” (like updating customer info) while others (like subtracting funds) are not without safeguards.

By using upserts, unique constraints, and careful transaction design, database writes can be made effectively idempotent.


Idempotency in Distributed Systems

Idempotency in Distributed Systems

In distributed systems, unreliable networks and concurrent execution make idempotency crucial.

Example

Consider an online payment: the client submits a charge but loses connection. The client retries, unsure if the first went through. Without idempotency, the user might be double-charged.

Designing the service to be idempotent prevents such issues.

Common Strategies

  • Unique request identifiers and deduplication.
  • Each request carries a globally unique ID.
  • The server or message processor sees a repeated ID and ignores it.

Distributed Message Queue

The consumer can track processed message IDs and skip any already handled.

If an operation is not naturally idempotent (like withdrawing money), the service can explicitly check for duplication.