Webhook Idempotency: Make Duplicate Delivery Harmless

Webhook idempotency means repeated delivery does not repeat the business effect. It is more than putting an event ID in a cache. The difficult case is a process that crashes between changing business state and recording that the event was handled.

Imagine an integration that grants a workspace access after a billing event. If the provider retries, the workspace should still have one entitlement. If your worker crashes, the event must remain recoverable. I would design around those two outcomes before choosing a queue or writing the endpoint.

Separate authenticity, receipt, and application

Stripe documents signature verification, duplicate events, asynchronous processing, and the possibility of events arriving out of order in its webhook guide. GitHub recommends validating webhook deliveries and responding promptly, with asynchronous processing where appropriate, in its webhook best practices.

These are provider-specific contracts. Do not assume that the identifier, signature format, retry schedule, or ordering behavior transfers unchanged from one provider to another.

For the hypothetical billing integration, I would separate three stages:

1. Verify the incoming delivery using the provider's documented method.
2. Persist the accepted event or durable processing job.
3. Apply the event's business effect with an explicit deduplication rule.

The HTTP success response belongs after durable acceptance. Returning success before placing the work somewhere recoverable creates a window in which the provider believes delivery succeeded but your application has lost it.

Give the database a role in deduplication

An illustrative receipt table is:

CREATE TABLE webhook_receipts (
  provider text NOT NULL,
  account_id text NOT NULL,
  event_id text NOT NULL,
  received_at timestamptz NOT NULL DEFAULT now(),
  applied_at timestamptz,
  PRIMARY KEY (provider, account_id, event_id)
);

This is a schema sketch, not a complete handler. A real inbox also needs enough verified payload or reference information to resume processing, plus retention and access controls appropriate to its contents.

For a local database effect, I would insert the receipt and update the entitlement in the same transaction. If the unique receipt already exists with a completed effect, return the existing outcome. If processing is deferred, distinguish accepted from applied; a row's presence must not automatically mean the business action succeeded.

The exact implementation depends on whether ingestion and application happen in one transaction or separate workers. Write that choice explicitly. A uniqueness constraint prevents duplicate receipt identities; it does not independently prove that all side effects occurred once.

Walk the crash windows

Here is the failure review I would require:

That last row is deliberately uncomfortable. Your database transaction cannot atomically roll back an email already accepted by an external service. Use an external idempotency mechanism where available, or design a durable outbound workflow with a documented duplicate policy.

I would avoid describing this entire pipeline as exactly-once delivery. “Duplicate deliveries produce one local entitlement transition” is a narrower claim that can be tested.

Event identity and business identity are different

Two different events may refer to the same underlying resource. Deduplicating only by delivery ID will not protect a business action that should occur once per invoice or subscription transition.

For this proposed product, granting access would be a state transition keyed by the workspace and entitlement version, not an unconditional “add thirty days” on every event. That design can make repeated notifications harmless while preserving an audit trail of what the provider actually sent.

Decide how to handle older events arriving after newer ones. One possible policy is to fetch current provider state before applying a change. Another is to use a provider-supported version or sequence field. Choose according to the provider's guarantees; timestamps alone may not establish the ordering your business needs.

Build a replay fixture before launch

I would create fixtures for a valid delivery, an invalid signature, the same event repeated, two distinct events for one resource, and reversed arrival order. Add process termination after durable acceptance and after local commit. Each case should have an expected final entitlement and receipt state.

A replay tool should operate only on verified stored events, require an explicit scope, and record who requested the replay. Replays are production operations with business consequences, not merely developer convenience buttons.

The structured API article explains why these machine interfaces deserve deliberate contracts. The website checklist discussion provides broader context for operational requirements that are easy to omit from a launch checklist.

Ship the webhook when you can stop the process at each boundary and still predict the final business state. That prediction is the useful guarantee.