Like 0

Interview prep notes

webhooks.md Raw

System Design Interview: Designing a Webhook Delivery System

Platforms like Stripe and GitHub notify customers by POSTing events to URLs the customer registers. Designing that reliably is a good interview question.

Requirements

  • Customers register endpoints and choose event types.
  • Every event is delivered at least once, even if the customer's server is down for hours.
  • One slow or broken endpoint must not delay everyone else.

Flow

  1. When a business event happens, the producing service writes it to an outbox table in the same transaction as its own change, so events are never lost.
  2. A relay publishes events to a queue. A fan-out service looks up subscriptions and creates one delivery record per (event, endpoint).
  3. Delivery workers POST the payload with a timeout (say 10 s). A 2xx means done; anything else schedules a retry.

Retries

Exponential backoff with jitter (1 min, 5 min, 30 min, 2 h ... up to ~3 days), then mark the delivery failed and alert the customer. Store attempts so customers can see history and trigger a manual redelivery.

Isolation

Keep a queue (or a rate-limited lane) per endpoint or hash endpoints into many queues, and cap concurrency per endpoint. Otherwise one endpoint that hangs for 10 s per request ties up all workers. Disable endpoints that fail for days.

Security

  • Sign each payload with an HMAC of the body and timestamp using a per-endpoint secret; receivers verify the signature and reject old timestamps to stop replays.
  • Resolve and block private IP ranges before connecting, to avoid SSRF into your own network.

Receiver expectations

Document that delivery is at-least-once and possibly out of order: include an event ID for dedup and a timestamp or version so receivers can ignore stale updates.


Disclosure: I work on Linkjob AI, an AI job interview assistant.