Webhook Replay Systems

Replay Models

FastHook stores evidence at three useful layers: the inbound Request, each routed Event branch, and each outbound Attempt. Replay should start at the layer where the mistake happened.

Provider redelivery is outside FastHook and arrives as new ingress. Request replay and event retry are FastHook operator actions, so they carry retry metadata and can be traced through metrics and bulk operations.

• Request replay starts from the stored inbound request and runs current routing, filters, transformations, deduplication, and pause state again.
• Event retry starts from one already-routed destination branch and sends that branch again.
• Bulk replay creates a batch operation from filters such as time range, source, destination, connection, status, and search term.
• Provider redelivery happens outside FastHook and should be treated as new ingress traffic.

Request replay
  -> source pipeline
  -> current routing
  -> new destination events

Event retry
  -> existing destination branch
  -> new delivery attempt path

When To Replay Requests

Use request replay when FastHook should evaluate the original provider request again. This is the right layer when the route graph or source boundary has changed since the request arrived.

Request replay can create different event branches than the first run because it uses current connections and current rules. That is useful after a fixed filter, but risky if production routing changed for unrelated reasons.

• A filter was too strict and ignored valid provider traffic.
• A transformation was fixed and current routing should be applied again.
• A source verification setting was corrected after rejected traffic.
• A new destination branch should receive old accepted requests.
• The original request payload is still available and safe to process.
• The receiver is idempotent enough to tolerate new delivery attempts for the same provider event.

curl -X POST "https://api.fasthook.io/v1/requests/req_xxx/retry" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

# request bulk retry supports accepted/rejected request status
curl -X POST "https://api.fasthook.io/v1/requests/bulk_operations" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "from": "2026-05-29T10:00:00.000Z",
    "to": "2026-05-29T10:30:00.000Z",
    "status": "accepted",
    "source_id": "src_xxx",
    "q": "stripe"
  }'

When To Retry Events

Use event retry when the route decision was correct and only one destination branch failed. Event retry keeps recovery scoped to the existing connection and destination instead of re-running every current route from the original request.

FastHook checks branch state before queuing the retry. The source and connection must be enabled, and the connection must be unpaused. Held events are released by unpause, not by event bulk retry.

• The source request and routing decision were correct.
• Only one destination branch failed.
• The receiver returned a recoverable 429, timeout, or 5xx response.
• The destination has been fixed and is ready to receive traffic again.
• The connection is enabled and unpaused.
• The destination rate limit is low enough to handle the retry traffic.

curl -X POST "https://api.fasthook.io/v1/events/evt_xxx/retry" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

# event bulk retry targets FAILED or CANCELLED events
curl -X POST "https://api.fasthook.io/v1/events/bulk_operations" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "from": "2026-05-29T10:00:00.000Z",
    "to": "2026-05-29T10:30:00.000Z",
    "status": "FAILED",
    "source_id": "src_xxx",
    "connection_id": "web_xxx",
    "destination_id": "des_xxx",
    "q": "timeout"
  }'

Bulk Operations

Bulk retry is a control-plane object, not a hidden loop. FastHook creates a batch operation with an id, status, total_count, queued_count, done_count, failed_count, and cancelable flag.

The dashboard exposes separate bulk retry modals for Requests and Events. Request bulk filters use request statuses such as accepted or rejected. Event bulk retry is for FAILED or CANCELLED events; HOLD is drained by unpausing the connection.

• Use single request replay or single event retry before creating a large bulk operation.
• Build the filter from the same list or metrics query that proved the failure window.
• Watch ongoing operations with GET /v1/requests/bulk_operations or GET /v1/events/bulk_operations.
• Cancel planned or running work with POST /v1/requests/bulk_operations/:id/cancel or POST /v1/events/bulk_operations/:id/cancel.
• Use the Events dashboard scope switch to compare retry events created by the operation with target events selected by the filter.

curl "https://api.fasthook.io/v1/events/bulk_operations?bucket=ongoing&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl -X POST "https://api.fasthook.io/v1/events/bulk_operations/bch_xxx/cancel" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/events?batch_operation_id=bch_xxx&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Evidence And Metadata

Replay is safer when operators can prove exactly what changed. Requests expose retry metadata such as is_retry, retry_kind, retry_parent_request_id, retry_operation_id, and retried_at. Events expose retry_parent_event_id and retry_operation_id for event retries.

Attempts show the delivery conversation: response_status, attempt_number, trigger, error_code, requested_url, http_method, latency fields, and bulk_retry_id. Use attempts to distinguish receiver failures from FastHook route or transform problems.

• Requests prove ingress status, source_id, verified, events_count, ignored_count, and request retry metadata.
• Events prove branch status, connection_id, destination_id, response_status, ignored_reason, and event retry metadata.
• Attempts prove trigger, response_status, body, error_code, requested_url, method, and timing.
• Metrics can filter by batch_operation_id while a bulk retry is running.
• Ignored events should be inspected before request replay when filters or deduplication were involved.

curl "https://api.fasthook.io/v1/requests/req_xxx/events" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/requests/req_xxx/ignored_events" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/attempts?event_id=evt_xxx&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Operational Runbook

During an incident, avoid jumping straight to bulk retry. First prove the failure layer, then make one narrow recovery action, then widen only if the receiver remains healthy.

If the receiver is failing now, pause the affected connection or lower the destination rate limit before replaying. Replaying into a broken receiver usually converts one outage into a backlog and retry storm.

• Find the window with request and event metrics.
• List exact requests or events with from, to, status, source_id, connection_id, destination_id, and q.
• Inspect one request's events, ignored events, and attempts.
• Fix the source, connection rule, transformation, destination, or receiver first.
• Retry one event or one request manually.
• Start a bulk operation only after the single retry behaves correctly.
• Watch metrics and cancel the bulk operation if failures rise again.

Replay Safety Checklist

Replay is an operational privilege. The controls below are the difference between recovery and accidental duplicate processing.

• Every receiver has an idempotency key before replay starts.
• Bulk filters are narrow enough to avoid unrelated traffic.
• Destination rate limits are set before replaying a large window.
• Paused connections are resumed deliberately, not bypassed.
• Operators can inspect the original request, replayed event, and new attempts.
• Metrics are watched by source, destination, connection, status, and batch operation.
• Request replay is used only when current routing should be applied again.
• Event retry is used when one existing destination branch failed.
• Bulk operation ids are recorded in the incident timeline.
• The runbook includes a cancel step and an owner who watches the receiver.

Next

• Webhook retry strategy
• Webhook idempotency
• Control throughput