Guide

Observability Of Event Flows

FastHook stores webhook evidence at each stage of the flow: inbound Requests, connection-specific Events, outbound delivery Attempts, metrics histograms, ignored branch records, and bulk operation progress.

Use this guide when a provider says it sent a webhook, a receiver says it did not process it, or an operator needs to decide whether to retry one event or replay a larger window.

Inspection Ladder

Debug from the broadest record to the narrowest record. A Request proves that FastHook received and verified provider ingress. Events prove which connection branches were created. Attempts prove what the destination actually returned.

Keep the ids from each step. The same source_id, connection_id, destination_id, request_id, event_id, and batch_operation_id are the filters you use in list endpoints, metrics, count endpoints, and bulk retry runbooks.

  • Requests show status accepted or rejected, verified true or false, source_id, events_count, ignored_count, retry metadata, and stored payload data when include=data is requested.
  • Events show one outbound branch per connection with request_id, webhook_id or connection_id, source_id, destination_id, uppercase delivery status, attempts, response_status, retry metadata, and timestamps.
  • Attempts show one receiver conversation with event_id, destination_id, response_status, attempt_number, trigger, body, requested_url, method, latency, and status.
  • Metrics show zero-filled request and event time series for the same ids and time range.
  • Bulk operations show the exact filter, total_count, queued_count, done_count, failed_count, status, and cancelable flag.
Trace one provider call through request, event, attempt, metrics, ignored-branch records, and bulk operation progress.
Evidence chain
Request req_xxx
  -> Event evt_billing
     -> Attempt att_billing_1
     -> Retry attempt att_billing_2

Request req_xxx
  -> ignored branch evidence
     /v1/requests/req_xxx/ignored_events

Start With Ingress

If a receiver reports missing work, first prove whether FastHook received the provider request. Search Requests by source, time range, request status, verified flag, retry metadata, and provider event id or external id through q/search_term.

Request status casing is lower-case: accepted or rejected. The verified flag describes source authentication only; it does not say anything about outbound destination delivery.

  • No request means the provider is not calling the FastHook source URL.
  • Rejected requests usually indicate method, source auth, timestamp, or provider signature mismatch.
  • Accepted requests with events_count greater than zero should be followed with /v1/requests/:id/events.
  • Accepted requests with events_count zero should be checked for ignored_count and /v1/requests/:id/ignored_events.
  • Use include=data only when you need stored payload data; avoid printing sensitive headers or bodies in incident logs.
Read the Request row first: missing, rejected, accepted without events, and accepted with events all lead to different next checks.
Ingress inspection
curl "https://api.fasthook.io/v1/requests?from=now-1h&to=now&source_id=src_xxx&status=accepted&verified=true&include=data&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/requests/req_xxx/events" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/requests/req_xxx/ignored_events" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Inspect Event Branches

Events are destination-branch records. One accepted request can create several events when a source fans out through multiple connections. Filter by connection_id or destination_id to avoid mixing receiver-specific evidence.

Event status casing is uppercase: SCHEDULED, QUEUED, HOLD, SUCCESSFUL, FAILED, or CANCELLED. Ignored branch evidence is separate from that status enum and is exposed through ignored_count, ignored_events, and ignored metrics.

  • Filter Events by source_id, connection_id, destination_id, status, and batch_operation_id.
  • Event records include webhook_id, source_id, destination_id, cli_id, next_attempt_at, successful_at, updated_at, and created_at.
  • SUCCESSFUL means FastHook received a successful receiver response and successful_at is set.
  • FAILED means the last delivery attempt failed; inspect Attempts for response body, response_status, and error_code.
  • SCHEDULED or QUEUED means work is waiting because of delay, retry, rate limit, or queue timing.
  • HOLD means the connection branch is paused and delivery is held until unpause.
  • CANCELLED means work was stopped and should be reviewed before retry.
Event status tells you whether to inspect attempts, wait for scheduling, unpause a branch, or review cancellation.
Event filters
curl "https://api.fasthook.io/v1/events?from=now-1h&to=now&connection_id=conn_xxx&status=FAILED&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/events/count?from=now-1h&to=now&destination_id=des_xxx&status=FAILED" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Read Attempts For Receiver Proof

Attempts are the most concrete delivery record because they preserve the receiver-facing HTTP exchange. Use them when a receiver returned a 4xx, 5xx, timeout, malformed response, or a successful status that still did not produce the expected side effect.

The trigger field explains why the attempt exists. INITIAL is the first delivery. AUTOMATIC is retry policy. MANUAL is a single operator retry. BULK_RETRY comes from a bulk operation. UNPAUSE comes from releasing held work.

  • Query attempts by event_id or repeated event_id[] values.
  • Sort by created_at desc when you want the latest receiver response first.
  • Compare attempt_number, response_status, body, error_code, delivery_latency, and response_latency.
  • Use requested_url and http_method when a receiver says FastHook called the wrong path.
  • A 200-level response is delivery success, not proof that the receiver completed downstream processing.
Attempt history
curl "https://api.fasthook.io/v1/attempts?event_id=evt_xxx&order_by=created_at&dir=desc&limit=100" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/attempts/att_xxx" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Use Metrics For The Shape

Metrics endpoints are for trends and windows, not individual proof. Use request metrics to prove provider ingress volume, and event metrics to prove delivery volume by status. Once a spike or gap is visible, use the same time range and ids in Requests, Events, Attempts, or count endpoints.

date_range[start] and date_range[end] are required for both request and event metrics. Choose granularity based on the incident window: 1m for short debugging, 1h for daily review, and wider buckets for long trend checks.

  • Request metrics support count, accepted_count, and rejected_count.
  • Event metrics support count, delivered_count, failed_count, queued_count, processing_count, hold_count, and ignored_count.
  • Request metrics can be filtered by source_id, batch_operation_id, status, and q.
  • Event metrics can be filtered by source_id, connection_id, destination_id, batch_operation_id, status, and q.
  • Use /v1/requests/count or /v1/events/count when pagination returns a lower-bound total and you need an exact count.
Use metrics to pick a narrow window, then list exact rows and inspect attempts before retrying.
Metrics windows
curl "https://api.fasthook.io/v1/metrics/requests?date_range[start]=2026-05-29T00:00:00Z&date_range[end]=2026-05-29T23:59:59Z&granularity=1h&source_id=src_xxx&measures[]=count&measures[]=accepted_count&measures[]=rejected_count" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/metrics/events?date_range[start]=2026-05-29T00:00:00Z&date_range[end]=2026-05-29T23:59:59Z&granularity=1h&connection_id=conn_xxx&measures[]=count&measures[]=delivered_count&measures[]=failed_count&measures[]=hold_count&measures[]=ignored_count" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Retry Evidence Safely

Request replay and event retry answer different questions. Replaying a request sends the stored inbound request back through current routing, which can create new branches if routing changed. Retrying an event targets one existing destination branch and requires the source and connection to be enabled and the connection to be unpaused.

Bulk retry should come after a single-event test. Build the filter from the same evidence you used in metrics and list endpoints: from, to, status, source_id, connection_id, destination_id, and optional q.

  • POST /v1/requests/:id/retry replays inbound source evidence.
  • POST /v1/events/:id/retry retries one outbound branch.
  • GET /v1/requests/bulk_operations shows request replay progress.
  • GET /v1/events/bulk_operations shows event bulk retry progress.
  • Cancel a planned or running bulk operation with the matching /cancel endpoint when the filter is wrong.
Retry and operation status
curl -X POST "https://api.fasthook.io/v1/events/evt_xxx/retry" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl "https://api.fasthook.io/v1/events/bulk_operations?bucket=ongoing&limit=20" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

curl -X POST "https://api.fasthook.io/v1/events/bulk_operations/bch_xxx/cancel" \
  -H "Authorization: Bearer fhp_xxx" \
  -H "x-team-id: tm_xxx"

Common Evidence Patterns

  • No request row: provider URL, provider environment, or source URL is wrong.
  • Rejected request: source auth, allowed method, timestamp, or provider signature is wrong.
  • Accepted request plus no event: no enabled route created a branch, or every branch was ignored by filter or dedupe.
  • HOLD event: the connection is paused; unpause releases held work.
  • SCHEDULED event: delay, retry, or destination rate limit is waiting.
  • FAILED attempt with 401 or 403: destination auth or receiver signature verification is mismatched.
  • FAILED attempt with timeout: receiver is slow, unreachable, or overloaded.
  • SUCCESSFUL event but receiver still missing side effect: inspect receiver logs using event id, request id, and idempotency key.

Observability Checklist

  • The runbook starts with Requests, not Events.
  • Request status filters use accepted or rejected.
  • Event status filters use uppercase delivery statuses.
  • Every incident query includes from, to, and the environment-specific source_id, connection_id, or destination_id.
  • Operators inspect /requests/:id/events and /requests/:id/ignored_events before retrying.
  • Attempts are checked for response_status, body, trigger, requested_url, and latency.
  • Metrics are used to find the window; list and count endpoints are used to identify exact records.
  • Bulk retry filters are copied from proven list queries and include a cancel path.
  • Receivers log idempotency keys so successful delivery can be reconciled with downstream processing.

Next