Webhooks

Webhooks push crawl events to your own HTTPS endpoint the moment they occur, so downstream systems can react without polling.

Registering

Open Settings > API Access in the app. Enter the URL of your endpoint and select the event types you want to receive. Only HTTPS endpoints are accepted.

On creation, a signing secret is displayed once. Copy it and store it securely — Consuela keeps only a hash, so the original cannot be retrieved later. If the secret is lost, delete the endpoint and create a new one.

Event payload

Every webhook delivery is a POST request with a JSON body. Two event types are supported:

Payload structure:

{
  "eventId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "eventType": "crawl.completed",
  "createdAt": "2026-07-30T09:00:00Z",
  "data": {
    "passId": "f9e8d7c6-b5a4-3210-fedc-ba0987654321",
    "succeeded": true
  }
}

Signature verification

Every delivery includes a consuela-signature header:

consuela-signature: t=1700000000,v1=5a2f3c...

The v1 value is the hex-encoded HMAC-SHA256 of the signing secret over the string <t>.<raw body>, where t is the Unix timestamp in the header and the raw body is the exact bytes of the request — not re-serialized JSON.

To verify a delivery:

1. Extract t and v1 from the header.
2. Compute HMAC-SHA256(secret, "<t>.<raw body>") and hex-encode the result.
3. Compare your computed value to v1 using a constant-time comparison.
4. Check that t is within 5 minutes of your server's clock. Reject the delivery if the difference exceeds 300 seconds.

Node.js example

import crypto from "node:crypto";

function verifyWebhook(secret, header, rawBody) {
  const parts = Object.fromEntries(
    header.split(",").map((p) => {
      const [k, ...v] = p.split("=");
      return [k, v.join("=")];
    }),
  );

  const timestamp = parts["t"];
  const signature = parts["v1"];

  // Reject if the timestamp is more than 5 minutes old
  const age = Math.abs(Date.now() / 1000 - Number(timestamp));
  if (age > 300) {
    throw new Error("Timestamp outside tolerance window");
  }

  const expected = crypto
    .createHmac("sha256", secret)
    .update(`${timestamp}.${rawBody}`)
    .digest("hex");

  if (!crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(signature))) {
    throw new Error("Invalid signature");
  }

  return JSON.parse(rawBody);
}

Delivery semantics

Webhook delivery is at-least-once. Your endpoint may receive the same event more than once, so deduplicate on eventId.

When a delivery attempt fails, Consuela retries up to 5 times with exponential backoff. The following responses trigger a retry:

• Any 5xx status code
• 408 Request Timeout
• 429 Too Many Requests
• Connection timeout or network error

All other 4xx responses are treated as permanent failures and are not retried.

Delivery log

The delivery log is available in the app alongside the endpoint configuration. Each entry shows the event type, the delivery result, and the attempt count.

Every delivery carries one of four states: