How Webhooks Work
The basic webhook flow is: you register a URL with a source system (your payment processor, your email provider, a third-party API). When a relevant event occurs in that system, it sends an HTTP POST request to your registered URL with a JSON body describing the event. Your server receives the request, validates it, and processes the event. The source system typically expects a 200 HTTP response within a few seconds to confirm receipt. If your server takes too long to respond or returns an error, the source system will retry the webhook delivery, typically with exponential backoff. The key implementation requirement on your side is that your webhook endpoint must respond quickly (under 2-3 seconds) and process the actual event asynchronously. If processing the event takes 10 seconds (because it triggers an LLM call, for example), you must acknowledge receipt immediately with a 200 response and submit the actual processing to a background job queue. Failing to do this results in timeout-related retry storms where the source system retries the same event multiple times, and your system processes it multiple times.
Webhook Security
Webhooks receive HTTP POST requests from the internet at a publicly accessible URL. Without security measures, anyone who discovers your webhook URL can send arbitrary payloads and trick your system into taking unintended actions. Proper webhook security uses signature verification: the source system signs each payload with a secret key shared between you and them (typically an HMAC-SHA256 signature). Your webhook handler verifies this signature before processing the payload. If the signature does not match, you reject the request immediately. Most webhook providers (Stripe, GitHub, Twilio, Clerk) implement this pattern. The provider gives you a webhook signing secret when you configure the webhook. You include their SDK or implement the verification logic yourself. Stripe's signature verification, for example, is a single function call that returns the verified event object or throws an error. Always implement signature verification. Never rely on the presence of a secret query parameter in the URL alone, as URLs can be logged and intercepted.
Idempotent Webhook Processing
Because webhook delivery is at-least-once (source systems retry on failure), your webhook handler must be idempotent: processing the same event twice must produce the same result as processing it once, without side effects from the second processing. The standard implementation is to record the webhook event ID when you first receive it and check for that ID before processing. If the event has already been processed, return 200 immediately without reprocessing. Most webhook providers include a unique event ID in every payload. Storing these IDs in a processed_events table with a unique constraint gives you idempotency with a single database insert. For AI products, idempotency is particularly important for events that trigger AI processing (a document upload that triggers embedding generation, an order that triggers AI analysis). Processing the same trigger twice means double the LLM API cost and potentially conflicting results in your database.
Sending Webhooks from Your AI Product
As your AI product grows, you will likely want to send webhooks to your customers, allowing them to receive real-time notifications when AI events occur in your product. A customer might want to know when an AI analysis job completes, when their AI-generated report is ready, or when an AI action was taken on their behalf. Building a webhook delivery system requires: a webhook registration mechanism (letting customers configure endpoint URLs and select which events they want to receive), event serialisation (defining a consistent JSON payload format for each event type), reliable delivery (queuing webhook deliveries with retry logic for failures), and a delivery log (showing customers which webhooks were sent and their delivery status). This is non-trivial to build correctly but is a common enterprise requirement for AI products integrating with existing customer workflows. Stripe's webhook architecture is a useful reference model.
Webhooks for AI Job Completion
A common AI product pattern is the request-webhook pattern: a customer submits a request (analyse this document, process this dataset, generate this report), receives a job ID immediately, and receives a webhook notification when the job completes with the result. This is preferable to long-polling (repeatedly checking a status endpoint) because it does not require the customer to write polling logic, and it delivers results immediately when ready rather than at the next polling interval. For AI products with longer processing times (multi-step agentic workflows, large document batches, complex data analysis), webhook delivery of results is the standard integration pattern for business customers. Implementing this pattern requires a job queue for AI processing, a webhook delivery system for result notifications, and clear documentation of the event payload format customers should expect.
Webhooks and GDPR
Webhook payloads often contain personal data: customer names, email addresses, order details, or AI-processed content derived from personal information. Under UK GDPR and EU GDPR, sending personal data to a customer's webhook endpoint constitutes disclosure of personal data to a third party, which must be covered by your data processing agreements and privacy notice. For B2B products where customers receive webhooks about their own users' data, this is typically covered by the DPA between you and the customer. For webhooks sent to third-party services on your customers' behalf (CRM integrations, Slack notifications), the data flow must be disclosed and each recipient must be listed as a sub-processor in your GDPR documentation. Ensure that webhook endpoints you send to are HTTPS (encrypted in transit) and consider whether payload data can be minimised: rather than including full personal data in the webhook payload, include only an ID that the recipient can use to fetch the data from your API.