architecture

WebSockets: Real-Time Communication for AI Products

A protocol providing full-duplex communication channels over a single TCP connection, enabling real-time bidirectional data exchange between client and server.

WebSockets are a communication protocol that provides a full-duplex, persistent connection between a client and a server over a single TCP connection. Unlike HTTP, where the client initiates every request and the server responds, a WebSocket connection allows both the client and the server to send messages to each other at any time, without the overhead of establishing a new connection for each exchange. For AI products, WebSockets are the enabling technology for real-time features: streaming LLM output to the user character by character, live collaboration where multiple users see changes simultaneously, and instant notifications when a background AI job completes. Understanding when to use WebSockets versus the simpler Server-Sent Events (SSE) alternative is important for UK AI product teams, because choosing the more complex option when SSE would suffice adds infrastructure cost without user-visible benefit. SSE is server-to-client only, which is sufficient for the most common AI streaming use case: sending LLM output to the browser as it is generated. WebSockets are the right choice when you need genuine bidirectionality: a collaborative AI workspace where multiple users interact in real time, or an AI agent that needs to receive user corrections mid-task. Next.js on Vercel supports SSE natively through streaming responses, making it the easiest starting point. WebSocket support on Vercel requires a managed service such as Ably, Pusher, or Supabase Realtime, which adds cost and an additional service dependency. SpeedMVPs implements the appropriate real-time mechanism for each product: SSE for LLM streaming interfaces, and managed WebSocket services for products that genuinely require bidirectional real-time communication.

How WebSockets Work

A WebSocket connection begins as a standard HTTP request with a special Upgrade header. The server responds with a 101 Switching Protocols response, and the connection upgrades from HTTP to the WebSocket protocol. From this point, both client and server can send frames of data to each other at any time without a new request. The connection stays open until either party closes it. This persistent connection is the fundamental difference from HTTP. In HTTP, every interaction requires a new request: the client opens a connection, sends a request, receives a response, and the connection closes (or is reused briefly). The server can never push data to the client unprompted. With WebSockets, the server can push data to the client the moment it is available, enabling genuinely real-time experiences. The connection is bidirectional: the client can also send data to the server at any time, making WebSockets suitable for interactive applications like collaborative editing, live chat, and interactive AI assistants.

WebSockets vs Server-Sent Events for LLM Streaming

For the specific use case of streaming LLM output to a browser, WebSockets and Server-Sent Events (SSE) are both commonly used, with different trade-offs. Server-Sent Events are a simpler, HTTP-based standard where the server sends a stream of text events over a persistent HTTP connection. The client cannot send data over an SSE connection: it is unidirectional, server to client only. SSE is natively supported in browsers without any JavaScript library, works with existing HTTP caching and proxy infrastructure, and is simpler to implement for purely server-to-client streaming. WebSockets are bidirectional, making them appropriate for interactive AI applications where the user sends messages and receives streaming responses, such as a chat interface. The connection is more efficient for high-frequency bidirectional communication because it avoids HTTP header overhead on each message. For most AI chat interfaces, SSE is sufficient and simpler. For collaborative real-time applications or AI interfaces with complex bidirectional interaction patterns, WebSockets are more appropriate.

Real-Time AI Features That Use WebSockets

Several AI product features are natural implementations for WebSocket connections. Streaming chat interfaces, where the AI response appears word by word as it is generated, use either SSE or WebSockets to push partial output from the server to the client as the LLM generates it. Collaborative AI workspaces where multiple users see each other's edits and AI suggestions in real time require WebSockets for the bidirectional synchronisation. Live AI analysis dashboards that update as new data is processed benefit from WebSocket push notifications rather than polling. Background AI job completion notifications, where a user triggers a long-running AI task and receives a notification when it finishes, can use WebSockets to deliver the result without the user refreshing the page. Multi-turn AI agent conversations where the agent is taking actions and reporting progress in real time need persistent connections to stream the agent's intermediate steps to the user.

Implementing WebSockets in Next.js

Next.js running on Vercel does not natively support long-lived WebSocket connections in its serverless function model because serverless functions terminate after the request completes. For WebSocket support in a Next.js application, the options are: use a third-party WebSocket service (Ably, Pusher, or Supabase Realtime) that handles the persistent connection infrastructure and provides a client SDK for the browser; deploy a separate Node.js WebSocket server alongside the Next.js application on a platform that supports persistent connections (Railway, Fly.io, a VPS); or use Vercel's own real-time offerings with PartyKit or similar integrations. For most AI MVPs, a managed WebSocket service like Ably or Pusher is the most practical choice. The managed service handles connection management, scaling, and geographic distribution, and you pay for usage rather than managing persistent server infrastructure. For SSE-based LLM streaming (the more common case), Next.js streaming responses work natively on Vercel without any additional infrastructure.

WebSocket Security and Authentication

WebSocket connections have security considerations that differ from standard HTTP requests. The initial WebSocket handshake is an HTTP request, so standard HTTP authentication (Bearer tokens in the Authorization header) applies at connection time. Once the connection is upgraded, no further authentication occurs unless you implement it at the application level within the WebSocket protocol. This means that if a user's authentication token expires mid-connection, the server needs to handle this gracefully. For AI products handling sensitive data, WebSocket connections should always use WSS (WebSocket Secure, the WebSocket equivalent of HTTPS) to encrypt the connection. For UK products processing personal data over WebSockets, the data in transit must be encrypted, which WSS provides. If personal data is sent over WebSockets, the DPA with your WebSocket service provider must cover this data processing.

When Not to Use WebSockets

WebSockets are not always the right tool. For straightforward request-response interactions, standard HTTP is simpler and sufficient. For server-to-client notifications that are infrequent, HTTP polling (checking for updates every few seconds) or SSE are simpler to implement and easier to debug. WebSocket connections consume server resources for the duration of the connection. At scale, many simultaneous open WebSocket connections require infrastructure that can handle them: load balancers configured for WebSocket passthrough, sticky sessions, and connection management. For AI products where the primary interaction pattern is asynchronous (user submits a task, comes back later to view the result), WebSockets may be more complex than the use case justifies. A simple polling mechanism or email/push notification on completion is often the right starting point, with WebSocket-based real-time updates added later if users request it.

Frequently Asked Questions

Do I need WebSockets for LLM streaming in my AI product?+

For most AI chat or text generation interfaces, Server-Sent Events (SSE) are simpler and sufficient. SSE provides server-to-client streaming natively in browsers without a library, works with existing HTTP infrastructure, and handles the LLM streaming use case cleanly. WebSockets are warranted when you need bidirectional real-time communication, such as a collaborative workspace where multiple users interact simultaneously or an AI agent that requires interactive input during task execution.

How do I implement WebSockets on Vercel with Next.js?+

Vercel's serverless functions do not support persistent WebSocket connections. Use a managed WebSocket service (Ably, Pusher, or Supabase Realtime) that handles the persistent connection while your Next.js application handles business logic. Alternatively, deploy a separate WebSocket server on a platform that supports persistent processes (Railway or Fly.io) alongside your Vercel Next.js deployment. For LLM streaming specifically, Next.js streaming responses with SSE work natively on Vercel without additional infrastructure.

What is the difference between WebSockets and HTTP/2 Server Push?+

HTTP/2 Server Push is a feature that lets a server proactively send resources (like CSS or JavaScript files) to a client that it anticipates the client will need. It is optimised for static asset delivery and has been deprecated in most modern browsers. WebSockets are a general-purpose bidirectional messaging protocol designed for real-time application data. They are entirely different technologies used for different purposes. HTTP/2 Server Push is not a viable alternative for real-time AI product features.

How many simultaneous WebSocket connections can a server handle?+

A single Node.js WebSocket server can typically handle tens of thousands of simultaneous connections on modest hardware, because WebSocket connections are largely idle (not consuming CPU) when no messages are being sent. The limiting factor is memory, approximately 10-50KB per connection depending on buffer sizes. At the scale of an AI MVP, WebSocket connection limits are not a practical concern. At scale, a managed WebSocket service automatically handles connection distribution across their infrastructure.

Does SpeedMVPs implement WebSocket-based real-time features?+

Yes. For AI products with chat interfaces, collaborative features, or real-time job status updates, we implement the appropriate real-time mechanism: SSE for one-directional LLM streaming, managed WebSocket services (Ably or Supabase Realtime) for bidirectional real-time features, and job queue polling as a simpler alternative when full WebSocket infrastructure is not yet justified. We select the approach based on your actual use case rather than building more infrastructure than the product needs. Get a free consultation at speedmvps.co.uk

Building an AI product that needs real-time streaming or live collaboration? Get a free consultation at speedmvps.co.uk

Get a Free Quote