How Serverless Functions Work
A serverless function is a piece of code that runs in response to a trigger, typically an HTTP request, a queue message, a scheduled cron event, or a storage event. The cloud provider manages all underlying infrastructure: it provisions the compute resources to run your function when a request arrives, executes your code, returns the response, and tears down the resources when the function is idle. You pay only for the compute time consumed during execution, measured in milliseconds, plus the number of invocations. This model has distinct advantages for AI products with variable or unpredictable traffic. An AI tool that processes documents uploaded by users may receive ten requests one day and ten thousand the next. Serverless functions scale to zero during periods of no traffic and scale out to handle traffic spikes without any manual intervention or pre-provisioned capacity sitting idle. The disadvantages are execution time limits, which range from 10 seconds on some platforms to 15 minutes on AWS Lambda with specific configuration, and cold starts, which add latency to the first request after a function has been idle.
Vercel Functions for Next.js AI Products
For AI products built on Next.js, Vercel Functions are the path of least resistance. Next.js API routes and server actions are automatically deployed as serverless functions when you deploy to Vercel, with no additional configuration. The function runs in the same codebase as your frontend, sharing types and utilities, which is a significant developer experience advantage over managing separate API infrastructure. Vercel Functions have a default execution timeout of 10 seconds on the Hobby plan and up to 300 seconds on Pro and Enterprise plans for functions using the Fluid compute architecture. For LLM inference calls that can take 30-60 seconds for long completions, streaming responses are the practical solution: the function streams tokens back to the client as they arrive rather than waiting for the complete response, which keeps the user experience responsive within the timeout window. Vercel's edge and Node.js runtimes serve different use cases: edge functions run globally with lower latency but have more restricted APIs; Node.js functions run in specific regions and have full Node.js capabilities including file system access and longer execution times.
AWS Lambda for AI Workloads
AWS Lambda is the most mature and flexible serverless function platform, supporting runtimes for Node.js, Python, Go, Java, and custom runtimes. For AI products already on AWS infrastructure, Lambda integrates naturally with other AWS services including SQS for queue-based processing, S3 for document storage triggers, API Gateway for HTTP APIs, and EventBridge for scheduled functions. Lambda's maximum execution time of 15 minutes, achievable through specific configuration, accommodates longer AI processing tasks that Vercel Functions cannot handle. Lambda also supports response streaming, allowing LLM token output to be streamed back to clients through the Lambda Function URL or through a streaming-capable API Gateway configuration. For AI inference workloads that need GPU access, Lambda does not support GPU instances. This is the primary limitation that pushes GPU-based inference to containers on ECS or EKS rather than Lambda. For CPU-based inference and all API orchestration tasks, Lambda is a well-proven choice with a mature ecosystem of tooling, monitoring, and operational patterns.
Cloudflare Workers for Edge AI
Cloudflare Workers run serverless functions at Cloudflare's global edge network, which spans over 300 locations worldwide. This means your function code runs geographically close to your users, reducing the network latency component of response time. Workers use the V8 JavaScript engine rather than a full Node.js runtime, which enables cold starts measured in microseconds rather than seconds, but also means that many Node.js-specific packages are not compatible without modification. Workers AI, Cloudflare's integrated AI inference service, allows you to run inference on open-source models such as Llama and Mistral directly within a Worker, without round-tripping to a third-party LLM API. For AI products where inference latency is critical and data sovereignty requirements can be met within Cloudflare's network, Workers AI is an interesting option for specific use cases. The platform limitations, primarily the restricted Node.js compatibility and the smaller set of available AI models compared to managed API providers, make it more suitable for specific edge AI functions than as a primary AI product backend.
The Cold Start Problem for AI APIs
A cold start occurs when a serverless function is invoked after being idle and the platform needs to provision a new execution environment before running the code. Cold starts add latency ranging from a few hundred milliseconds to several seconds depending on the platform, runtime, and function size. For AI products where users are already experiencing LLM inference latency, cold starts add to an already noticeable wait time. Strategies for reducing cold start impact include keeping function bundle sizes small by minimising dependencies and using tree shaking, using platform-specific features that keep functions warm such as Vercel's Fluid compute or Lambda Provisioned Concurrency, implementing streaming responses so the user sees activity immediately rather than waiting for the full response, and designing the user experience to acknowledge that processing is happening rather than showing a static loading state. For functions handling authentication or routing where sub-100ms response times are expected, cold starts are more problematic than for LLM inference functions where users already expect a wait.
When to Move from Serverless to Containers
Serverless functions are the right starting point for most AI product APIs, but specific requirements create good reasons to move to containers. If your AI processing consistently takes longer than the maximum timeout available on your serverless platform, containers with no timeout are necessary. If you need to run a background process that is always on rather than request-triggered, serverless functions are the wrong model. If you need to cache a large ML model in memory between invocations to avoid reloading it on every request, serverless functions make this difficult because execution environments are ephemeral. If you need GPU access for self-hosted model inference, serverless platforms do not provide it and containers on GPU-equipped infrastructure are required. If your function invocation volume is consistently very high, the per-invocation billing model of serverless can become more expensive than reserved container capacity. The common pattern for AI products is to start with serverless functions, identify specific workloads that hit these limitations, and migrate only those workloads to containers while keeping the rest on the simpler serverless model.