functions as a service
AWS introduced Lambda at re:Invent 2014: upload code, attach an event source, and pay per invocation and per 100 ms of execution (since changed to per millisecond). In 2018 the maximum duration was raised from 5 to 15 minutes.
Architecture Patterns · lesson 3 of 7 · about 28 min
Serverless
Functions on demand and what they cost you: why the first call after a quiet spell is slow, when pay-per-use beats an always-on server and when it does not, how concurrency limits and database connections limit scale-out, what to do with jobs longer than the time limit, which workloads fit, and the bill from a function that triggers itself. Tap any for a plain-language card.
0 / 9 sections done
Functions as a service
Nobody called the function for ten minutes. The next caller waits for a machine to be created.
After a call finishes, the provider keeps the environment warm for a while (minutes) and reuses it. So cold starts matter most for spiky or rare traffic: with one call per 100 seconds almost every call is cold; with a hundred a second almost none are. Mitigations include smaller packages, lighter runtimes, provisioned (pre-warmed) concurrency at a fixed price, and not putting the function in a latency-critical user path unless it is hot. Simulate the traffic.
At one call per 100 seconds, most calls are cold; at 10 a second almost none. Shorten the keep-warm time to see how fast it degrades.
Invocations simulated (1 hour)3,684
Cold starts5 (0.1%)
Average added latency1 ms
Slowest 1% of calls wait≈ 0
At 1 calls a second with environments kept warm for 10 minutes, 0.1% of invocations start cold, each paying about 800 ms. Traffic is steady enough that nearly every call reuses a warm environment.
Seeded one-hour simulation: Poisson arrivals; an invocation is cold if no warm idle environment exists; environments expire after the keep-warm time (real providers do not publish exact values and may reclaim earlier).
The bill
Pay only for what runs. Until what runs is most of the day.
There is a break-even rate, and it depends on the duration and memory of each call and the price of the alternative. Set the request rate and the cost of an always-on instance (the instance price is your own number; the bars use two cores at 60% utilisation). Memory also sets CPU: a function gets CPU in proportion to its memory, roughly one full vCPU at 1.8 GB, so the default here is sized to do the same work as one core of the instance. The function cost rises linearly with traffic; the instances rise in steps.
At one or two requests a second the function costs a fraction of an instance. Raise the traffic to a few dozen a second and the always-on fleet becomes the cheaper choice.
Serverless, per month$41
Always-on (1 instance), per month$58
Cheaper optionserverless
5 requests a second of 100 ms at 1.8 GB costs about $41 a month as functions and $58 on 1 always-on instance: serverless is cheaper at this traffic; the lines cross somewhere below 20 requests a second.
Function cost uses Lambda-style list prices ($0.20 per million requests, $0.0000166667 per GB-second). The instance price is your assumption; instances needed = traffic ÷ (cores ÷ time per request × 60% utilisation).
Scaling out, instantly
The platform scales to a thousand copies in seconds. Your database does not.
The sharper problem is downstream. Each copy may open its own connections to your database, so a burst of 1,000 copies means thousands of connections to a database sized for a few hundred, which is the scale-out spiral from the autoscaling lesson, only faster. The usual cures: a connection proxy or pooler (RDS Proxy, PgBouncer), an HTTP-style data API, or a reserved concurrency limit on the function to protect what it calls.
500 calls a second at 2 s each needs 1,000 copies; with 5 connections each that is 5,000 connections against a database that allows 2,000.
Concurrent copies needed1,000
Requests throttled0%
Database connections requested5,000
500 calls a second lasting 2.0 s need 1,000 copies running at once. With 5 connections each, 5,000 connections hit a database that accepts 2,000: the function fleet takes the database down. Add a connection pooler or cap the function's concurrency.
Concurrency = rate × duration, capped by the account limit; connections = running copies × connections per copy.
Limits
A function may run for at most 15 minutes. The report takes 45.
Splitting can make the job faster than one long run, because the chunks run side by side, at the cost of start-up overhead per chunk and a more complex design. If the work is long-running, needs a GPU, a large in-memory dataset or a persistent connection, a container or VM is simply the better tool. Set the job and the limit.
A 45-minute job with a 15-minute limit needs 4 chunks; run 10 in parallel and it finishes in about 12 minutes.
Fits in one function call?no
Chunks needed4
Wall-clock time12.1 min
A 45-minute job cannot finish within 15 minutes, so it is split into 4 chunks of at most 12 minutes, run 10 at a time; the whole thing completes in about 12.1 minutes, but needs shared storage and a workflow to track the chunks.
Chunks = ⌈job ÷ (0.8 × limit)⌉ so each stays inside the limit; wall time = ⌈chunks ÷ parallel⌉ rounds × (chunk length + overhead).
Where it fits
Serverless is a good fit for some shapes of work and an expensive mistake for others.
It also changes how you build: small functions wired together by events, state in managed services, observability across many short-lived invocations, and the risk of lock-in to one provider's triggers and limits. Pick a workload and see what each model costs per month, using the list prices from the cost scene.
Compare the nightly job (pennies) with the steady API (the fleet wins) and the GPU model (functions simply cannot).
Webhook receiver (a few per minute, bursty). Rare, unpredictable events with no latency-critical user.
As functions, per month$1.22
On an always-on instance, per month$58
Fitgood
Webhook receiver: about $1.22 a month as functions against $58 for an always-on instance. Functions win because the work is small or rare enough that the idle server would be the expense.
Per-workload request rates are illustrative; function cost uses 1.8 GB (about one full vCPU) at list prices; instance count assumes 60% utilisation.
Serverless or not?
Six workloads, four homes.
Functions suit rare, spiky, short, stateless work; steady heavy traffic and long-lived connections want always-on compute; commodity capabilities want a managed service. Each answer is worth 20 XP.
Ask: how steady is the traffic, how long does one unit of work run, and does it hold state or connections?
Question 1 / 6
Resize an image when it is uploaded; about 200 uploads a day.
Tip: press 1 to 4 to answer, Enter for the next one.
Who lives by this
From Lambda to the Berkeley view.
Four published sources on serverless.
microVMs for Lambda and Fargate
Firecracker is a lightweight virtual machine monitor open-sourced by AWS and used under Lambda; it is designed to start a microVM in around 125 milliseconds with a small memory footprint, so thousands can run on one host with VM-level isolation.
V8 isolates at the edge
Cloudflare Workers run JavaScript and WebAssembly in V8 isolates rather than containers, trading a restricted environment for start-up in milliseconds and very dense packing on edge machines.
'Cloud Programming Simplified: A Berkeley View on Serverless Computing'
The paper argued that serverless would become the default way to program the cloud, while listing limits of the first generation: slow shared state and storage access, no fast coordination between functions, and unpredictable performance, which later work aims to fix.
Interview angle
What interviewers actually ask.
Six questions, each worth 20 XP. Serverless questions probe whether you know the limits as well as the convenience.
Mention cold starts, concurrency and connection limits, time limits, and the cost crossover with always-on compute.
Question 1 / 6
What is a cold start?
Tip: press 1 to 3 to answer, Enter for the next one.
Break it
A function resizes images when they land in a bucket. It writes the thumbnail to the same bucket. Monday morning there is a bill nobody expected.
Defences: write outputs to a different bucket or prefix from the trigger's; filter triggers by key pattern; set a reserved concurrency limit on every function so a runaway cannot scale past what you can afford; add billing alarms and budgets that notify (or cut off) at a threshold; and alert on invocation counts per function. Set the scene.
1,000 stuck chains of 1 second at 3 GB, left for a weekend (48 hours): 173 million invocations and about $8,700. A concurrency cap of 20 makes it about $170.
Concurrent runs1,000
Invocations per second1,000
Invocations in total173 million
Cost$8,675
1,000 self-triggering chains of 1000 ms keep 1,000 copies busy, 1,000 invocations a second. After 48 hours that is 173 million invocations and about $8,675, with no error anywhere.
Each chain is one invocation after another; invocations = concurrency ÷ duration × seconds; cost at Lambda-style list prices. Concurrency is limited by the account limit (1,000 by default), so the damage is bounded only by time and memory.