What a service boundary costs

benchmarks project 01 · four measurements, each run twice

tldr

A request that used to be a function call costs 36 µs here at best and 317 at worst. Nine times across the whole ladder, not the hundred the argument assumes. Turning on HTTP keep-alives was worth 2.2x by itself, more than any step down that ladder. And none of it matters much next to the receiving side's own code, which moved the median by two orders of magnitude.

What was actually run

Everything below runs on one box: two cores of a Ryzen 9 5950X, Linux 6.1.0-43, Go 1.27.1 and CPython 3.11. The governor is unreadable from inside the container and a neighbour comes and goes, so orderings are the result here and absolute numbers are not.

Four separate measurements, each one changing a single thing and pinning everything else:

The ladder. A client sends a request, a server does one small piece of work, the reply comes back. Five different stacks carry that same request. How long does the round trip take?
Contention. One server, one fixed stream of requests, and periodic heavy work landing on it. What happens to latency when the far side gets busy, and does the receiver's own code change the answer?
Composition. One calculation graph, written four ways. Does wiring it as channels and goroutines make it faster than an ordinary function call graph?
The unit. How long does a single arithmetic operation take here, so the microseconds above can be converted into work.

Every measurement runs each side twice: once written the obvious way, and once with that platform's own documented fix applied. The second row is not optional. A comparison that leaves out the other side's official remedy is a strawman, and the gap between the two rows turned out to be the most useful number in the project.

The ladder

The client sends 8 bytes of request id plus a payload. The server computes a CRC-32 over that payload and returns 12 bytes. Arrivals are closed loop with exactly one request in flight, so nothing queues and the number is pure round trip. The same payloads come from one splitmix64 stream, the client and the server each get their own core, and every row calls the same timing function. Nagle is off on every TCP row.

The five stacks, each in both its obvious and its fixed form:

stack	written the obvious way	with its documented fix
Python HTTP	stdlib server left at HTTP/1.0, new connection per request	HTTP/1.1, one persistent connection
Python WebSocket	`compression="deflate"`, the library default	`compression=None`
Go gRPC	one unary RPC per request, stock client	one bidirectional stream, tuned windows
Go UDP	unconnected socket, destination named per send	`DialUDP`, enlarged socket buffers
Go HTTP	`DisableKeepAlives`, server closes each response	keep-alives on, one idle connection held

Round-trip p50 in microseconds, with each stack in its fixed form. Two runs, shown run 1 / run 2, taken with the pinned cores 26% and 86% busy. Running them under very different load is the cheap way to find out whether an ordering belongs to the code or to the machine.

stack	128 B	512 B	2 KiB	8 KiB	32 KiB
Go, UDP	36.5 / 35.3	38.4 / 38.0	38.0 / 40.5	40.6 / 40.1	48.4 / 45.2
Go, gRPC	59.6 / 59.5	61.3 / 59.2	58.8 / 60.1	63.5 / 62.6	202.3 / 183.7
Go, HTTP	68.3 / 70.8	69.5 / 69.4	73.1 / 71.6	81.3 / 84.8	96.3 / 86.0
Python, WebSocket	102.1 / 98.6	96.8 / 98.8	98.2 / 96.2	101.0 / 102.2	124.5 / 129.1
Python, HTTP	180.0 / 181.1	172.2 / 187.5	183.7 / 162.3	189.7 / 187.9	199.2 / 195.4

Python HTTP written the obvious way is 316.6 µs at 128 bytes. Go UDP with its fix is 36.5. That 8.7x is the full width of the ladder.

The Go HTTP row is there to split a confound. The five stacks change language and transport at the same time, because that is how the choice really arrives. Nobody picks Python and then writes a custom UDP protocol. But a confounded ladder is a language benchmark wearing an architecture label, so Go HTTP uses the same transport as the Python HTTP row and the same language as the UDP row. Comparing across it: the runtime is worth about 2.3x and the socket about 1.9x, at every payload size, in both runs. The runtime is the bigger half.

Two places the ordering breaks

Python WebSocket without its fix beats Python HTTP without its fix at small payloads and loses at large ones. They cross at 8 KiB, and both runs put the ratio there at 1.01 and 0.99, which is about as cleanly as a crossover ever gets located. The cause is deflate compressing random bytes: it saves nothing and burns CPU at both ends. Switching compression off is worth 8x at 32 KiB.

Tuned gRPC is second fastest up to 8 KiB and then falls apart, 3.2x and 2.9x worse across a single 4x step in payload. I expected it to degrade. I did not expect it to degrade past plain Go HTTP. Exactly how far it falls did not reproduce between the two runs, so that part is not worth quoting.

The far side costs more than the wire

This is the result I did not expect, and it makes most of the above secondary.

The setup: one server takes 50,000 requests per second, released open loop on a 1 ms tick, for 12 seconds. Most requests are light. Each one runs a filter-and-reduce over a window of 8,192 numbers, adding each element once if it is below a threshold and three times if it is not. Then, one request in every N, a heavy one arrives and runs a 100 ms chain of splitmix64. That heavy burst is byte-identical in every row. Sweeping N sweeps how often the server is busy.

The only thing that changes between rows is how the light request's reduce is executed: one number at a time, the same code compiled for a newer instruction set, or four at a time with SIMD. Making the light path cheaper lowers total utilisation at the same burst rate, and that is not a confound. It is the mechanism being measured.

Two cores is a capacity of 2.0. Here is what each burst rate asks for:

bursts/s	cores asked for, scalar	cores asked for, SIMD	outcome
2	0.68	0.32	both fit
4	0.88	0.52	both fit
8	1.28	0.92	both fit, scalar is close
12	1.68	1.32	scalar queueing
16	2.08	1.72	scalar over capacity
20	2.48	2.12	both over capacity

Between 4 and 8 bursts per second the scalar receiver's median goes from 3.6 ms to 326 ms. The SIMD receiver goes from 588 µs to 635 µs. Same requests, same wire, same machine. One of them ran out of headroom and the other did not.

Peak memory makes the queue visible: 161 MB against 33 MB, because a request you cannot serve yet is still sitting in memory waiting.

So a latency budget written against transport cost is measuring the wrong thing. A 36 µs crossing lands on a receiver whose median is either 588 µs or 326 ms, and the crossing did not decide that.

The top of that sweep is unusable and the program says so. Latencies are stored in a 32-bit nanosecond counter that saturates at 4.295 s, and past 8 bursts per second up to 276,497 of 600,000 scalar samples hit the ceiling. Each row prints its own clamp count, so those points are excluded rather than quoted.

Channels buy you a core, not cheaper work

The setup: a derived-value graph over a stream of ticks, the shape you get in market data or any incremental recalculation engine. A tick arrives, the nodes that depend on it recompute, the graph settles, the next tick arrives. Six levels of four nodes, each node joining two parents below it, 26 nodes in all. Every node keeps state across ticks, so a tick's value depends on the whole history and the ticks cannot be run in parallel with each other.

Four ways to wire that same graph. The ordinary version walks it in topological order on one goroutine. Its documented fix runs each level as a parallel-for with a barrier between levels, which is the only parallelism the shape allows. The channel version gives every node its own goroutine and an unbuffered channel, moving 52 messages per tick. Its documented fix buffers the channels and sends 32 ticks per message.

In wall-clock time the batched channel version wins the whole sweep, 1.81x to 2.10x. That looks like a clear argument for the structure, until you multiply each row by the number of cores it actually used.

Then it is level with the plain single-threaded walk, 0.95x to 1.11x, at every point and in both runs. The direct walk uses 1.0 cores. Everything else uses 1.8 to 2.0. The channel graph did not make the work cheaper. It spent a second core, and it charged 200x to 544x in residence latency for the privilege.

Which also undercuts a common argument for splitting a service. If channels inside one process do not make the work cheaper, moving the same graph across a socket will not either. You pay the crossing on top of work that has not shrunk.

One surprise worth keeping: the relative tail is tighter in the channel versions, not wider. p99.9 over p50 is 20.2x for the direct walk and 1.6x to 2.1x for the channel rows. A 260 ns median turns any interruption into a 20x outlier, while a queue quietly absorbs it. If you write your budget against p99.9, you may prefer the version with the slower median.

What a microsecond buys

The last measurement prices the unit. The kernel is gonum's element-wise divide, dst[i] = x[i] / y[i], run scalar and again with Go 1.27's SIMD package, swept over the number of elements.

One divide costs about 1.0 ns scalar and 0.46 ns vectorised. So a 36.5 µs crossing is worth roughly 36,000 divides. Nine times is not much spread across the ladder, and a boundary is still expensive in work terms however well you build it. Both are true, and they answer different questions.

Vectorising only pays between about 8 elements and the point where the working set leaves L3. Below 8 it is up to 4x worse, because a masked partial vector costs more than the scalar iterations it replaces. The two runs disagreed about the upper edge, so I am not claiming one.

Why it is built this way

Splitting a program into services buys real things: deployment independence, team autonomy, blast radius control, scaling one part without the rest. Those are worth paying for, and this measures none of them. They are the half of the trade that gets argued loudly while the price gets asserted rather than measured, so this prices the other half and leaves the benefits alone.

The hardest objection is that a ping-pong is not a service call, and it stands. One request in flight, no TLS, no real object being marshalled, no retries, no pool contention. Several of those cost more than the socket does. So every crossing number here is a floor: the least it can cost once you have decided to have a boundary. A floor is still worth knowing, because an argument for a boundary that fails at its own floor fails everywhere else too.

The second objection was that contention inflates the spread, and replication settled it. The two ladder runs started with the pinned cores 26% and 86% busy, and the headline read 8.67x and 8.97x. Tripling the neighbour load moved it by 3%. An earlier version of this page blamed the slow rows for holding more cores, quoting figures the ladder does not actually record. That claim is withdrawn.

Why you can believe any of this

Every variant folds its work into a checksum and the runner refuses to print results when they disagree. The ladder gates one checksum per payload across two languages and four transports, so the Python and Go rows provably computed the same thing.
Cores are pinned, and a run that cannot pin raises instead of continuing. An unpinned run looks identical afterwards, which is what makes it dangerous.
The SIMD binaries are disassembled at build time and the instruction counts go into the result file. Scalar rows hold zero packed divides, vector rows hold six.
Each comparison keeps the losing side's own documented fix. That is how I learned GOAMD64=v3 buys almost nothing here.
Predictions went in before the runs and got scored after, wrong ones included. The transport model called both crossovers and put every absolute cost too high. The contention model had the sign backwards: it expected the SIMD advantage to shrink under load, and it grew.
Every result file carries its own CPU, kernel, governor, pinned cores, starting load, the toolchain versions that built the binaries, and the checksum each gated point agreed on — all written by the runner, never by hand.
The ladder has now run three times, with the pinned cores 26.4%, 85.9% and 99.5% busy at the start. The headline spread reads 8.67x, 8.82x and 8.97x, and both ends keep their identity across all three. A 3.8x change in neighbour load moves the result 3.5%, which is the strongest evidence here that the orderings survive this box.

No result in this project is verified. The runner judges that on the busy fraction of the pinned cores against a 10% limit, and those cores have never been under 25% busy when a run started. So every file says structure, and this box cannot certify a baseline at all. An earlier gate judged the load average instead, which counts runnable threads rather than CPU time and so graded the workload instead of the machine — a variant that spawns sleeper threads drove it to 81.6 on a two-CPU box with eleven tasks runnable.

What I have not measured

The specialist transports this ladder was aimed at: a custom UDP cast at 8.8 µs, MoldUDP64, SoupBinTCP, KCP, Aeron. Rung three here is still plain UDP.
The channel comparison in anything but Go. A Python rung now exists and folds the same checksum as the Go rows, but it has not been swept, so I still cannot say whether the sign survives a different runtime.
The cross-language rows of the contention measurement. Their Rust twins now exist and every case agrees with Go on a checksum, but the inherited operating points ask for 3 and 6 cores against this box's 2, so they saturate the latency counter and need retuning before they mean anything.
A quiet machine. One measurement has a run that started under the load limit. The rest are filed as structure, which means shape rather than baseline.