benchmarks

an experiment log

tldr

A place to ask one measurable question at a time and keep the answer. Every comparison is two real programs doing byte-identical work, proved by a checksum, on pinned cores. The finding is never a single number. It is the size, depth or load at which the winner changes.

Most benchmarks I read fail in one of three ways. They compare two programs that are not doing the same work. They report a mean and hide the tail. Or they quote one number that was true on one machine on one afternoon, and it rots from there.

So the rules here are dull on purpose. One axis moves and everything else is pinned. Both sides fold their work into a checksum and the runner refuses to print anything when the checksums disagree. The losing side keeps its own documented fix in the table, because a comparison that omits the other side's remedy is a strawman. And the prediction goes in writing before the run, then gets scored afterwards, wrong ones included.

Projects

01 — what does a service boundary cost?

four measurements, three of them run twice — partly answered

The ladder from a function call to Python over HTTP is far shorter than the argument usually assumes. The largest wins came from fixing a wrong default rather than changing stacks, so the first answer is tune it and not rewrite it. And the receiving side's own code mattered more than the wire did. The figures are on the project page, each beside the conditions that qualify it.

How a number earns its place

Twin implementations fold a checksum over the work they did. Mismatch means no report, not a footnote.
Cores are pinned. A run that cannot pin raises rather than continuing, because an unpinned run looks identical afterwards.
Low, median and high, never a mean alone. The spread is usually telling you about the scheduler rather than the code.
An ordering that does not survive a second run under different load is not quotable, and saying so is the honest result.
Every result file carries the CPU, kernel, governor, toolchain, pinned cores and starting load, written by the runner rather than by hand.

This box is a two-CPU slice of a Ryzen 9 5950X with an unreadable governor and neighbours that never stop. No result here is verified and none can be: the runner judges that on the busy fraction of the pinned cores against a 10% limit, and those cores have never been under 25% busy when a run started. Every file says structure, so no absolute figure here is quotable.

What survives is the ordering, and it survives well. The transport ladder has run three times with the pinned cores 26.4%, 85.9% and 99.5% busy, and its headline spread moved 3.5% across that. The orderings are the result.