🔥 The DB Grill 🔥

Well now, this was a delightful trip down memory lane. It's always a treat to see the old "best practices" from the lab get written up as if they're some kind of universal truth. It truly warms my heart.

The server classification—small, medium, large—is a particularly bold move. It’s so refreshing to see someone cut through all that confusing noise about CPU architecture, cache hierarchy, and memory bandwidth to deliver a taxonomy with such elegant simplicity. Fewer than 10 cores? Small. I'm sure the marketing team loved how easy that was to fit on a slide.

And the decision to co-locate the benchmark client and the database server? A masterclass in pragmatism. I remember when we first discovered that little trick. You see, when you put the client on the same box, you completely eliminate that pesky, unpredictable thing called "the network." It's amazing how much faster your transaction commit latency looks when it doesn't have to travel more than a few nanoseconds across the PCIE bus. It makes for some truly heroic-looking graphs. Why would you want to simulate a real-world workload where users aren't running their applications directly on the database host? That just introduces... variance. And we can't have that. Plus, as the author so wisely notes, it’s "much easier to setup." I can almost hear the sound of a VPE of Engineering nodding sagely at that one. 'Ship it!'

But the real gem, the part that truly brought a tear to my eye, is the guidance on concurrency. The insistence on setting the number of connections to be less than the number of CPU cores is just... chef's kiss.

Finally, I usually set the benchmark concurrency level to be less than the number of CPU cores because I want to leave some cores for the DBMS to do the important background work, which is mostly MVCC garbage collection -- MyRocks compaction, InnoDB purge and dirty page writeback, Postgres vacuum.

This is such a wonderfully candid admission. For those not in the know, let me translate. What's being said here is that you must gently cordon off a few cores and put up a little velvet rope, because the database's own housekeeping is so resource-intensive and, shall we say, inefficiently implemented, that it can't be trusted to run alongside actual user queries without grinding the whole machine to a halt.

It reminds me of the good old days. We had a name for it internally: "feeding the beast." You couldn't just run the database; you had to actively reserve a significant chunk of the machine's capacity just to keep it from choking on its own garbage. The user-facing work must graciously step aside so the system can frantically try to not eat itself. It's less a "benchmark" and more a "managed demolition."

It's a beautiful strategy, really. You get to publish numbers showing fantastic single-threaded performance while conveniently ignoring the fact that the system requires a dedicated support crew of CPU cores just to stay upright.

Anyway, this was a delightful read. It brought back so many memories of roadmap meetings where we'd plan to "fix" the background work in the next release. And the one after that. And the one after that.

Great stuff. I will now be setting a filter to ensure I never accidentally read this blog again. Cheers