Architecture matters, and yes Virginia, there are no silver bullets for performance

Time and time again, the day job had been asked to discuss how the solutions are differentiated. Time and time again, we showed benchmarks on real workloads that show significant performance deltas. Not 2 or 3 sigma measurements. More often than not, 2x -> 10x better. Yet … yet … we were asked, again and again, how we did it. We pointed to our architecture. But, they complained, isn’t it the same as X (insert your favorite volume vendor here)? No, we pointed out. It isn’t. We described the differences. Showed them precisely how the differences manifested. Showed them that the results are normal, repeatable, and generally different from what others made claims about. Often in the past, we’ve heard that (insert random vendor here) has comparable systems. And when the real world measurements come out, we hear a very different message. Or when a customer eschewed our solution, went for the brand name version (at much higher cost), they rapidly discovered that engineering by spec sheet is a very … very bad thing to do. It doesn’t work (engineering by spec sheet). Yet, we heard this all the time. In the past, I’ve railed against the notion of silver bullets. A silver bullet is a magical component, hardware or software, that will suddenly make something go much faster, and you know, give a competitor an unfair competitive advantage. Marketing people love their silver bullets. They don’t work, but hey, they are fun to talk about. How do we know they don’t work? Easy. Decades of benchmarking against them. Running real applications against them and our kit. We designed and built something quite good. It enabled us to build parallel IO engines, tune the heck out of the engines. Move tremendous amounts of data between process/memory complex, storage, and RAM, without bottlenecks of other designs. Despite protestations and spec sheets to the contrary, measurements I and many others have done have demonstrated sustained and profound advantages of superior architectures over the silver bullet enhanced architectures. I see spec sheets and marketing blurbs on products proclaiming them to be “the fastest” stuff in the west, with numbers that are lower … lower … than numbers we surpassed more than 3 years ago. Yet, we are told by some that these products are comparable. Or, even more (unwittingly) humorously, that there really is no difference, even though, in a number of cases, we had just demonstrated a profound (nearly order of magnitude) difference. It astounds me. No, confounds me … this may be a better way to articulate it. We’ve not simply created a better mousetrap, we’ve tried to tell the world about it. And been ignored. And we tried to get folks to invest in this. And been ignored. All the while the market is going on validating our ideas (dense and high performance systems), and we see VCs investing in things like, I dunno … Secret? This gets to you after a while. You start questioning a number of premises you had held to be truth. So here we are, with (what I’d argue what is) a fantastic architecture second to none. And despite the simplicity and obviousness of our message, our (many and repeatable and sustained) measured results … we get people reading off a marketing spec sheet telling us we are not all that different. Though we are. This is one of those inflection points in a company’s existence. I’ve been asked multiple times in recent months to estimate what we could do if we took our stack to other hardware. Apart from the significant performance (and likely stability) loss, such that we’d be like everyone else, not much. I’ve also been asked multiple times to “divulge our secrets”, though our architecture is open, our kernels are available online. As I said, it gets to you. I am thinking hard about this battle, and whether or not I want to keep fighting it. Our kit is obviously, objectively better. And not by a little bit. But it doesn’t matter if we can’t sell it, because people read spec sheets and think the numbers printed on it are what they will get in normal operational states, versus the best case scenarios that are specifically set up for those parts. A friend noted the fallacy of engineering by spec sheet a while ago. They are right. smh