On expectations

This has happened multiple times over the last few months. Just variations on the theme as it were, so I’ll talk about the theme. The day job builds some of the fastest systems for storage and analytics in market. We pride ourselves on being able to make things go very … very fast. If its slow, IMO, its a bug. So we often get people contacting us with their requirements. These requirements are often very hard for our competitors, and fairly simple for us to address. We’ll get inquiries like this:

We'd like 250TB of storage, replicated, and we need to sustain 10GB/s writes, and 10GB/s reads.    Can you do this?

I made up those numbers, but they are around the same order of magnitude in many cases, and the first digits are also quite similar. We know what is possible. We know what homebrew/self built systems behave as. We know the ins and outs of making this work. So we start with a spec, work up a few config/design variants to address this, and offer a spectrum to the person whom contacted us. A quick segue here. Very high performance, very high efficiency is hard. You can’t simply slap components together and hope it will work. As you quickly discover, it doesn’t. Moreover, it is worth noting that most people read spec sheets and presume … really … presume … that they are going to get the maximum performance of the device … all the time, under all conditions. Many people don’t quite have a mental model of the connection between the IO/computing/network load patterns and the perceived performance. And also, as part of this segue, they don’t really … have a clue as to how much performance implementations will cost. They look at the consumer grade SSD with 80GB of write per day, do a quick bit of math in their heads, and come out with a number they think will work. And then they come to us. Back to the expectations discussion. So the people with this number of what they think the cost for their 10GB/s read/write system would be. And they tell you. Since we design and build these things, we actually have a pretty good idea of the actual costs involved. The costs … our cost for materials that can actually meet the requirements when assembled into a system … are often significantly larger than their perceived cost. Its … almost … depressing. Not that we are going to lose their business. We run a fairly tight ship, and are very aggressive on our pricing. We like repeat customers … this is how we live and grow. But … But … in these instances, we would have to subsidize 3/4 or more of the unit for them. What makes this sadder is that these are often very well funded startups or large companies doing this. In the past it has been universities and research labs. I do canvass the market fairly regularly, to see if I am missing something, and to see if someone magically came out with a 10 drive write per day SSD at under $0.05/GB that sustains 500-1GB/s and 100k IOPs on 12g SAS. There seems to be a disconnect between what people believe they’d like to pay, and what it actually costs (even in raw materials). I know, prices are not really firm. A market is made when a buyer and a seller agree upon a price, and the price may not necessarily reflect portions of the cost of the item. Performance is a valuable feature. More-so than ever in the past. Being able to design and build high sustained performance systems, and deliver appliances that provide this high performance is a valuable service. I am not quite sure what to think about this disconnect between reality and people’s expectations. I’m respectful and open with the people doing the inquiry. I help them to understand where they should be looking at for their budget. But we can’t afford to pay people to take our solutions. A few years ago, a potential partner had come to us with an opportunity at a national lab for a sizable system. We looked at the specs, and then the budget. The lab wanted the highest end kit, of course. You know that. Their requirements specifically called out what you could or could not do. Then came the budget. When we looked at it … the pricing was below the lowest end raw disks in market (dense consumer grade drives) that we could get in bulk. Speaking on the side with some of the OEMs, and they completely blanched at providing low margin consumer grade units at these prices, never mind the high margin, highest end units. Someone did eventually “win” this business. But these wins are pyrrhic. Enough of them and they will go out of business. They had a layoff sometime after this was delivered, who knows if there was a connection. End user is happy because they got a fresh new system at high spec, for a price well under the market rate … well under the actual part costs in the system. The vendor isn’t happy, as they not only lost money on the deal, but thanks to the language around these deals, they can’t do any real marketing, so the win is … well … of low value/quality. We read the spec, and did a no-bid. We can’t afford “wins” like that. I dunno. This stuff bugs me. Real performance will cost some money, and you need to likely have a range of performance concepts in mind to compare to your budget.