Will CCS dominate all?

John West asks this question in a short article on HPCWire. In it he posits a few things. It is worth looking into them.

First, his title is provocative, “Windows CCS and the End of *nix in HPC”. Not that this is an issue, he needs to draw readers in. Second, he shows what he is interpreting to be the fundamental conflict, that is, between end users who just want to get things done, as typified in his quote from the person who bought a windows cluster, and Don Becker, who did a pretty good job of jump starting the industry. I disagree that this is the fundamental conflict, or even that these two people are typical (sorry Don!) of users. More about that in a second. Third, John quotes from someone at Unisys. This quote is again, fairly typical of what we hear from marketing folk, it is yet another attempt to draw a box around a group, attach a label as in “it is like a religion for them”. I disagree with this characterization. Will go back to that in a bit as well. John also takes issue with those who “blithely dismiss Windows CCS as a marketing stunt” indicating that they are missing the major shift in the industry. I am going to disagree with this, quite strongly, and will back that up in a moment as well. He is right in that HPC has been pushing out of its traditional boundaries for quite some time. It has been going downstream for decades. It will continue to do so. First with large monolithic supers, then supermicros, then clusters. Is this the end of the line? Nope. He is right as well, in that Enterprises run windows. The also run an increasing amount of Linux. A non-trivial increasing amount of Linux. Which is, by the various measures available (IDC, Gartner, and others who aren’t exactly friendly or thrilled about this, but have to grudgingly admit it), is taking share not just from nix, but from Win. He is right that there is no element of computer research in this. They just want to use HPC. With as little pain as possible. With as much payback as possible. This is what our customers tell us. This might be why more than a few of them have stopped using windows on their desktops are focusing upon using Linux on their clusters. We are aware of users who will not or cannot change due to corporate dictates. One OS for all, as that is what the CIO believes will be the lowest cost to support. It has nothing to do with the user in the enterprise, and everything to do with the standards that the CIO and other responsible parties push. Well CIO’s come and go. So do standard platforms, standard players, and so on. Enterprise is more of a process than a specific product. Its how do you make sure you can support and continue operations. How can you replicate operations. How can you sustain operations. All without breaking the bank. Enterprise is not “how do we deploy Microsoft solutions.” Its about payback on investment. So if one investment has a lower upfront cost and higher returns than another (due to performance issues), why wouldn’t it be considered? This question is what is driving Linux, hard, into enterprise. As John notes correctly, those ignoring it, do so at their own peril. Further he indicates that for some, CCS will be just what they need, and will create an “immediate huge commercial driver for hardware vendors to develop solutions that support CCS.” I take issue with the latter portion of this. An immediate huge commercial driver for software and hardware. On the software side, the current list of applications on CCS is about the same (several new ones, but not many) as when CCS released. I argue that they “cheated” on the release in that they used applications that already existed on the desktop. And this is, as I have pointed out, their albatross. Why should a software vendor increase their costs in order to support a CCS specific version of their application, when they cover windows via the desktop version? The CCS version needs to sell well in order to justify that effort. Or Microsoft needs to pay them to port (which is happening from what I understand, though this is 3rd hand, so take it with a grain or two of NaCl). The paying-to-port model is not sustainable in the long term, even if you have infinite cash, like Microsoft. CCS selling well is what will drive ISVs there. This is precisely the question we have been asking. Available data suggests rather strongly that it is not selling well (IDC sound-bites and others). More about this in a bit. John also posits that the huge commercial driver in the hardware space will be the demand, and that will kill the old line HPC manufacturers. The first part of this appears not to be true, Microsoft is apparantly paying IHVs (independent hardware vendors) to push CCS. Providing them marketing money. Again, that infinite bank account. The huge commercial driver has nothing to do with customer demands, or we would see it in the uptake numbers. It has to do with bank account size. This is what could turn the tide. It will set some serious expectations in the IHV side, and yes, will skew the market badly, but I disagree with what he posits here. Once the payments stop, what happens to the IHVs who have been getting say $3M/year? Thats a loss of revenue. Lots of revenue. Now do this across an entire market. “Skew” may not be a strong enough word. As for what is killing the old-line HPC companies? Simple. A deep and thorough misunderstanding of the market, what drives it, where it is going. A deep belief that customers are willing to pay more for performance. They are not. They get performance free by waiting. They have for a long time. They want disruption. They want an order-of-magnitude or better performance, for the same price. Or less. None of this has anything to do with OS, apart from the OS that helped enable the HPC revolution being the OS that has been driving its growth for the last decade, and shows no signs of letting up. John does have a good point here, which I will also take issue with

Several implicit assumptions are being made in here. Will get to those. But first it is worth noting that we can replace the phrase “Windows CCS” with Linux, and curiously, we have what appears (from IDC, Gartner, others) to be what is in fact happening. The implicit assumption is that “Windows CCS will grow”. More specifically it will grow faster than demand in this market (the replacement philosophy). Current data puts the growth in the noise in this market. Strong commercial drivers would have given us a signal already. Strong adoption would have pushed ISVs to say “hey this is a good market, we should serve it as well, as our desktop version won’t be adequate here.” Microsoft will have funded some of the latter, but as I like to point out, SGI had Irix in a dominant position in the scientific computing market at one point in time, in part by helping to defray costs of support and maintenance. Help me spot the Irix support today. OTOH, no one pays to have Linux support in the ISVs hands. As noted, this asymmetry will have long term consequences: ISVs choose to support Linux, as it dominates HPC. They may be paid to support CCS. They all but ignore Solaris. There are reasons for all of this. For that matter, current HPC is a $10B+ market. Clusters are ballpark, 60% of it. And growing hard and fast. More than 20%/year. Have been for quite a while. But it all comes down to the customers. We see our customers using Linux in more and more sites, not just the clusters, but the back end servers, and now the desktops. IDC’s and others numbers bear this out. A plan to support CCS is a good idea. We follow our customers lead, and we see where they are taking us. If they pull to CCS, great, we will support it. If they pull to Solaris, great, we will support it. If they pull to BSD, OSX, DOS, yes, we will support it. But they aren’t now, are they. Moreover, they show little sign of doing this in the future. As a solutions provider, we have to be ahead of the curve to an extent. We will follow where our customers want to go, but we also have to look at adoption of platforms, and this gets to our earlier point. For a P&L; center to be successful, obviously, the P’s have to be larger and sustainable as compared to the L’s. We can’t go build our codes (accelerated computing bits) on a platform which doesn’t sell. You won’t see us building our apps on Solaris without Sun paying us, as there is just no demand. We will not invest in platforms for which we don’t have a good “warm and fuzzy” on our return. For Linux, it is a no-brainer, the market is growing fast, and there is strong demand and market forces pulling adoption along, diffusing it into the rest of the enterprise. For windows, this is a desktop market. CCS would be a tiny tiny fraction. Microscopic. This is the problem Microsoft faces, as the desktop product may be what runs on CCS. In which case, why should we invest in CCS when the desktop product is so obviously the large market? Why should we incur additional investment costs, and not likely see a corresponding return? Put another way, the business model is broken for this. An ISV who looks at providing a CCS code needs to do it in addition to providing a desktop code. They might get a return on the desktop code investment. They need CCS to sell strongly, and provide them a market for recouping the investment in the CCS code. With Linux, I run the same code on the desktop, and on the cluster. This is not lost on the ISVs. This provides good performance on both. It lowers their costs. I respect John, I just disagree with his thesis and a fair number of his points. The CCS business model is broken as currently construed, and I am concerned about it. I think I know how to fix it, but Microsoft hasn’t made a habit out of listening to its customers, and potential partners. That and we have given away enough “free advice” (you know, worth every penny!) over time. The business model isn’t sustainable as a separate P&L; entity. Selling thousands of compute nodes worth as Patrick had insinuated gives us a feeling for the size systems that they have sold, and the percent market share from a bottom up approach. The top down numbers agree with the analysis within about a factor of 2 or so. Being generous, and calling it 10k nodes over the last year. Assuming a cluster size of 32 nodes (on average), means 313 clusters. This 10k nodes also translates into $4.7M in OS revenue. Assuming $5k nodes, this comes out to about $50M in windows clusters last year. Remember that the cluster portion of HPC was more than $6B last year. Lets ignore growth and call it $6B this year during the time when CCS was being pushed. $0.05B/$6B is 0.8% of the market. IDC had predicted “a few percent”, as had Gartner and a few others. Yeah. It is a few percent. The noise signal (error bars) is larger than the data signal. If there were strong commercial drivers, we would expect this number (data signal, not noise signal) to be higher. Say 100k nodes sold last year. They we would be talking 8% of the market. And that would be news, and that would be something we would be excited about as a new platform. But reality on the ground has a neat habit of squashing people’s conceptions of what reality should be. Or might become. I will watch this market again this year. Like many other solutions providers, the vast majority not being paid by Microsoft for ports, we are taking a wait-n-see attitude. I don’t think this market is theirs to lose. On the contrary, they see that the broader enterprise market is in play when they thought they had it locked. The desktop market is coming into play, one which they had locked up. I don’t have any illusions on Linux, something better cheaper faster comes along (I keep hearing *BSD), then it will fall by the way side. Its economics pure and simple. The market demands what the market demands. Sales follow that demand. Companies well positioned to meet that demand will thrive. Companies scratching their heads as to why demand went that way will not, or at least their products won’t. The market demands the intersection of decreasing costs, increasing performance, and security. The market sees that Linux provides all of these, across HPC, Enterprise, and Desktop. The old guard on desktops is having issues grasping this. So they try to spread into HPC after noticing rapid encroachment in Enterprise. I do agree though, ignoring market realities is done at one’s own peril. Don Becker might not be a typical user, nor might the other person be. Our typical user just wants to get stuff done. They don’t care about the OS. It is just a platform to them. We don’t care in as much as the OS choice does not get in the way of them doing their work. Increasingly we are seeing a pull in of Linux desktops to replace windows desktops. Sometimes this is for interoperability, sometimes it is for the convenience of interface. But it isn’t about “religion” as the Unisys person states. Its about using the right tools for the job. It always has been, and bringing some sort of fanatical devotion to a platform in as an argument obfuscates what is really happening. It is a gross overbearing generalization, and as the recursive joke goes, it tends to be incorrect. Windows CCS as a business has a weak case for it at best. We pointed this out an analysis above and previously here. Our concern is that if we invest in it, our money is gone, our investment is non-recoverable due to the size and realistic growth opportunities for this market. Every single ISV needs to ask precisely the same question. Many are. And that is why they are taking a wait and see. Paraphrasing John, but using the corrected numbers above

Yeah, I agree. It is the end of something, but as Mark Twain once quipped, the rumors of death are greatly exaggerated. Moreover, ignoring Linux is done at the peril of the IHVs and ISVs. This is where the market growth is; it is large and growing fast. Declaring it “dead” seems to go against data that I am aware of from multiple sources. Adding to this, lots of ISVs appear to be aware of it as well, as surveys of them (posted earlier here) seem to show that they are more interested in developing new products for Linux target markets. Now why would that be? Again, I respect John, I simply disagree with the points in his article.