TCO: The sorta kinda but not-really argument, part 1 (the TCO study)

One of the more annoying parts of writing stuff for consumption online is that, every now and then, someone with an agenda, a very obvious agenda, will go off with weak arguments. One of those weak arguments that we saw yesterday was TCO. Not that TCO is a minor concern, it is a real, significant concern for management. How much something costs in the end is a sorely needed datum for any enterprise, company, entity, to make rational and realistic decisions. That said, there are ways to use TCO, and there are ways to abuse and FUD with TCO. We saw my attempted fisker (I was the fiskee? I dunno, we need a new jargon here) hand wave, poorly, the TCO argument. We saw a weak argument, that companies now must hire new staff to support Linux. Allow me to deconstruct and eviscerate this argument. It is wrong, it is illogical, it is weak. It is also the most common FUD we hear, so it is worth spending time to do this.

The major point of those raising this FUD is that Linux costs more to administer and run over the lifetime of the system. Assuming the hardware costs are the same, they start attacking the administrative costs. Their arguments fall into several broad categories.

1. The TCO study said so
2. Everyone knows how to administer Windows systems, and no one knows how to administer Linux systems
3. You will have to hire more expensive staff to handle these systems

There may be others that I have missed, feel free to leave a comment, and we will cover them.

The TCO study said …

As we noted, there are several TCO studies out there. Numerous. Some of them (2004 time frame and before), such as the Yankee group indicate that Windows simply costs less to administer. The claims are varied, but one of the take home messages is

Unfortunately for this study, its methodology appears to be questionable. [

](http://www.techworld.com/security/news/index.cfm?newsid=3435) More to the point, BusinessWeek did a better job of outing the problems and the bias of the major author, Laura DiDio

Moreover, specific methodological problems are indicated. From this link we see

Ok. It is a self selecting study, on a windows focused list, of windows admins, responding to what they think of Linux vs Windows. This needs to be understood by anyone reading that. If it helps, the first 5 people wanting to read the study, if you send me your snail mail address, I will send you something that you ought to take with this study. The BusinessWeek blog goes on.

And more to the point

Remember, the TCO study take home message was that Linux was more expensive to deploy and manage. But their own data appears to contradict that. The article that all these quotes came from was a blog posting that did a fairly good job at eviscerating that TCO study. But those aren’t the only folks that took issue with it. So did TechWorld. Their criticisms are also quite apropos, and biting

So lets do the same thing with say, I dunno, AIX and Solaris. Lets ask AIX admins whether or not a switch to Solaris will cost more. Such a survey would be as valid as the Yankee Group version, which is to say, it is of dubious value at best (it is a stretch to assign dubious value to it, I am being generous). The survey of course ignored many other elements of the benefits analysis, specifically elements of the democratization of support (e.g. creating real competition for support services)

Thats right, there is no vendor lock in on the OS. We have switched from RedHat to SuSE in the past. We are examining a switch to or an addition of Canonical (Ubuntu). This is some of what the Yankee group study missed. The bigger picture and how TCO fits into it. The horizontal scaling (e.g. replicating the Linux is free, in Windows, you have to pay for more licenses and CALs). Add in the bias of the principal analyst, the weak methodology that many others have done a much better job of discussing than I. So when advocates and marketeers haul out this study, and they will, please, feel free to point them to the multitudes of critiques of it. Lets simply say that a fork has been stuck in this particular study. It is done, shown to be problematic in methodology and conclusions. It is not a study that someone seeking to make a valid point would knowingly or willingly rely upon to make their point. Paraphrasing Monty Python, it is an “ex-study”. Next, the Forrester study. I did not see this one hauled out, but the TechWorld article linked above also decimates this. There is a study published by Cybersource, that they updated after the Yankee group study came out. I looked for online criticism of the study but found none with a googling. If you find any, please bring it to my attention. In short this study created a model organization and examined the costs to support it. In the study, Cybersource notes that not all organizations are the same, and as theirs is a model of a particular size, may not accurately reflect all real situations. Hey, thats brutal honesty. They are telling you that you need to be careful in drawing conclusions which you extrapolated to from this study without a detailed accounting of your own costs and design plugged in. Also, Cybersource comes out and indicates, in the executive summary, where they are coming from. Go read the report, I won’t summarize. One of the elements I really enjoyed about it was this:

That my gentle reader, takes guts. Putting out the model, the datum, and the methodology, not hiding it, and showing every step is the hallmark of something well thought out. Don’t trust. Verify. But that whole line about “As an organisation considering the TCO differences between Linux and Windows, you should ultimately not trust anyone’s TCO study but your own” is critical. Advocates for windows solutions are going to haul out TCO after TCO that claims to show Linux will cost more. If they haven’t studied your organization, they really don’t know what it will cost you. If your organization and other elements are not well modeled by their assumptions (most are not), then you can’t point to them as being typical of what you would get. Just like standard benchmarks, you really need to take any TCO study that isn’t your own (well designed and modeled on your organization, measuring things of real value for you), with a few kilograms of salt. Anyone at all trying to convince you that their TCO report is valid is trying to sell you something. In this Cybersource study that they tell you not to trust, but to verify on your own, we see significantly lower costs for Linux deployments and support costs as compared to Windows. Again, their model may not reflect yours. You can’t read this (or any other study) and claim it is universal and the definitive word. But do read this study. Really. They have a well thought out and well reasoned set of arguments and detailed discussion on other TCO studies, the good, the bad, and the ugly. Cybersource notes that while you should not think of any study as conclusive, their review of Soreon’s study, also mentioned in a previous post, appears to provide similar methodology, independence, and comes to similar conclusions.

Some of the more amazing aspect of this study, the one you should not take as gospel, is that they modeled the costs for Windows and Linux consultants differently. Their words

Read that last paragraph again. And again. Now remember Dan’s little FUD gem from yesterday. And remember that in their model, this study found, for the modeled organization, the costs to be lower for Linux over the lifetime. Remarkable. As I indicated, this study is worth the read. Yes, Cybersource is an open source solutions company. They indicate as such throughout their document. Their study provides all the goes-in-to’s and goes-out-of’s (pronounced gah-Zin-tas, and gah-Zout-uhfs), that is the things that go into the model, the things that come out of the model, and the model. You don’t have to believe it, you can crunch it your self. None of this, the open source solutions company, the model, etc. invalidates their work. Whether or not their work is applicable to your organization is not as relevant as the fact that they walk you through how to do the TCO analysis. You can use their results (not advisable for any TCO study as they note), or crunch them yourself. But this isn’t the only set of studies that conclude that Linux is less expensive. Several studies that you will not hear the windows people talk about include the following. The Robert Francis group study is one of the major ones. IBM talks about it on a web site. Do remember that IBM sells far more windows than linux. One might argue that this fact biases them in that direction. The RFG study has on it, title page, in big bolded letters so that there can be no misunderstanding:

IBM sponsored this study and analysis. This document exclusively reflects the analysis and opinions of the Robert Frances Group (RFG) author, who has final control of its content.

Yes, IBM paid them to study it. Draw your own conclusions as to whether or not is biased, given that IBM sells more windows than anything else. IBM writes:

This is specific to J2EE application servers. Whether or not it carries over to other systems is unknown. But there are some issues for our next area, we will touch on in a moment. IBM also mentioned a study by Pund-IT. This examined use cases of several organizations. This study is not as detailed as I would like. It does represent an overview of real world use and deployment cases though, and is useful in that context. To be fair, Microsoft will likely have several counter studies, that show how windows saved them time and money … er … over windows, and Linux of course. Again, useful datum may be found in these reports, as long as they were not scrubbed by marketing prior to publication. Every now and then, one of these comes out with some interesting real facts buried within, that somehow have escaped the de-information machinations. Your job, should you choose to accept it, is to explore this, and see if you can spot them. Use case can be informative, but you have to remember you may be reading it through a marketing RCG (read as: Rose Colored Glasses) filter. Back to the IBM article. There are some interesting notes in here from the interview of the authors of these studies.

Paraphrasing Bill Clinton’s election mantra “Its the cost, stupid.” TCO is an aspect. There are holistic elements that the TCO for a single server will not model. Such as the horizontal scaling. Such as hardware reuse (e.g. “older” fully depreciated hardware which is still supportable costs you very little to run once you pay the depreciation). It’s the cost. TCO is a factor in it. Continuing on the TCO thread.

So there we have the TCO survey roundup. As I pointed out, as others have noted, TCO white papers ought to be taken with considerable skepticism. Specifically the Yankee Group report has been effectively eviscerated. If they won’t present their raw data, their model, their assumptions, their rationalization and reasoning, they cannot be trusted to be well representing what they actually saw. Such internal inconsistencies appear to plague the Yankee Group report as indicated in the BusinessWeek article. Other TCO studies of the same ilk are suspect at best. The TCO studies I noted above are in fact available for perusal, and do not appear to hide raw data or methodology. Whether or not you agree with their conclusions, they present their analysis in highly specific detail for you to evaluate them. Regardless, if you insist on believing these studies without going through a detailed study on your own, note that it appears that the majority are showing slight to significant benefit in TCO to Linux. That is, if you go by the studies, windows advocates wouldn’t say

But of course, this assumes that such advocates are in fact aware of the controversy around the studies they hawk, and the thorough evisceration these studies have been dealt in the press. Take home message: Try not to buy into such studies without thoroughly understanding them, and their limitations, and their applicability to your situation. Doing otherwise will leave you with egg on your face. Lets call this one done.