Setting expectations for SSDs versus Flash

Nomenclature: SSD is a physical device that plugs into an electrical disk slot. Flash is a PCIe card. Both use the same underlying back end storage technology (flash chips of SLC, MLC, and related). I’ve had a while to do some testing with a large number of SSD units in a single device. I can give you a definite sense of what I’ve been observing. First: SSDs are, of course, fast for certain operations. Second: there’s a whole lotta er … marketing numerology … around SSDs. Ok. So imagine we have 48x very late model Sandforce 22xx equipped SSDs, in a single chassis. Call this thing an SSD array. Imagine that we’ve done some experimentation on various RAID cards. Including some from a vendor that has not been announced/released. Including some dumber HBAs. If we believe the underlying theory behind SSDs, and aggregates of the same, for reasonable configurations of SSD (that does not mean RAID0s, but RAID5’s with a smaller chunk size), we should be able to approach the theoretical maximum number of IOPs, assuming that the RAID calculation engine can keep up with the SSD. This assumption is, sadly, incorrect. Theoretical maximum IOP rate for this unit, assuming the vendors don’t … er … embellish too badly … is about 2.4M IOPs. After putting them into RAID5’s best possible case should be 2.1M IOPs. What do we achieve?

Roughly 1/10th that number. For 8k random reads across files much larger than RAM. Well … thats using the HBAs. Using the RAID cards, its 1/30th that number. And we see that 1/10th number again and again in our measurements. Doesn’t matter what we measure, the IOP rates never come close to the theoretical max, and are always hovering around 10% of it. This suggests a nice rule of thumb. Take whatever they promise, and shift the decimal once to the left. That will be more along the IOP rate you will achieve. What about with Flash cards? There we have seen numbers from 100k-1M IOPs. For the Virident cards, we’ve measured, the numbers quoted were what we achieved without heroics. Remember I indicated I found a very legitimate reason for PCIe cards? Well, its IOPs. Streaming … not so much, we can get decent stream performance out of spinning rust. No real advantage to SSD or Flash there. But IOPs … yeah … IOPs. SSDs are in the $2.50 +/- 0.5 USD/GB region today. SLC flash ranges up to $50/GB, and MLC flash is on the order of call it $20/GB or less. Actually this varies quite a bit as well. I can’t talk publicly about some of the pricing we’ve seen, but its getting interesting. So, a $2/GB SSD unit is, in reality in the 5-10k IOP region for applications (not meaningless benchmarks). A $50/GB Flash unit (25x the pricing) is in the 300k-1M IOP range for applications. 50-100x the performance. If you look at it this way, for IOP heavy apps, the Flash pricing isn’t outrageous. But you have to compare similar things to understand this. Don’t compare them to 7200RPM spinning rust for bulk data service. Thats not what flash is good at. Compare it to the IOP heavy apps, where that poor spinning rust disk is maxing out at 110 IOPs, and the SSD is crusing along at 50-100x that, while the PCIe flash cruises along at 25-50x that of the SSD. We have to set expectations correctly. Getting the nomenclature and real performance measures correct is very important. Avoiding the marketing numbers … also very important.