JackRabbit-M update

JackRabbit-M [you will be able to order these online in short order from our store]. More tests, more burn in. I will describe this unit in a moment. We are taking this one out to the test track. Running a few time trials. Cracking the throttle. Wide open.

[queue Don Felder’s “Heavy Metal": this unit masses north of 60 kg … ] raw write speed:

[root@jackrabbit ~]# dd if=/dev/zero of=/big/big.file ...
2500+0 records in
2500+0 records out
83886080000 bytes (84 GB) copied, 74.7441 seconds, 1.1 GB/s

raw read speed:

[root@jackrabbit ~]# dd if=/big/big.file of=/dev/null ...
2500+0 records in
2500+0 records out
83886080000 bytes (84 GB) copied, 52.9713 seconds, 1.6 GB/s

Ok, what does this mean … and what is the context? This is our 2.6.23.14Scalable kernel first off. We had been playing with 2.6.26.1 but a file system bug (and the patch for it) is still a problem, so we are resorting to a known quantity. The newer kernel is a bit faster on writes. The units which make up this file system are RAID6’s. Of the 24 disks in it (yup, you read that right), 2 are hot spares, leaving 22 drives for use. Of these 22, we have 2x RAID6 of 11 drives each, giving 9 effective spindles per RAID, or 18 effective spindles total. Now lets see what this means: At read speeds of 1.6 GB/s (ok, if you calculate it out it is 1.58 GiB/s, dd rounds a bit), these 18 spindles are providing on average 87.98 MB/s read speed per spindle. On these drives, we have measured individually, right about 93 MB/s sustained reads. Others claim they have measured 105 MB/s (these are 7200 RPM SATA II units), though we haven’t seen that on these drives. That means we are driving these units at about 94.6% read efficiency. At write speeds of 1.1 GB/s (well, 1.12 GiB/s if you calculate it out), we are seeing 62.4 MB/s. Considering that we see 70 MB/s on the write speed for these drives outside this array, this means we are 89.1% efficient on writes. Not bad. Ok, what about this machine. This is a 64 GB RAM (FBDIMM DDR2-800) machine (note that the tests were all greater than this size, so we are actually really testing disk speed, and not cache speed), with 8 Xeon 5482 3.2 GHz processor cores. We can put up to 128 GB RAM in this machine, though it would be … expensive (look at the price of 8GB DIMMs… owie!). Now there are many ways we can calculate performance, a fair number of which our competitors do use. For example. Suppose I simply took the 24 drives and pretended I could write perfectly in parallel across all of them at once (which is not possible no matter which OS is used). This would net me about 1680 MB/s. Well, one of our competitors does this calculation with 48 drives and comes out with 3.2 GB/s. It doesn’t mean they measure that (they don’t, you can see measurements online from various places, as long as they aren’t using a certain OS). The funny thing is that this is the number that we are dealing with in their marketing literature. The reality is (quite a bit) less than the marketing number. As we measure our numbers, we are pretty sure they are where we report them to be. Some customers have commented to us privately that their benchmarks were “dead-on” ours. Go figure. Considering we measure them, it doesn’t surprise me.