Upped sustained speeds on new JackRabbit unit

I forgot to mention this. Odd. Our updated JackRabbit (JR4 f.k.a JRM) unit being burnt in over the last few days for a customer. Putting obscene loads on it. Trying hard to crash it. Really.

From fio (apart from the 4M buffer size issue, I really like fio)

streaming-write: (groupid=0, jobs=1): err= 0: pid=12270
  write: io=6,397GiB, bw=1,498MiB/s, iops=365, runt=4479113msec
    clat (msec): min=1, max=4,560, avg= 2.73, stdev=10.33
    bw (KiB/s) : min=    0, max=2427840, per=101.23%, avg=1552322.90, stdev=2228
30.41
cpu          : usr=0.09%, sys=8.08%, ctx=3264682, majf=0, minf=256

Let me walk you through the parsing of this. 1 MiB = 1024 KiB = 1048576 B. So 1498 MiB/s = 1571 MB/s or 1.57 GB/s. Which as you might notice, was sustained for 4479 seconds. A bit more than an hour. With a CPU load of 0.09% in user space, 8% in system space, and a few million context switches. You might also notice the maximum bandwidth measured during an interval. 2.48 GB/s. We didn’t sustain it, but it burst to that. Remember, this is direct IO, uncached IO. So this isn’t cache speed. For each of the 9 storage disks in each raid (ok 11 disks, but 9 of them are in use for data storage in RAID6), this represents about 138 MB/s. We see some instances of these drives bursting this high, but not sustaining there. This is from tests outside of this environment. We see typically 87 MB/s sustained write speed, though often on more modern firmware revs, we are seeing 110 MB/s. These drives, we are seeing close to that on writes. So the fluctuations were really fluctuations and not stable measurements … not a signal per se. Of course, reads went up as well.

streaming-read: (groupid=1, jobs=1): err= 0: pid=12422
  read : io=6,400GiB, bw=1,593MiB/s, iops=388, runt=4212362msec
    clat (msec): min=1, max=2,977, avg= 2.57, stdev= 3.13
    bw (KiB/s) : min= 4894, max=1845493, per=100.18%, avg=1634342.17, stdev=94968.36
  cpu          : usr=0.08%, sys=13.23%, ctx=3277615, majf=0, minf=1273

Again, working on our parsing … 1593 MiB/s = 1670 MB/s = 1.67 GB/s. Based upon other measurements (crude dd bits) we have been talking about 1.2 GB/s writes and 1.5 GB/s reads. I think I can comfortably say that we are measuring (sustained) 1.57 GB/s writes and 1.67 GB/s sustained reads, uncached (e.g. straight to/from disk). But this is RAID6, not JBOD, and is configured in customer deployable configurations.