Massive unapologetic storage firepower part 4: On the test track with a Forte unit ... vaaaaROOOOOOMMMMMMM!!!!!
By joe
- 4 minutes read - 822 wordsI am trying to help people conceptualize the experience. Here is a video depicting very fast, very powerful cars and their sound signatures.
This is a good start. Take one of those awesome machines, and turn off half the engine. So it is literally running with 1/2 of its power turned off. Remember this. There will be a quiz. As we flippantly noted in the video, this is face-melting performance. Had I any hair left, it would have been blown way back. This is (for a number of reasons) a 1/2 configuration of a Forte NVMe storage unit. We’ve said it is quite likely the fastest storage platform in market, not by a little bit. And we are really … really … not kidding. After conditioning the unit for a few hours, I turned on the parallel file system (BeeGFS), and started doing a customer run. The Forte unit is attached to a Unison unit as a client, over a single (yes, really) 100Gb link. The network traffic is RDMA so it won’t show up on dstat output below. Watch as I run writes …
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 100 0 0 0| 0 0 | 458B 1642B| 0 0 |4169 530
0 0 100 0 0 0| 588k 0 | 488B 2056B| 0 0 |4195 755
0 0 100 0 0 0| 0 0 | 398B 1560B| 0 0 |4179 610
0 0 100 0 0 0| 0 0 | 398B 1532B| 0 0 |4171 576
0 0 100 0 0 0| 0 0 | 398B 1472B| 0 0 |4179 603
0 0 100 0 0 0| 0 0 | 338B 1390B| 0 0 |4182 699
0 0 100 0 0 0| 0 0 | 398B 1532B| 0 0 |4189 637
7 20 72 0 0 1| 312k 48k| 278B 1192B| 0 0 | 41k 1148k
10 33 56 0 0 0| 0 84k| 218B 1126B| 0 0 | 54k 1558k
11 32 56 0 0 1| 96k 7526M| 218B 1110B| 0 0 | 84k 1462k
10 34 47 9 0 1| 80k 18G| 158B 452B| 0 0 | 106k 1206k
0 9 45 46 0 0| 0 9850M| 158B 452B| 0 0 | 82k 131k
0 7 60 33 0 0| 0 1524M| 218B 1110B| 0 0 | 76k 429k
0 6 60 35 0 0| 0 1393M| 158B 436B| 0 0 | 68k 289k
0 6 72 22 0 0| 0 2371M| 158B 436B| 0 0 | 59k 291k
1 5 74 20 0 0| 0 1421M| 158B 436B| 0 0 | 56k 269k
then reads …
----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 3 97 0 0 0| 0 0 | 218B 1110B| 0 0 |4173 1177
0 3 98 0 0 0| 0 0 | 218B 1590B| 0 0 |4163 1228
0 3 98 0 0 0| 0 0 | 218B 1110B| 0 0 |4155 1156
0 3 98 0 0 0| 0 0 | 218B 1110B| 0 0 |4175 1164
0 2 98 0 0 0| 64k 260k|5095B 4259B| 0 0 |4222 1653
0 0 100 0 0 0| 320k 0 | 218B 1126B| 0 0 |4188 685
0 0 100 0 0 0| 64k 0 | 278B 1192B| 0 0 |4186 630
0 0 100 0 0 0| 0 0 | 278B 1192B| 0 0 |4153 532
0 0 100 0 0 0| 0 0 | 278B 1192B| 0 0 |4192 1366
0 0 100 0 0 0| 0 0 | 278B 1192B| 0 0 |4191 896
0 0 100 0 0 0| 0 0 | 278B 1192B| 0 0 |4162 536
5 6 81 8 0 0|4575M 0 | 278B 1192B| 0 0 | 127k 211k
9 12 62 17 0 0|9098M 0 | 218B 1110B| 0 0 | 190k 421k
10 11 62 16 0 0|9092M 0 | 218B 1110B| 0 0 | 203k 421k
9 12 62 17 0 0|9089M 0 | 218B 1110B| 0 0 | 203k 420k
1 2 95 2 0 0|1036M 0 | 218B 1110B| 0 0 | 25k 50k
0 0 100 0 0 0| 0 0 | 218B 1110B| 0 0 |4151 552
0 0 100 0 0 0| 0 0 | 218B 1590B| 0 0 |4163 618
The longer tail actually comes from straggling writers/readers. 24 threads, similar performance from 2k to 1024k blocks. The take home message is that we are filling up the network, and it is our bottleneck. We are filling the 100Gb network. With 1/2 our box. And its 2/3rds idle during the reads, and about 1/2 idle during the writes. These are very reasonably priced boxen. And available in volume today. Reach out to the day job if you would like more info. More tests soon. Very soon.