Seriously … no more cheap switches as the central point of information flow in storage or computing clusters. The money you save will be blown in the first hour you pay for down time or architectural changes you need to actually move your data without tossing packets on the ground …
… because while standard network codes don’t care so much if they need to retransmit or lose data, cluster file systems get very … very … testy when data doesn’t arrive when and where it is supposed to, in the right order, because the cheap-ass switch was too busy tossing packets on the floor.
I can’t tell you how many times I’ve seen this. I could name off the vendors of these cheap-ass switches, but they are so damned prevalent … you run into them in so many places, that you have to engineer around them, so that they aren’t impacting you.
I just handled a call with a customer with one of these cheap-ass switches as their core. Its easy to blame the cluster file system in this case, as its failure modes are a thing of legend … but the fault really resides in the cheap-ass switch. The one we recommended they not get, but darn it, it cost less than the good switch.
There is a price for every decision. If you buy crappy gear, chances are your results will be suboptimal.
Or as I’ve been saying for 30 years or so, “systems designed to fail, often will.”