Its not perfect, but we have CentOS/RHEL 7.2 and Lustre integrated into SIOS now

Lustre is infamous for its kernel specificity, and it is, sadly, quite problematic to get running on a modern kernel (3.18+). This has implications for quite a large number of things, including whole subsystems with a partial back-porting to earlier kernels … which quite often misses very critical bits for stability/performance. I am not a fan of back porting for features, I am a fan of updating kernels for features. But that is another issue that I’ve talked about in the past. A large government lab customer wanted a Lustre version of our NVMe system. I won’t discuss specifics of this scenario other than to note that we have to get Lustre onto the system. SIOS is our OS and boot loader for diskless/stateless systems, and our stateful systems. We don’t like using the distro native installers, due to their sheer fragility (anaconda is something you really want to avoid at almost all costs). Debian’s is large and unweildly, we cannot get it to do what we want. So SIOS handles putting bits down on disk/ramdisk image. This part works beautifully. The issue is on startup, debian’s initramfs is a very well engineered/designed system, and it just works correctly. I don’t have to mess around with very much to make it do what we need. Dracut on the other hand … is … well … dracut. If there was an option to rip it out and replace it with debian’s initramfs, I’d do that. I spent the better part of the last week working on debugging dracut, finding places where reality and the documentation did not match up. Tracing booting one step at a time. Re-building our dracut module (first built in 2012 for CentOS/RHEL6 issues of a similar nature), and discovering the the documentation over what gets called, and when it gets called … is woefully wrong. It took a week. A whole week to debug the monstrosity. I needed a mixture of grub options, and dracut.conf and other bits to make it work. Its not fragile. With our current stable kernel, there is a nice dracut “bug” which causes significant spamming of the logs, and really long boot times. Testing it on the NVMe machine now. I wonder how much performance and stability I am giving up going back to an ancient buggy kernel that we need to use, in order to support Lustre. It would be much better if the Lustre patches for 3.18 and beyond were easily accessible (they aren’t , you have to hunt them down). Would also be good if we didn’t need to mess with the whole kernel build process in order to build Lustre. No problems if the user space depends upon the kernel bits and headers. Definitely a problem that you have to build them all together, all at once. I seem to remember grousing about this 8+ years ago as well. /sigh