started playing with SmartOS for the day job

This is a very cool concept, something that meshes perfectly with our Tiburon based siCluster philosophy. That is, compute nodes should boot diskless, there should be very little state on each node, and stuff that you need to do should be made absolutely as simple as possible. SmartOS is a project of Joyent. Joyent, for those not familiar with them, are a cloud company, building a nice public cloud for end users to build on. Their tools are largely open, and they are building some good stuff. Like node.js . Imagine specifying your VM as a JSON file. As dumb as it sounds, this meshes … almost PERFECTLY with what we’ve developed for our boot management server.

<code>
# config-file-type: JSON 1
{
   "boot_servers" : {
      "default" : "192.168.1.250:3000"
   },
   "targets" : {
     "nfs" : {
        "Centos6.2_x64" : {
           "boot_args" : "ip=dhcp root=nfs:192.168.1.250:/data/nfs/Centos6.2_x64 rw debug verbose rdshell rd_NO_DM rd_NO_MD rd.ip=auto selinux=0  ",
           "boot_server" : "192.168.1.250:3000",
           "initrd" : "http://192.168.1.250:3000/images/Centos6.2_x64/initramfs-2.6.32-220.13.1.el6.x86_64.img",
           "kernel" : "http://192.168.1.250:3000/images/Centos6.2_x64/vmlinuz-2.6.32-220.13.1.el6.x86_64",
                          },
     },
   "install" : {
      "Centos6.2_x64" : {
                "boot_args" : "keeppxe text xfs nfsroot=192.168.1.250:/data/nfs/OS/Centos6.2_x64 ks=http://192.168.1.250:3000/install/Centos6.2_x64/compute-node-6.2.ks.cfg nomodeset ksdevice=eth2 ",
                "boot_server" : "192.168.1.250:3000",
                "initrd" : "http://192.168.1.250:3000/images/Centos6.2_x64/initrd.img",
                "kernel"  : "http://192.168.1.250:3000/images/Centos6.2_x64/vmlinuz "
                        },
      "Centos6.2_x64_sirouter" : {
                "boot_args" : "keeppxe text xfs nfsroot=192.168.1.250:/data/nfs/OS/Centos6.2_x64 ks=http://192.168.1.250:3000/install/Centos6.2_x64/storage-node-6.2.ks.cfg nomodeset ksdevice=eth0 verbose debug ",
                "boot_server" : "192.168.1.250:3000",
                "initrd" : "http://192.168.1.250:3000/images/Centos6.2_x64/initrd.img",
                "kernel"  : "http://192.168.1.250:3000/images/Centos6.2_x64/vmlinuz "
                        },
      "Centos6.2_x64_storage" : {
                "boot_args" : "keeppxe text xfs nfsroot=192.168.1.250:/data/nfs/OS/Centos6.2_x64 ks=http://192.168.1.250:3000/install/Centos6.2_x64/storage-node-6.2.ks.cfg nomodeset ksdevice=eth0  ",
                "boot_server" : "192.168.1.250:3000",
                "initrd" : "http://192.168.1.250:3000/images/Centos6.2_x64/initrd.img",
                "kernel"  : "http://192.168.1.250:3000/images/Centos6.2_x64/vmlinuz "
                        },
               },
    "biosupdate" : {
    "jr0" : {
         "kernel"    : "memdisk",
         "boot_args" : "keeppxe initrd=biosupdate/16MB-bios-update.img",
         "initrd"    : "biosupdate/16MB-bios-update.img"
        }
    },
   },
   "machines" : {
      "default" : {
        "type" : "install",
        "name" : "Centos6.2_x64"
#        "name" : "Ubuntu11.04"
      },
      "00:25:90:27:db:b4" : {
         "type" : "nfs",
         "name" : "Centos6.2_x64"
      },
...
</code>

I could simply incorporate their VM description directly into our DB, without changing anything. And we can serve it from Tiburon, our siCluster management system. Damn … I like it! Underneath SmartOS is OpenSolaris, really the Illumos kernel. ZFS and dtrace are in there. No messing about with licensing issues. Ok, some things won’t work well. OFED stack has been ported, but its state is unknown. Not sure about cluster file systems (GlusterFS, etc.). But for what we are looking at doing with it, this could be a hand-in-glove scenario. What Joyent has done is to package up OpenSolaris so that its an appliance OS. Very easy to boot, very easy to configure. Its purpose in life is to be used as a base for running their VMs. This is, curiously, exactly how I thought Solaris would survive into the future (and if you read that post, and google thumper images, you might get a sense that JackRabbit got its name as a riff on thumper ). Unfortunately, VMs are notoriously bad at running 10GbE and IB. There really no VM cut through you can do to preserve performance. Which is a shame for low latency stuff. But we may be able to keep enough performance on the IO side that even with all the inefficiencies, its still fast. If so, one of the other things we do could get a nice boost in capability … again, with us changing very little. I like this. This is a well thought out system. And I didn’t even mention the community. Sorta like the linux community of old. Back when having 5+ different ways to accomplish the goals was a good thing. This is something to watch. In no small part because you can boot pretty much any OS in their VMs. And they make it (IMO) painless to create/boot new OSes.