Infiniband

I’m building a 6-node cluster, using Xen at the moment. For the last few days, I tried my setup in a virtual machine, simply because VM’s boot much faster than the real hardware. However, certain things you can only replicate on the real hardware (for example, the InfiniBand interfaces, as well as certain nfs-stuff). So I spent most of the day to replicate my configurations onto the hardware. After getting all done, the moment of the first boot … kaput! Doesn’t boot, just keeps hanging before booting the real kernel. Now what ? I removed the Xen vga parameters and rebooted (waited ~2 minutes in the process) until I finally saw the root cause for my trouble: ...

As I mentioned yesterday, I’m currently doing some project work. Said project includes InfiniBand technology. Apparently we bought a “cheap” InfiniBand switch, which comes without a subnet manager. So, in order to communicate between the nodes, you need to install the subnet manager (opensm in my case) on each node. 1 2 3 4 echo 'deb http://pkg-ofed.alioth.debian.org/apt/ofed ./' > /etc/apt/sources.list.d/openfabrics.list; aptitude update; aptitude install opensm In order to utilize the InfiniBand interface you need to do a few things first though: ...