Welcome to BAFM#
This blog is a collection of thoughts, experiences, and technical insights from a sysadmin’s perspective. Here you’ll find posts about system administration, infrastructure challenges, troubleshooting adventures, and the occasional philosophical rambling about technology and its role in our daily work.
Whether you’re a fellow sysadmin looking for solutions, someone curious about the behind-the-scenes work that keeps systems running, or just stumbled upon this corner of the internet – welcome! Feel free to explore, and don’t hesitate to reach out if you have questions or want to share your own experiences.
Follow me through my journey through life with all it’s neat little tricks, caveats and side-quests.
Quick Links#
Last updated: January 2026
Yeah, yeah .. I know, it’s weekend. But I usually can think much better when no one is rattling my cage. So I had another look at my replication problems.
Don’t you never ever change InnoDB settings when migrating between hardware, because InnoDB is rather sensitive regarding those parameters. When you’re setting up the replication (don’t ask me why) and copying over the database to the second replication partner, be aware if you’re using wild cards you’re gonna get seriously bitten in the back. Now, let’s look at the constellation.
...
You know, I’m not getting any younger. It’s getting harder remembering every damn command … so here is how you get information out of your EXT3 filesystem:
1 2 3 4 5 6 7 8 9 10 sles10sp2 ~ [0] > tune2fs -l /dev/sda2 | grep "^Filesystem" Filesystem volume name: <none> Filesystem UUID: 8eec8235-4d9e-4b58-acf9-3c68c977d5ea Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal resize_inode filetype needs_recovery sparse_super large_file Filesystem state: clean Filesystem OS type: Linux Filesystem created: Tue May 27 10:48:56 2008
For people, who are as click and point-lazy as me, here is how you restart the service without using the service management applet.
1 2 net stop "NSClientpp (Nagios) 0.3.5.2 2008-09-24 w32" net start "NSClientpp (Nagios) 0.3.5.2 2008-09-24 w32"
Well, since I had to brood about this (again I might add), I’m gonna write it down this time …
Setting up the InnoDB raw device isn’t that hard, just make sure the device has proper permissions (either add mysql to the disk group or create a udev rule).
1 KERNEL="sdb2", OWNER="mysql", GROUP="mysql" Now after that (and a reboot/udevcontrol reload_rules later), you should be able to initialize the InnoDB device. Yes, the InnoDB device needs initializing.
...
Today I ended up working out the details on what we want to monitor regarding our BladeCenter. The most interesting details (for us that is) are these:
Fan speeds for Chassis Cooling/Power Module Cooling Bay(s) Temperature Power Domain utilization It wasn’t * that* hard to implement. Only trouble(s) I ran into, were ( 1) IBM did a real shitty job with the MIB’s. If you look closely into the mmblade.mib, you’re gonna notice, that not a single OID is specified for the events. ( 2) As the MIB’s weren’t documented anywhere, I had to look them up via snmpwalk (which I had never used before). So as a reminder (to myself), here’s how it is done:
1 snmpwalk -v1 -c public -O n 10.0.0.35 .1.3.6.1.4.1.2.3.51.2.2 This will get you a list, with a lot of output (5154 lines to be exact). Lucky me, the web interface of the management module/ssh interface is rather verbose, so all you need to do is compare those values with what you are looking for.
So for myself (and anyone interested) read ahead for the list of checks we are currently running on the management module.
In our current fight against the BladeCenter switches, we’re currently facing the problem that the blades ain’t able to send/receive DHCP-traffic.
So in order to move forward, we had to use static IP addresses. And since SLES10 ain’t straight forward on that, I had to look it up. Now, here’s for me (and everyone else tired of searching) how to do it:
1 2 3 4 5 6 7 8 9 install=http://ftp.barfoo.org/install/SLES10-x64/CD1 netdevice=eth0 hostip=10.0.1.240 netmask=255.255.255.0 gateway=10.0.1.1 nameserver=10.0.1.2 domain=home.barfoo.org insmod=bnx2 splash=verbose
Well, we finally had our maintenance window today, in which we planned the hardware exchange for our current Dell Blade Chassis (don’t ask!). The exchange went fine, but as we started exploring the components (like the IBM BladeCenter SAN switches – which are in fact Cisco MDS 9100) we hit a few road blocks.
First, the default user name/password combo for the Cisco MDS 9100 for the BladeCenter is USERID/PASSW0RD (just as the rest of the password combinations).
...
As I did some switching today (between the new lin_tape version by IBM and our own lin_tape version), I ended up writing those lines a dozen times. Here is (just for me, if you don’t care .. skip ahead) on how to generate a list of commands:
1 2 3 4 5 6 for i in $( seq -w 1 32 ); do token="${i/0/}" path_prefix="/dev/lt/IBMtape12245775" echo "DELETE PATH TSM1 VTL1_DR$i SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=VTL1" echo "DEFINE PATH TSM1 VTL1_DR$i SRCTYPE=SERVER DESTTYPE=DRIVE LIBRARY=VTL1 DEVICE=$path_prefix$((token+11))A1" done which should get you a list like this:
...
Well, I just stumbled about this again (and I don’t know right now whether or not this is documented inside a RedBook or not) today, so I thought maybe I’m gonna write it down.
Slot-Amount Property of a Virtual Tape Library
Please keep in mind, when creating the virtual library to think hard about the amount of slots you might need. It ain’t that bad, you just can’t decrease the amount anymore. So if you think about creating 50 different virtual tape libraries with 500 slots each on your TS7530, think again. The current software level only supports 25.000 slots on a global level.
Recently, we got the recommendation from our system partner to use static allocated tape cartridges instead of dynamic allocated ones. Apparently using dynamic allocating cartridges comes with a performance penalty if more than a few nodes are backing up a large amount of data at once.
And yet again, I noticed that the IBM Virtualization Engine Console (aka Falconstor Software) is really error prone.
In order to change the allocation type, we had to shred the old cartridges first (500 x ~100M up till now), chance the allocation type at the virtual tape library level, and then recreate the 500 cartridges with a fixed size (500x 102400MB). Now, as I was kinda optimistic, I decided to create all 500 cartridges at once.
...