Mysql on BAFM

MySQL: Beware of sync_binlog on EXT3

Mon, 23 Feb 2009 21:35:38 +0000

Well, I just glazed again over my my.cnf for our web-cluster because I just moved a database from one cluster to another and getting quite different performance from it. So, as I expected, there is a slight difference between both configuration files:

1
2
3
4
5


@@ -55,8 +58,6 @@
 innodb_log_group_home_dir = /var/lib/mysql/db
 innodb_log_file_size = 512M
 innodb_thread_concurrency = 8
-sync_binlog = 1

And apparently, according to the MySQL Performance Blog that’s really, really bad (as well, we’re currently running without write caching, as the battery module of the storage is dead).

MySQL: Replication and hostname wild cards

Sun, 15 Feb 2009 19:06:13 +0000

Yeah, yeah .. I know, it’s weekend. But I usually can think much better when no one is rattling my cage. So I had another look at my replication problems.

Don’t you never ever change InnoDB settings when migrating between hardware, because InnoDB is rather sensitive regarding those parameters.
When you’re setting up the replication (don’t ask me why) and copying over the database to the second replication partner, be aware if you’re using wild cards you’re gonna get seriously bitten in the back.

Now, let’s look at the constellation.

MySQL: Setting up an InnoDB raw device

Wed, 11 Feb 2009 13:34:45 +0000

Well, since I had to brood about this (again I might add), I’m gonna write it down this time …

Setting up the InnoDB raw device isn’t that hard, just make sure the device has proper permissions (either add mysql to the disk group or create a udev rule).

`1`	`KERNEL="sdb2", OWNER="mysql", GROUP="mysql"`

Now after that (and a reboot/udevcontrol reload_rules later), you should be able to initialize the InnoDB device. Yes, the InnoDB device needs initializing.

Defragmenting all fragmented MyISAM tables

Thu, 16 Oct 2008 20:18:22 +0000

I just had another look at what I wrote the week before last (you know, being home-sick/on vacation has it’s advantages) and additionally read up on " OPTIMIZE TABLE" again. The comments in the manual mention " SHOW TABLE STATUS", which gives you a complete list, but it doesn’t allow you to filter certain kinds of things out (like I only wanted to see MyISAM tables in the list, I only wanted database and table).

Been a while

Sun, 17 Feb 2008 08:53:16 +0000

Well, it’s been quite a while since most of the people last heard a word from me. The last few months I’ve been extremely busy with work-related tasks (and as a side-effect of that, didn’t want to spend much time in front of the computer after 9 hours of work). I also started spending more and more time in the gym, like nearly two hours every Tuesday and Thursday.

I finally fixed our replication issues, we do now have a working! MySQL Multi-Master ( 1. Node, 2. Node -- bear in mind, this boxes are only serving MySQL and nothing else, so don’t use these configurations on mixed setups) Replication Setup as database back end for our TYPO3-vHosts.
all the web nodes are now serving the content from a clustered, shared SAN volume (is that a good thing ? 😛 - don’t know yet …)
our VI environment is getting more and more acceptance (even if you hear some complaints now and then, like “awww, damn that crap my 4GiB RAM, 2x3.0GHz Windows 2008 is running soooo choppy” - simple answer, don’t use Windows Server 2008 and/or Windows Vista!)
I finished prepping our VM templates (at least the Windows ones)
we’re still putting together the plans on whether or not invest into a VDI solution.

The next few weeks are gonna be as frantic as the weeks before, I still have to migrate a lot of TYPO3 installations to our new cluster (which sadly needs time, as we need to wait for DNS changes to propagate). Honestly, I might be ending up extending the SAN volume for the MySQL data storage, as even with only three somewhat busy sites, the binary log of the last 5 days is about 2GiB in size. And we still have ~ 20 other busy sites on a separate box.

TYPO3 and MySQL replication

Sat, 08 Sep 2007 20:33:13 +0000

Apparently the TYPO3 version we are using, doesn’t play too nice with the MySQL MasterMaster replication.

Sometimes, something like this is going to happen:

1
2


070826 0:44:32 [ERROR] Slave: Error 'Duplicate entry '75-222419149' for key 1' on query. Default database: 't3nb'. Query: 'INSERT INTO cache_pagesection
070826 0:44:32 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'dbc-mysql1.000192' position 611861372

Well, as you can see from the last line in the log, the Slave-SQL thread found a duplicate entry and thought it is smart to just turn off the thread instead of disregarding the just made entry. So from now on, both databases drift since there ain’t no replication anymore until someone kick starts the replication again (someone being me).

Continuing on SLES10

Sat, 16 Jun 2007 16:14:15 +0000

OK, it turns out that I was rather stupid when configuring the my.cnf. As it turned out, the effect I was seeing was due to the presence of two log-bin lines, which looked like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


[mysqld]
port = 3306
datadir = /mysql/dbase
log = /mysql/logs/dbc-mysql1.log
log-error = /mysql/logs/dbc-mysql1.err
socket = /var/lib/mysql/mysql.sock
bind = 172.16.234.31

# custom paths for binary logs
log-bin = /mysql/binlogs/dbc-mysql1
log-bin-index = /mysql/binlogs/dbc-mysql1.idx
relay-log = /mysql/binlogs/dbc-mysql1.relay

And some lines down there was this:

Back at SLES10

Tue, 12 Jun 2007 19:42:22 +0000

Here I am, sitting at my desk on a Thuesday evening thinking about what happened the last few days.

I finally got to play around with our PacketPro 450 Cluster(nifty LoadBalancing appliance)
We reworked the network the way we want it (and not that tool of a wannabe sysadmin)
We mostly figured out how to do the LoadBalancing right, we just need to find some bugs in the LoadBalancer software (like the thing is failing over to its slave from time to time, but keeping the IP address for himself) or let the guys at teamix do their work and hopefully get a working release within the next week or so
I figured out how to setup interface bonding with SLES10 (it was quite straight forward, thanks to the excellent in-kernel documentation), and we’re using an active-backup mode for now
I still need to figure out how to do the MySQL Master<->Master replication right .. I’m currently building fresh RPM’s on one of those Dell blades (yes, they ROCK!) which will hopefully be finished till I’m at the office tomorrow.
Pt. 5 also includes figuring out how to pass MySQL a custom location for the binary-log, at least that’s what the handbook says in Chapter “5.11.3. The Binary Log” …

When started with the –log-bin[=base_name] option, mysqld writes a log file containing all SQL commands that update data. If no base_name value is given, the default name is the name of the host machine followed by -bin. If the basename is given, but not as an absolute pathname, the server writes the file in the data directory. It is recommended that you specify a basename; see Section B.1.8.1, “Open Issues in MySQL”, for the reason.

Waiting

Wed, 28 Feb 2007 14:58:05 +0000

We are still waiting for the money promised by the state and the country for our HBFG (again, it’s “Hochschulbauförderungsgesetz”), that hopefully is reducing or eliminating our storage/SAN problem we have currently. Right now we have to Cisco MDS9216 (that’s a 16-port 2GBps SAN-switch, two for redundancy), which means we only have 16 SAN-ports. That isn’t much, but still is to less, as we have like 30 machines or so, that really need access to the SAN, so we either end up unplugging some of them from the SAN or merge them onto some big machines (like our x366).