Lenz Grimmer's blog

The UK Oracle User Group (UKOUG) is an independent non-profit organisation of users of Oracle products in the UK, which organizes over 80 events per year all across the UK and Ireland. They have more that 15.000 members and were established over 25 years ago. Currently, they are in the final stretch of the preparations for their annual Technology & E-Business Conference, which will take place from November, 29th - December, 1st in Birmingham (UK).

For the first time, the conference will host a dedicated stream of presentations related to MySQL, which is scheduled for Wednesday (Dec. 1st). Targeted at database administrators, developers as well as users new to MySQL, there will be presentations from well-known speakers and participants of the MySQL community as well as members of the MySQL Team at Oracle.

Attendees will learn about best practices and gain valuable insight into the MySQL Server and related applications and tools.

Highlights include a keynote about the Oracle strategy and plans for MySQL by Richard Mason, Vice President of MySQL Sales and Consulting at Oracle, talks about High Availability with MySQL and MySQL Cluster as well as InnoDB Performance Tuning best practices. Other sessions will cover related products and projects from the MySQL ecosystem, that help to make the most of MySQL. See the full agenda on the conference web site for details and speaker information.

Registration for the event is still open. Get your tickets now!

Over the weekend I updated my Drupal 7 test appliance in SUSE Studio to the Drupal 7.0-beta2 release, which was released on Oct. 23rd. I also added phpMyAdmin upon a user request, to provide a web-based method to work with the MySQL instance, if needed.

In addition to the lightweight "headless" appliance (which can only be accessed and configured via a remote network connection), I've now also created a GUI-based version. This appliance starts a minimal GNOME desktop and a Mozilla Firefox browser, which in turn opens the Drupal installation page by default. I hope you will find this useful if you want to toy around and test Drupal 7 without having to go through the entire OS and LAMP stack configuration yourself. In fact, you can even test this appliance via the recently added test drive option from right out of your web browser!

The appliance is now also available in OVF format. SuSE Studio now also builds Amazon EC2 images, which don't seem to be available for download from the SUSE Gallery yet. I assume this is a recent addition to the continuously improving SUSE Studio functionality, hopefully these images will be made available soon.

The Drupal community just recently released another alpha test release of their upcoming Drupal 7 version, to shake out the remaining bugs and to encourage more users to test it.

If you would like to give it a try, but you don't have a free server handy, how about using a virtual machine instead? Using the fabolous SuSE Studio, I've created an appliance based on openSUSE 11.3, Drupal 7.0-alpha7 and MySQL 5.1 with the InnoDB plugin and strict mode enabled (both for the SQL mode and InnoDB mode. Using this configuration helps to ensure that Drupal works well with the current version of MySQL/InnoDB and does not use any "questionable" SQL statements. This might be especially interesting for additional modules - Drupal core did not reveal any problems using strict mode so far.

You can download disk images for VMware/Virtualbox/KVM or XEN from the SUSE Gallery (free login required). Just boot the appliance in your virtualization application of choice, choose your keyboard layout and step through the network configuration and Time Zone selection. Once the appliance has booted up and the login: prompt appeared, point your web browser to the appliance's IP address to start the Drupal installation/configuration. MySQL has been pre-configured, there is an empty database named "drupal" and a user "drupal" with the same password to access it. You just need to enter this information in the Drupal Database configuration dialogue during the installation. Anything else can be configured to your liking.

After you have finished the installation, you can toy around with a fresh Drupal 7 installation! Install additional modules, change the themes, add content. And make sure to report all bugs that you run into while doing so! Have fun.

Shortly after I created the initial packages of embedded InnoDB on the OpenSUSE Build Service, Oracle/Innobase released an updated version (1.0.3.5325). In addition to many improvements and bug fixes, they slightly changed the versioning scheme to better indicate what version of the InnodDB plugin their code is based on (see Vasil's posting on the InnoDB Forums for more information).

I've now updated my InnoDB packages on the Build Service to this version as well - please note that the naming scheme of the shared library package has been changed from "embedded_innodb1" to "libinnodb2" — RPM will take care of replacing the old package during update, even though the name has changed.

Oracle/InnoBase announced the availability of the embedded version of InnoDB at this year's MySQL Conference & Expo, but I have not seen a lot of comments or reviews about it so far. Which surprises me, because I think this is a very interesting piece of technology!

In my opinion it might actually hit the sweet spot for application developers seeking an alternative embedded database solution. SQLite is nice and popular, but it seems to have concurrency issues when used in multi-threaded applications. An embedded MySQL server would be an alternative - this is what the Amarok developers decided to go with, for example. But this approach has its issues, too, especially the lack of a shared library version of libmysqld poses some challenges when distributing binaries.

This is where I think the embedded version of InnoDB might have an edge. It's pretty lightweight in comparison to a full-blown MySQL server, provides excellent crash-recovery (which is essential for desktop applications), transactions (useful in environments with high concurrency) and foreign key constraints. I'm not sure how important these are for embedded use cases, it probably depends on the complexity of the data to be stored. On the downside, Embedded InnoDB does not "speak" SQL. In order to store and retrieve values, you need to use the InnoDB API. See the chapter Concepts and Architecture for more details and an overview.

Another possible reason for the low popularity might be that it's currently not part of any Linux distribution (yet) and that Oracle only provides binary tarball packages for Linux and a Windows binary for download from the web site.

Therefore I've now created a spec file to build RPMs of Embedded InnoDB and added it to my repository on the openSUSE Build Service, which now provides Embedded InnoDB packages for a wide range of RPM-based Linux distributions. I hope that the spec file will be included in the next source distribution. I've posted it (and a patch to fix a few problems with the examples) to the newly created InnoDB mailing list, but to be sure I added a note to the Embedded InnoDB Forum as well.

After a long hiatus, I am happy to announce that mylvmbackup version 0.12 has now been released. This release includes a large number of improvements, minor code cleanups, as well as some new functionality. In particular, I would like to thank Matthew Boehm, Tim Stoop, Baron Schwartz, Ville Skyttä and Ronald Bradford for their contributions.

Some notable highlights from the ChangeLog:

• Removed the absolute path names to external tools (make sure $PATH is correct)
• Added --log-err to the startup options of the recovery instance to avoid cluttering the server's error log
• Added support for hooks written as Perl Modules. (Matthew Boehm)
• Added support for date/time-formatted path names for backupdir and mountdir (Matthew Boehm)
• Backupdir and mountdir are now created automatically (Matthew Boehm)
• Added new hook "logerr" when an error is logged. (Matthew Boehm)
• Added Option --keep-mount... (Tim Stoop)
• Removed the bind mount, now requires LVMv2
• Support reading login/password from ~/.my.cnf (Baron Schwartz)
• Documentation fixes and improvements (Ville Skyttä) (Bug #302144)

XtraBackup is an Open Source online (non-blockable) backup solution for the InnoDB and XtraDB storage engines. It works with both MySQL 5.0 and 5.1 (and possibly 5.4 as well) and is distributed under the GPLv2.

Some weeks ago Vadim announced the availability of xtrabackup-0.7, stating that they consider it stable enough now to label this version a "Release Candidate". I've been maintaining RPM packages of xtrabackup on the fine openSUSE Build Service for quite some time now, RPMs of 0.7 for a number of distributions are now available for download. Please report any bug reports via the bug tracker on Launchpad.

Today I gave my first MySQL University session as a speaker, talking about Backing up MySQL using file system snapshots. The talk went quite well (at least that was my impression) and we had ~10 people attending. The slides (PDF) and a recording of the session are now available from the Wiki page. Unfortunately the recording lacks the audio track, which is a bit of a bummer. We've submitted a support request with the DimDim folks, so hopefully they can provide us with a complete recording.

There was one question during the session that I was not able to answer myself, so I'm asking for your insights here:

Consider we're using InnoDB and MyISAM tables on a file system that can be snapshotted (e.g. Linux LVM or ZFS) and we're performing the following operations:

1. FLUSH TABLES WITH READ LOCK (yes, this won't help for the InnoDB tables)
2. Create the snapshot
3. Store the output of SHOW MASTER/SLAVE status in a file to be part of the backup
4. UNLOCK TABLES
5. Mount the snapshot
6. Start a second MySQL instance that accesses the tables on the snapshot, let InnoDB perform its table recovery
7. Shut down the second instance and perform the backup of the snapshot

The question that came up was if this actually still is a consistent backup, considering that InnoDB rolled back uncommited transactions. Does the state of the tables still match the binary log positions we noted before? I assume yes, as long as the transaction does not involve modifications non-transactional tables.

Another suggestion that came up was to change InnoDB's configuration variable innodb_max_dirty_pages_pct to "0" prior to performing the snapshot, to minimize the amount of dirty pages that have not been written to disk (and thus reducing the time required for recovering later). I wonder if this would make a difference...

What other InnoDB variables might have a noteworthy effect in the context of snapshot backups? I am looking forward to your comments.