Lenz Grimmer's blog

After a long and uneventful flight from HAM to SFO via FRA, I arrived safely in Santa Clara yesterday. Today we spent the day in San Francisco, for some sightseeing (Downtown, Chinatown, Pier 39 and Fishermen's Wharf) and a bit of shopping. It was a very nice and sunny day, the sunlight helped a lot with getting over the jetlag for a while.

But now I'm pretty tired - I look forward to meeting friends, colleagues and members of our community at the MySQL Conference, which starts next week. I will also give a talk together with Colin Charles about MySQL Server Backup, Restoration, and Disaster Recovery Planning as well as moderating a BoF about MySQL Code Contributions, where we will talk about the recent developments and changes that we've implemented (or plan to put in place) to make it easier for developers to contribute code to the MySQL project.

I will try to take lots of pictures during the conference, which will be posted to my MySQL Conference Photo Set on Flickr every once in a while.

On early Tuesday morning, I made a quick trip to Paris, France, to attend and speak at the Solutions Linux / Open Source 2009 Conference. I've never been to this conference before and was quite surprised about its size - it's actually the largest Open Source event in France and it reminded me a lot of LinuxTag in Germany. Many well-known vendors (e.g. Sun, Novell, Canonical, Bull, etc.) were exhibiting. The also was a large "DotOrg" section for various Open Source projects and I was very happy to see that LeMUG.fr, the official MySQL User Group of France, had a table there, too! A big Thank You goes to Pascal Borghino, who manned that table on his own most of the time and answered questions about MySQL. I walked around the exhibition floor and took some pictures, which I have now posted to my Flickr account.

In addition to the exhibition, there were several parallel tracks with sessions. I was invited to speak about MySQL HA Solutions in the "Aquarium". Unfortunately I had the last slot at that day and they were running a bit behind schedule, so I had only 15-20 people in the audience. But I still had a great time and I received several positive comments about my presentation. I travelled back home early the next day - I wish I had scheduled some more time to attend the conference. I look forward to going there again next year, it was a nice event.

My travel schedule is getting quite crowded for the next months - I just received confirmation that I will be speaking at the phpDay in Verona, Italy on May 15-16th. I'll be talking about bzr - The Bazaar source revision control system as well as MySQL Backup and Security - Best practices in the developer track of the conference. I've never been to Verona, but it seems like it's a beautiful city. I look forward to being there!

At this year's CeBit 2009 trade fair, there was a dedicated exhibition area called the Open Source Forum. In addition to providing booths for various Open Source projects, they also had a stage for presentations about different topics. Every talk was recorded and available as a live stream during the show.

My dear colleague Simon Phipps spoke about "Adoption-Led: The Third Wave of Open Source" on Wednesday, I gave a presentation about "MySQL Backup and Security Best Practices" on Friday. The recording of these and all other talks as well as the Linux New Media Award ceremony are now available from the archives. Enjoy!

Version 1.0.3-alpha of the MySQL Connector/C++ has just been announced by Lawrin Novitsky and is now available for download. This driver is licensed under the GPL and is a new implementation of the MySQL Client/Server protocol. Instead of wrapping the C API calls in C++ methods, the implementation mimics the JDBC API, which hopefully feels much more "native" to a seasoned C++ developer.

The driver has been ported to a wide range of platforms and is about to hit the beta test phase. So if you're writing a C++ application that needs to connect to a MySQL Server, give it a try! The developers are always looking for feedback.

More information and examples on how to use it can be found on this detailed Wiki page on the MySQL Forge.

Today I gave my first MySQL University session as a speaker, talking about Backing up MySQL using file system snapshots. The talk went quite well (at least that was my impression) and we had ~10 people attending. The slides (PDF) and a recording of the session are now available from the Wiki page. Unfortunately the recording lacks the audio track, which is a bit of a bummer. We've submitted a support request with the DimDim folks, so hopefully they can provide us with a complete recording.

There was one question during the session that I was not able to answer myself, so I'm asking for your insights here:

Consider we're using InnoDB and MyISAM tables on a file system that can be snapshotted (e.g. Linux LVM or ZFS) and we're performing the following operations:

1. FLUSH TABLES WITH READ LOCK (yes, this won't help for the InnoDB tables)
2. Create the snapshot
3. Store the output of SHOW MASTER/SLAVE status in a file to be part of the backup
4. UNLOCK TABLES
5. Mount the snapshot
6. Start a second MySQL instance that accesses the tables on the snapshot, let InnoDB perform its table recovery
7. Shut down the second instance and perform the backup of the snapshot

The question that came up was if this actually still is a consistent backup, considering that InnoDB rolled back uncommited transactions. Does the state of the tables still match the binary log positions we noted before? I assume yes, as long as the transaction does not involve modifications non-transactional tables.

Another suggestion that came up was to change InnoDB's configuration variable innodb_max_dirty_pages_pct to "0" prior to performing the snapshot, to minimize the amount of dirty pages that have not been written to disk (and thus reducing the time required for recovering later). I wonder if this would make a difference...

What other InnoDB variables might have a noteworthy effect in the context of snapshot backups? I am looking forward to your comments.

Shortly after I blogged about my upcoming talks and events, I got informed that one of my session proposals for the next09 conference was accepted. This event will take place in Hamburg, Germany on Tuesday and Wednesday (5th and 6th of May), so at least there isn't much travel involved.

My talk about "Working for a Virtual Company: How we do it at MySQL" has been scheduled for 9:00 am on Wednesday morning, parallel to a keynote by Brian Solis from FutureWorks. I'm not sure how many people will actually show up, but I look forward to giving a talk about a less technical subject for a change! I'm going to share my experiences how it's like working in a virtual company of our scale and talk about some of the key aspects of what makes this model work. If you happen to be in Hamburg at that time, consider attending the conference!

The conference program looks quite interesting and there are quite a few well-known names on the speakers roster. Considering that I'll also be giving two talks about MySQL at AMOOCON on Monday that very same week, it'll be some busy times...

My calendar for the upcoming months is already filling up with conferences, trade fairs and other events at which I'll speak about MySQL. Here's a quick overview:

• This coming Thursday at 15:00 CET, I'll be speaking about "Backing up MySQL using file system snapshots" at the MySQL University. The session will be hosted live using DimDim, which is a great online conferencing and presentation system (Flash required). Attendance is free, so come and join me if you want to learn more about this backup technique!
• On Friday, 6th of March at 15:15 I'll give a presentation about "MySQL Backup and Security" in the CeBit Open Source Forum in Hannover, Germany. By the way, Sun will have a large presence themed "Open Source for Open Minds" in Hall 6, Booth E36. There'll be a MySQL info pod as well, which I will help manning from Friday until Sunday, 8th. So make sure to stop by and say hello!
• On March 23-24, there'll be a PHP/MySQL Conference in Warsaw, Poland, organized by IDG. I submitted three talks about various MySQL topics, currently I am still waiting for the confirmation which of these they selected.
• On April 20-23, there is of course the MySQL Conference & Expo in Santa Clara, to which I am very much looking forward to. I'm helping to organize the DotOrg Pavilion in the exhibition hall and will likely give a BoF about a project I'm currently involved in.
• In May 4-5 there is the AMOOCON in Rostock, Germany, which is a conference about Open Source VoIP and communication. I'll talk about "MySQL High Availability Solutions" and "Backup & Security Best Practices" (in German). My colleague Geert will be there as well, giving an in-depth talk about MySQL Cluster.

There is a number of additional events in the pipeline, I'll blog about these once my session proposals have been accepted.

Sorry for the downtime of this site - until around a week ago I hosted my home page on a trusty Genesi Pegasos II system (powered by a PowerPC G4 Processor clocked at 1GHz, using Debian 4.0 PPC with 512 MB of RAM), serving these pages from my home DSL connection. Unfortunately this system provided no means of redundancy - the hard disk drive died.

Luckily I perform frequent backups, so I moved most parts of the site to a shared hosting space now - the picture gallery is unfortunately too big to fit into the space that I have there. I'll try to move the pictures into my Flickr account instead, but this will take some time.

Note that the primary domain name of this site is now lenzg.net - lenzg.org, (the domain that I tried to promote as the official domain for my site) used to redirect to the home machine at lenz.homelinux.org. Both now redirect to the new address instead. I've initiated the move of the lenzg.org domain to the other provider as well, so soon this site will be available from both the .org and .net domain. Please don't link to lenz.homelinux.org anymore, as that site will eventually go out of service. Until then, a small openSUSE Linux box (Intel PIII, 500 MHz, 192 MB of RAM) running lighttpd will perform the URL redirection.

We've now concluded our call for papers for the MySQL Developer Room at FOSDEM 2009 in Brussels, Belgium, which will be open on Sunday, 8th of February from 09:00-17:00.

We received some excellent proposals and I am very excited about the schedule. Here's the quick summary of the talks:

• Vladimir Kolesnikov: Practicing DBA's Guide to the PBXT Storage Engine
• Kris Buytaert: Monitoring MySQL
• Geert Vanderkelen: MySQL Cluster
• Roland Bouman: MySQL 5.1 Plugins
• Kaj Arnö: MySQL, powering and using Social Networks
• Ewen Fortune: Percona MySQL patches and the XtraDB storage engine
• Giuseppe Maxia: Boost performance with MySQL 5.1 partitions
• Jurriaan Persyn: Database Sharding

See the Schedule page on the MySQL Forge for the detailed agenda, including the detailed session abstracts and speaker bios. These talks will soon appear in the general FOSDEM schedule, too. If you are interested in MySQL and any of the topics above, consider visiting us in Room AW1.126! Participation and attendance is totally free, though the organizers happily accept donations and sponsorships.

In addition to the Developer Room, MySQL will share a project desk with the OpenSolaris community. We are still looking for at least one more volunteer that would help us with manning the desk! If you are interested in helping out (2 hours at a minimum), please drop me a line!

A (slightly belated) Happy New Year to you! I just returned from my Christmas vacation two days ago, which I spent mostly at home and with my parents-in-law in St. Radegund, Austria. Now I am busy catching up with what has piled up during my absence (I managed to resist the temptation to check my work email during the time off).

Some MySQL-related news that came up in the past weeks and are worth sharing:

• My talk about MySQL HA solutions has been accepted in the main FOSDEM conference track
• The FOSDEM organizers also accepted my lightning talk proposal about Bazaar - it will take place on Saturday, 14h20 (tentative)
• MySQL will have a project stand, that we will share with the OpenSolaris Community. We're still looking for one or two more volunteers that would help us manning the table! If you would like to help out, please register here!
• MySQL will have a developer room at FOSDEM on Sunday. We will be in Room AW1.126 (72 seats) and the session schedule looks great already! We're still working on the fine tuning and scheduling.
• We've almost finished with creating a Japanese version of Planet MySQL (some minor encoding quirks remain to be fixed) and I finally got rid of the remaining parts of code that still used MagpieRSS (only in the admin backend parts) - now everything uses SimplePie instead.
• Dups added the MySQL Buzz to Planet MySQL
• I'm also working on sending out invitations to various MySQL-related projects to represent and showcase their work at the DotOrg Pavilion of the MySQL Conference Exhibition Area. Currently we have confirmation for attendance from Drupal, phpMyAdmin and Sphinx. What other projects would you like to see there? I am open for suggestions/applications

Quite a lot of exciting stuff going on, and more to come. This is a great start into the new year!

Now that MySQL 5.1 has finally been labeled and released as "GA", we have noticed quite a significant increase in the download numbers for 5.1. I find this quite promising, despite the comments from some people that suggested to stay away from it for now. I performed updates of 5.0 to 5.1 several times already, and did not observe any serious problem so far. But then again, I am by no means a power user, so your mileage may vary...

I assume that many of the users are already using some older versions of MySQL and will have to perform an upgrade from the previous version to the new one. As many things have been changed and improved between 5.0 and 5.1, an upgrade needs to be planned carefully. The MySQL reference manual has an entire chapter devoted to this subject. The best general advice is probably to create a backup of your data first before attempting to upgrade!

 

I performed an upgrade test on an openSUSE 11.0 system that was running on the MySQL version shipped with the distribution (5.0.51a). I first loaded it with some databases that I copied from other live systems (via SQL dumps). Then I grabbed the Server, Client and shared library RPMs the "Linux x86 generic RPM (dynamically linked)" section of the 5.1 downloads page and installed them with rpm -Uhv MySQL-server-5.1.30-0.glibc23.i386.rpm MySQL-client-5.1.30-0.glibc23.i386.rpm MySQL-shared-5.1.30-0.glibc23.i386.rpm. The packages were installed without a hitch and properly replaced the existing 5.0 packages (even though they use different package names). I then restarted the server and ran mysql_upgrade afterwards.

 

Success! I was pretty impressed how smooth this upgrade was performed. So far, I did not notice any regression or incompatibility. But I would be interested in hearing stories and comments from other users about their upgrade experiences!

So here are some questions I would like you to answer — I would appreciate your feedback either via this blog or by email to firstname at sun dot com:

• What are your expectations and most important requirements for performing MySQL upgrades?
• Have you upgraded to MySQL 5.1 yet?
• If not, why not?
• If yes, what was your upgrade experience like? Any serious issues or problems? What platform and package format did you use?
• Did you use the documentation in our reference manual to perform the update? Was it helpful? What could be improved?
• Do you have any other comments/suggestions on the topic of upgrading MySQL?

I am looking forward to your comments. Thanks and Happy Holidays!

FYI, we've now added an Italian section on Planet MySQL: http://it.planetmysql.org

If you are a MySQL enthusiast from Italy and would like to start blogging about it in your native language, please consider submitting your feed for inclusion!

Giuseppe just recently started blogging in Italian as well and has already added his feed there.

The FOSDEM 2009 organizers have been very kind to us this year: in addition to a Developer Room on Sunday, we now also have a project desk (1 table) on both days! Thank you very much! In summary, this means that there will be the following MySQL-related activities:

• We will have a Project Stand (one table) on both days
• On Sunday, we will have a MySQL Developer Room, allowing us to schedule our own track of talks about MySQL and related projects
• I have been invited to give a talk about "MySQL High Availability Solutions" in the main conference track
• I also submitted two proposals for lightning talks ("What's new in MySQL 5.1" and "Why you should use Bazaar for maintaining your OSS project") (pending approval)

In addition to employees from Sun/MySQL, we would also like to encourage and invite members of the MySQL Community to contribute to making the MySQL project's presence a success. We are looking for your support and contribution! Here is how you can help:

Give a talk about MySQL in the developer room

The MySQL Developer Room will be open on Sunday, 8th of February, from 9:00-17:00. We would like to set up a schedule of talks related to MySQL. As the audience will mostly be developers and DBAs, we are looking for in-depth technical sessions about the MySQL Server and related projects/tools (e.g Cluster, Proxy, Connectors, etc.). Each session will last for 45 minutes, followed by 15 minutes of Q&A.

If you would be interested in giving a talk, please submit your proposal via this page now! The deadline for proposing a talk is January 15th, 2009. We look forward to your submission!

After the call for papers has been closed, we will comment and vote on all proposals and will get back to you about further details!

Record/transcribe sessions

For people that could not make it to FOSDEM or some of the sessions, it would be nice to be able to provide video recordings or transcriptions (e.g. live blogging) from these talks. We're looking for volunteers to record the sessions! If you have a video camera or are an expert blogger, we'd like to hear from you.

Help us manning the project stand

We are also looking for MySQL users that are willing and excited to talk about MySQL to other people and would like to assist us with manning our project desk. Your task would be to be present at the project table, talk with other users and developers about all things MySQL, hand out merchandise (in case we get some) and marketing material as well as providing them with additional useful information (e.g. links to further information, contact information).

So if you are a MySQL enthusiast and plan to attend FOSDEM, consider sparing at least two hours and join us to man the table! I've set up a quick registration form in which you can leave your name, contact information and times you would be available. We will then get back to you about further details. Thank you!

Contact / Mailing list

To facilitate the organization and for further discussion, please join our public Community mailing list! I've also set up a Wiki page on the MySQL Forge to track and document our FOSDEM 2009 activities. I am looking forward to your feedback and suggestions!

Some days ago, I released version 0.11 of mylvmbackup a Perl script that performs consistent backups of a MySQL server by using LVM filesystem snapshots. The source archive as well as a generic RPM can be found on the project home page, packages for many Linux distributions are available on the openSUSE Build service.

This release includes some new functionality as well as numerous bug fixes and improvements, most notably:

• Added support for using rsnap as a backup backend (Matt Lohier)
• The documentation is now maintained in POD style instead of asciidoc (Matthew Boehm)
• Support using non-GNU tar and additional compression methods (e.g. bzip2, lzma) (Alexander Skwar)
• Code cleanups, improved error handling and logging

I would like to thank all the contributors for their support! More details about the changes in this release, directly from the ChangeLog:

• Added new option "--quiet" that suppresses informal logging output (warnings and errors will still be printed/logged)
• Applied patch from "kjetilho" that makes mylvmbackup more robust and paranoid when it comes to handling errors/failures (Bug #298175) Now the script aborts cleanly in case of any error.
• Fixed error handling in case of a failed DB connection (patch submitted by Matthew Boehm, Bug #280989)
• Merged changes from Alexander Skwar: Fix for Bug #278478 and implemented Blueprint "Improve the tar backup backend of mylvmbackup"
• Fixed Bug #271671: "overloading parameters does not work" by removing the default values for host and port from the configuration file and removing the unnecessary check for passing both host and socket at the same time. Updated documentation and configuration file comments accordingly.
• Applied patch suggested by "Jonas" to fix Bug #267944: "backup returns successfully when snapshot creation fails"
• Code cleanup: moved flushing of tables in a separate subroutine flush_tables()
• Code cleanup: moved log messages into the respective subroutines
• Code cleanup: use return values of subroutines instead of updating global variables
• Improved some log messages to explain what was DONE or FAILED
• Code cleanup: build up long command strings in a $command variable before passing it to system()
• Renamed subroutine create_snapshot() to create_lvm_snapshot()
• Merged patch from Matthew Boehm: Removed old asciidoc documentation in favor of POD style. This removes the dependency on the external program a2x for creating documentation and uses the 'built-in' pod2html and pod2man instead. Updated the Makefile to accommodate the change.
• Applied patch from Matthew Boehm to make the backup file name suffix configurable via a "--suffix" option. Updated the man page accordingly.
• Applied patch from Matt Lohier to support rsnap as a backup backend 
• Moved the list of contributors from the man page into a separate CREDITS file, added missing names

Enjoy!