Lenz Grimmer's blog

It's been a while since my last post on this blog; I definitely need to get back into the habit! One of the reasons for my radio silence was that I switched roles here at Oracle. After having been with the MySQL team for 9 years, I felt it was time for a change. Fortunately I did not have to look far – I'm now a member of the Oracle Linux product management team and I am having a lot of fun there.

However, I realized that while I was an active Linux user on the desktop, quite a lot has happened on the enterprise and data center side of things. Linux has really come a long way and I am glad to be back in this field, drinking from the firehose and learning a lot about recent developments and technologies. For me, this is kind of going "back to my roots", as I have been deeply involved with Linux at SuSE before I joined MySQL in 2002.

Anyway, I'm still alive and you can expect to hear more about Linux and less about MySQL from me on this blog in the future. I also started writing for the official Oracle Linux Blog as well as the OTN Garage blog – watch these spaces for future posts from me, too.

In addition to that, my travel schedule is also filling up again: at the end of this month I'll be speaking at the fisl12 conference in Brasil and Oracle OpenWorld in San Francisco later this year. I also submitted talk proposals to the DOAG conference and the UKOUG conference. Looking forward to it!

Congratulations to the Drupal community for getting version 7.0 released! This is a major mile stone and an excellent reason to celebrate!

If you want to give Drupal 7 a try without having to install anything, I've now updated my Drupal 7 appliances on SuSE Studio to the latest release. The appliance is based on openSUSE Linux 11.3 and is available in two variants:

• A text-mode only appliance to which you connect using your local web browser via the network.
• A GUI version that starts up the Firefox browser in a minimalistic GNOME desktop to perform the installation locally. Remote network access is available, too.

The database backend is MySQL 5.1, with the InnoDB plugin and strict mode enabled. phpMyAdmin has been added to support web-based administration of the MySQL server. You can access it via http://localhost/phpMyAdmin/. I also added drush, the Drupal command line shell and scripting interface and some additional packages (yast2-http-server, bind-utils, php5-ctype, patch). I also bumped up the appliance's version number to 7.0.0, to match the version number of Drupal included.

The appliance is available in various formats:

• A live raw disk image, ready to be written to an USB stick or flash drive
• A live ISO image, to be burned on a CD or used in a virtual machine
• A hard disk image, to be dumped on a hard disk drive
• Various virtual disk formats, e.g. OVF, VMWare/VirtualBox/KVM and Xen

Please see the installation instructions provided on the download pages for details on how to use the various image formats.

So congratulations to the Drupal developer community for reaching this goal and thanks to SuSE/Novell/Attachmate for providing the infrastructure for creating such appliances. I also would like to especially thank Richard Bos for the testing and many suggestions for improvement of these appliances!

Over the weekend I updated my Drupal 7 test appliance in SUSE Studio to the Drupal 7.0-beta2 release, which was released on Oct. 23rd. I also added phpMyAdmin upon a user request, to provide a web-based method to work with the MySQL instance, if needed.

In addition to the lightweight "headless" appliance (which can only be accessed and configured via a remote network connection), I've now also created a GUI-based version. This appliance starts a minimal GNOME desktop and a Mozilla Firefox browser, which in turn opens the Drupal installation page by default. I hope you will find this useful if you want to toy around and test Drupal 7 without having to go through the entire OS and LAMP stack configuration yourself. In fact, you can even test this appliance via the recently added test drive option from right out of your web browser!

The appliance is now also available in OVF format. SuSE Studio now also builds Amazon EC2 images, which don't seem to be available for download from the SUSE Gallery yet. I assume this is a recent addition to the continuously improving SUSE Studio functionality, hopefully these images will be made available soon.

The Drupal community just recently released another alpha test release of their upcoming Drupal 7 version, to shake out the remaining bugs and to encourage more users to test it.

If you would like to give it a try, but you don't have a free server handy, how about using a virtual machine instead? Using the fabolous SuSE Studio, I've created an appliance based on openSUSE 11.3, Drupal 7.0-alpha7 and MySQL 5.1 with the InnoDB plugin and strict mode enabled (both for the SQL mode and InnoDB mode. Using this configuration helps to ensure that Drupal works well with the current version of MySQL/InnoDB and does not use any "questionable" SQL statements. This might be especially interesting for additional modules - Drupal core did not reveal any problems using strict mode so far.

You can download disk images for VMware/Virtualbox/KVM or XEN from the SUSE Gallery (free login required). Just boot the appliance in your virtualization application of choice, choose your keyboard layout and step through the network configuration and Time Zone selection. Once the appliance has booted up and the login: prompt appeared, point your web browser to the appliance's IP address to start the Drupal installation/configuration. MySQL has been pre-configured, there is an empty database named "drupal" and a user "drupal" with the same password to access it. You just need to enter this information in the Drupal Database configuration dialogue during the installation. Anything else can be configured to your liking.

After you have finished the installation, you can toy around with a fresh Drupal 7 installation! Install additional modules, change the themes, add content. And make sure to report all bugs that you run into while doing so! Have fun.

I've been working in a fully distributed work environment for almost 8 years now (I joined MySQL AB in April, 2002). Therefore I've been reading Toni Schneider's blog post about the "5 reasons why your company should be distributed" with great interest – he raised several points that I fully agree with and which I covered in my talks about "Working for a virtual company - how we do it at MySQL" at last year's next09 conference (slides, video) and at FrOSCon 2009 (video).

However, Toni draws a profusely positive picture here, or, as my dear colleague Dean pointed out "The blog overly simplifies the realities of a distributed workforce, making it sound like it's all ponies and rainbows".

I recently received a question from Robin Schumacher at Calpont, the makers of the InfiniDB analytics database engine for MySQL: "How would you recommend we try and get bundled in with the various Linux distros?"

Since this question has come up several times before, I thought it might make sense to blog about my take on this.

First of all, please note that there is a difference between "being part of the core distribution" and "being available from a distributor's package repository". The latter one is relatively easy, the former can be hard, as you need to convince the distributor that your application is worth devoting engineering resources to maintain and support your application as part of their product. It's also a space issue – distributions need to make sure that the core packages still fit on the installation media (e.g. CD-ROMs or a DVD). Therefore they take a very close look at each package and if it's really needed to be part of the installation medium or if it's fine to provide it for download from a package repository instead.

Distributors prefer to keep their core product small and restricted to the "basic OS building blocks". While MySQL might still be considered to be a part of this, this probably does not apply to the various plugins and extensions that are available for it. Therefore the best approach is to invest some engineering time and start doing the packaging yourself, either by hiring an engineer capable of creating and maintaining the packages, or by finding someone in your community who has the required experiences and is willing to do it.

While it's of course possible to set up and maintain your own build and package hosting infrastructure for that, I recommend to make use of the existing services provided by the distributors.

The top tier distributors all provide means of offloading the maintenance of "non-core" packages to their community, offering various options for packages to be made available. For example, Novell/openSUSE provide the free "Build Service", which is capable of building packages for other distributions as well (e.g. Fedora, Mandriva, Debian/Ubuntu, etc.). In addition to automating the builds, the Build Service also takes care of the distribution via their download mirror network and ensures that your application can be found via their package search interface.

Red Hat/Fedora provide something similar, named "Koji" – but it's "Fedora only". Here's a HOWTO that outlines the process of becoming a Fedora package maintainer.

Ubuntu/Canonical have "Personal Package Archives (PPAs) – if your project is hosted on Launchpad already, that might be something to look into for providing Debian/Ubuntu packages. Alternatively you could join the Debian project and start building and maintaining your package there. They maintain a list of "Work-Needing and Prospective Packages", a description of the process on how to become a new maintainer is outlined here.

If you'd like to target Solaris/OpenSolaris as well, there is the OpenSolaris Source Juicer – a web service which allows OpenSolaris community developers to build packages (using RPM spec files) and publish them for review, so they will be included in an official package repository. The Software Porters Community Group coordinates, advocates, encourages and helps with the porting of Software from multiple Platforms to the OpenSolaris Platform.

So you're a small startup company, ready to go live with your product, which you intend to distribute under an Open Source License. Congratulations, you made a wise decision! Your developers have been hacking away frantically, getting the code in good shape for the initial launch. Now it's time to look into what else needs to be built and setup, so you're ready to welcome the first members of your new community and to ensure they are coming back!

Keep the following saying in mind, which especially holds true in the Open Source world: "You never get a second chance to make a first impression!". While the most important thing is of course to have a compelling and useful product, this blog post is an attempt to highlight some other aspects about community building and providing the adequate infrastructure. This insight is based on my own experiences and my observations from talking with many people involved in OSS startups and projects.

The summer break seems to be over and the event season is heating up again! There is a number of conferences and events coming up in the next months — here is a quick summary of the events that I plan to attend.

This Friday I will attend an event here in Hamburg: the "Silpion Sommerfest", organized by Silpion (a local IT solutions provider which is a partner of Sun Microsystems as well). I will be there to network and talk about MySQL.

This coming weekend (2009-09-12/2009-09-13), there will be the PHP Unconference here in Hamburg, Germany . It will consist of two days of Barcamp-style sessions about PHP. Sun/MySQL are sponsors of the event and I expect several of my team mates to be there as well. With more than 180 participants, the event is already sold out.

The following week I will be attending the openSUSE Conference in Nuremberg, Germany on 2009-09-17/2009-09-20. I will give the opening keynote on Thursday morning. Titled "Working in a Virtual Community", I will talk about the pros and cons of working in a virtual organization, giving an overview about some of the technical and social aspects that play a role in working with virtual communities.

On December 10th and 11th I will be attending the 4. Secure Linux Administration Conference 2009 (SLAC) in Berlin, Germany. I've been invited to talk about MySQL and will give two sessions about MySQL Backup & Security as well as MySQL High Availability Solutions. The Call for Papers for this event is still open, so if you have a technical, "best practice" talk that might be relevant for system administrators, consider submitting your proposal!

 

I admit it — I'm a fan of simulation software, particularly flight simulators. Probably the best Open Source Flight Simulator out there is FlightGear — it provides an impressive level of reality and you can download and install many additional plane models and terrains. There are packages of FlightGear 1.0.0 in the games repository of the openSUSE Build Service, which works quite well and I have been enjoying it a lot. However, the FlightGear project released version 1.9.x quite a while ago (1.9.1 was published in January 2009) and I was itching on giving the new version a try (just take a look at the screenshots and you know what I mean). However, building FlighGear on Linux is quite a complex task with many dependencies, and so held off from doing it myself, waiting for someone else to perform the update...

Well, this weekend I finally bit the bullet and did it myself - FlightGear 1.9.1 has now been added to my home:LenzGr build repository. I based my packages on the ones included in the games repository, but I plan on cleaning them up a bit and splitting them into separate packages (currently the FlightGear source RPM contains SimGear and fgrun as well). I also "borrowed" the OpenSceneGraph sources and spec file from the PackMan repository, in order to have a functional build. Unfortunately FlightGear currently only builds on a very limited list of distributions so far (namely OpenSUSE 11.0, just what I needed) — I haven't had time to adapt the spec files for FlightGear and OpenSceneGraph to match the appropriate build dependencies for the other distributions yet and "02-check-gcc-output" gives me some grief on platforms where it actually builds but generates compiler warnings (but patches are welcome!)...

Shortly after I created the initial packages of embedded InnoDB on the OpenSUSE Build Service, Oracle/Innobase released an updated version (1.0.3.5325). In addition to many improvements and bug fixes, they slightly changed the versioning scheme to better indicate what version of the InnodDB plugin their code is based on (see Vasil's posting on the InnoDB Forums for more information).

I've now updated my InnoDB packages on the Build Service to this version as well - please note that the naming scheme of the shared library package has been changed from "embedded_innodb1" to "libinnodb2" — RPM will take care of replacing the old package during update, even though the name has changed.

Oracle/InnoBase announced the availability of the embedded version of InnoDB at this year's MySQL Conference & Expo, but I have not seen a lot of comments or reviews about it so far. Which surprises me, because I think this is a very interesting piece of technology!

In my opinion it might actually hit the sweet spot for application developers seeking an alternative embedded database solution. SQLite is nice and popular, but it seems to have concurrency issues when used in multi-threaded applications. An embedded MySQL server would be an alternative - this is what the Amarok developers decided to go with, for example. But this approach has its issues, too, especially the lack of a shared library version of libmysqld poses some challenges when distributing binaries.

This is where I think the embedded version of InnoDB might have an edge. It's pretty lightweight in comparison to a full-blown MySQL server, provides excellent crash-recovery (which is essential for desktop applications), transactions (useful in environments with high concurrency) and foreign key constraints. I'm not sure how important these are for embedded use cases, it probably depends on the complexity of the data to be stored. On the downside, Embedded InnoDB does not "speak" SQL. In order to store and retrieve values, you need to use the InnoDB API. See the chapter Concepts and Architecture for more details and an overview.

Another possible reason for the low popularity might be that it's currently not part of any Linux distribution (yet) and that Oracle only provides binary tarball packages for Linux and a Windows binary for download from the web site.

Therefore I've now created a spec file to build RPMs of Embedded InnoDB and added it to my repository on the openSUSE Build Service, which now provides Embedded InnoDB packages for a wide range of RPM-based Linux distributions. I hope that the spec file will be included in the next source distribution. I've posted it (and a patch to fix a few problems with the examples) to the newly created InnoDB mailing list, but to be sure I added a note to the Embedded InnoDB Forum as well.

XtraBackup is an Open Source online (non-blockable) backup solution for the InnoDB and XtraDB storage engines. It works with both MySQL 5.0 and 5.1 (and possibly 5.4 as well) and is distributed under the GPLv2.

Some weeks ago Vadim announced the availability of xtrabackup-0.7, stating that they consider it stable enough now to label this version a "Release Candidate". I've been maintaining RPM packages of xtrabackup on the fine openSUSE Build Service for quite some time now, RPMs of 0.7 for a number of distributions are now available for download. Please report any bug reports via the bug tracker on Launchpad.

Today I attended the Drizzle Developer Day which took place in the auditorium of the Sun Campus in Santa Clara.

Many of the the Drizzle core hackers as well as several other people interested in the development attended this event, hacking away and discussing various issues. Jeremy Zawodny gave a presentation about Craigslist's needs for Drizzle, Jay Pipes gave an overview over Google's protocol buffers library. I took a number of pictures, which you can find in my Flickr photo set.

I joined a group of people that haven't built Drizzle from source by themselves so far, helping them with installing Bazaar and the required libraries. As Drizzle requires several third-party libraries that sometimes are not included in the common linux distributions (or only in outdated versions), we spent some time in getting these build requirements fulfilled.

One of the requirements for building Drizzle is libdrizzle - the client & protocol library. So one first has to download and compile this one, before the actual build of the server can proceed. I noticed that the libdrizzle source distribution contained an RPM spec file already, so I've been working on adding libdrizzle to the openSUSE build service today. The packages for various distributions (Fedora, openSUSE, RHEL, Mandriva) will be available for download shortly. Along the way I also fixed several small issues in the spec file and created a libdrizzle-devel subpackage. The patches are now proposed for merging on Launchpad, I hope Eric will take a look at these shortly.

On early Tuesday morning, I made a quick trip to Paris, France, to attend and speak at the Solutions Linux / Open Source 2009 Conference. I've never been to this conference before and was quite surprised about its size - it's actually the largest Open Source event in France and it reminded me a lot of LinuxTag in Germany. Many well-known vendors (e.g. Sun, Novell, Canonical, Bull, etc.) were exhibiting. The also was a large "DotOrg" section for various Open Source projects and I was very happy to see that LeMUG.fr, the official MySQL User Group of France, had a table there, too! A big Thank You goes to Pascal Borghino, who manned that table on his own most of the time and answered questions about MySQL. I walked around the exhibition floor and took some pictures, which I have now posted to my Flickr account.

In addition to the exhibition, there were several parallel tracks with sessions. I was invited to speak about MySQL HA Solutions in the "Aquarium". Unfortunately I had the last slot at that day and they were running a bit behind schedule, so I had only 15-20 people in the audience. But I still had a great time and I received several positive comments about my presentation. I travelled back home early the next day - I wish I had scheduled some more time to attend the conference. I look forward to going there again next year, it was a nice event.

Sorry for the downtime of this site - until around a week ago I hosted my home page on a trusty Genesi Pegasos II system (powered by a PowerPC G4 Processor clocked at 1GHz, using Debian 4.0 PPC with 512 MB of RAM), serving these pages from my home DSL connection. Unfortunately this system provided no means of redundancy - the hard disk drive died.

Luckily I perform frequent backups, so I moved most parts of the site to a shared hosting space now - the picture gallery is unfortunately too big to fit into the space that I have there. I'll try to move the pictures into my Flickr account instead, but this will take some time.

Note that the primary domain name of this site is now lenzg.net - lenzg.org, (the domain that I tried to promote as the official domain for my site) used to redirect to the home machine at lenz.homelinux.org. Both now redirect to the new address instead. I've initiated the move of the lenzg.org domain to the other provider as well, so soon this site will be available from both the .org and .net domain. Please don't link to lenz.homelinux.org anymore, as that site will eventually go out of service. Until then, a small openSUSE Linux box (Intel PIII, 500 MHz, 192 MB of RAM) running lighttpd will perform the URL redirection.