Skip to content

New package: pine-gpg-filter

After getting very annoyed about the behaviour of pinepg in combination with gpg2 on my openSUSE 10.3 beta test system, I have now scratched my itch and switched to an alternative tool: pine-gpg-filter:

The distinguishing characteristic of this package (when compared against similar pine and gpg wrappers) is its ability to handle multiple roles or identities (i.e. different keys for different email addresses). Unlike some of the other pine and gpg wrappers, this one performs no passphrase caching (consider using gpg-agent in gnupg2).

They provide a "noarch" RPM directly from that site, but I've now also added it to my home:LenzGr repository in the openSUSE build service. Enjoy! 

Binary builds of MySQL Proxy available via the openSUSE build service

In addition to the binary downloads that we provide from our site, Linux RPM builds of the MySQL Proxy (both the latest stable version as well as SVN snapshot releases, named mysql-proxy-snapshot) are now available for download from the server:/database repository of the openSUSE build service. It provides packages for a number of Linux distributions, e.g. Fedora 5/6, SLES 9/10 SuSE/openSUSE 10.x. By the way, this repository also contains RPMs of the current 5.0.45 MySQL Community Server for the distributions mentioned above. Thanks a lot to Darix for the initial checkin of the proxy package!

 

Updated packages in the openSUSE Build Service

JFYI, I recently updated several packages in my home:LenzGr and the devel:tools:scm repositories of the openSUSE Build Service:

Enjoy! And Kudos to the folks that develop and maintain the build service, this is a great service to the Community.

Comparing Web2.0 with Open Source

This thought has been floating around my head for quite some time now and I finally bit the bullet and released it from the draft state it had been sitting in for too long: there are quite many similarities between Open Source Software (OSS) projects and most of today's popular Web 2.0 sites, but there is also one odd difference that I wonder about.

For both worlds, the concept of collaboration, participation and giving more power to their users is a key component. OSS projects need contributors for patches and bug reports, but also for feedback, translations, artwork, advocacy in order to be popular and healthy. The project's developers need to be open for suggestions, listening to their user base on where the project should be heading. They also usually strive for open standards and APIs, allowing easy combination and interaction with other Open Source Software so that other projects can build their work on top of theirs or compliment it. A successful OSS application gains popularity by the snowball effect: users of the product find it useful and recommend it to other users seeking for similar functionality. These too start to contribute to the project in various ways, making it in turn more useful and attractive for even more users and so on.

The same is true for Web 2.0 sites: they depend on a user base that creates and provides content that makes their site useful and popular, e.g. blog postings, pictures, articles and other content, but also by providing feedback and bug reports about the site itself and recommending it to others. A successful Web2.0 site needs to be as open to user feedback as a regular OSS project in order to remain attractive and maintain a loyal user base. So it's a lot about participation and empowering the users to contribute and control the content - the part that actually makes a site useful and attractive. Many of these Web2.0 sites also provide public APIs (e.g. SOAP, REST, XML-RPC or RSS, just to name a few acronyms) that allow others to access their services and functionality and reuse or combine them with their own work. And by using technologies like AJAX, most of these sites actually look and feel like regular desktop applications already.

But there is a strikingly difference: even while most Web2.0 sites are based on Open Source technology (e.g. the LAMP stack or Ruby on Rails), only very few ones actually carry on the OSS philosophy and make their own code available as Open Source as well. Not that they would have to do so, as there is this so-called ASP loophole in the GPL and most other OSS licenses: web sites are not actually distributing their application code, so they are not required to disclose the source of their applications.

But why do they break the chain here? One could argue that the obvious reason for that is that most web sites are created and maintained by companies, that are doing this for business reasons and want to maintain their competitive edge over others. However, many sites started as a hobby project by an individual or a small group and were converted into commercial entitities once they became successful. Even in these early stages it's rare that sites provide their source code. But isn't Web2.0 mostly about the users and content and not so much about the technology behind it? Would it really make a difference if the source code of a site is freely available?

In the OSS world, users could just fork off a project, if they don't feel the project is going the right direction or it has been abandoned by the original author. This sometimes leads to much better applications than what the original project came up with. This also leads to a healthy competition and to much better software with less duplication of work in the long run. And this is being recognized by more and more software companies as well - there is a large and growing number of OSS applications that are not maintained by a group of volunteers, but by a commercial entity.

In the Web2.0 world, people start creating new sites from scratch again and again, resulting in a vast amount of sites that serve a similar purpose, all with their own warts and deficiencies, splitting up the user base and available content. This causes fragmentation and makes it much harder to create one useful resource. And they are all on their own in maintaining and improving their sites, to fulfill the requests of their user base and keeping them happy and loyal.

I wonder if and when Web2.0 developers will embrace and extend the concept of sharing their code as well, to let others help them to become even more powerful and popular. Once a site has gained a certain popularity, there is plenty of new features that their users will ask for, or bugs and deficiencies that are in need of fixing. Making the source code available would allow others to jump in here, opening yet another channel of possible contributions and community-building. The Open Source community provides numerous lessons to learn from. True, others may then be able to take the code and create a site that provides the same services and experience. But they still would have to gain traction by creating a community and content before they would become a serious competition.

One great example of how this could be done is Wikipedia - they maintain Mediawiki (the application that powers their site) as a regular OSS project, which has made it one of the most popular and powerful Wiki applications around and has allowed others to create Wiki sites about all kinds of topics on their own.

And while it's not really true Web2.0 company, I also welcome Linden Lab's decision to open up the Second Life client source code. I am confident that the experiences they make with this will encourage them to consider opening the sources of the server code as well at some point...

But they both have grasped that it's not so much about the technology and software, but much more about the users and the content created by the community that makes them popular and successful. This is something one can't simply take and create a competing offering, even though the source code is available.

Presenting the DotOrg Pavilion exhibitors of the MySQL Conference & Expo 2007

In two weeks from now, the MySQL Conference & Expo 2007 will take place in Santa Clara, California. Our team has been very busy with working with the conference management team from O'Reilly to prepare for this event, especially Jay who is in charge of organizing and arranging the more than 110 sessions that will be held in several parallel tracks. The conference program looks very exciting, I am impressed by the wide range of topics and quality of speakers that we have lined up.

In addition to the presentations and tutorials, there will also be an Expo, where partner companies will exhibit their products related to MySQL. You will notice a number of well-known names in the industry there, e.g. hardware vendors like Dell, Dolphin Interconnect, HP or Sun as well as software companies like Alfresco, BitRock, Google, Pentaho, Red Hat and many more.

One of my tasks for this conference is to arrange the DotOrg Pavilion, an exhibition area inside the expo hall that is reserved for Open Source Community projects that are based on or support MySQL in some way. The conference web page does not yet list them all, therefore I'd like to give you a full rundown of all the OSS projects that will be present there:
  • Bugzilla is an open source bug tracker used by thousands of businesses to track defects and enhancements. Bugzilla, the standard in open source bug tracking, is a versatile and powerful web based system with a minimal footprint, proven to support groups of any size from small projects to multinational enterprises.
  • dotProject is a Web-based project management framework that includes modules for companies, projects, tasks (with Gantt charts), forums, files, a calendar, contacts, tickets/helpdesk, multi-language support, user/module permissions, and themes. It is translated into 17 languages and has a modular design that allows extra modules (such as time sheets and inventory) to be added in easily.
  • Drupal is a powerful, scalable and full-featured content management system that allows an individual or a community of users to easily publish, manage and organize content and run an online community. Tens of thousands of people and organizations use Drupal to power scores of different web sites. A large, healthy and diverse community of developers, users and consultants drives a rapid pace of innovation and a wide variety of feature development.
  • Eventum is user friendly and extensible issue tracking software from MySQL that can be used by a support department to track incoming technical support requests, a software development team to quickly organize tasks and bugs or anyone else looking to organize their work.
  • HeidiSQL is an easy-to-use interface for developers using MySQL, allowing simple management of their databases and tables. Previously known as MySQL-Front, the name has been switched in 2006 by Ansgar Becker, the first developer, that put his old sources into a new open source project at SourceForge. Currently this tool is available on Windows and Linux.
  • Joomla! is one of the most outstanding content management systems in the world. This is not only because of his approach of providing one of the most well structured and simple user interfaces for managing your media, text, image and other public information materials in the web. With Joomla! you are able to provide a huge information base for your professional audience as simple as you build up your own family community website. The new refactored content framework gives you the confidence that your site is based on a reliable, flexible, future proven and easy to manage plattform.
  • LinuxQuestions.org is a free, friendly and active Linux Community with over 180,000 members and over 1,7000,000 posts. Founded in 2000 LQ offers forums, reviews, a Linux hardware compatibility list, a Linux knowledge base in wiki format, Linux tutorials, a free Linux download site, a Linux podcast, a Linux radio show and more. LQ has forums for everything from Linux Newbies to Linux in the Enterprise and has over 20 officially recognized Linux distribution forums.
  • OTRS is an Open source Ticket Request System (also well known as trouble ticket system) with many features to manage customer telephone calls and e-mails. The system is built to allow your support, sales, pre-sales, billing, internal IT, helpdesk, etc. department to react quickly to inbound inquiries. Do you receive many e-mails and want to answer them with a team of agents? You're going to love the OTRS!
  • Phorum, is the original PHP/MySQL message board software. Started in 1997 by Brian Moon, Phorum has been used by sites such as Zend.com and currently is the message board of choice for MySQL.com. Phorum's developers focus on speed and expandability through its flexible module system.
  • phpMyAdmin is a tool written in PHP intended to handle the administration of MySQL over the Web. Currently it can create and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields, manage privileges,export data into various formats and is available in 50 languages.
  • SnapLogic is an Open Source project that enables developers to easily and rapidly integrate applications and other data sources through a Data Integration Network. SnapLogic embraces the simplicity and scalability of the Web to collapse the complexity of data integration for Internet-scale integration.
  • Sphinx is fast and powerful full text search engine designed with indexing database content in mind. It offers high performance, good scalability (with multi-server distributed searching), great search quality, and other advanced features such as efficient filtering, sorting and grouping. Both native search APIs and a Pluggable MySQL Storage Engine search client are available, which makes Sphinx easy to integrate into existing applications.
  • WordPress is a state-of-the-art semantic personal publishing platform with a focus on aesthetics, web standards, and usability. What a mouthful. WordPress is both free and priceless at the same time. More simply, WordPress is what you use when you want to work with your blogging software, not fight it.
  • XAMPP is a free software package containing the Apache web server, MySQL database and necessary tools to use the PHP and Perl programming languages. Currently, XAMPP is available for Windows, Linux, Solaris and Mac OS X (the X in its name could stand for any one of these operating systems).
I hope I have gathered an interesting mix of projects and look forward to meeting with these people in person soon! If you attend the conference, please make sure to stop by the DotOrg pavilion and have a chat with the representatives from these projects!

Linux and OSS print magazines in Germany

I took this picture at Hamburg Airport while waiting for Kaj to arrive for our meeting last week. I was browsing through the magazines in a news shop in the arrival area and was pretty impressed about the amount of available magazines about Linux and Open Source Software. It amazes me how they all manage to produce enough content to fill a magazine on a frequent basis. I take this as a very positive sign - Linux has become a mainstream OS. I wonder how many people consider switching to Linux instead of going with MS Vista - the numerous CDs/DVDs that are included in these magazines significantly reduce the acceptance threshold for new users. Live CDs make it easy to give Linux a try without having to touch the installed operating system. The support and automatic detection and configuration of hardware in Linux has become much better nowadays, too. Granted, there is still a lot of room for improvement (especially when it comes to video card and WLAN chipset support, it seems) - but Linux distributions have reached a level of maturity that make them a viable alternative to Mac OS X or MS Vista for the most common use cases. And all the required applications are already included out of the box and are free (as in beer)! So what is missing to world domination?

To start or not start the MySQL server during the RPM installation?

So far, the MySQL Server RPM packages as provided by MySQL AB used to automatically start the mysqld process after the package has been installed. It has been like that since the very beginning and we think of it as a convenience for our users when they want to get up and running quickly.

However, Kristian raised an interesting point in BUG#27072 where he points out that automatically starting mysqld during the RPM installation might not always be the desired behaviour, especially in automatic installation environments or during a fresh installation (where the system might not be fully configured yet). Therefore he proposes to change this behaviour to not start mysqld as part of the installation.

While I personally agree with his proposal, this is of course a tricky decision: our users are familiar with this behaviour and it's never a good idea to cause surprises. Therefore this change could only be done in future versions of MySQL, where they can be properly announced and documented and don't cause too much confusion.

Then again, there are several options here. Should we just disable it for the Enterprise Linux RPMs (RHEL/SLES) and keep it enabled in the "generic" RPMs? This would be inconsistent and harder to document/explain. Should the server never be started at all, or should we keep the behaviour for updates at least (when the server was already running before the package update)?

We would like to get your take on this. Please post your comments and suggestions to the bug report (preferred) or leave a comment on my blog, so we can get an impression on what you would expect to be the correct behaviour here. Thanks!

FrOSCon 2007 Call for Papers now open!

I just stumbled over Sebastian's blog entry:

The second Free and Open Source Conference "FrOSCon" takes place on August, 25th and 26th 2007 in Sankt Augustin, near Bonn, Germany. The conference is once again hosted by the faculty of computer science of the University of Applied Sciences Bonn-Rhein-Sieg in collaboration with the student body and the Linux/Unix User Group Sankt Augustin.

I attended FrOSCon last year and found it a very pleasant conference to be at. I gave two sessions about MySQL Backup and Security and the MySQL Business model, which both had a good audience and went quite well. I will definitely consider going there again this year, still pondering about the topics of my talks... The CfP ends on June, 4th, so make sure to turn in your suggestions in time!

If you are looking for a very well organized OSS event with lots of good technical content in a relaxed atmosphere, consider visting FrOSCon this year! I look forward to being there again.

Giving a lightning talk at FOSDEM: schedule now online

FOSDEM 2007

FOSDEM, the Free and Open Source Software Developers' European Meeting will again take place in Brussels, Belgium on February, 24th & 25th. I went there last year and really had a great time. This year, I submitted a proposal for a lightning talk, which has been accepted! I've just been informed that the lightning talk schedule is now published. My talk "What's new at MySQL AB" will be on Saturday, at 17:00. I look forward to being there!

openSUSE Build Service news: added google-perftools and build on openSUSE 10.2

After a longer pause I finally had a moment to work on my packages in the openSUSE Build Service. I now added RPMs for the Google Performance Tools and enabled building on openSUSE 10.2. From the perftools project page:

Perf Tools is a collection of a high-performance multi-threaded malloc() implementation, plus some pretty nifty performance analysis tools.

I hope you will find them useful. It would be interesting to see, if a KDE desktop linked against tcmalloc() is significantly faster than using the default glibc malloc()...

How to recover accidentally deleted MySQL database files

Recently I stumbled over a posting on the German MySQL Forum from a user that accidentally removed all table files from a MySQL Server's data directory with a misbehaving shell script. He was surprised to find out that the server happily continued to serve requests and his web site was still fully operational, even though /var/lib/mysql/<database> was completely emtpy! The reason for this in a nutshell: the rm command only removed the reference to the table files from the database directory, the files itself were not removed from the file system yet as the mysqld process still had the files opened. So as long as a process keeps a file open, the kernel will not release the disk space occupied by the file and it will remain intact, albeit no longer visible.

Of course, the user was now desperate to recover the deleted tables files and was asking for help. Fortunately the recovery in this case is pretty simple. You should first shut down your application to avoid further activitiy on the affected database. Important: you must not shut down the MySQL Server, as this would close the last open reference to the table files! Now you can simply use mysqldump --opt <database> > database.sql to perform an SQL dump of the deleted tables. As the MySQL server still can access the open table files, the dump will contain the entire content and can then be used to restore the database again. Now you should restart the MySQL server so it closes the still open file descriptors of the deleted tables files. Alternatively, you could use DROP TABLE <table> or DROP DATABASE <database> to properly remove the references, in case you don't want to shut down the entire server. Now you can restore your missing tables from the SQL dump as usual and can restart your application!

Note that this trick only works on table files that were removed on the file system level, not after you used DROP TABLE/DATABASE, so it's not a magic undo function for these commands - only restoring from a recent backup (e.g. performed with mylvmbackup, hint, hint) will help in this case. In addition to that the MySQL server must have had opened the tables before. A freshly started MySQL server has not opened any table files apart from the ones in the mysql system database.

By the way, there is a related article "Bring back deleted files with lsof" on Linux.com that covers the subject of recovering deleted (but still open) files on a more general level and also provides some more background information about the Linux internals. Worth a read!

mylvmbackup 0.3 now released

I am happy to announce version 0.3 of mylvmbackup, a tool that performs consistent backups of a MySQL server's tables using Linux LVM snapshots.

Special thanks go to Fred Blaise, who contributed the majority of the new features that have been added to this new release:

  • It is now possible to use an external configuration file /etc/mylvmbackup.conf to store the options. This is probably more convenient than having to pass a slew of options on the command line or having to hack the script itself to change the default values. This new feature requires the Config::IniFiles Perl module to be installed, a sample configuration file is included in the package.
  • The logging to the console has been visually enhanced by including a time stamp and the message category (e.g. Info, Warning or Error). In addition to that, it is now possible to log messages to a local or remote syslog server. This feature requires the Sys::Syslog Perl module.
  • The man page has now been converted into an asciidoc file, which makes it easier to generate other document formats as well, e.g. a HTML version.
  • Several small bugs have been fixed, too: see the ChangeLog for details.
If you are looking for a convenient backup tool to create fast and consistent MySQL backups, please give mylvmbackup a try! You feedback is appreciated. A tarball and RPM are now available for download from the project's home page. Thanks!

Hot-swapping the CD-ROM drive in my Thinkpad T42 UltraBay with SUSE Linux 10.1

While browsing the fabolous ThinkWiki pages I stumbled over this little gem: the lt_hotswap kernel module finally allows me to eject the CD-ROM drive in my Thinkpad T42's UltraBay without having to shut down Linux first. This module makes sure that the device is properly unregistered from the kernel's device list. Now I can replace it with e.g. a second battery on the fly and don't have to shutdown the OS completely! The installation on my SUSE Linux 10.1 system was quite easy: after downloading the source tarball from the SourceForge.net download page I performed the following commands:
tar zxvf lt_hotswap-0.3.6.tar.gz
cd lt_hotswap-0.3.6
make
sudo make install
sudo rcacpid restart
modprobe lt_hotswap

You need to have the kernel-source package installed, as well as the gcc compiler, of course. Now when I eject the little lever that allows me to pull out the CD-ROM drive from the UltraBay, the following message appears in /var/log/messages:

lt_hotswap: Requesting IDE eject!
lt_hotswap: Attempting to eject

Now the CD-ROM drive can be safely pulled from the UltraBay and e.g. exchanged with an additional battery. Inserting the battery yields the following kernel message:

kernel: ACPI: Battery Slot [BAT1] (battery present)

The battery info also appears in /proc/acpi/battery and is fully operational. Unfortunately battery monitoring tools like KPowersave or gkrellm don't display the second battery, but at least the kernel is happily using it. I can remove the battery again, but this event does not seem to get registered properly - the second battery remains visible in /proc/acpi/battery, but shows zero capacity and the "present" state does not change. But at least replacing the CD-ROM drive with the battery now workes fine, which was my major concern. And it is still possible to re-insert the CD-ROM drive again:

kernel: ide1: BM-DMA at 0x1868-0x186f, BIOS settings: hdc:pio, hdd:pio
kernel: Probing IDE interface ide1...
kernel: hdc: HL-DT-STCD-RW/DVD DRIVE GCC-4242N, ATAPI CD/DVD-ROM drive
kernel: ide1 at 0x170-0x177,0x376 on irq 15
kernel: hdc: ATAPI 24X DVD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)

If you want to enable the lt_hotswap module at bootup, you need to add its name to the MODULES_LOADED_ON_BOOT variable in the /etc/sysconfig/kernel configuration file.

If your Laptop uses an external docking station to host a CD-ROM drive, there are some good news for you: according to Andreas Jaeger's blog, the upcoming openSUSE 10.2 will actually provide a generic Dockutils framework that will support docking/undocking functionality for different laptops and vendors. It will be interesting to find out if this covers UltraBay devices as well.

mylvmbackup version 0.2 has been released

I am happy to announce that version 0.2 of the mylvmbackup tool is now available!

mylvmbackup is a Perl script for quickly performing backups of a MySQL server's databases using the Linux Logical Volume Manager (LVM). It creates a consistent LVM snapshot of the server's data directory which is then backed up without further blocking the server's operation.

After version 0.1 was published in May this year, I did not really get much feedback about it. I had some ideas for improvements (see the TODO file included in the package), but never got around to actually start working on them.

Thanks to Robin H. Johnson from the Gentoo project for contributing a number of new options and features as well as some code cleanups. His changes motivated me to make a few more modifications and improvements by myself, which have now been rolled into a new release.

The new options provide some more flexibility in the way the script handles the logical volumes and how the backup files are being created. I also overhauled the building and packaging and added a Makefile to automate these procedures. For details, please refer to the ChangeLog and check the manual page and the README for additional info.

A tarball and RPM of version 0.2 can now be downloaded from the project page.

The SVN repository can now be browsed using WebSVN as well.

Please give it a try! Your feedback is very welcome.

Lars to talk about Linux HA at this Thursday's SAGE@GUUG Hamburg meeting

If you happen to live close to Hamburg, Germany, and you are keen on learning more about High Availability Solutions on Linux, you don't want to miss Lars Marowsky-Brée's talk about "Cluster and Data Center Automation with Linux HA". It will be held this Thursday at 19:00 at the SAGE@GUUG Hamburg Meeting. If you want to participate, please RSVP via OpenBC. See you there!
tweetbackcheck