Lenz Grimmer's blog

As noted in my previous blog posting, I manged to revive my old Logitech TrackMan Marble FX on Linux (openSuSE 11.1b4), using a Serial-to-USB dongle with a Prolific PL2303 chip. But I also use OpenSolaris on my Laptop quite frequently (currently testing the upcoming 2008.11 release), so I investigated if it would be possible to enable the trackball there as well.

Luckily, the Driver Manager listed the plugged in adapter and the correct driver (usbsprl) was loaded already. Now the real challenge was finding out which device node to use. Some research revealed that the driver actually comes with a manual page , which indicated that /dev/term/0 was the correct device name.

Lo and behold, I copied the InputDevice section from my Linux xorg.conf file into the OpenSolaris one, replaced the Device parameter with the appropriate one and restarted the X server. Immediate success! Now I can enjoy using my most favourite input device on OpenSolaris as well.

I am probably different than most users, but I am a a fan of unusual input devices. I prefer Laptops with trackpoints - I immediately disabled the touchpads on my Lenovo laptops (a T61 and T42) in the BIOS when I received them. My first Laptop (a Toshiba Portege 3440CT) didn't even have a touchpad to begin with. It's a pity that trackpoints seem to a dying breed.

And I don't like using regular mice on my desktop, either! Actually, my most favourite input device is a trackball - I purchased a Logitech TrackMan Marble FX a long time ago, and used it for years. Then computers stopped having serial or PS/2 connectors, and I replaced the trackball with an USB mouse. I never really got the hang of using mice, but Logitech (or other vendors) somehow never came up with a suitable replacement model for the Marble FX with a USB port. I recently looked at the Logitech Trackman Optical, but was not convinced by the reviews I read, and the fact that it requires batteries (a trackball is a stationary device, so a cable does not really interfere here!). The Microsoft Trackball Explorer might have been an option, but it seems to be impossible to get nowadays.

After experimenting with several mouse models (Cherry, Microsoft), I decided to revive the TrackMan Marble again. It comes with a PS/2-connector by default, so I first tried to connect it to my PC using a PS/2-to-USB converter dongle. This actually worked without any tweaking, but had two limitations: the fourth mouse button was not detected anymore (I could have lived with that) and the Trackball stopped responding after it had been idle for a while, requiring me to restart the X server to get it working again.

So using the PS/2-to-USB dongle was ruled out and I tried an Serial-to-USB dongle instead:

lenz@thebe:~> lsusb|grep Serial
Bus 002 Device 006: ID 067b:2303 Prolific Technology, Inc. PL2303 Serial Port

When plugged in, udev creates a new serial device /dev/ttyUSB0 which I needed to inform the X server about. Fortunately it's possible to define multiple input devices in the xorg.conf configuration file. Using the serial port actually had another advantage - I was able to add some trackball-specific tweaks that would have collided with the settings of the "regular" PS/2 mouse section that I needed for the builtin trackpoint of my Thinkpad. It required some tweaking and testing, but this is what I added to xorg.conf to be able to use the TrackMan Marble FX in addition to the builtin pointing device:

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Trackball" "SendCoreEvents"
    Option         "Clone" "off"
    Option         "Xinerama" "off"
EndSection

Section "InputDevice"
    Driver       "mouse"
    Identifier   "Trackball"
    Option       "Device" "/dev/ttyUSB0"
    Option       "Name" "TrackMan Marble FX"
    Option       "Protocol" "Intellimouse"
    Option       "Vendor" "Logitech"
    Option       "AngleOffset" "10"
    Option       "Buttons" "8"
    Option       "Emulate3Buttons" "off"
    Option       "EmulateWheel" "true"
    Option       "EmulateWheelButton" "8"
    Option       "YAxisMapping" "4 5"
    Option       "EmulateWheelInertia" "8"
EndSection

You may wonder about the 8 mouse buttons, as the device only has four physical buttons. Interestingly, the fourth button on the TrackMan reported itself as "button 8" when I probed it with "xev", so I needed to make sure the server is aware of it. When pressed, the trackball now acts like a mouse wheel and allows me to quickly scroll across long documents - very handy! Now I just hope that the button switches in the TrackMan last for another while - until some vendor eventually comes up with a suitable replacement...

Thankfully my RSS reader has builtin-search capability, so I can quickly skim the full RSS feed from blogs.sun.com for keywords of interest. I found the following articles quite helpful:

• Setting up MySQL Cluster using Solaris Zones by Hashamkha Pathan walks through the steps involved of setting up a virtual MySQL Cluster setup on a single Solaris instance (for evaluation purposes only - this is probably not the best idea for a productive HA environment)
• Mixing SQL and shell commands in MySQL by Amit Saha explains how to write scripts that execute both SQL statements and unix shell commands using the MySQL command line client
• Improving filesort performance in MySQL by Neelakanth Nadgir provides some background information about MySQL's filesort operation that is being performed when records have to be sorted using ORDER BY.
• Ubuntu, PHP, NetBeans - part I by Petr Pisl is the first part of an article series that will explain how to develop PHP/MySQL applications using NetBeans on Ubuntu Linux. This part describes the basic installation of the required LAMP stack components.

If you are would like to learn more on how to develop on NetBeans with using a MySQL Server and how to create web apps that use MySQL as a backend, take a look at these two tutorials:

• Connecting to a MySQL Database
• Creating a Simple Web Application Using a MySQL Database

Over on the Novell Cool Solutions pages (which are powered by Drupal, by the way), I found this introduction on how to set up MySQL replication by Damian Myerscough. It walks you through the steps involved in setting up a classical master-slave replication setup.

 

This article describes how to install the Drupal 6.2 CMS on MySQL 6.0, using the Falcon Storage Engine. The operating system is a default Ubuntu 8.04 "Hardy Heron" (x86) installation.

I will make a few assumptions here, in order to keep the instructions simple: a fresh OS install, no other MySQL databases or web services are running or have already been installed. Both MySQL and the web server are installed on the same host. You should be able to become root to install packages and to have access to the local file system and the system configuration.

This article will explain how to install and configure Apache/PHP, MySQL 6.0 and Drupal 6.2.

Last week at CeBit I once again had a conversation with an application developer that was not aware that it's possible to embed the MySQL Server inside an application instead of running it as a separate process. I experienced something similar at FOSDEM recently, while talking with the Amarok developers - how come that this feature is so little known? In fact, there are two possibilities for embedding MySQL (and both of them are licensed under the GPL): the first one is using the libmysqld embedded library, which is suitable if you are developing C/C++ applications:

The embedded MySQL server library makes it possible to run a full-featured MySQL server inside a client application. The main benefits are increased speed and more simple management for embedded applications.

The embedded server library is based on the client/server version of MySQL, which is written in C/C++. Consequently, the embedded server also is written in C/C++.

The API is identical for the embedded MySQL version and the client/server version.

Especially consider the last sentence - it's actually fairly easy to create an application that supports connecting to both an embedded or a standalone server instance! Here are some related links to get you going:

• libmysqld, the Embedded MySQL Server Library
• MySQL C API functions
• Embedding the MySQL Server Library in a C Program by Russell Dyer
• Embedded MySQL Discussion Forum

If you're a Java developer, you should consider taking a look at our Connector/MXJ product - this allows you to embed a MySQL server in your Java application:

MySQL Connector/MXJ is a Java Utility package for deploying and managing a MySQL database. Deploying and using MySQL can be as easy as adding an additional parameter to the JDBC connection url, which will result in the database being started when the first connection is made. This makes it easy for Java developers to deploy applications which require a database by reducing installation barriers for their end-users.

MySQL Connector/MXJ makes the MySQL database appear to be a java-based component. It does this by determining what platform the system is running on, selecting the appropriate binary, and launching the executable. It will also optionally deploy an initial database, with any specified parameters.

Included are instructions for use with a JDBC driver and deploying as a JMX MBean to JBoss.

Some related links to further information/help:

• MySQL Connector/MXJ documentation
• MySQL Connector/MXJ Downloads
• JDBC and Java discussion forum
• MySQL and Java Mailing list

Does your application make use of an embedded MySQL Server already? Please let me know! I'd love to learn more about the use cases and experiences you have made.

 

 

 

When it comes to handling my work-related email, many people will probably conclude that I am an old fart in Linux terms: I still use the console-based Pine as my MUA of choice, as my fingers have been hardcoded to its key combinations over the years (my first encounter with Pine was around 1994 on a shared DEC Ultrix box in my university). So far, I have not found any other application that allows me to process email as quickly as by using this tool.

I admit that I do use Thunderbird for my personal email, though, to gather experience with it (and to toy around with the various extensions, especially Nostalgy is a gift from heaven for people like me!). And of course because the mail volume there is less critical to cope with! But I am not going to start a holy war here - this article is supposed to explain how I have configured my mail setup to rotate the log files that keep track of all my incoming mails.

While helping a user with some questions about the geospatial extensions of MySQL on the #mysql-dev IRC channel on Freenode, I stumbled over this blog: How to use MySQL Spatial Extensions. There currently is just one post, but it was exactly what we were looking for: "Using Circular Area Selection". Nice work! I hope the author will soon provide more examples of how to make use of this functionality.

By the way, there is work in progress to improve the GIS functionality in MySQL - if you are looking for new GIS functions that do not use MBRs, take a look at our GIS feature preview on the MySQL Forge. We plan to publish a new set of preview binaries once the 5.1.23 release has been published and merged into this code base. 

 

Since the very early days, the MySQL distribution packages contain a very useful commandline-tool named replace. As the name implies, it can be used to replace strings in text files. From the documentation:

The replace utility program changes strings in place in files or on the standard input. Invoke replace in one of the following ways:

shell> replace from to [from to] ... -- file [file] ...
shell> replace from to [from to] ... < file

from represents a string to look for and to represents its replacement. There can be one or more pairs of strings. Use the -- option to indicate where the string-replacement list ends and the filenames begin. In this case, any file named on the command line is modified in place, so you may want to make a copy of the original before converting it. replace prints a message indicating which of the input files it actually modifies. If the -- option is not given, replace reads the standard input and writes to the standard output. replace uses a finite state machine to match longer strings first. It can be used to swap strings. For example, the following command swaps a and b in the given files, file1 and file2:

shell> replace a b b a -- file1 file2 ...

This comes in very handy, especially if you need to replace a similar text in a large number of files, or if you need to replace multiple different strings in a file at the same time! So next time you need to replace strings in text files, take a look at this hidden gem!

Unfortunately the string matching does not support regular expressions - but the source code actually documents some additional options:

Special characters in from string:
  ^  Match start of line.
  \$  Match end of line.
  \b  Match space-character, start of line or end of line.
        For end \b the next replace starts locking at the end space-character.
        An \b alone or in a string matches only a space-character.
  \r, \t, \v as in C.

The programs make a DFA-state-machine of the strings and the speed isn't
dependent on the count of replace-strings (only of the number of replaces).
A line is assumed ending with \n or \0.
There are no limit exept memory on length of strings.

 

 

 

 

Now that the source tree for the new Falcon Storage Engine is finally public, here's a quick HOWTO on how to compile the server from source. This procedure is described in more detail in the MySQL Manual. I assume you use Linux and have the required development toolchain installed.

You first should get the free BK client from http://www.bitmover.com/bk-client2.0.shar, unpack and install it:

$ wget http://www.bitmover.com/bk-client2.0.shar
--17:34:34--  http://www.bitmover.com/bk-client2.0.shar
           => `bk-client2.0.shar'
Resolving www.bitmover.com... 192.132.92.2
Connecting to www.bitmover.com|192.132.92.2|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17,676 (17K) [application/x-shar]

100%[========================================================>] 17,676        33.56K/s

17:34:35 (33.50 KB/s) - `bk-client2.0.shar' saved [17676/17676]

$ sh ./bk-client2.0.shar
x - creating lock directory
x - creating directory bk-client2.0
x - extracting bk-client2.0/Makefile (text)
x - extracting bk-client2.0/bkf.c (text)
x - extracting bk-client2.0/demo.sh (text)

$ cd bk-client2.0
$ make
cc -O2 -Wall -Wno-parentheses    bkf.c   -o bkf
$ sudo install -m755 bkf /usr/local/bin

Now that the BK client is installed, we can create a local clone of the Falcon development tree:

$ bkf clone bk://mysql.bkbits.net/mysql-5.2-falcon mysql-5.2-falcon

This will now perform a check out the most recent source tree from the mysql-5.2-falcon repository (~83 MB of source code). You can now enter the newly created directory and start the build:

$ cd mysql-5.2-falcon
$ ./BUILD/compile-pentium-debug-falcon

If you have a x86_64 Linux system, use compile-amd64-debug-falcon instead. Depending on the performance of your system, this step will take a while, as it performs a full compile of the MySQL server and all related tools. Once the build completed with no errors, you can create a binary tarball distribution:

./scripts/make_binary_distribution

This should leave you with a tarball mysql-5.2.0-falcon-alpha-linux-i686.tar.gz that you can now install to, say, /usr/local:

$ tar zxvf mysql-5.2.0-falcon-alpha-linux-i686.tar.gz -C /usr/local
$ cd /usr/local
$ ln -s mysql-5.2.0-falcon-alpha-linux-i686 mysql
$ cd mysql
$ ./configure

The last command will install the privilege tables and start up the mysql server. You can now fire up the mysql client:

$ ./bin/mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 5.2.0-falcon-alpha-debug

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

Now make sure that falcon has indeed been included:

mysql> SHOW VARIABLES LIKE 'have_falcon';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| have_falcon   | YES   |
+---------------+-------+
1 row in set (0.01 sec)

Looks good! Now it's time to actually create a table that uses the Falcon engine:

mysql> use test;
Database changed
mysql> CREATE TABLE names (id INT, fname VARCHAR (20), lname VARCHAR (20)) ENGINE=Falcon;
Query OK, 0 rows affected (2.84 sec)

mysql> describe names;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(11)     | YES  |     | NULL    |       |
| fname | varchar(20) | YES  |     | NULL    |       |
| lname | varchar(20) | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.01 sec)

mysql> INSERT INTO names VALUES (0, 'Lenz', 'Grimmer');
Query OK, 1 row affected (0.03 sec)

mysql> INSERT INTO names VALUES (1, 'Monty', 'Widenius');
Query OK, 1 row affected (0.06 sec)

mysql> SELECT * FROM names WHERE lname='Widenius';
+------+-------+----------+
| id   | fname | lname    |
+------+-------+----------+
|    1 | Monty | Widenius |
+------+-------+----------+
1 row in set (0.00 sec)

Congratulations, you are ready to further test and explore this new storage engine! Have fun with testing it - we would appreciate your feedback and experiences! Please share your impressions via the Falcon Forum or report bugs via our Bug Database. Make sure to read the bug reporting guidelines on the MySQL Forge Wiki first!

UPDATE: The people at Bitkeeper updated the free BK client to version 2.0 after I wrote this. I updated the described procedure above accordingly. Currently, the repository seems to have a locking problem that only appears when using the free BK client. We are working with the BitKeeper team on resolving this issue.

Recently I stumbled over a posting on the German MySQL Forum from a user that accidentally removed all table files from a MySQL Server's data directory with a misbehaving shell script. He was surprised to find out that the server happily continued to serve requests and his web site was still fully operational, even though /var/lib/mysql/<database> was completely emtpy! The reason for this in a nutshell: the rm command only removed the reference to the table files from the database directory, the files itself were not removed from the file system yet as the mysqld process still had the files opened. So as long as a process keeps a file open, the kernel will not release the disk space occupied by the file and it will remain intact, albeit no longer visible.

Of course, the user was now desperate to recover the deleted tables files and was asking for help. Fortunately the recovery in this case is pretty simple. You should first shut down your application to avoid further activitiy on the affected database. Important: you must not shut down the MySQL Server, as this would close the last open reference to the table files! Now you can simply use mysqldump --opt <database> > database.sql to perform an SQL dump of the deleted tables. As the MySQL server still can access the open table files, the dump will contain the entire content and can then be used to restore the database again. Now you should restart the MySQL server so it closes the still open file descriptors of the deleted tables files. Alternatively, you could use DROP TABLE <table> or DROP DATABASE <database> to properly remove the references, in case you don't want to shut down the entire server. Now you can restore your missing tables from the SQL dump as usual and can restart your application!

Note that this trick only works on table files that were removed on the file system level, not after you used DROP TABLE/DATABASE, so it's not a magic undo function for these commands - only restoring from a recent backup (e.g. performed with mylvmbackup, hint, hint) will help in this case. In addition to that the MySQL server must have had opened the tables before. A freshly started MySQL server has not opened any table files apart from the ones in the mysql system database.

By the way, there is a related article "Bring back deleted files with lsof" on Linux.com that covers the subject of recovering deleted (but still open) files on a more general level and also provides some more background information about the Linux internals. Worth a read!

While browsing the fabolous ThinkWiki pages I stumbled over this little gem: the lt_hotswap kernel module finally allows me to eject the CD-ROM drive in my Thinkpad T42's UltraBay without having to shut down Linux first. This module makes sure that the device is properly unregistered from the kernel's device list. Now I can replace it with e.g. a second battery on the fly and don't have to shutdown the OS completely! The installation on my SUSE Linux 10.1 system was quite easy: after downloading the source tarball from the SourceForge.net download page I performed the following commands:

tar zxvf lt_hotswap-0.3.6.tar.gz
cd lt_hotswap-0.3.6
make
sudo make install
sudo rcacpid restart
modprobe lt_hotswap

You need to have the kernel-source package installed, as well as the gcc compiler, of course. Now when I eject the little lever that allows me to pull out the CD-ROM drive from the UltraBay, the following message appears in /var/log/messages:

lt_hotswap: Requesting IDE eject!
lt_hotswap: Attempting to eject

Now the CD-ROM drive can be safely pulled from the UltraBay and e.g. exchanged with an additional battery. Inserting the battery yields the following kernel message:

kernel: ACPI: Battery Slot [BAT1] (battery present)

The battery info also appears in /proc/acpi/battery and is fully operational. Unfortunately battery monitoring tools like KPowersave or gkrellm don't display the second battery, but at least the kernel is happily using it. I can remove the battery again, but this event does not seem to get registered properly - the second battery remains visible in /proc/acpi/battery, but shows zero capacity and the "present" state does not change. But at least replacing the CD-ROM drive with the battery now workes fine, which was my major concern. And it is still possible to re-insert the CD-ROM drive again:

kernel: ide1: BM-DMA at 0x1868-0x186f, BIOS settings: hdc:pio, hdd:pio
kernel: Probing IDE interface ide1...
kernel: hdc: HL-DT-STCD-RW/DVD DRIVE GCC-4242N, ATAPI CD/DVD-ROM drive
kernel: ide1 at 0x170-0x177,0x376 on irq 15
kernel: hdc: ATAPI 24X DVD-ROM CD-R/RW drive, 2048kB Cache, UDMA(33)

If you want to enable the lt_hotswap module at bootup, you need to add its name to the MODULES_LOADED_ON_BOOT variable in the /etc/sysconfig/kernel configuration file.

If your Laptop uses an external docking station to host a CD-ROM drive, there are some good news for you: according to Andreas Jaeger's blog, the upcoming openSUSE 10.2 will actually provide a generic Dockutils framework that will support docking/undocking functionality for different laptops and vendors. It will be interesting to find out if this covers UltraBay devices as well.

A few days ago I received a neat little gadget: a Yakumo QuickStick DVB-T, which allows me to watch TV via the USB-2 port on my Thinkpad. Amazon sold it for 52 EUR, so I could not resist, after making sure that it was listed in the list of supported devices on the very informative LinuxTV Wiki pages. All what was left to be done for me was to download the firmware and putting it into /lib/firmware. The driver and firmware were loaded automatically when I plugged in the device:

Mar 20 19:44:05 metis kernel: usb 1-4: new high speed USB device using ehci_hcd and address 8
Mar 20 19:44:05 metis kernel: dvb-usb: found a 'WideView WT-220U PenType Receiver (and clones)' in cold state, will try to load a firmware
Mar 20 19:44:05 metis kernel: dvb-usb: downloading firmware from file 'dvb-usb-wt220u-01.fw' to the 'Cypress FX2'
Mar 20 19:44:05 metis kernel: dvb-usb: WideView WT-220U PenType Receiver (and clones) successfully initialized and connected.
Mar 20 19:44:07 metis kernel: usb 1-4: USB disconnect, address 8
Mar 20 19:44:07 metis kernel: dvb-usb: generic DVB-USB module successfully deinitialized and disconnected.
Mar 20 19:44:08 metis kernel: usb 1-4: new high speed USB device using ehci_hcd and address 9
Mar 20 19:44:08 metis kernel: dvb-usb: found a 'WideView WT-220U PenType Receiver (and clones)' in warm state.
Mar 20 19:44:08 metis kernel: dvb-usb: will use the device's hardware PID filter (table count: 15).
Mar 20 19:44:08 metis kernel: DVB: registering new adapter (WideView WT-220U PenType Receiver (and clones)).
Mar 20 19:44:08 metis kernel: DVB: registering frontend 0 (WideView USB DVB-T)...
Mar 20 19:44:08 metis kernel: dvb-usb: schedule remote query interval to 300 msecs.
Mar 20 19:44:08 metis kernel: dvb-usb: WideView WT-220U PenType Receiver (and clones) successfully initialized and connected.

Now I fired up the KDE video player Kaffeine 0.7.1 and configured it to scan for available channels. That's all there was to it! I am positively surprised.

I have been struggling with that for ages: how can I configure my slides to not display all bullet points at once when I switch to the next slide, but rather display one at a time?

I prefer not not distract my audience by having them read through the entire slide in advance instead of listening to me still elaborating on the previous bullet points. Some Google research finally pointed me to this article, which also mentions how to accomplish this:

• Mark the frame that holds the bullet points
• In the Tasks pane, select "Custom Animation"
• Click "Add" and select the Entrance Effect you want to use. For displaying one item after another, either the plain "Appear" or "Fade In" effect is probably least distracting. Apply the Effect with OK.
• Small progress: now all bullet points will be displayed at once, but after the slide itself has been displayed.
• This can be changed with the "Effect Options" Button in the "Custom Animation -> Modify Effect" dialogue of the Tasks pane (it's well hidden as the button is only labelled as "..."). Click on the button and choose the "Text Animation" tab in the appearing dialogue. In there, change the "Group Text" option from "As one object" to "By 1st level paragraphs". If you have sublevels of bullets, choose the appropriate level that suits your needs.
• Now click OK and test the presentation again. Each bullet should now be appear individually. You're done!