The views or opinions expressed on this blog are my own and do not necessarily reflect the views or opinions of Oracle Corporation. The views or opinions expressed by visitors on this blog are theirs solely and may not reflect mine.
Thursday, November 5. 2009
This blog post is a by-product of my preparation work for an upcoming talk titled "Why you should be using a distributed version control system (DVCS) for your project" at SAPO Codebits in Lisbon (December 3-5, 2009). Publishing these thoughts prior to the conference serves two purposes: getting some peer review on my findings and acting as a teaser for the actual talk. So please let me know — did I cover the relevant aspects or did I miss anything? What's your take on DVCS vs. the centralized approach? Why do you prefer one over the other? I'm looking forward to your comments!
Even though there are several distributed alternatives available for some years now (with Bazaar, git and Mercurial being the most prominent representatives here), many large and popular Open Source projects still use centralized systems like Subversion or even CVS to maintain their source code. While Subversion has eased some of the pains of CVS (e.g. better remote access, renaming/moving of files and directories, easy branching), the centralized approach by itself poses some disadvantages compared to distributed systems. So what are these? Let me give you a few examples of the limitations that a centralized system like Subversion has and how these affect the possible workflows and development practices.
Continue reading "Aspects and benefits of distributed version control systems (DVCS)"
Sunday, September 6. 2009
I am happy to announce that mylvmbackup version 0.13 has now been released. This release includes a fix for a nasty bug in on of the recently added Perl hooks (precleanup.pm) and some added functionality (better support for remote rsync backups).
From the ChangeLog:
Saturday, June 20. 2009
After a long hiatus, I am happy to announce that mylvmbackup version 0.12 has now been released. This release includes a large number of improvements, minor code cleanups, as well as some new functionality. In particular, I would like to thank Matthew Boehm, Tim Stoop, Baron Schwartz, Ville Skyttä and Ronald Bradford for their contributions.
Some notable highlights from the ChangeLog:
Thursday, February 26. 2009
Concluded my first MySQL University Session about MySQL backups using file system snapshots - some questions remained unanswered...
Today I gave my first MySQL University session as a speaker, talking about Backing up MySQL using file system snapshots. The talk went quite well (at least that was my impression) and we had ~10 people attending. The slides (PDF) and a recording of the session are now available from the Wiki page. Unfortunately the recording lacks the audio track, which is a bit of a bummer. We've submitted a support request with the DimDim folks, so hopefully they can provide us with a complete recording.
There was one question during the session that I was not able to answer myself, so I'm asking for your insights here:
Consider we're using InnoDB and MyISAM tables on a file system that can be snapshotted (e.g. Linux LVM or ZFS) and we're performing the following operations:
The question that came up was if this actually still is a consistent backup, considering that InnoDB rolled back uncommited transactions. Does the state of the tables still match the binary log positions we noted before? I assume yes, as long as the transaction does not involve modifications non-transactional tables.
Another suggestion that came up was to change InnoDB's configuration variable innodb_max_dirty_pages_pct to "0" prior to performing the snapshot, to minimize the amount of dirty pages that have not been written to disk (and thus reducing the time required for recovering later). I wonder if this would make a difference...
What other InnoDB variables might have a noteworthy effect in the context of snapshot backups? I am looking forward to your comments.
Sunday, February 22. 2009
Sorry for the downtime of this site - until around a week ago I hosted my home page on a trusty Genesi Pegasos II system (powered by a PowerPC G4 Processor clocked at 1GHz, using Debian 4.0 PPC with 512 MB of RAM), serving these pages from my home DSL connection. Unfortunately this system provided no means of redundancy - the hard disk drive died.
Luckily I perform frequent backups, so I moved most parts of the site to a shared hosting space now - the picture gallery is unfortunately too big to fit into the space that I have there. I'll try to move the pictures into my Flickr account instead, but this will take some time.
Note that the primary domain name of this site is now lenzg.net - lenzg.org, (the domain that I tried to promote as the official domain for my site) used to redirect to the home machine at lenz.homelinux.org. Both now redirect to the new address instead. I've initiated the move of the lenzg.org domain to the other provider as well, so soon this site will be available from both the .org and .net domain. Please don't link to lenz.homelinux.org anymore, as that site will eventually go out of service. Until then, a small openSUSE Linux box (Intel PIII, 500 MHz, 192 MB of RAM) running lighttpd will perform the URL redirection.
Monday, December 1. 2008
Some days ago, I released version 0.11 of mylvmbackup a Perl script that performs consistent backups of a MySQL server by using LVM filesystem snapshots. The source archive as well as a generic RPM can be found on the project home page, packages for many Linux distributions are available on the openSUSE Build service.
This release includes some new functionality as well as numerous bug fixes and improvements, most notably:
I would like to thank all the contributors for their support! More details about the changes in this release, directly from the ChangeLog:
Saturday, September 20. 2008
I am happy to announce that mylvmbackup version 0.10 has been released.
You can download the updated package from the project home page or via the openSUSE Build Service.
This version fixes some bugs and includes new functionality:
Friday, July 11. 2008
I am happy to announce that a new version (0.9) of mylvmbackup has been released. This is the first release since the source code has been moved from Subversion to Bazaar and is now hosted on Launchpad.net. I would like to thank Robin H. Johnson and Patrick Hahn for providing the patches that contributed to this new release!
mylvmbackup is a tool for quickly creating backups of MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, makes an LVM snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the server can continue normal operations, while the actual file backup proceeds.
From the ChangeLog:
You can download a source tarball or RPM from the project home page. Additional packages for various Linux distributions can be obtained from the openSUSE Build Service. Packages for Gentoo and Debian should appear shortly, too.
Wednesday, June 25. 2008
This will hopefully make it easier for contributors to work on the code and share their modifications with others, removing me as the bottleneck for applying and testing patches for new releases. I chose Bazaar primarily because I wanted to get some more hands-on practice with it, now that the MySQL Server source trees have been transferred to it as well (see Kaj's announcement for details).
As mylvmbackup is closely related to the MySQL Server project, it made sense to choose the same platform and enjoy the cross-pollination effects and the infrastructure that Launchpad provides. Additionally, the distributed nature of Bazaar makes it more convenient to work with the code history and commiting changes locally without having to be online and connected to the SVN server.
The "trunk" branch is now hosted on Launchpad. I assume that I will soon open up a development branch, that will receive heavier modifications first. I also plan to use the site for bug tracking and keeping track of feature requests (via Blueprints).
To create a local branch of the "trunk" repository, you can use the following command:
bzr branch lp:mylvmbackup
I also maintain a copy of that branch on my home server, just in case: http://www.lenzg.net/bzr/mylvmbackup/
To avoid confusion, I removed the Subversion repository on http://www.lenzg.net. Please use the Bazaar tree on Launchpad from now on. Thanks!
Thursday, April 24. 2008
JFYI, I now placed a PDF of my MySQL Conference talk slides about "Performing MySQL backups using Linux LVM Snapshots" on my MySQL talks page. Enjoy!
Wednesday, April 16. 2008
While reading Colin's post about LugRadio Live, I stumbled over the Zumastor Linux Storage Project. Going through the project home page and their HOWTO got me curious - could this eventually become an alternative to using DRBD (for replicating data) and LVM snapshots (for performing backups)?
I assume it's not ready for production use yet, but it would sure be interesting to investigate on how to utilize it for the purpose of running MySQL on top of it...
I will keep an eye on this project, I wonder if I will have to add support for Zumastor snapshots to mylvmbackup at some point?
Friday, April 11. 2008
I am happy to announce the release of mylvmbackup version 0.8. mylvmbackup is a tool for quickly creating backups of a MySQL server's data files. To perform a backup, mylvmbackup obtains a read lock on all tables and flushes all server caches to disk, makes an LVM snapshot of the volume containing the MySQL data directory, and unlocks the tables again. The snapshot process takes only a small amount of time. When it is done, the server can continue normal operations, while the actual file backup proceeds.
Below is the list of changes since version 0.6. You may wonder what happened to version 0.7 - it had a rather short life cycle as I was informed about a bug that I fixed quickly before I made a wider release announcement of 0.7.
Updated package are available from the home page and via the openSUSE Build Service as usual. Updated packages for Debian/Ubuntu and Gentoo Linux should also be available shortly. Enjoy!
Speaking of LVM snapshot backups: I will be giving a talk about this subject at our MySQL Conference 2008 in Santa Clara, CA next week. If you are curious about how MySQL can be backed up using this technology, please consider to stop by!
Saturday, January 19. 2008
While browsing the many blog entries on blogs.sun.com about the MySQL Acquisition (thanks a lot for the very warm welcome!), I stumbled over this (Python-based) utility: SnapBack, a tool that uses ZFS snapshots to perform physical backups of MySQL databases on Solaris. Very cool! This is actually something I was wanting to add to the mylvmbackup script, too - I have to take a closer look at how this is done (I tried to install OpenSolaris on a VirtualBox instance, but it caused it to crash the emulator).
Sunday, October 28. 2007
Version 0.6 of mylvmbackup, a script to perform backups of a MySQL server using Linux LVM snapshots, has now been released.
In addition to various code cleanups and documentation improvements, many new features have been added to this version. I'd like to specially thank Robin H. Johnson from the Gentoo project for contributing many of the improvements to this release!
Version 0.6 is now available for download from http://www.lenzg.net/mylvmbackup/ (source tarball and RPM). I also provide RPM packagages for a number of additional platforms via my home:LenzGr repository on the fabolous openSUSE Build Service.
Enjoy! Feedback, patches and suggestions are welcome - please consider joining the mylvmbackup mailing list to discuss your experiences with this tool.
Wednesday, May 9. 2007
Eric Bergen from Proven Scaling (which I had the pleasure to meet in person during the MySQL Conference & Expo in Santa Clara last month) was kind enough to send me a patch for the mylmbackup tool, which justifies a new release:
Attached is a patch file for mylvmbackup that adds the ability to use
This is a very useful addition, as it significantly reduces the time required to recover a MySQL server from a snapshot backup. Thank you, Eric! Note that you need to use LVM2, as LVM1 does not support the required writing to snapshot volumes.
Version 0.5 is now available for download from the mylvmbackup project page.I also would like to announce that I have set up a dedicated mailing list for this tool: if you want to discuss the usage/future of mylvmbackup, propose patches or ask for help, there now is a mailinglist, hosted on FreeLists.org. To subscribe, either enter your email address on the mylvmbackup list information page or send an email with the subject "subscribe" to firstname.lastname@example.org. The list is archived here. See you there!
(Page 1 of 2, totaling 20 entries) » next page
Show tagged entries
Original content in this work is licensed under a Creative Commons License