Steve’s Ubuntu Weblog

Mainly (but not only) about Ubuntu

Ubuntu. Simple backup that works.

UPDATE 10.08.2009, please read comment at foot of this post

Having recently been let down by mondorestore in Gutsy (see below) I have been researching a new backup strategy. I didn’t want to again leave myself trusting in a tool which lets you down when you really need it, so the stategy had to use readily available and trustworthy software. The solution I have decided upon and tested by doing a complete restore is as follows:

Backing up.
Firstly you need to copy your home directory using sudo grsync (preserve time, owner, permissions and group) or

sudo rsync -r -t -p -o -g -v --progress -l /home/ /media/disk/home/

Keep a copy of this somewhere so that you can find it when your system dies ie on an external drive. Sync it regularly, rsync will only update changes to the original so after the first copy has been made, updates usually only require a few seconds.

Then make a list of all of the packages in your installation using the command

sudo dpkg --get-selections | grep '[[:space:]]install$' | \awk '{print $1}' > package_list

Copy this file together with /etc/apt/sources.list to the same medium you have used for your /home/ directory backup.

You could initiate a cron job to keep this list up-to-date using by putting the following script in /etc/cron/dailypackage_list and making it executable.

#!/bin/sh
sudo dpkg --get-selections | grep '[[:space:]]install$' | \awk '{print $1}' > package_list

Restoring.
First boot the live cd and install Ubuntu.

After Ubuntu has rebooted to the HD version set up your restricted drivers for your graphics card if applicable, there is no need to reboot yet. Then replace the existing sources list with the one from your lost system. assuming that it is on an external disk (together with the /home/ directory) you would use

sudo mv /media/disk/sources.list /etc/apt/sources.list
sudo apt-get update

Then update the packages using:

cat /media/disc/package.list | xargs sudo apt-get install

If you are using some unauthorized packages you will get the following warning

“WARNING: The following packages cannot be authenticated!
libxxx libxx libx xxxplayer
skype-common skype xxxcodecs
Install these packages without verification [y/N]? E: Some packages could not be authenticated”

for some reason you cannot answer yes or no to this, it’s a bug. So simply install all of these packages using

sudo apt-get install libxxx libxx libx xxxplayer skype-common skype xxxcodecs

When that has completed run

cat /media/disc/package.list | xargs sudo apt-get install

again and this time it should complete without problems.

Now copy your /home directory to the new system using sudo grsync (preserve time, owner, permissions and group) or

sudo rsync -r -t -p -o -g -v --progress -l /media/disk/home/ /home/

That’s it! You need to reboot and your system should be exactly as it was. I have tested this three times without problems.

UPDATE 10.08.2009
The method of building and restoring a list of installed packages described above no longer works. Instead to build the list use:
sudo dpkg --get-selections > package.list
To restore the packages use:
sudo dpkg --set-selections < package.list
sudo apt-get dselect-upgrade

19 November, 2007 - Posted by steveyoung | Ubuntu | , | 17 Comments

17 Comments »

  1. Steve, is rsync with -n option going to do any real work, i.e. move some files. Isn’t -n dry-run?

    Jerry

    Comment by Jerry Fowler | 20 November, 2007 | Reply

  2. Hi Jerry you are right. Actually I used grsync and then did a simulation to get the rsync command to past into the blog. I will edit the blog to reflect your correction.

    Thanks.

    Comment by steveyoung | 20 November, 2007 | Reply

  3. with this method, what about all of the configuration in /etc? Should that be grabbed as well?

    Comment by slestak | 8 January, 2008 | Reply

  4. Most of the pertinent information in /etc will be re-written when you re-install the packages from package_List. Some files, such as cron jobs, will be missing so you could back those up if you want to but I wouldn’t backup the whole of the /etc file.

    Comment by steveyoung | 11 January, 2008 | Reply

  5. Won’t sudo rsync change the permissions for the files you’re backing up to root? I checked the files I backed up when I sudo rsynced my home directory and it was set too root.

    Comment by Jimbo | 21 April, 2008 | Reply

  6. The -p option preserves permissions and therefore it must be included in the command, as in my example. If you use Grsync click on “Preserve permissions” in the Basic options tab.

    Comment by steveyoung | 21 April, 2008 | Reply

  7. It definitely changes the file permissions at my end if I use sudo rsync, it might be different with Grsync so I’ll try that out. Also there are errors in the “sudo dpkg –get-selections…..” commands you listed, probably because of the blogging software changes the characters. I’ve seen this happen before.

    Comment by Jimbo | 21 April, 2008 | Reply

  8. What exactly is the rsync command you are using?

    Comment by steveyoung | 21 April, 2008 | Reply

  9. I’m using “sudo rsyn -arvu /home /media/disk/backup” which isn’t much different from yours, I’ve tried your way as well. But I’ve not had a go with the grsync. Thank you.

    Comment by Jimbo | 22 April, 2008 | Reply

  10. Sorry, its actually “sudo rsync -arvu /home/ /media/disk/backup”

    Comment by Jimbo | 22 April, 2008 | Reply

  11. Actually it’s very much different from mine! You are not using -p in the command and so the permissions are not preserved.

    Comment by steveyoung | 22 April, 2008 | Reply

  12. Are you sure?? man rsync says: -a, –archive archive mode; same as -rlptgoD (no -H, -A)

    Comment by Jimbo | 22 April, 2008 | Reply

  13. Anyway I’m going to try out the graphical frontend. Thanks for the advice.

    Comment by Jimbo | 22 April, 2008 | Reply

  14. Then update the packages using:

    cat /media/disc/package.list | xargs sudo apt-get install

    If you are using some unauthorized packages you will get the following warning

    “WARNING: The following packages cannot be authenticated!
    libxxx libxx libx xxxplayer
    skype-common skype xxxcodecs
    Install these packages without verification [y/N]? E: Some packages could not be authenticated”

    for some reason you cannot answer yes or no to this, it’s a bug. So simply install all of these packages using

    You should use the -y flag after. Thanks for the posting.

    cat /media/disc/package.list | xargs sudo apt-get install -y

    Comment by Moi | 19 July, 2008 | Reply

  15. Thanks for the tip.

    Comment by steveyoung | 19 July, 2008 | Reply

  16. I am currently evaluating backup solutions and came across your site. Can you elaborate on how mondo failed you as mentioned? It seems like this utility has been around a
    while and looks like stabilty is what everyone else seems
    to point to as being one of the best attributes.

    I was thinking of using partimage or modorescue to do an initial backup of my entire partition every so often and something else for incrementals:

    rsync / grsync
    rdiff-backup
    sbackup

    Seems like there are a lot of options. I want the partition backed up because I’ve built a lot of libraries from source and don’t want to go recompile everything again. Also, want something daily or weekly for my source code that is in flux.

    Comment by Christian | 20 February, 2009 | Reply


Leave a comment