Rsync, Level: Expert

Learning all kinds of new things, like how to escape spaces in filenames. When I tried the command

$ sudo rsync -avr --progress /etc/ /media/FreeAgent Backup/etc/

I got only error messages. What works is quoting,

sudo rsync -avr --progress /etc/ "/media/FreeAgent Backup/etc/"

or escaping:

sudo rsync -avr --progress /etc/ /media/FreeAgent\ Backup/etc/

. I back up /etc as it's small, and may be useful. The main thing to backup, of course, is /home:

sudo rsync -avr --progress /home/valorie/ "/media/FreeAgent Backup/home/"

where /home is the source, and /media/FreeAgent Backup/ is the destination. Thanks to sbeattie and maco in #ubuntu-women for showing me this.

Also, empty trash BEFORE running a backup of /home!

sudo rm -r /home/valorie/.local/share/Trash/*

. In #ubuntu-women, JanC suggested using "the GUI or a special commandline tool to empty the trash, as it keeps references around etc. The trash-cli package contains a command

empty-trash

." When I asked what she meant by references, JanC explained, "references as to what file has to be restored to where (if you ever want to do so). And of course there are separate trash directories on every disk you have etc. AFAIK trash-cli implements the XDG spec about this, just like GNOME & KDE do."

In #kubuntu-offtopic, James147 suggested that rather than just emptying the trash, I use --exclude=, ie rsync ... --exclude=*/Trash/* --exclude=*.tmp --exclude=*.bak ... etc. He suggested "useful patterns might be: *.tmp *.bak *.backup *~ *.swp */lost+found/*. I really wish I had thought to do this, and I definitely will in future. Watching that stuff scroll past as I waited was not fun!

I asked, "so each of the excludes needs to be separate, like: --exclude=*/Trash/* --exclude=*.tmp --exclude=*.bak"

[15:27] <james147> yes
[15:27] <james147> each pattern you want to exclude needs a separate --exclude=
[15:27] <james147> ^^ can get long but thats what scripts are for :D
[15:28] <valorie> that maybe my next step
[15:28] <valorie> for now, a good string I can copy/paste will be good

[15:29] <james147> valorie: a script isn't hard to do...

[15:29] <valorie> the up-arrow in bash makes things pretty easy
[15:30] <james147> valorie: can be as simple as

1. #!/bin/bash
   2. rsync ... --exclude=*.tmp --exclude=/Trash/* ... # add and edit this line till it suits your needs

[15:30] ^^ then you just need to run "chmod +x scriptname"
[15:31] then run it with ./scriptname or bash scriptname or sh scriptname
[15:31] (use sudo if it needs root)
[15:31] valorie: bash scripts can just be a bunch of commands you want to run

I'm not quite ready to wrap all this into a script, but will definitely think about do so before my next backup. http://www.linux.com/news/enterprise/storage/8200-back-up-like-an-expert-with-rsync discussing scripting the backup process too. For now, on to Natty Narwhal!

Rsync and Backup Success

A blog post started long ago, back in Oct. 2010:

If you are a reader of my blog, you might have suffered along with me through a non-existent backup. Tomorrow, I'm setting off for UDS in Orlando, Florida, and I wanted to be sure to have an up-to-date backup. So I started up my 1.5 T backup, newly formatted, and looked up a reliable article about using rsync. No more relying on GUI front-ends to it! I wanted to use it direct from the command-line, so I could see the errors right away. http://www.linux.com/news/enterprise/storage/8200-back-up-like-an-expert-with-rsync is simple and understandable, even for me. "The basic syntax for rsync is simple enough -- just run rsync [options] source destination to copy the file or files provided as the source argument to the destination."

Written in the now:

Unfortunately, I didn't finish this blog post after returning from Florida, but today a major disaster became a minor disaster by means of 1. a small screwdriver, 2. an inexpensive disk drive enclosure ($14.99 at Fry's), and 3. this simple command:

rsync -av /media/home/valorie/Music /home/valorie/Music

Disaster was my laptop power supply dying on me, without a recent backup. I know, shame on me! Instead, I used the screwdriver to take out the hard drive and attempted to put it into the HD enclosure I already have. However, it didn't fit. So I wrapped up the HD securely, and made a trip to Fry's. I showed the salesman the drive, to ensure I got the right kind of enclosure (SATA).

Once home, I put the drive into the enclosure, hooked it up to my netbook (thanks, ZaReason!) and began transferring my music from my old laptop to my new one, while continuing to work on the Amarok Handbook, which is *almost done*! I took the opportunity of a freshly installed Amarok to make some screenshots with which to finish the book! http://userbase.kde.org/Amarok/Manual

An update: One trouble I've had, besides some error messages that Sre-su helped me to figure out (thank you again, Sre-su!), is that the drive kept un-mounting. I don't know why -- bad USB connection, bad enclosure, bad HD -- but the solution is to just run the command again. Rsync checks the files it has already done, and then just charges ahead. I added -r (recursive) to be sure all music files were copied, and --progress so I can see at a glance what is going on. Also, --dry-run is cool -- it shows you what will happen, quick as a flash. Also, sre-su pointed out that I needed sudo before the command, which is now:

sudo rsync -avr --progress /media/home/valorie/Music /home/Music

I didn't catch my error in the above command until today, when I wondered why my music was STILL not in my Amarok collection! What must have happened is that while my original command was correct, the USB drive is touchy, and I needed to run it a few times. Not knowing this, I changed it, and put my Music not in /home/valorie/Music where it belongs, but in /home/Music/. Bleah.

Proper command, after all, is:

rsync -avr --progress /media/home/valorie/Music /home/valorie/Music

Learning New Skills: Reformat, Partition, Backup, Virtualize

Too much peace and quiet is too much, I guess. As soon as the current skill-set is working, I seem to need to add more complexity. Tomorrow is Ubuntu Global Jam, designed to get *buntu users together to test new software, the next release, work on bugs, clean out stale wiki pages, and other useful stuff. Seattle-area members of the Washington LoCo are gathering at the Rainier Beach Library tomorrow for Global Jam, so I thought I would try out Kubuntu Maverick, which is at the end of alpha. I figured backing up my current ~/home would be wise, so I bought a hard-drive at Costco for $99. For under a hundred dollars, I got 1.5 terrabytes!

Unfortunately, I hadn't done my research ahead of time, and it turns out that this Seagate model, the Free Agent, doesn't really like Linux, and isn't supported. Plus it was NTFS, so I figured out how to re-format it into EXT3, here: http://www.linuxquestions.org/questions/linux-newbie-8/how-to-format-a-ntfs-usb-hard-drive-to-fat32-or-ext3-621585/. Basicly, use sudo fdisk -l to figure out what you have, and then mkfs.ext3 /dev/sdb1 to re-format.

I asked some friends what the best Kubuntu partition manager was, and it turns out to be Partition Manager, hahaha. On the other hand, I'm not sure I got it successfully partitioned, but I decided that I could always do that later, if necessary. So after doing a bit of research, I settled on LuckyBackup to manage the backup, because it's a nice light GUI on top of rsync, which rocks. Set it up last night, and let it back up during the night. It feels GREAT to start out a day with success! I let it update ~/home this morning, and also backed up /usr.

Next step, ask the developers in #kubuntu-devel what is most useful for tomorrow. So, now downloading (very slowly) the Maverick 64-bit ISO, and investigating how to install Virtualbox for testing. Using http://www.ubuntugeek.com/create-and-manage-virtual-machines-using-virtualbox.html a guide to getting it going. It really was very easy!

I found another guide for those who are testing inside the environment, at http://www.mikesmullin.com/development/programmers-virtualbox-is-a-better-development-environment/. The seamless mode (http://www.makeuseof.com/tag/virtualboxs-seamless-mode-combine-operating-systems-desktop/ seems like it is worth trying out also.