Setting disk UUIDs

December 14, 2009

After setting up mounting by UUID as described here, I realized I had a problem.

I wanted to swap disks out (that is, have multiple versions of my backup drive). However, since the drives are being mounted by UUID, I wanted to have only one entry in /etc/fstab for all my “backup” drives.

It appears that dd would work, but that seemed inelegant, especially if future drives are different sizes. So I did a bit of searching and found the discussion here.

In short, I could use

sudo tune2fs /dev/sdb1 -U new-uuid-number

to set the UUID of a file system to whatever I want.

After I’d done that, I couldn’t figure out how to refresh the UUID in /dev/disk/by-uuid/ – even after ejecting and re-inserting the SATA disk, the old UUID still showed up. I ended up rebooting (sigh) and now I can swap drives out with abandon. I probably should have specified the UUID when I did the mkfs. This looks like this bug.

I haven’t tried it, but I would guess having two disks with the same UUID in the machine at the same time would be a Bad Thing.

In retrospect, it would probably have been better if I’d used disk labels instead of UUIDs.

Changing Unix/Linux filenames to handle DOS conventions

December 12, 2009

Recently, I had a bunch of files with characters in their names that made them hard to handle under Windows. Although Unix has no problems with :, ? and others, DOS / Windows doesn’t handle them well. This made using them on my CIFS share a bit of a pain.

To rename them, I hacked up a quick Groovy script. It maps all “weird” characters to _, which is pretty much universal.

Note that this script doesn’t currently handle the case where two files map to the same name. If you have :foo and ?foo, they’ll both get mapped to _foo and you’ll lose one of them.

I called it dosname because it was late and I wasn’t feeling creative.

#!/usr/bin/groovy -
if(args.length == 0) {

dirName = args[0];
new File(dirName).eachFile() { file ->
    def matcher = (file.getName() =~ /[\?\:\"\\]/)
    def newName = matcher.replaceAll("_")
    File newFile = new File(dirName + "/" + newName.toString())
    println file.getName() + " -> " + newFile.getName()

def printArgs() {
	println("Usage: dosname [dirname]")
	println(" Note: I don't handle the case where two files map to the same filename yet!")

The interesting bit is in the line

def matcher = (file.getName() =~ /[\?\:\"\\]/)

That line defines the regular expression which contains a group of characters which will be replaced by _. Right now it’s [?:”\] (all quoted with \ because that’s what you have to do in regex).

Run it with “dosname directoryname“. There’s no way to reverse its effects, so try not to use it on useful directories like /etc/.

Also, I needed to install OpenJDK6 JDK (not just the runtime) in order to run Groovy on Ubuntu 9.04. Go figure.

Since I don’t know Groovy, I stole most of this code from here.

Using rsync to back up a Windows box to Ubuntu Server 8.04

December 10, 2009

After getting rsync working so well for backing up hard drive to hard drive, I naturally wanted to back up from my Windows box to my Ubuntu server. Luckily, Cygwin has an rsync for Windows – so I started by installing that. (I actually already had it installed – Cygwin is nice.)

I found some good instructions here, and used them as the basis for what I did.

The steps I went through:

  1. First I created /etc/rsyncd.conf. I wanted to share a single directory called shared, so I had just one entry:
    comment=Shared directory
    read only=false
    auth users=rsyncuser
    secrets file=/etc/rsyncd.secret

    I picked group sambashare just to be consistent with my Samba configuration. rsyncuser does not exist on the machine; it is instead mapped to andrew (uid)

  2. Next I created /etc/rsyncd.secret:
  3. After that, I needed to enable rsync (both right now and after reboot). First, I edited /etc/default/rsync and changed RSYNC_ENABLE to true:

    Next I started the rsync daemon:

    sudo /etc/init.d/rsync restart
  4. The /etc/inetd.conf of a plain Ubuntu 8.04 Server install was empty, which was a surprise to me. There appears to be a utility called “update-inetd” to update the file and then restart inetd. Because I couldn’t be bothered to find out the syntax, I just edited /etc/inetd.conf and put in:
    rsync  stream  tcp     nowait  root    /usr/local/bin/rsync rsyncd --daemon

    Then I used update-inetd from the command line to restart inetd for me:

    sudo update-inetd --disable rsync
    sudo update-inetd --enable rsync
  5. At this point, I could telnet to port 873 (the rsync port) and see a connection, but rsync itself still didn’t want to run. Instead, it failed (even when I entered the correct password) with this message:
    @ERROR: auth failed on module shared
    rsync error: error starting client-server protocol (code 5) at main.c(1383)

    Andrew Tridgell has awesome utilities, but he really has a problem with displaying meaningful messages.

    It turns out that rsync really wants the secrets file (in my case /etc/rsyncd.secret) to be readable only by the rsync user. There is a global option “strict modes” that can be set to false to allow you to get around this, but I decided why not just do the right thing:

    sudo chown root.root /etc/rsyncd.secret
    sudo chmod 400 /etc/rsyncd.secret
  6. Finally, I needed a command for the rsync on my Windows box. Here was one I came up with:

    @echo off
    set RSYNC_PASSWORD=secretrsyncpassword
    rsync -vrtz --delete --delete-excluded --exclude "Temp/" --exclude "*.tmp" --exclude "parent.lock" --exclude "UsrClass.dat*" --exclude "NTUSER.DAT" --exclude "ntuser.dat.LOG" --exclude "Cache/" "/cygdrive/c/Documents and Settings/" "rsyncuser@myserver::shared/win2k-backup/Documents and Settings"

    This command preserves timestamps (t) and prints out what it’s doing (v) as it recursively (r) goes through Documents and Settings and backs up the files, using compression so it’s faster over the net (z). It excludes a bunch of things.

I should probably have used a file to specify what’s excluded (–exclude-from), but I was lazy and just kept adding to the command line. I’m excluding cache files, as well as lock/log files that are kept open by Windows 2000 (they report as errors if you try to back them up).

At some point I’ll probably add a “hosts allow” line to my /etc/rsyncd.conf, as soon as I decide which machines I want to let backup to the server.

Automatically mounting drives with UUIDs

December 6, 2009

Until now, I’ve always mounted drives by accessing their devices. However, I ran into a situation where this wouldn’t work. Luckily, Ubuntu has the ablility to access drives by UUID – which solved my problem.

I have a drive that holds networked data, which I mount on Ubuntu 8.04 Server as /data/. I also back that drive up to a drive which is normally read-only as /databackup/.

Both of these drives are SATA – meaning they could be unplugged at any time. If both are unplugged, whichever drive gets plugged in first becomes /dev/sda1 – and the other becomes /dev/sdb1. This means I can’t rely on mounting /dev/sda1 on /data and /dev/sdb1 on /databackup.

To get around this, I mounted the drives using UUID in /etc/fstab. First, I had to figure out what the UUIDs of the drives were. To start with, I killed Samba and unmounted both – I knew /dev/sda1 was /data and /dev/sdb1 was /databackup. Then I obtained the UUIDs of both drives:

$ sudo vol_id --uuid /dev/sda1
$ sudo vol_id --uuid /dev/sdb1

Next, I replaced the device mounts (/dev/sda1 and /dev/sdb1) in /etc/fstab with:

UUID=7b932326-717b-4ba6-bef2-fedfbafcabe6 /data ext3...
UUID=94efd7bd-8498-46f3-ab6d-cb706c413567 /databackup ext3...

Under Ubuntu 9.10, it appears that vol_id has merged into blkid – so now you would use:

$ sudo blkid /dev/sda1
/dev/sda1: UUID="f30ba2a3-9da6-48b1-8ab5-75952ef26cc4" TYPE="ext3"

to determine the UUID, and then update /etc/fstab as you’d do for 8.04.

Simple backup using rsync

December 3, 2009

A simple way to back up your data from one drive to another is rsync. The magical incantation (assuming the data drive is /dev/sda1 mounted on /data and the backup drive is /dev/sdb1 mounted on /databackup, and you normally keep the backup drive mounted read-only):

sudo mount -o remount,rw /dev/sdb1 /databackup
sudo rsync -a --delete --progress /data/ /databackup
sudo mount -o remount,ro /dev/sdb /databackup