Time Machine, Meet Netatalk

2009-10-06

Introduction

My family has started using MacOS on laptops. Apple’s been shipping a nifty
little backup utility, Time Machine - it makes backups and restores to easy,
the wife can use it (and believe me, this is a big improvement from things
like dirvish, or
amanda, or other things that require
a more extensive knowlege of UNIX). For more info, check out
Apple’s
site.

Time Machine requires either a local disk, or a special (Apple-specific)
networked disk called a Time Capsule. Since I’ve got a FreeBSD machine in
the basement with a good-sized ZFS pool, I’d rather leverage that. In
theory, if the laptops’ Time Machines can backup to the basement ZFS pool,
they’ll get the benefit of dynamically grown disk space and offsite backups
(well, once I figure out an offsite backup solution for the pool, at least).
Sadly, making Time Machine use a non-Apple widget for storage is decidedly
non-intuitive, and I can never seem to find appropriate documentation on
this when I need to. Below are my notes on the subject, for future
reference.

---

Server Setup

Personally, I’m using a FreeBSD file server these days (primarily for the
availability of ZFS, and the lack of Solaris). These instructions are rather
general, but be warned that you may have different results under different
operating systems.

You’ll need two pieces on the server:
netatalk for serving the data,
and Avahi for advertising your share.

Netatalk

Netatalk implements Apple’s File Protocol (AFP) under UNIX. Install it
however your OS installs software (it’s in FreeBSD’s ports, and is a package
for most Linux distributions). Netatalk uses PAM for authentication, so
take a look in /etc/pam.d and make sure it looks like netatalk
will authenticate.

We care about two config files, AppleVolumes.default (which
lists the available shares) and afpd.conf (which controls the
file sharing service). Netatalk can handle a variety of other ancillary
AFP tasks (there’s a whole set of protocols for naming things, for instance),
but we really don’t care about that now. The format of both files is simple:
one directive per line, and lines starting with # are comments.

I declare that a particular subdirectory off my ZFS pool is available as an
AFP volume named “fmep-Tardis” (what else do you call fmepnet’s time machine?)
That’s the name of the disk as it appears in Time Machine Preferences on your
clients. Relevant AppleVolumes.default parts:

# The “~” below indicates that Home directories are visible by default.
# If you do not wish to have people accessing their Home directories,
# please put a pound sign in front of the tilde or delete it.
#~
/pool/backup/time_machine       “fmep-Tardis”

I haven’t had need to modify afpd.conf from default. Since your
default may vary, here’s what I’m using:

# default:
# - -transall -uamlist uams_clrtxt.so,uams_dhx.so -nosavepassword
- -transall -uamlist uams_clrtxt.so,uams_dhx.so -nosavepassword

Consult the afpd.conf man page for details. Basically, this sets
the default options for all servers (as clearly indicated by the server name
“-“), and allows both PAM-based and Diffie-Hellman key exchanges
for password authentication.

Avahi

Avahi is an mDNS responder, for zero-configuration service advertisements.
Ever wonder how your Mac finds other Macs on the network? Here you go.
Again, install as best suits your OS - it’s a FreeBSD port, and a package on
a variety of Linuxes.

Avahi is a layer-2 protocol - it doesn’t use routeable IP addresses. You’ll
want to run this on something that’s on the same network as your Mac clients.
If you have a separate wireless and wired network in the house, as I do,
you’ll want to put Avahi on something connected to each network (well, each
network with clients, at least). There are no problems routing AFP over TCP,
so it’s perfectly permissible to use Avahi to advertise an AFP server on a
different subnet - in fact, that’s what I’m doing right now.

Avahi has a top-level config file, avahi-daemon.conf, and a
config file for every advertised service. Here’s my
avahi-daemon.conf:

# See avahi-daemon.conf(5) for more information on this configuration
# file!

[server]
#host-name=foo
#domain-name=local
#browse-domains=0pointer.de, zeroconf.org
use-ipv4=yes
use-ipv6=no
#check-response-ttl=no
#use-iff-running=no
#enable-dbus=yes
#disallow-other-stacks=no
#allow-point-to-point=no

[wide-area]
enable-wide-area=yes

[publish]
#disable-publishing=no
#disable-user-service-publishing=no
#add-service-cookie=yes
#publish-addresses=yes
#publish-hinfo=yes
#publish-workstation=yes
#publish-domain=yes
#publish-dns-servers=192.168.50.1, 192.168.50.2
#publish-resolv-conf-dns-servers=yes

[reflector]
#enable-reflector=no
#reflect-ipv=no

[rlimits]
#rlimit-as=
rlimit-core=0
rlimit-data=4194304
rlimit-fsize=0
rlimit-nofile=30
rlimit-stack=4194304
rlimit-nproc=3

Note that this is basically the default config on many OSes. Be sure to enable
ipv4!

When you installed Avahi, it should have created a “services“
directory. Files in there are XML service descriptions, usually one per
service name. Here’s my afp.service:

<?xml version=”1.0” standalone=’no’?><!–-nxml-–>
<!DOCTYPE service-group SYSTEM “avahi-service.dtd”>

<!– $Id: time_machine.html,v 1.5 2009/10/06 03:47:50 shuey Exp $ –>

<!–
  This file is part of avahi.

  avahi is free software; you can redistribute it and/or modify it
  under the terms of the GNU Lesser General Public License as
  published by the Free Software Foundation; either version 2 of the
  License, or (at your option) any later version.

  avahi is distributed in the hope that it will be useful, but
  WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
  General Public License for more details.

  You should have received a copy of the GNU Lesser General Public
  License along with avahi; if not, write to the Free Software
  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
  02111-1307 USA.
–>

<!– See avahi.service(5) for more information about this configuration file –>

<service-group>

  <name replace-wildcards=”yes”>AFP on fergus</name>

  <service>
    <type>_afpovertcp._tcp</type>
    <port>548</port>
    <host-name>server.in.my.basement.fmepnet.org</host-name>
  </service>
  <service>
    <type>_device-info._tcp</type>
    <port>0</port>
    <txt-record>model=Xserve</txt-record>
  </service>

</service-group>

Most fields above are self-explanatory. The host-name field
can be replaced with an IP address, if you don’t have DNS service for internal
machines around the house. The _device-info service stanza
contains metadata for the service. In this case, the model=Xserve
causes more recent versions of MacOS to think your AFP service is an Apple
Xserve, so you’ll get a pretty little icon (an Xserve RAID picture, I believe).

File System Bits

Obviously, make sure your Time Machine directory, referenced in Netatalk’s
AppleVolumes.default, actually exists. It’ll also need to be
writable by whatever user or group that should be using the volume - group
read/write/execute permissions for “users“ is probably a good
idea.

You’ll also need to create an empty file named
.com.apple.timemachine.supported. This did it for me:

touch /pool/backup/time_machine/.com.apple.timemachine.supported

---

Client Configuration

The bad news: Because you’re using a non-Apple server, this isn’t going to be
very intuitive. The good news: Once a client is set up, it will just work as
normal - just like if it was an Apple server on the other end. Open a
terminal window, and let’s get started.

Netatalk may be AFP, but it’s not quite supported. In a terminal window, run
this:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

Now go into System Preferences, into the Time Machine config. You should be
able to see your shiny new Netatalk share (assuming your Avahi and Netatalk
configurations are correct, and both sets of daemons are running, and nothing
else is wrong), so you can configure Time Machine normally. Go ahead, start
your first backup. Then watch it fail.

Time Machine is trying to create a sparse bundle disk image - that’s a set of
files that pretend to be a MacOS disk, and grow (or increase in numbers) as
files are copied into the disk. That’s MacOS’s way of handling
less-than-supported volumes. Problem is, Netatalk doesn’t support a couple
of the AFP operations necessary to finish off the disk image. You can
work around it by creating your own disk image locally, then copying the
result to the Netatalk server.

Fire up Disk Utility. Use File->New Blank Image… to create a new disk image.
Fill out the form as follows:

• Volume Name: computername_macaddr.sparsebundle


• Volume Size: Custom (enter a size larger than your client’s disk)


• Volume Format: Mac OS Extended (Journaled)


• Encryption: none


• Partitions: Single partition - Apple Partition Map


• Image Format: sparse bundle disk image

Note that the volume name is your client’s hostname, an underscore, followed
by the MAC address of the on-board ethernet interface (in hex, all lower case,
with no :s), with the .sparsebundle extension. To
find out our machine’s MAC address, just do an ifconfig -a and
look for the “media“ line in the en0 stanza.

If you have issues with the GUI, you may want to try the CLI version:

hdiutil create -library SPUD -size $SIZE -fs Journaled HFS+ -type SPARSEBUNDLE -volname $MACHINENAME_$MAC_ADDRESS.sparsebundle

Once you’ve created a sparse bundle, copy it (scp works for me)
over to your Netatalk server. The sparse bundle is actually just a directory
of information, so this should work fine.

With the sparsebundle in place, open Time Machine’s preferences and do another
backup. Things should click, your Netatalk server should mount, and data
should start flowing. Yay!

---

Caveats

Server Disk Fills Up

If your server’s disk fills up, Time Machine will not be happy. Early versions
of Time Machine would nuke all old backup images (except maybe the latest one).
Supposedly this has been fixed, but….well, caveat emptor.

Restores

If you’re like me, you use a networked file server to back up your laptop.
If your laptop’s disk dies, you’ll have trouble restoring - the Mac OS X
install disk can “Restore from Time Machine Backup”, but it won’t find yours
(since it’s on an unsupported volume). Supposedly, you can look under the
Utilities menu and open up a Terminal, then run this:

mount -t afp afp://username:password@hostname/ShareName /Volumes/ShareMount

Obviously, customize the above for your setup. If you can ls
/Volumes and see your share, with your sparsebundle in it, it’s
mounted correctly. Once that’s done, close the terminal and try “Restore
from Time Machine Backup” again, and things should work.

Well, in theory, at any rate.

Acknowledgements

Google is extremely helpful, as always. Also, Matthias Kretschmann’s blog
has proven pretty handy, especially
this entry.

apple