In this article, we’ll take a look at using Leopard’s launchd and rsync to develop a automated data backup methodology. In addition to rsync and launchd, we will also integrate Bernhard Baehr’s SleepWatcher utility.
In our environment, we use NetApp filers to host our network shares and are using CIFS network shares. However, with NetApp and Mac OS X, those file shares could be NFS, just as well. This will also work with your AFP shares.
Components
-
- launchd
- launchd is Apple’s (not-so-new) system startup program. launchd is more powerful than older programs such as init and cron, since it can not only start and stop programs but can also be triggered by other events, such as availability of network shares etc, which we will use in this setup.
-
- rsync
- Most people reading this article will already know what rsync is. rsync is a program that behaves in much the same way that rcp does, but has many more options and uses the rsync remote-update protocol to greatly speed up file transfers when the destination file is being updated. In this setup, rsync does the heavy lifting of actually performing the backups.
-
- SleepWatcher
- is a niftly command line tool (daemon) developed by Bernhard Baehr for Mac OS X that monitors sleep, wakeup and idleness of a Mac. It can be used to execute a Unix command when the Mac or the display of the Mac goes to sleep mode or wakes up or after a given time without user interaction. We will be using SleepWatcher to do just that – to automatically mount and unmount network shares.
-
SleepWatcher
As mentioned above, SleepWatcher monitors the sleep and wake states of a Mac (among other things). We’ll be using both of the SleepWatcher’s components, the SleepWatcher command line tool and the SleepWatcher StartupItem. As part of the tool’s functionality, SleepWatcher employs two hidden files,.wakeupand.sleepin the user’s home directory to execute user specific commands when a Mac wakes up from sleep or goes to sleep, as the case may be.First, we need a wakeup script to automatically mount our network shares when the laptop/Mac wakes up from sleep.
/Users/username/.wakeup
#!/bin/bash filer="bitbucket.example.com"; test "$SHELLOPTS" && shopt -s xpg_echo function checkWorkNetwork() { # sleep for a bit, wait for network to show up sleep 30 count=$(/sbin/ping -c 3 -o $filer | grep "received" | awk -F',' '{ print $2 }' | awk '{ print $1 }') if [ $count -eq 0 ]; then # 100% failed - try one more time after sleeping for 30 secs sleep 30 count=$(/sbin/ping -c 3 -o $filer | grep "received" | awk -F',' '{ print $2 }' | awk '{ print $1 }') if [ $count -eq 0 ]; then /usr/bin/logger -t SleepWatcher:wakeup "filer not reachable...Exiting". exit; fi else /usr/bin/logger -t SleepWatcher:wakeup "filer is reachable...Continuing". fi } function doMounts { vol=$1; mnt_cnt=$(/sbin/mount | grep -ic "/Volumes/$vol") if [ $mnt_cnt -le 0 ]; then /usr/bin/osascript <<EOT tell application "Finder" to mount volume "cifs://$filer/$vol" EOT fi } checkWorkNetwork doMounts Backups exit 0In the above wakeup script, first we check to see if we are on our work network using the function
checkWorkNetwork. and mount the network resources. This function can easily be modified to adapt to a home network. At home, if we are on a wireless network, a function like the following may be used.function checknMountHomeNetwork { # sleep for a bit, wait for network to show up sleep 60 homelan=$(/usr/sbin/system_profiler SPAirPortDataType | awk -F': ' '/Current Wireless Network/{print $2}') if [ "$homelan" == "JewelThief-Fake" ]; then /usr/bin/osascript <<EOT tell application "Finder" to mount volume "cifs://192.168.1.3/Documents" EOT /usr/bin/logger -t SleepWatcher:wakeup "Mounted Home Volumes...Exiting" exit 0 else /usr/bin/logger -t SleepWatcher:wakeup "Cannot ascertain home network...Exiting" fi }Next, we need a sleep script to trigger the stoppage of any running rsync backups and unmount the network shares.
The sleep script needs to execute and finish within 15 seconds. After this timeout, the system forces the sleep mode and your script may not finish properly.
/Users/username/.sleep
#!/bin/bash test "$SHELLOPTS" && shopt -s xpg_echo function doUmounts { /usr/bin/logger -t SleepWatcher:sleep "Going to sleep...unmounting network volumes" /usr/bin/osascript -e 'tell application "Finder" to eject (every disk whose local volume is false)' } function kill_rsync { /usr/bin/logger -t SleepWatcher:sleep "Stopping rsync" pid=`ps auxw | grep rsync | grep -v grep | awk '{print $2}'` while [ ! -z "$pid" ]; do echo -n "." for pid in $pid do kill -9 $pid done sleep 3 pid=`ps auxw | grep rsync | grep -v grep | awk '{print $2}'` done } kill_rsync doUmounts exit 0In the above sleep script, we first kill off any running rsync jobs using the
kill_rsyncfunction and unmount the network shares using thedoUmountsfunction. -
/usr/local/laptop_rsync.sh
#!/bin/sh PROG=$0 SRC="$HOME/Documents/" DST="/Volumes/Backups/$USER/" CMD="/usr/bin/rsync -q --numeric-ids -E -axzS --exclude-from=$HOME/.rsync_excludes.txt $SRC $DST" if [ ! -r $SRC ]; then /usr/bin/logger -t $PROG "Source $SRC not readable - Cannot start the sync process" exit; fi if [ ! -w $DST ]; then /usr/bin/logger -t $PROG "Destination $DST not writeable - Cannot start the sync process" exit; fi /usr/bin/logger -t $PROG "Start rsync" $CMD /usr/bin/logger -t $PROG "End rsync" exit 0
In this script, we are using rsync to synchronize the Mac user’s Documents/ directory to a network share. We are also using a exclude list here called
.ib_backup_excludes.txtto prevent any unwanted/personal files and directories from being synchronized to the network share.$HOME/.rsync_excludes.txt
/tmp/*
/Network/*
/cores/*
*/.Trash
/afs/*
/automount/*
/private/tmp/*
/private/var/run/*
/private/var/spool/postfix/*
/private/var/vm/*
/Previous Systems.localized
.Spotlight-*/
._*
Personal
personal
Most importantly, if we create a directory called Personal or personal within our Documents/ directory and place all of our personal files, they will not be synchronized to our network files share.
- Lastly, we need way to automatically run this rsync script (and not run it when the network shares have disappeared). Here’s where Apple’s launchd comes in. We will give control of the above laptop_rsync.sh script to launchd by creating a launchd agent file. From the launchd.plist man page, the agent files can be deployed in several locations, depending on whether this is done by a regular user or by an administrator. In our example, I am using the administrator location and installing the launchd agent file in
/Library/LaunchAgents/.The rsync launchd agent file is
/Library/LaunchAgents/name.rajeev.laptop_rsync.plist<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>KeepAlive</key> <dict> <key>PathState</key> <dict> <key>/Volumes/Documents/</key> <true/> </dict> </dict> <key>Label</key> <string>name.rajeev.laptop_rsync</string> <key>Program</key> <string>/usr/local/laptop_rsync.sh</string> <key>StartInterval</key> <integer>3600</integer> <key>ThrottleInterval</key> <integer>3600</integer> <key>WorkingDirectory</key> <string>/usr/local/</string> </dict> </plist>In the above plist agent file, note that we are using the key
PathState. launchd monitors for this path specified by this key to become available and launches the script once the path becomes available. In our example, once this path becomes available (via the wakeup script which is in turn launched by SleepWatcher), the rsync backup job is kicked off. Also note that we are using aThrottleIntervalto prevent launchd from starting the rsync script too fast.This agent file can itself be loaded upon restart by using the
$HOME/.launchd.confor/etc/launchd.conf. The command to load and unload this agent into launchd is below:launchd load /Library/LaunchAgents/name.rajeev.laptop_rsync.plist. This will load this agent into launchdlaunchd unload /Library/LaunchAgents/name.rajeev.laptop_rsync.plist. This will unload this agent into launchd
Now that we have the wakeup and sleep scripts, our network shares will be automatically mounted and unmounted. We will now need a way to detect these network shares transitions and periodically launch our backup scripts. For this, we need a rsync backup script and a launchd hook to manage the automatic launch of this script.
Let’s first take a look at the rsync script itself. In our example, the script is installed in /usr/local/.
