Incremental Backup Script Explained

In an earlier post I laid out why I wanted yet another layer of backups for my important documents and I provided the script without much explanation. As an exercise for myself, I decided to post a step by step explanation for why I wrote the script the way I did. This will be educational for me, and hopefully interesting for you.

Alright, lets start at the beginning.

#! /bin/bash
# 

set -ue

 

I am highly influenced by David Pashley’s superb article “Writing Robust Bash Shell Scripts.” Pashley is not only a great programmer, he is a superb writer who can really explain his reasoning in a way that I find appealing.

Since I want this script to be robust, adding “set -ue “on the first line is a must. The “u” breaks the script if a variable is unset. This is critical as human error can easily turn a script into a vicious beast that will consume your data. The “e” breaks the script on any error. This ensures that everything behaves as expected. Unless there is a good reason not to, always “set -ue”

For reference here is the explain shell for the command.

Here is the first function:

function build_vars
{
# Current Date / Time
epoch=$(date +%s)
month=$(date +%m)
year=$(date +%Y)
day=$(date +%d)

# Building Environment: Directories
configdir="$config_dir"
yeardir="$backup_dir"/"$year"
monthdir="$yeardir"/"$month"
daydir="$monthdir"/"$day"
backupdir="$daydir"

# Building Environment: File Names
backupfile="$backupdir"/"$prefix"."$epoch".tar.bz2
logfile="$configdir"/"$prefix".log
lastepochfile="$configdir"/"$prefix".last
}

 

Basically this function sets up all of the variables which will be used in the rest of the script. Putting them in a single function improves readability and makes maintenance easy.

Here we also see the first crumbs of how the backup system works. By finding the current Unix epoch, the script discovers the current time in a non-timezone dependent way. It can then use this as reference point and as a means of ultimately building unique file names.

In addition, this function builds all of the directory names that will be used throughout the remained of the script.

I could probably make this more efficient by only running “date” once, but all of the target systems are multi-Ghz so I can endure a couple of wasted cycles.

Here is the second function:

function check_epoch
{
#Checking Time Since Last Run
if [ -n "$(find "$lastepochfile" -mtime -1)" ]; then 
  echo "Less than 24 hours since last archive"
  exit 0
fi
}

 

This function uses “find” to check if the file “$lastepochfile” is older than 1 day. As “$lastepochfile” is the last thing created in the script this effectively checks to see if the script successfully completed in the last 24 hours. Since I ended up setting a cron job that runs once an hour, this ensures that even if a system is only powered on for short times during the day the job will still get done but not OVER done which would waste cycles and file space.

For reference here is the explain shell for the find command used.

Here is the third function:

function check_env
{
#Checking Configuration Directory
if [ ! -d "$configdir" ]; then
  mkdir "$configdir"
  echo "$epoch" Making "$configdir" >> "$logfile"
fi

#Checking Year Directory
if [ ! -d "$yeardir" ]; then
  mkdir "$yeardir"
  echo "$epoch" Making "$yeardir" >> "$logfile"
fi

#Checking Month Directory
if [ ! -d "$monthdir" ]; then
  mkdir "$monthdir"
  echo "$epoch" Making "$monthdir" >> "$logfile"
fi

#Checking Day Directory
if [ ! -d "$daydir" ]; then
  mkdir "$daydir"
  echo "$epoch" Making "$daydir" >> "$logfile"
fi
}

 

While this covers a lot of lines, it is really doing something very simple, it checks to see if the year / month / day directory structure exists for the backup snap-shot that will ultimately be produced. If it does not exist then it makes it.

Effectively what this means is that each snapshot will be in its own directory which will be easy to browse to and find.

Here is the fourth function:

function write_file
{
#Incremental Tar & Compression
echo "$epoch" Writing "$backupfile" >> "$logfile"
find "$target_dir" -type f ! -name ".*" -newer "$lastepochfile" -print0 | \
tar czvf "$backupfile" --null -T -

#Log Success
echo "$epoch" Success "$backupfile" >> "$logfile"
}

 

This is the payload of the entire script. Everything else has just been leading up to this. First “find” is used to search “$target_dir” for all files that are newer than “$lastepochfile” and do not match the pattern .* . The reasoning for this should be pretty obvious, but I should not that I added the exclusion (by using the !) so that I wouldn’t needlessly copy lockfiles etc.

To ensure proper streaming of the files to tar I used the -printO.

It is important to note that this only searches out changed files and copies only their directory structure. My first approach copied the entire directory structure resulting in mostly empty directories… this resulted in overly large files as all those empty directories eat up space.

The tar line is pretty standard, note I use –null to ensure that it properly deals with null-delimited output and -T sets tar up to receive the filenames.

For reference here is the explain shell for the find and tar commands used.

Success is then logged.

Here is the fifth function

function write_epoch
{
# Unprotect Last Epoch File
chmod 600 "$lastepochfile"

# Write Last Epoch File
echo "$epoch" > "$lastepochfile"

# Protect Last Epoch File
chmod 400 "$lastepochfile"
}

 

In a way this is the most important part. First it unprotects “$lastepochfile” then it writes the epoch to it and finally the script protects “$lastepochfile.” As the entire system depends on the accuracy of “$lastepochfile” the protection should prevent casual deletion. Worst thing comes to worst, the epoch of the last run is saved in the log file and a new “$lastepochfile” could be created with the date.

Writing the epoch to the file isn’t necessary, but I think it provides a nice backup record and isn’t really that different than simply touching the file.

There is, likely, a way of doing this that does not require an external file (instead referencing the last created archive) but I like that this approach is easy to debug and trivial to understand.

Finally the actual program

# Configuration
prefix=important
target_dir=/home/user/Important
backup_dir=/home/user/ImportantArchive
config_dir=/home/user/ImportantArchive/.kb

#Program
build_vars
check_epoch
check_env
write_file
write_epoch

exit 0

 

As you can see, I’m really just using functions as a way of organizing the script. I prefer to think in terms of modules which I can replace as needed and this provides that. Also I like that “forcing” an update would be as simple as commenting out “check_epoch”.

So there you have it, it is no masterwork of programming but this script has now been comfortably running on two different systems automatically using CRON for 4 months without a single glitch. I can’t really imagine a better testament to its stability than that!

So what could I improve about it? I’m no expert at scripting so I’d love any feedback that anyone has.

Advertisements

Incremental Backup Script

I am Paranoid! (When it comes to Backups)

There I said it. I am a paranoid person but thankfully most of my paranoia is restricted to backups of important files. When you think about it, an academic truly is their files… my main document directory represents thousands of hours of time and years of my life. Scary thought, right?

So I pursue a multi-layered approach to backing up my critical files. My current approach utilizes manual backups, p2p backups / synchronization among my systems, and backups to Google Drive. However, even that wasn’t enough for me to be comfortable. What if I overwrote a file by accident? Only the p2p network (which uses Syncthing and includes rudimentary versioning) could save me. But what if the conflict resolution system made an incorrect decision and preserved the wrong copy of the file? What then?

Like I said, I’m paranoid. So I decided that each major work system that I use should make incremental backups on a daily basis. Essentially with the command line you can easily search for changed files, then tar them. This produces a snap-shot style backup which could, in theory, be used to regenerate your files as they were at any given time.  Write a quick cron task and now it is scheduled and automated.

However, just to make things more complicated I had three more requirements:

  1. While I want no more than one incremental, snap-shot style backup per day, I cannot guarantee that each system will be on at a given time.
  2. I want the snap-shots to be easy to navigate.
  3. As this is a script that I intend to run automatically for years, it has to have robust error handling and logging.

So this is now more complex than a single line of BASH, time for some scripting!

I am attaching the solution I came up with but I’ll explain my reasoning for each thing more in another post

Here is the script I came up with. There is nothing innovative about it but it has been running once an hour (it is scheduled with cron) for 4 months without a single hiccup or fault–my definition of success! If it is useful to you feel free to use/modify/abuse it… consider it under the MIT License.

Thanks for reading and I’d love to hear any thoughts that people have on how to improve it

#! /bin/bash
# 

set -ue

function build_vars
{
# Current Date / Time
epoch=$(date +%s)
month=$(date +%m)
year=$(date +%Y)
day=$(date +%d)

# Building Environment: Directories
configdir="$config_dir"
yeardir="$backup_dir"/"$year"
monthdir="$yeardir"/"$month"
daydir="$monthdir"/"$day"
backupdir="$daydir"

# Building Environment: File Names
backupfile="$backupdir"/"$prefix"."$epoch".tar.bz2
logfile="$configdir"/"$prefix".log
lastepochfile="$configdir"/"$prefix".last
}

function check_epoch
{
#Checking Time Since Last Run
if [ -n "$(find "$lastepochfile" -mtime -1)" ]; then 
  echo "Less than 24 hours since last archive"
  exit 0
fi
}

function check_env
{
#Checking Configuration Directory
if [ ! -d "$configdir" ]; then
  mkdir "$configdir"
  echo "$epoch" Making "$configdir" >> "$logfile"
fi

#Checking Year Directory
if [ ! -d "$yeardir" ]; then
  mkdir "$yeardir"
  echo "$epoch" Making "$yeardir" >> "$logfile"
fi

#Checking Month Directory
if [ ! -d "$monthdir" ]; then
  mkdir "$monthdir"
  echo "$epoch" Making "$monthdir" >> "$logfile"
fi

#Checking Day Directory
if [ ! -d "$daydir" ]; then
  mkdir "$daydir"
  echo "$epoch" Making "$daydir" >> "$logfile"
fi
}

function write_file
{
#Incremental Tar & Compression
echo "$epoch" Writing "$backupfile" >> "$logfile"
find "$target_dir" -type f ! -name ".*" -newer "$lastepochfile" -print0 | tar czvf "$backupfile" --null -T -

#Log Success
echo "$epoch" Success "$backupfile" >> "$logfile"
}

function write_epoch
{
# Unprotect Last Epoch File
chmod 600 "$lastepochfile"

# Write Last Epoch File
echo "$epoch" > "$lastepochfile"

# Protect Last Epoch File
chmod 400 "$lastepochfile"
}

# Configuration
prefix=important
target_dir=/home/user/Important
backup_dir=/home/user/ImportantArchive
config_dir=/home/user/ImportantArchive/.kb

#Program
build_vars
check_epoch
check_env
write_file
write_epoch

exit 0