OSE Site Backup

Page Status

Hint: As of 2017, all of this information appears to be invalid. See OSE Server#Backups for current backup info.

See OSE Server#Backups for current backup info.

Overview

Our website uses mbackup as a backup tool. mbackup uses duplicity as a back-end.

Introduction

mbackup performs module-oriented backups.

Example:

mbkp.sh forum-db
mbkp.sh forum-root
mbkp.sh forum-extra

The commands above will backup 3 different modules: one for the forum database, one for the forum root files (which don't change frequently) and one for extra forum files (uploaded files)

You define what each module should back up using these module configuration files:

• module-name.pre - executes before the backup starts
• module-name.post - executes after the backup ends
• module-name.files - defines which files should be included and excluded, using the same syntax as duplicity
• module-name.priv - contains sensitive information, like database passwords

Pre-Requisites

OSE's site is located on a shared server. We had to install duplicity and its libraries on a separate folder (hence identified by $D_ROOT). Before running mbackup, we need to prepare some environment variables by running $D_ROOT/init.sh:

% cat $D_ROOT/init.sh 

BASEDIR="$(dirname $0)"
export LD_LIBRARY_PATH=$D_ROOT/lib:$D_ROOT/usr/lib
export PYTHONPATH=$D_ROOT/share/pyshared:$LD_LIBRARY_PATH
export PATH=~/bin:$D_ROOT/bin:$PATH
export HGRCPATH="$HOME/.hgrc"

Files and Folders

mbackup itself is installed on a folder identified as $MBKP_HOME.

On ~/.config/mbackup/ we have configuration files:

-rw-r--r-- 1 a_user a_group 1 2011-06-05 15:43 mbackup-config.sh
drwxr-xr-x 2 a_user a_group 4 2011-06-05 16:03 modules/
drwx------ 2 a_user a_group 4 2011-06-05 16:03 priv/

On ~/.config/mbackup/modules we have the configuration for each module to be backed up:

blog-db.pre      drupal-db.post         forum-db.pre       wiki-db.post
blog-db-var.pre  drupal-db.pre          home.files         wiki-db.pre
civicrm-db.pre   drupal-sites.excludes  limesurvey-db.pre  wiki-root.excludes

On ~/.config/mbackup/priv we have sensitive configuration data (like db passwords):

altroot.priv     drupal-db.priv     home.priv             wiki-extra-1.priv
blog-cache.priv  drupal-root.priv   limesurvey-db.priv    wiki-extra-2.priv
blog-db.priv     drupal-sites.priv  limesurvey-root.priv  wiki-root.priv
blog-extra.priv  forum-db.priv      _mbackup.priv
blog-root.priv   forum-extra.priv   site-root.priv
civicrm-db.priv  forum-root.priv    wiki-db.priv

mbackup-config.sh is the main configuration file:

cat ~/.config/mbackup/mbackup-config.sh

#!/bin/sh
FORUM_ROOT="path/to/forum/root"
BLOG_ROOT="path/to/blog/root"
OSE_ROOT="path/to/main/site"
# MBKP_BASE_TARGET="sftp://user@backup.domain.com/mbackup-modules"
MBKP_BASE_TARGET="file://$MBKP_LOCAL_DATA"

_mbackup.priv contains other configurations:

cat ~/.config/mbackup/priv/_mbackup.priv
# Please choose the default passwords you want to use
# for protecting your backups.
set +x
# Default Gpg passphrase for Duplicity
PASSPHRASE='the-passphrase'
# Default password for ssh / sftp
FTP_PASSWORD="the-ftp-password"
# Delete the line below in order to use mbackup:
# MBKP_FIRST_RUN=1

~/.local/share/mbackup/data/ contains a folder for each module, storing duplicity archives. Said folder is rsynce'd to a remote server.

~/.local/share/mbackup/cache/modules contains cache files used during backup (like database dumps)

Performing a Backup

We have some cron jobs that perform our backups. See the daily backup job below:

. $D_ROOT/init.sh
#
~/bin/mbkp.sh limesurvey-db
~/bin/mbkp.sh limesurvey-root
#
~/bin/mbkp.sh civicrm-db
#
~/bin/mbkp.sh drupal-db
~/bin/mbkp.sh drupal-root
~/bin/mbkp.sh drupal-sites
#
~/bin/mbkp.sh forum-db
~/bin/mbkp.sh forum-root
~/bin/mbkp.sh forum-extra
#
~/bin/mbkp.sh blog-db
~/bin/mbkp.sh blog-root
~/bin/mbkp.sh blog-extra
#
~/bin/mbkp.sh wiki-db
~/bin/mbkp.sh wiki-root
~/bin/mbkp.sh wiki-extra-1
~/bin/mbkp.sh wiki-extra-2
#
# Sync to backup account
~/bin/sync2bkpaccount.sh

See Also

• Website
• OSE Server#Backups
• Amazon Glacier
• Backblaze