CHG-2018-02-04 deprecate vanilla forums

From Open Source Ecology
Jump to: navigation, search

Purpose

This change will do the following for the forums site:

  1. entirely migrate the 'forums' wordpress site from hetzner1 (shared hosting) to hetzner2 (dedicated hosting)
  2. changes the domain from 'http://forum.opensourceecology.org/' to 'https://forum.opensourceecology.org'
  3. deprecates the site without loosing the content. we're replacing our extremely out-of-date vanilla codebase with purely static content made from a wget scrape
  4. keeps a backup of the dynamic codebase & database on the server, but 1 directory _above_ the docroot.

Points of Contact

Change being performed by: Michael Altfield

Service owners: Marcin Jakubowski

Backups

On 2018-07-11, during the backup stage of the change to deprecate hetzner1, a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on Amazon Glacier. Because this backup size was manageably small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I (User:Maltfield) put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/ 

[root@hetzner2 forum.opensourceecology.org]# date
Wed Jul 11 19:16:55 UTC 2018
[root@hetzner2 forum.opensourceecology.org]# pwd
/var/www/html/forum.opensourceecology.org
[root@hetzner2 forum.opensourceecology.org]# du -sh *
955M    final_backup_before_hetzner1_deprecation_oseforum_20180706-230007
2.7G    htdocs
4.0K    readme.txt
173M    vanilla_docroot_backup.20180113
[root@hetzner2 forum.opensourceecology.org]# du -sh final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/*
853M    final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_home.tar.bz2
46M     final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_mysqldump-oseforum.20180706-230007.sql.bz2
57M     final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_webroot.tar.bz2
[root@hetzner2 forum.opensourceecology.org]#

Apply to Production

#############################
# run on hetzner1 (osemain) #
#############################

# STEP 0: CREATE BACKUPS
source /usr/home/osemain/backups/backup.settings
/usr/home/osemain/backups/backup.sh

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /usr/home/osemain/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner1/*'

###############################
# run on hetzner1 (oseforum) #
###############################

# DECLARE VARIABLES
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
vhostDir_hetzner1='/usr/www/users/osefourm'
dbName_hetzner1='ose_oseforum'
 dbUser_hetzner1="oseforum"
 dbPass_hetzner1="CHANGEME"

# STEP 1: BACKUP DB
mkdir -p ${backupDir_hetzner1}/{current,old}
pushd ${backupDir_hetzner1}/current/
mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/
time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases | bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}

# STEP 2: BACKUP FILES
time nice tar -czvf ${backupDir_hetzner1}/current/${backupFileName_files_hetzner1} ${vhostDir_hetzner1}

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

# DECLARE VARIABLES
source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupDir_hetzner2="/var/tmp/backups_for_migration_from_hetzner1/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
dbName_hetzner1='oseforum'
dbName_hetzner2='oseforum_db'
 dbUser_hetzner2="oseforum_user"
 dbPass_hetzner2="CHANGEME"
vhostDir_hetzner2="/var/www/html/forum.opensourceecology.org"
vanilla_backupdir_hetzner2="${vhostDir_hetzner2}/vanilla_docroot_backup.${stamp}"
docrootDir_hetzner2="${vhostDir_hetzner2}/htdocs"

# STEP 1: COPY FROM HETZNER1

mkdir -p ${backupDir_hetzner2}/{current,old}
mv ${backupDir_hetzner2}/current/* ${backupDir_hetzner2}/old/
scp -P 222 oseforum@dedi978.your-server.de:${backupDir_hetzner1}/current/* ${backupDir_hetzner2}/current/

# STEP 2: ADD DB

# create backup before we start changing the sql file
pushd ${backupDir_hetzner2}/current
cp ${backupFileName_db_hetzner1} ${backupFileName_db_hetzner1}.orig

# extract .sql.bz2 -> .sql
bzip2 -dc ${backupFileName_db_hetzner1} > db.sql

# verify the first 2 (non-comment) occurances of $dbName meet the naming convention of "<siteName>_db
vim db.sql

 time nice mysql -uroot -p${mysqlPass} -sNe "DROP DATABASE IF EXISTS ${dbName_hetzner2};" 
 time nice mysql -uroot -p${mysqlPass} -sNe "CREATE DATABASE ${dbName_hetzner2}; USE ${dbName_hetzner2};"
 time nice mysql -uroot -p${mysqlPass} < "db.sql"
 time nice mysql -uroot -p${mysqlPass} -sNe "GRANT ALL ON ${dbName_hetzner2}.* TO '${dbUser_hetzner2}'@'localhost' IDENTIFIED BY '${dbPass_hetzner2}'; FLUSH PRIVILEGES;"

# STEP 3: Add backup of dynamic-content files

mv ${vhostDir_hetzner2}/* ${backupDir_hetzner2}/old/
time nice tar -xzvf ${backupFileName_files_hetzner1}

mkdir -p ${docrootDir_hetzner2}
time nice rsync -av --progress usr/www/users/oseforum/.* "${vanilla_backupdir_hetzner2}/"

# set ['Database'] Name/User/Password
vim "${vanilla_backupdir_hetzner2}/conf/config.php"

# set permissions
chown -R apache:apache "${vanilla_backupdir_hetzner2}"
find "${vanilla_backupdir_hetzner2}" -type d -exec chmod 0750 {} \;
find "${vanilla_backupdir_hetzner2}" -type f -exec chmod 0640 {} \;
chown apache:apache-admins "${vanilla_backupdir_hetzner2}/conf/config.php"
chmod 0440 "${vanilla_backupdir_hetzner2}/conf/config.php"

# STEP 4: Add static content files to docroot

# create static html content by scraping existing site
mkdir -p ${backupDir_hetzner2}/{wget}
pushd ${backupDir_hetzner2}/{wget}
time nice wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains "forum.opensourceecology.org" "http://forum.opensourceecology.org"

# copy static content to docroot
mkdir -p "${docrootDir_hetzner2}"
time nice rsync -av --progress wget/* "${docrootDir_hetzner2}/"
time nice rsync -av --progress wget/.** "${docrootDir_hetzner2}/"
Retrieved from "https://wiki.opensourceecology.org/index.php?title=CHG-2018-02-04_deprecate_vanilla_forums&oldid=218670"