CHG-2018-01-03 migrate fef to hetzner2

Status

2017-12-21

Catarina validated the ephemeral clone of the existing site running as staging on hetzner 2. She approved downtime for the migration on 2018-01-03 12:00 ET, followed by a 1 week content freeze.

Purpose

This change does the following for fef

1. changes the domain from 'http://opensourceecology.org/fef/' to 'https://fef.opensourceecology.org'
2. update core wp from v4.2.18 to v4.9.1
3. changes the docroot to a subversion repo of the wp codebase, making it easier to update in the future without sacrificing security (ie: permitting wordpress to update itself) for ease-of-use
4. updates the vhost config to block access to svn files, and more generally: ".*\.(svn|git|hg|bzr|cvs|ht)/.*"
5. update plugin 'akismet' from v3.1.5 to v4.0.1
6. update plugin 'cyclone-slider-2' from v2.10.0 to 3.2.0
7. install & activate plugin 'google-authenticator' to provide 2FA
8. install & activate plugin 'google-authenticator-encourage-user-activation' to require users to use 2FA
9. install & activate plugin 'force-strong-passwords' to prevent users from using a poor password, as determined by the built-in wp core password strength feature
10. install & activate plugin 'rename-wp-login' to obfuscate the wordpress login page (reduces server load & alert emails from HIDS blocking brute force attempts)
11. install & activate plugin 'vcaching' so wordpress & varnish communicate to clear the cache when needed
12. updates the vhost config to block all access to ".*wp-login.php"
13. enable https via nginx
14. enable cache via varnish

Points of Contact

Change being performed by: Michael Altfield

Service owners: Catarina Mota & Marcin Jakubowski

Apply to Production

####################
# run on hetzner1 #
####################

# STEP 0: CREATE BACKUPS
source /usr/home/osemain/backups/backup.settings
/usr/home/osemain/backups/backup.sh

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /usr/home/osemain/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner1/*'

# DECLARE VARIABLES
source /usr/home/osemain/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/osemain/tmp/backups_for_migration_to_hetzner2/fef_${stamp}"
backupFileName_db_hetzner1="mysqldump_fef.${stamp}.sql.bz2"
backupFileName_files_hetzner1="fef_files.${stamp}.tar.gz"
vhostDir_hetzner1='/usr/home/osemain/opensourceecology.org/fef'
dbName_hetzner1='ose_fef'
 dbUser_hetzner1="${mysqlUser_fef}"
 dbPass_hetzner1="${mysqlPass_fef}"

# STEP 1: BACKUP DB
pushd ${backupDir_hetzner1}/current/
mkdir -p ${backupDir_hetzner1}/{current,old}
mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/
time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases | bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}

# STEP 2: BACKUP FILES
time nice tar -czvf ${backupDir_hetzner1}/current/${backupFileName_files_hetzner1} ${vhostDir_hetzner1}

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

# DECLARE VARIABLES
source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/osemain/tmp/backups_for_migration_to_hetzner2/fef_${stamp}"
backupDir_hetzner2="/var/tmp/backups_for_migration_from_hetzner1/fef_${stamp}"
backupFileName_db_hetzner1="mysqldump_fef.${stamp}.sql.bz2"
backupFileName_files_hetzner1="fef_files.${stamp}.tar.gz"
dbName_hetzner1='ose_fef'
dbName_hetzner2='fef_db'
 dbUser_hetzner2="fef_user"
 dbPass_hetzner2="CHANGEME"
vhostDir_hetzner2="/var/www/html/fef.opensourceecology.org"
docrootDir_hetzner2="${vhostDir_hetzner2}/htdocs"

# STEP 1: COPY FROM HETZNER1

mkdir -p ${backupDir_hetzner2}/{current,old}
mv ${backupDir_hetzner2}/current/* ${backupDir_hetzner2}/old/
scp -P 222 osemain@dedi978.your-server.de:${backupDir_hetzner1}/current/* ${backupDir_hetzner2}/current/

# STEP 2: ADD DB

# create backup before we start changing the sql file
pushd ${backupDir_hetzner2}/current
cp ${backupFileName_db_hetzner1} ${backupFileName_db_hetzner1}.orig

# extract .sql.bz2 -> .sql
bzip2 -dc ${backupFileName_db_hetzner1} > db.sql

# verify the first 2 (non-comment) occurances of $dbName meet the naming convention of "<siteName>_db
vim db.sql

 time nice mysql -uroot -p${mysqlPass} -sNe "DROP DATABASE IF EXISTS ${dbName_hetzner2};" 
 time nice mysql -uroot -p${mysqlPass} -sNe "CREATE DATABASE ${dbName_hetzner2}; USE ${dbName_hetzner2};"
 time nice mysql -uroot -p${mysqlPass} < "db.sql"
 time nice mysql -uroot -p${mysqlPass} -sNe "GRANT ALL ON ${dbName_hetzner2}.* TO '${dbUser_hetzner2}'@'localhost' IDENTIFIED BY '${dbPass_hetzner2}'; FLUSH PRIVILEGES;"

# STEP 3: Add vhost files
mv ${vhostDir_hetzner2}/* ${backupDir_hetzner2}/old/
tar -xzvf ${backupFileName_files_hetzner1}
content_dir=`find ${backupFileName_db_hetzner2} -name wp-content -type d | sort | head -n1`
htaccess_file=`find ${backupFileName_db_hetzner2} -name '.htaccess' -type f | sort | head -n1`
wp_config_file=`find ${backupFileName_db_hetzner2} -name 'wp-config.php' -type f | sort | head -n1`

mkdir -p ${docrootDir_hetzner2}

pushd ${docrootDir_hetzner2}
currentWpVersion=`curl -i https://core.svn.wordpress.org/tags/ | grep '<li>' | tail -n1 | cut -d\" -f2`
svn co https://core.svn.wordpress.org/tags/${currentWpVersion}/ ${docrootDir_hetzner2}
popd

rsync -av --progress ${wp_config_file} ${vhostDir_hetzner2}/
rsync -av --progress ${htaccess_file} ${docrootDir_hetzner2}/
rsync -av --progress ${content_dir} ${docrootDir_hetzner2}/

# make sure this is sudomain, not subdir now
vim ${docrootDir_hetzner2}/.htaccess

# update WP_HOME/WP_SITEURL/DB_NAME/DB_USER/DB_PASSWORD/DB_HOST/
# add/replace salts https://api.wordpress.org/secret-key/1.1/salt/
vim ${vhostDir_hetzner2}/wp-config.php

chown -R apache:apache "${vhostDir_hetzner2}"
find "${vhostDir_hetzner2}" -type d -exec chmod 0750 {} \;
find "${vhostDir_hetzner2}" -type f -exec chmod 0640 {} \;
find "${docrootDir_hetzner2}/wp-content" -type f -exec chmod 0660 {} \;
find "${docrootDir_hetzner2}/wp-content" -type d -exec chmod 0770 {} \;
chown apache:apache-admins "${vhostDir_hetzner2}/wp-config.php"
chmod 0440 "${vhostDir_hetzner2}/wp-config.php"

# INSTALL PLUGINS

# install & configure 2FA plugins
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install google-authenticator --activate
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install google-authenticator-encourage-user-activation --activate
defaultOtpAccountDescription="`basename ${vhostDir_hetzner2}` wp"
pushd ${docrootDir_hetzner2}/wp-content/plugins/google-authenticator
sed -i "s^\$GA_description\s=\s__(\s[\"'].*[\"']^\$GA_description = __( '$defaultOtpAccountDescription'^" google-authenticator.php
popd

# install 'force-strong-passwords' plugin
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install force-strong-passwords --activate

# install rename-wp-login plugin
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install rename-wp-login --activate

# install "SSL Insecure Content Fixer" pugin
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install ssl-insecure-content-fixer --activate

# install "Varnish Caching" pugin
sudo -u wp -i wp --path=${docrootDir_hetzner2} plugin install vcaching --activate

# finally, log into the new wordpress site (use '/login' instead of '/wp-login.php'. After authenticating, wp will ask you to update, if necessary. Then update settings:
# 1. "Settings" -> "Permalinks" -> "Rename wp-login.php" -> "Login url" = 'ose-hidden-login'
# 2. "Settings" -> "General" -> "Google Authenticator - Encourage User Activation" = "Force the user"
# 3. "Settings" -> "SSL Insecure Content" and then [a] uncheck the "WooCOmmerce" checkbox and [b] change the HTTPS detection from the default "standard WordPress function" to "HTTP_X_FORWARDED_PROTO"
# 4. "Varnish Caching" and then [a] check the "Enable" checkbox, [b] enter "86400" for the "Homepage cache TTL", [c] enter "86400" for the "Cache TTL", [d] enter "127.0.0.1:6081" for "IPs", [e] check the "Dynamic host" checkbox