Wordpress

Wordpress Plugins

2FA

We use the following two plugins to achieve 2FA auth in Wordpress:

# "Google Authenticator" plugin by Henrik Schack
# "Google Authenticator – Encourage User Activation" plugin by Ian Dunn

The above plugins were selected because

1. They support the TOTP standard defined in RFC6 238
2. The code is small & lightweight.
3. The software is popular with a large community of users. As of 2017, there are 30,000+ active installs.
4. The software is well-reviewed. As of 2017, there are 4.5/5 stars with 108 reviews.
5. The code is self-contained (it does not make external calls to google's servers, for example). This is important, as most of these 2FA plugins require access to google to generate QR codes. If google changes something, it could break our site. This plugin doesn't have this risk as it includes a script to generate the qr code locally.
6. The second plugin allows us to enforce 2FA for all users

For more information, see 2FA

Force Strong Passwords

The wp core software includes a Javascript-powered password strength indicator (based on the zxcvbn Password Strength Estimator by Dropbox since v3.7), but it does not have a way to enforce passwords to meet the recommended strength.

We use the "Force Strong Passwords plugin by Jason Cosper plugin, which simply leverages the already-built-in zxcvbn Password Strength Estimator, and makes it actually _require_ strong passwords when users (with publish_posts, upload_files and/or edit_published_posts) attempt to create or update their password.

Rename wp-login.php

We use the wordpress plugin "Rename wp-login.php" by Ella Iseulde Van Dorpe. This is a simple plugin that allows us to change the URL used for logins. This is an important mitigation to brute force attacks.

We were detecting dozens of brute-force attacks per day (attackers attempting to login to our site by repeatedly trying common username & password combinations). OSSEC's Active Response (our Host-based Intrusion Detection System) would actively ban these ip addresses & send an alert email. To cut the attack off sooner (and reduce alert email volumes without us ignoring targeted brute-force attacks), this plugin allows us to simply respond with "403 forbidden" (or drop the requests entirely--if the web server supports it) to *all* requests for 'wp-login.php'.

Note that there are many "wordpress security suite" plugins that include this feature. This plugin was selected because it's smaller & more lightweight. The bigger plugins make more sense to implement only if you're on a shared server. In our case, we have a dedicated server with root access, and it therefore makes more sense to implement our security walls at the Apache or OS-level (they're more powerful, flexible, and cut off an attacker earlier). Therefore, a simpler wordpress plugin without the unnecessary features (and runtime overhead) is better.

Other

• No Self Pings - remove self pings
• Contact Form 7 - easy to maintain contact form
• Efficient Related Posts - shows related posts by tags
• Enhanced Recent Posts - shows recent posts, can exclude posts from categories
• Like - Facebook like button for posts
• MailChimp Widget - easy and free newsletter by Mailchimp
• Really Simple CAPTCHA - needed for Contact Form 7
• Semisecure Login Reimagined - for more secure login into your Wordpress
• Social Media Widget - Adds links to all of your social media and sharing site as a widget
• W3 Total Cache - for caching (performance)
• XCloner - Backup and Restore - schedule backups with cron jobs

OSE Website - Known Bugs

• Fields not filled in -

Proper File/Directory Ownership & Permissions

This section will describe how the file permissions should be set on an OSE wordpress site.

For the purposes of this documentation, let's assume:

1. vhost dir = /var/www/html/osemain
2. wp docroot = /var/www/html/osemain/htdocs

Then the ideal permissions are:

1. Files containing passwords (ie: wp-config.php) should be located outside the wp docroot with apache:apache-admins 0440
2. Files in the wp-content dir should be apache:apache 0660
3. Directories in the wp-content dir should be apache:apache 0770
4. All other files in the vhost dir should be apache:apache 0640
5. All other directories in the vhost dir should be apache:apache 0750

This is achievable with the following idempotent commands:

vhostDir="/var/www/html/osemain"
wpDocroot="${vhostDir}/htdocs"

chown -R apache:apache "${vhostDir}"
find "${vhostDir}" -type d -exec chmod 0750 {} \;
find "${vhostDir}" -type f -exec chmod 0640 {} \;
find "${wpDocroot}/wp-content" -type f -exec chmod 0660 {} \;
find "${wpDocroot}/wp-content" -type d -exec chmod 0770 {} \;
chown apache:apache-admins "${vhostDir}/wp-config.php"
chmod 0440 "${vhostDir}/wp-config.php"

Updating Wordpress

First of all, it is not uncommon for an attempt to update wordpress to result in an entirely broken site. If you do not have linux and bash literacy, do not attempt to update wordpress. Moreover, you should be well-versed in how to work with mysqldump, tar, rsync, chmod, chown, & sudo. If you are not confident in how all of these commands work, do not proceed. Hire someone with sysops experience to follow this guide; it should take them less than a couple hours to update and/or revert if the update fails.

Step 0: Trigger Backup Scripts for System-Wide backup

For good measure, trigger a backup of the entire system's database & files:

sudo time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

When finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding

sudo bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

Step 1: Set variables

Type these commands to set some variables, which will be used by the commands in the sections below. Replace 'osemain' with the corresponding directory for the wp site you're updating.

export vhostDir=/var/www/html/osemain

Step 2: Make Vhost-specific backups

The backups made in the previous step are huge. Because it's easier to work with vhost-specific backups, let's make a redundant copy available in /var/tmp/:

sudo su -

dbName=osemain_db
dbUser=osemain_user
 dbPass=CHANGEME
 rootDbPass=CHANGEME

stamp=`date +%Y%m%d_%T`
tmpDir=/var/tmp/dbChange.$stamp
mkdir $tmpDir
chown root:root $tmpDir
chmod 0700 $tmpDir
pushd $tmpDir
service httpd stop

# create backup of all DBs for good measure
 time nice mysqldump -uroot -p$rootDbPass --all-databases | gzip -c > preBackup.all_databases.$stamp.sql.gz

# dump wp DB contents
 time nice mysqldump -u$dbUser -p$dbPass --database $dbName > $dbName.$stamp.sql

# files backup
rsync -av --progress "${vhostDir}" "vhostDir.${stamp}.bak/"

Step 3: Permissions

TODO link to other section

Step 4: SVN Update

TODO update with svn

Step 5: Update Plugins and Themes

Run the following commands to have wp-cli update your plugins & themes.

# update wp plugins
sudo -u wp -i wp --path=${vhostDir}/htdocs plugin update --all

# update wp themes
sudo -u wp -i wp --path=${vhostDir}/htdocs theme update --all

Step 6: Database Upgrade

TODO: login to wp & update (or is it upgrade?)

Step 7: Validate

TODO describe a test for sanity of successful upgrade

Revert

TODO restore procedure

WP-CLI

Wordpress has been officially developing a tool for managing their software over CLI called wp-cli. This has been around since 2011, it's funded by big hosting companies that offer cheap wordpress shared hosting plans, and is actively developed (as of 2017-08-26, at the time of writing, the last commit was a few days ago, and releases come out every few months).

Unfortunately, wp-cli expects php to be more open than our hardened config allows, so we can't use it for updating the wp core software (instead we use svn), but it works great for most plugins & themes.

Here's how we install wp-cli on our server:

useradd wp
gpasswd -a wp apache
gpasswd -a wp apache-admins
su - wp
mkdir -p $HOME/.wp-cli
cd $HOME/.wp-cli
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
mkdir -p $HOME/bin
ln -s $HOME/.wp-cli/wp-cli.phar $HOME/bin/wp
chown wp:wp -R /$HOME/.wp-cli
find $HOME/.wp-cli -type d -exec chmod 0700 {} \;
find $HOME/.wp-cli -type f -exec chmod 0600 {} \;
chmod 0700 $HOME/.wp-cli/wp-cli.phar

Note that we should *not* be running wp as root. Doing so may allow a wp site that's been owned (which is very common) to escalate itself to having root privileges. This is why we use a special user 'wp', and all documentation on this wiki for `wp` command is preceded with `sudo -u wp -i wp ...`, so that someone following this guide doesn't accidentally run `wp` as root.

Here are some example wp-cli commands:

sudo -u wp -i wp --path=/var/www/html/osemain/htdocs theme list
sudo -u wp -i wp --path=/var/www/html/osemain/htdocs theme update --all

sudo -u wp -i wp --path=/var/www/html/osemain/htdocs plugin list
sudo -u wp -i wp --path=/var/www/html/osemain/htdocs plugin update --all

CLI guides

This section will provide commands to achieve certain actions for managing wordpress

replace strings everywhere in wp database backend

Use these commands to safely backup & do a string replacement for all occurrences from a given $fromString to $toString. This was used to replace links in the wordpress posts/pages to the ip address instead of using the domain name (which creates migration issues & https issues)

dbName=obi_db
dbUser=obi_user
 dbPass=CHANGEME
 rootDbPass=CHANGEME

fromString=138.201.84.223
toString=openbuildinginstitute.org

stamp=`date +%Y%m%d_%T`
tmpDir=/var/tmp/dbChange.$stamp
mkdir $tmpDir
chown root:root $tmpDir
chmod 0700 $tmpDir
pushd $tmpDir
service httpd stop

# create backup of everything for good measure
 time nice mysqldump -uroot -p$rootDbPass --all-databases | gzip -c > preBackup.all_databases.$stamp.sql.gz

# dump obi wordpress db contents
 time nice mysqldump -u$dbUser -p$dbPass --database $dbName > $dbName.$stamp.sql

# make backup
cp $dbName.$stamp.sql $dbName.$stamp.sql.orig

# sed
sed -i "s/$fromString/$toString/g" $dbName.$stamp.sql

# verify
grep "$fromString" $dbName.$stamp.sql | less
grep "$toString" obi_db.$stamp.sql | less

# delete db tables
 mysql -u$dbUser -p$dbPass $dbName -sNe 'show tables' | while read table; do mysql -u$dbUser -p$dbPass -sNe "DROP TABLE $dbName.$table;"; done

# verify
 mysql -u$dbUser -p$dbPass $dbName -sNe 'show tables' 

# import
 mysql -u$dbUser -p$dbPass < obi_db.$stamp.sql

# verify
 mysql -u$dbUser -p$dbPass $dbName -sNe 'show tables' 
service httpd start

create staging clone of production wordpress

Use these commands to safely create a clone of a production wordpress site for testing changes

sudo su -

####################
# define variables #
####################

prodVhostDir=/var/www/html/openbuildinginstitute.org
stagingVhostDir=/var/www/html/staging.openbuildinginstitute.org
prodDbName=obi_db
stagingDbName=obi_staging_db
stagingDbUser=obi_staging_user
 stagingDbPass=CHANGEME
 rootDbPass=CHANGEME

##########################
# step 0: create backups #
##########################

# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

##############
# copy files #
##############

cd
rm -rf "${stagingVhostDir}"
mkdir -p "${stagingVhostDir}/htdocs"
rsync -av --progress "${prodVhostDir}/"* "${stagingVhostDir}/"

find "${stagingVhostDir}" -type d -exec chmod 750 {} \;
find "${stagingVhostDir}" -type f -exec chmod 640 {} \;
chown -R apache:apache "${stagingVhostDir}"

# set permissions
chown -R apache:apache "${stagingVhostDir}"
find "${stagingVhostDir}" -type d -exec chmod 0750 {} \;
find "${stagingVhostDir}" -type f -exec chmod 0640 {} \;
find "${stagingVhostDir}/htdocs/wp-content" -type f -exec chmod 0660 {} \;
find "${stagingVhostDir}/htdocs/wp-content" -type d -exec chmod 0770 {} \;
chown apache:apache-admins "${stagingVhostDir}/wp-config.php"
chmod 0440 "${stagingVhostDir}/wp-config.php"

###########
# copy db #
###########

stamp=`date +%Y%m%d_%T`
tmpDir="/var/tmp/dbChange.${stamp}"
mkdir "${tmpDir}"
chown root:root "${tmpDir}"
chmod 0700 "${tmpDir}"
pushd "${tmpDir}"
 time nice mysqldump -uroot -p${rootDbPass} --all-databases | gzip -c > "preBackup.all_databases.${stamp}.sql.gz"
 time nice mysqldump -uroot -p${rootDbPass} --databases $prodDbName > "${prodDbName}.${stamp}.sql"
cp "${prodDbName}.${stamp}.sql" "${stagingDbName}.${stamp}.sql"

# replace the first 2 (non-comment) occurances of $prodDbName with $stagingDbName
vim "${stagingDbName}.${stamp}.sql"

 time nice mysql -uroot -p${rootDbPass} -sNe "DROP DATABASE IF EXISTS ${stagingDbName};" 
 time nice mysql -uroot -p${rootDbPass} -sNe "CREATE DATABASE ${stagingDbName}; USE ${stagingDbName};"
 time nice mysql -uroot -p${rootDbPass} < "${stagingDbName}.${stamp}.sql"
 time nice mysql -uroot -p${rootDbPass} -sNe "GRANT ALL ON ${stagingDbName}.* TO '${stagingDbUser}'@'localhost' IDENTIFIED BY '${stagingDbPass}'; FLUSH PRIVILEGES;"

popd

# change WP_HOME, WP_SITEURL, DB_NAME, DB_USER, & DB_PASSWORD
vim "${stagingVhostDir}/wp-config.php"

#############
# add vhost #
#############

vim /etc/httpd/conf.d/staging.openbuildinginstitute.org

# add vhosts dir to php.ini 'open_basedir'
vim /etc/php.ini

######################
# updates (optional) #
######################

# update wp core
#TODO

# update wp plugins
sudo -u wp -i wp --path="${stagingVhostDir}/htdocs" plugin update --all

# update wp themes
sudo -u wp -i wp --path="${stagingVhostDir}/htdocs" theme update --all

#########################
# set permissions again #
#########################

# set permissions
chown -R apache:apache "${stagingVhostDir}"
find "${stagingVhostDir}" -type d -exec chmod 0750 {} \;
find "${stagingVhostDir}" -type f -exec chmod 0640 {} \;
find "${stagingVhostDir}/htdocs/wp-content" -type f -exec chmod 0660 {} \;
find "${stagingVhostDir}/htdocs/wp-content" -type d -exec chmod 0770 {} \;
chown apache:apache-admins "${stagingVhostDir}/wp-config.php"
chmod 0440 "${stagingVhostDir}/wp-config.php"

#################
# reload apache #
#################

# test apache configs
httpd -t

# apply apache configs
service httpd reload

########
# test #
########

# test

Changes

As of 2017-09, we have no ticket tracking or change control process. For now, everything is on the wiki as there's higher priorities. Hence, here's some articles used to track production changes:

1. CHG-2017-09-25 - first major obi production change by Michael Altfield

See Also

1. OSE Server
2. 2FA