CHG-2025-XX-XX deprecate fef

From Open Source Ecology
Jump to: navigation, search

Status

2024-12-30 17:59 UTC

Initial Ticket draft created on wiki (WIP)

Change Info

Scheduled Time

This change will take place on 2025-??-?? ??:00 UTC

  • = 2025-??-?? ??:00 Kansas City, US
  • = 2025-??-?? ??:00 Guayaquil, EC

https://www.timeanddate.com/worldclock/converter.html?iso=20240727T160000&p1=405&p2=1440&p3=93

Purpose

This change will do the following for the fef site:

  1. entirely migrate the OSE fef site from hetzner2 (EOL CentOS7) to hetzner3 (Debian 12)
  2. changes site from apache v2.4.6 to v2.4.62
  3. changes site from varnish v4.0.5 to v7.1.1
  4. changes site from nginx v1.20.1 to v1.22.1
  5. deprecates the site without loosing the content. we're replacing our fef site (now with an extremely out-of-date & broken wordpress theme) with purely static content made from a wget scrape
  6. keeps a backup of the dynamic codebase & database on the server, but 1 directory _above_ the docroot.

Points of Contact

Change being performed by: Michael Altfield

Service owners: Catarina Mota & Marcin Jakubowski

Time Length

This whole process is expected to take 6-24 hours.

Some systems could be impacted for days, if issues are encountered.

Systems Impacted

This change impacts both hetzner2 and hetzner3. It's possible that we'll need to restart one or both of these servers during this migration, which could affect the availability of all OSE & OBI websites.

Staging Test

n/a

Change Steps

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
source /root/backups/backup.settings
/root/backups/backup.sh

# when finished, verify that the whole system backup was successful before proceeding
sudo rclone lsl b2:ose-server-backups | grep -i $(date +%Y%m%d)

vhost_name='fef.opensourceecology.org'
dbName='fef_db'
 dbUser="CHANGEME"
 dbPass="CHANGEME"

source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner2="/var/tmp/backups_for_migration_to_hetzner3/${vhost_name}_${stamp}"
backupDir_hetzner3="/var/tmp/backups_for_migration_from_hetzner2/${vhost_name}_${stamp}"
backupFileName_db_hetzner2="mysqldump_${vhost_name}.${stamp}.sql.bz2"
backupFileName_files_hetzner2="${vhost_name}_files.${stamp}.tar.bz2"
vhostDir="/var/www/html/${vhost_name}"

# STEP 1: Bring site down on hetzner2
mkdir -p ${backupDir_hetzner2}/{current,old}
pushd ${backupDir_hetzner2}/current

# backup nginx config
cp /etc/nginx/conf.d/${vhost_name}.conf nginx_${vhost_name}.${stamp}.conf

# restrict website to local requests only
grep 'deny' /etc/nginx/conf.d/${vhost_name}.conf || sed --in-place=.`date "+%Y%m%d_%H%M%S"` 's%^\(\s*\)server_name\(.*\)%\1server_name\2\n\tallow 127.0.0.1;\n\tallow 2a01:4f8:172:209e::2;\n\tallow 138.201.84.243;\n\tdeny all;\n%' /etc/nginx/conf.d/${vhost_name}.conf

nginx -t && service nginx reload

# STEP 2: BACKUP DB
mkdir -p ${backupDir_hetzner2}/{current,old}
pushd ${backupDir_hetzner2}/current/
mv ${backupDir_hetzner2}/current/* ${backupDir_hetzner2}/old/

time nice mysqldump -u"${dbUser}" -p"${dbPass}" ${dbName} | bzip2 -c > ${backupDir_hetzner2}/current/${backupFileName_db_hetzner2}

# STEP 3: BACKUP FILES
time nice tar -cjvf ${backupDir_hetzner2}/current/${backupFileName_files_hetzner2} ${vhostDir}

# STEP 4: GENERATE STATIC SITE
mkdir wget
pushd wget
time nice wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains "${vhost_name}" "${vhost_name}"

# STEP 5: COPY TO HETZNER3
ssh -p 32415 maltfield@hetzner3 sudo mkdir -p ${backupDir_hetzner3}/{current,old}
ssh -p 32415 maltfield@hetzner3 sudo mv ${backupDir_hetzner3}/current/* ${backupDir_hetzner3}/old/
rsync -av --progress --rsync-path="sudo rsync" -e "ssh -p 32415" ${backupDir_hetzner2}/current/* maltfield@hetzner3:${backupDir_hetzner3}/current/

####################
# run on hetzner3 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, verify that the whole system backup was successful before proceeding
rclone lsl b2:ose-server-backups | grep -i $(date +%Y%m%d)

# DECLARE VARIABLES
vhost_name='fef.opensourceecology.org'
dbName='fef_db'
 dbUser="CHANGEME"
 dbPass="CHANGEME"

source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner2="/var/tmp/backups_for_migration_to_hetzner3/${vhost_name}_${stamp}"
backupDir_hetzner3="/var/tmp/backups_for_migration_from_hetzner2/${vhost_name}_${stamp}"
backupFileName_db_hetzner2="mysqldump_${vhost_name}.${stamp}.sql.bz2"
backupFileName_files_hetzner2="${vhost_name}_files.${stamp}.tar.bz2"
vhostDir="/var/www/html/${vhost_name}"
docrootDir="${vhostDir}/htdocs"

# STEP 1: ADD DB

# first, get rid of the old vhost dir
mv "${vhostDir}" "${backupDir_hetzner3}/${vhost_name}.$(date "+%Y%m%d_%H%M%S")"
mkdir -p "${vhostDir}"

# put a copy of the db file in the vhost dir (outside docroot)
pushd ${backupDir_hetzner3}/current
cp ${backupFileName_db_hetzner2} ${vhostDir}/
chown root:root ${vhostDir}/*.sql.bz2
chmod 0400 ${vhostDir}/*.sql.bz2

 time nice mysql -uroot -p${mysqlPass} -sNe "DROP DATABASE IF EXISTS ${dbName};" 

# STEP 2: Add vhost files backup to vhost dir (outside docroot)
cp ${backupFileName_files_hetzner2} ${vhostDir}/
chown root:root ${vhostDir}/*.tar.bz2
chmod 0400 ${vhostDir}/*.tar.bz2

# STEP 3: Add static html files to docroot
time nice rsync -av --progress wget/${vhost_name}/ ${docrootDir}/

# STEP 4: Create README about site deprecation and how to restore it
cat << EOF > ${vhostDir}/README.txt
In 2024, this website was no longer updated, the code required to render it was broken, and the decision was made to deprecate support for the site. The content is still accessible in as static-content; new content is not possible.

For more information, please see:

 * https://wiki.opensourceecology.org/wiki/CHG-2025-XX-XX_deprecate_fef

In 2025, during the backup stage of the change to deprecate this website, a backup of the wordpress webroot directory and database dump were created for upload to long-term backup storage on hetzner3. Because this backup size was manageable small, I put a "hot" copy of this dump in this website's apache vhost dir (but outside the htdocs docroot, of course).

-- Michael Altfield <michael@michaelaltfield.net> $(date -u --rfc-3339=seconds)

EOF

# STEP 5: Add all uploaded files (fix missing from wget)
tar -xjvf ${backupFileName_files_hetzner2}
rsync -av --progress var/www/html/${vhost_name}/htdocs/wp-content/uploads/ ${vhostDir}/htdocs/wp-content/uploads/

# STEP 6: Fix assets

# wget html-encoded question marks in versioned assets, which causes "MIME type mismatch" errors; change it back
find ${vhostDir}/htdocs/ -type f -iname '*\.html' -exec sed --in-place=.`date "+%Y%m%d_%H%M%S"` 's/\.js%3F/.js?/' '{}' \;
find ${vhostDir}/htdocs/ -type f -iname '*\.html' -exec sed --in-place=.`date "+%Y%m%d_%H%M%S"` 's/\.css%3F/.css?/' '{}' \;

# also rename the assets as back to their original filenames
js_files=$(find ${vhostDir}/htdocs/ -type f -iname '*\.js\?*')
for f in $js_files; do mv $f $(echo $f | sed 's/\.js\?.*/.js/'); done
css_files=$(find ${vhostDir}/htdocs/ -type f -iname '*\.css\?*')
for f in $css_files; do mv $f $(echo $f | sed 's/\.css\?.*/.css/'); done

# SET PERMISSIONS

# first pass, whole site
chown -R not-apache:www-data "/var/www/html"
find "/var/www/html" -type d -exec chmod 0050 {} \;
find "/var/www/html" -type f -exec chmod 0040 {} \;

#############
# WORDPRESS #
#############

wordpress_sites="$(find /var/www/html -type d -wholename *htdocs/wp-content)"

for wordpress_site in $wordpress_sites; do

	wp_docroot="$(dirname "${wordpress_site}")"
	vhost_dir="$(dirname "${wp_docroot}")"

	chown -R not-apache:www-data "${vhost_dir}"
	find "${vhost_dir}" -type d -exec chmod 0050 {} \;
	find "${vhost_dir}" -type f -exec chmod 0040 {} \;

	chown not-apache:apache-admins "${vhost_dir}/wp-config.php"
	chmod 0040 "${vhost_dir}/wp-config.php"

	[ -d "${wp_docroot}/wp-content/uploads" ] || mkdir "${wp_docroot}/wp-content/uploads"
	chown -R not-apache:www-data "${wp_docroot}/wp-content/uploads"
	find "${wp_docroot}/wp-content/uploads" -type f -exec chmod 0660 {} \;
	find "${wp_docroot}/wp-content/uploads" -type d -exec chmod 0770 {} \;

	[ -d "${wp_docroot}/wp-content/tmp" ] || mkdir "${wp_docroot}/wp-content/tmp"
	chown -R not-apache:www-data "${wp_docroot}/wp-content/tmp"
	find "${wp_docroot}/wp-content/tmp" -type f -exec chmod 0660 {} \;
	find "${wp_docroot}/wp-content/tmp" -type d -exec chmod 0770 {} \;

done

###########
# phpList #
###########

phplist_sites="$(find /var/www/html -maxdepth 1 -type d -iname *phplist*)"

for vhost_dir in $phplist_sites; do
 
	for dir in ${vhost_dir}; do chown -R not-apache:www-data "${dir}"; done
	for dir in ${vhost_dir}; do find "${dir}" -type d -exec chmod 0050 {} \;; done
	for dir in ${vhost_dir}; do find "${dir}" -type f -exec chmod 0040 {} \;; done
 
	for dir in ${vhost_dir}; do [ -d "${dir}/public_html/uploadimages" ] || mkdir "${dir}/public_html/uploadimages"; done
	for dir in ${vhost_dir}; do chown -R not-apache:www-data "${dir}/public_html/uploadimages"; done
	for dir in ${vhost_dir}; do find "${dir}/public_html/uploadimages" -type f -exec chmod 0660 {} \;; done
	for dir in ${vhost_dir}; do find "${dir}/public_html/uploadimages" -type d -exec chmod 0770 {} \;; done

done

#############
# MediaWiki #
#############

vhost_dir="/var/www/html/wiki.opensourceecology.org"
mw_docroot="${vhost_dir}/htdocs"

chown -R not-apache:www-data "${vhost_dir}"
find "${vhost_dir}" -type d -exec chmod 0050 {} \;
find "${vhost_dir}" -type f -exec chmod 0040 {} \;

chown not-apache:apache-admins "${vhost_dir}/LocalSettings.php"
chmod 0040 "${vhost_dir}/LocalSettings.php"

[ -d "${mw_docroot}/images" ] || mkdir "${mw_docroot}/images"
chown -R www-data:www-data "${mw_docroot}/images"
find "${mw_docroot}/images" -type f -exec chmod 0660 {} \;
find "${mw_docroot}/images" -type d -exec chmod 0770 {} \;

[ -d "${vhost_dir}/cache" ] || mkdir "${vhost_dir}/cache"
chown -R www-data:www-data "${vhost_dir}/cache"
find "${vhost_dir}/cache" -type f -exec chmod 0660 {} \;
find "${vhost_dir}/cache" -type d -exec chmod 0770 {} \;

Validation Steps

  1. TODO: copy from previous CHGs

Revert Steps

TODO

See Also

  1. CHG-2018-02-05_migrate_osemain_to_hetzner2 Last wordpress migration from hetzner1 to hetzner2
  2. CHG-2024-07-26 yum update Last (possible) update to hetzner2
  3. List of other CHG "tickets"
  4. CHG-2018-02-04_deprecate_vanilla_forums
Retrieved from "https://wiki.opensourceecology.org/index.php?title=CHG-2025-XX-XX_deprecate_fef&oldid=303199"