CHG-2025-XX-XX deprecate fef

Status

2024-12-30 17:59 UTC

Initial Ticket draft created on wiki (WIP)

Change Info

Scheduled Time

This change will take place on 2025-??-?? ??:00 UTC

• = 2025-??-?? ??:00 Kansas City, US
• = 2025-??-?? ??:00 Guayaquil, EC

https://www.timeanddate.com/worldclock/converter.html?iso=20240727T160000&p1=405&p2=1440&p3=93

Purpose

This change will do the following for the fef site:

1. entirely migrate the OSE fef site from hetzner2 (EOL CentOS7) to hetzner3 (Debian 12)
2. changes site from apache v2.4.6 to v2.4.62
3. changes site from varnish v4.0.5 to v7.1.1
4. changes site from nginx v1.20.1 to v1.22.1
5. deprecates the site without loosing the content. we're replacing our fef site (now with an extremely out-of-date & broken wordpress theme) with purely static content made from a wget scrape
6. keeps a backup of the dynamic codebase & database on the server, but 1 directory _above_ the docroot.

Points of Contact

Change being performed by: Michael Altfield

Service owners: Catarina Mota & Marcin Jakubowski

Time Length

This whole process is expected to take 6-24 hours.

Some systems could be impacted for days, if issues are encountered.

Systems Impacted

This change impacts both hetzner2 and hetzner3. It's possible that we'll need to restart one or both of these servers during this migration, which could affect the availability of all OSE & OBI websites.

Staging Test

n/a

Change Steps

# TODO: actually verify these commands

#############################
# run on hetzner1 (osemain) #
#############################

# STEP 0: CREATE BACKUPS
source /usr/home/osemain/backups/backup.settings
/usr/home/osemain/backups/backup.sh

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /usr/home/osemain/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner1/*'

###############################
# run on hetzner1 (oseforum) #
###############################

# DECLARE VARIABLES
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
vhostDir_hetzner1='/usr/www/users/osefourm'
dbName_hetzner1='ose_oseforum'
 dbUser_hetzner1="oseforum"
 dbPass_hetzner1="CHANGEME"

# STEP 1: BACKUP DB
mkdir -p ${backupDir_hetzner1}/{current,old}
pushd ${backupDir_hetzner1}/current/
mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/
time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases | bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}

# STEP 2: BACKUP FILES
time nice tar -czvf ${backupDir_hetzner1}/current/${backupFileName_files_hetzner1} ${vhostDir_hetzner1}

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

# DECLARE VARIABLES
source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupDir_hetzner2="/var/tmp/backups_for_migration_from_hetzner1/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
dbName_hetzner1='oseforum'
dbName_hetzner2='oseforum_db'
 dbUser_hetzner2="oseforum_user"
 dbPass_hetzner2="CHANGEME"
vhostDir_hetzner2="/var/www/html/forum.opensourceecology.org"
vanilla_backupdir_hetzner2="${vhostDir_hetzner2}/vanilla_docroot_backup.${stamp}"
docrootDir_hetzner2="${vhostDir_hetzner2}/htdocs"

# STEP 1: COPY FROM HETZNER1

mkdir -p ${backupDir_hetzner2}/{current,old}
mv ${backupDir_hetzner2}/current/* ${backupDir_hetzner2}/old/
scp -P 222 oseforum@dedi978.your-server.de:${backupDir_hetzner1}/current/* ${backupDir_hetzner2}/current/

# STEP 2: ADD DB

# create backup before we start changing the sql file
pushd ${backupDir_hetzner2}/current
cp ${backupFileName_db_hetzner1} ${backupFileName_db_hetzner1}.orig

# extract .sql.bz2 -> .sql
bzip2 -dc ${backupFileName_db_hetzner1} > db.sql

# verify the first 2 (non-comment) occurances of $dbName meet the naming convention of "<siteName>_db
vim db.sql

 time nice mysql -uroot -p${mysqlPass} -sNe "DROP DATABASE IF EXISTS ${dbName_hetzner2};" 
 time nice mysql -uroot -p${mysqlPass} -sNe "CREATE DATABASE ${dbName_hetzner2}; USE ${dbName_hetzner2};"
 time nice mysql -uroot -p${mysqlPass} < "db.sql"
 time nice mysql -uroot -p${mysqlPass} -sNe "GRANT ALL ON ${dbName_hetzner2}.* TO '${dbUser_hetzner2}'@'localhost' IDENTIFIED BY '${dbPass_hetzner2}'; FLUSH PRIVILEGES;"

# STEP 3: Add backup of dynamic-content files

mv ${vhostDir_hetzner2}/* ${backupDir_hetzner2}/old/
time nice tar -xzvf ${backupFileName_files_hetzner1}

mkdir -p ${docrootDir_hetzner2}
time nice rsync -av --progress usr/www/users/oseforum/.* "${vanilla_backupdir_hetzner2}/"

# set ['Database'] Name/User/Password
vim "${vanilla_backupdir_hetzner2}/conf/config.php"

# set permissions
chown -R apache:apache "${vanilla_backupdir_hetzner2}"
find "${vanilla_backupdir_hetzner2}" -type d -exec chmod 0750 {} \;
find "${vanilla_backupdir_hetzner2}" -type f -exec chmod 0640 {} \;
chown apache:apache-admins "${vanilla_backupdir_hetzner2}/conf/config.php"
chmod 0440 "${vanilla_backupdir_hetzner2}/conf/config.php"

# STEP 4: Add static content files to docroot

# create static html content by scraping existing site
mkdir -p ${backupDir_hetzner2}/{wget}
pushd ${backupDir_hetzner2}/{wget}
time nice wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains "forum.opensourceecology.org" "http://forum.opensourceecology.org"

# copy static content to docroot
mkdir -p "${docrootDir_hetzner2}"
time nice rsync -av --progress wget/* "${docrootDir_hetzner2}/"
time nice rsync -av --progress wget/.** "${docrootDir_hetzner2}/"

Validation Steps

1. TODO: copy from previous CHGs

Revert Steps

TODO

See Also

1. CHG-2018-02-05_migrate_osemain_to_hetzner2 Last wordpress migration from hetzner1 to hetzner2
2. CHG-2024-07-26 yum update Last (possible) update to hetzner2
3. List of other CHG "tickets"
4. CHG-2018-02-04_deprecate_vanilla_forums