CHG-2018-02-04 deprecate vanilla forums - Revision history

Maltfield: Maltfield moved page CHG-2018-02-04 to CHG-2018-02-04 deprecate vanilla forums

2020-04-27T12:22:01Z

Maltfield moved page CHG-2018-02-04 to CHG-2018-02-04 deprecate vanilla forums

← Older revision	Revision as of 12:22, 27 April 2020
(No difference)

Maltfield: /* Backups */

2018-07-11T19:23:04Z

Backups

← Older revision		Revision as of 19:23, 11 July 2018
Line 26:		Line 26:
	4.0K readme.txt		4.0K readme.txt
	173M vanilla_docroot_backup.20180113		173M vanilla_docroot_backup.20180113
			[root@hetzner2 forum.opensourceecology.org]# du -sh final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/*
			853M final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_home.tar.bz2
			46M final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_mysqldump-oseforum.20180706-230007.sql.bz2
			57M final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007_webroot.tar.bz2
	[root@hetzner2 forum.opensourceecology.org]#		[root@hetzner2 forum.opensourceecology.org]#
	</pre>		</pre>

Maltfield: /* Backups */

2018-07-11T19:21:33Z

Backups

← Older revision		Revision as of 19:21, 11 July 2018
Line 14:		Line 14:
	=Backups=		=Backups=

	On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on [[Amazon Glacier]]. Because this backup size was manageably small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/		On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on [[Amazon Glacier]]. Because this backup size was manageably small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I ([[User:Maltfield]]) put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/

	<pre>		<pre>
			[root@hetzner2 forum.opensourceecology.org]# date
			Wed Jul 11 19:16:55 UTC 2018
			[root@hetzner2 forum.opensourceecology.org]# pwd
			/var/www/html/forum.opensourceecology.org
			[root@hetzner2 forum.opensourceecology.org]# du -sh *
			955M final_backup_before_hetzner1_deprecation_oseforum_20180706-230007
			2.7G htdocs
			4.0K readme.txt
			173M vanilla_docroot_backup.20180113
			[root@hetzner2 forum.opensourceecology.org]#
	</pre>		</pre>

Maltfield: /* Backups */

2018-07-11T19:20:47Z

Backups

← Older revision		Revision as of 19:20, 11 July 2018
Line 14:		Line 14:
	=Backups=		=Backups=

	On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on [[Amazon Glacier]]. Because this backup size was ~~manageable~~ small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/		On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on [[Amazon Glacier]]. Because this backup size was manageably small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/

	<pre>		<pre>

Maltfield: /* Backups */

2018-07-11T19:20:19Z

Backups

← Older revision		Revision as of 19:20, 11 July 2018
Line 14:		Line 14:
	=Backups=		=Backups=

	On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on ~~hetzner1~~. Because this backup size was manageable small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/		On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on [[Amazon Glacier]]. Because this backup size was manageable small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/

	<pre>		<pre>

Maltfield: /* Backups */

2018-07-11T19:19:30Z

Backups

← Older revision		Revision as of 19:19, 11 July 2018
Line 14:		Line 14:
	=Backups=		=Backups=

	On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on hetzner1. Because this backup size was manageable small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/		On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation \| change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on hetzner1. Because this backup size was manageable small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/

	<pre>		<pre>

Maltfield at 19:12, 11 July 2018

2018-07-11T19:12:02Z

← Older revision		Revision as of 19:12, 11 July 2018
Line 11:		Line 11:

	Service owners: [[User:Marcin\|Marcin Jakubowski]]		Service owners: [[User:Marcin\|Marcin Jakubowski]]

			=Backups=

			On 2018-07-11, during the backup stage of the [[CHG-2018-07-06_hetzner1_deprecation change to deprecate hetzner1]], a backup of the vanilla forums home directory, webroot directory, and database dump were created for upload to long-term backup storage on hetzner1. Because this backup size was manageable small (1G, which is actually smaller than the 2.7G of static content currently live in the forum's docroot), I put a "hot" copy of this dump in the forum's apache dir (but outside the htdocs root, of course) located at hetzner2:/var/www/html/forum.opensourceecology.org/final_backup_before_hetzner1_deprecation_oseforum_20180706-230007/

			<pre>
			</pre>

	=Apply to Production=		=Apply to Production=

Maltfield: /* Apply to Production */

2018-02-09T02:19:18Z

Apply to Production

@@ Line 47: / Line 47: @@
 # STEP 2: BACKUP FILES
 time nice tar -czvf ${backupDir_hetzner1}/current/${backupFileName_files_hetzner1} ${vhostDir_hetzner1}
 ####################

Maltfield at 18:10, 4 February 2018

2018-02-04T18:10:29Z

← Older revision		Revision as of 18:10, 4 February 2018
Line 140:		Line 140:
	mkdir -p "${docrootDir_hetzner2}"		mkdir -p "${docrootDir_hetzner2}"
	time nice rsync -av --progress wget/* "${docrootDir_hetzner2}/"		time nice rsync -av --progress wget/* "${docrootDir_hetzner2}/"
			time nice rsync -av --progress wget/.** "${docrootDir_hetzner2}/"
	</pre>		</pre>

	[[Category: CHGs]]		[[Category: CHGs]]

Maltfield: Created page with "=Purpose= This change will do the following for the forums site: # entirely migrate the 'forums' wordpress site from hetzner1 (shared hosting) to hetzner2 (dedicated hosting)..."

2018-02-04T18:02:56Z

Created page with "=Purpose= This change will do the following for the forums site: # entirely migrate the 'forums' wordpress site from hetzner1 (shared hosting) to hetzner2 (dedicated hosting)..."

New page

=Purpose=
This change will do the following for the forums site:
# entirely migrate the 'forums' wordpress site from hetzner1 (shared hosting) to hetzner2 (dedicated hosting)
# changes the domain from 'http://forum.opensourceecology.org/' to 'https://forum.opensourceecology.org'
# deprecates the site without loosing the content. we're replacing our extremely out-of-date vanilla codebase with purely static content made from a wget scrape
# keeps a backup of the dynamic codebase & database on the server, but 1 directory _above_ the docroot.

=Points of Contact=

Change being performed by: [[User:Maltfield|Michael Altfield]]

Service owners: [[User:Marcin|Marcin Jakubowski]]

=Apply to Production=
<pre>
#############################
# run on hetzner1 (osemain) #
#############################

# STEP 0: CREATE BACKUPS
source /usr/home/osemain/backups/backup.settings
/usr/home/osemain/backups/backup.sh

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /usr/home/osemain/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner1/*'

###############################
# run on hetzner1 (oseforum) #
###############################

# DECLARE VARIABLES
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
vhostDir_hetzner1='/usr/www/users/osefourm'
dbName_hetzner1='ose_oseforum'
dbUser_hetzner1="oseforum"
dbPass_hetzner1="CHANGEME"

# STEP 1: BACKUP DB
mkdir -p ${backupDir_hetzner1}/{current,old}
pushd ${backupDir_hetzner1}/current/
mv ${backupDir_hetzner1}/current/* ${backupDir_hetzner1}/old/
time nice mysqldump -u"${dbUser_hetzner1}" -p"${dbPass_hetzner1}" --all-databases | bzip2 -c > ${backupDir_hetzner1}/current/${backupFileName_db_hetzner1}

# STEP 2: BACKUP FILES
time nice tar -czvf ${backupDir_hetzner1}/current/${backupFileName_files_hetzner1} ${vhostDir_hetzner1}

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

####################
# run on hetzner2 #
####################

sudo su -

# STEP 0: CREATE BACKUPS
# for good measure, trigger a backup of the entire system's database & files:
time /bin/nice /root/backups/backup.sh &>> /var/log/backups/backup.log

# when finished, SSH into the dreamhost server to verify that the whole system backup was successful before proceeding
bash -c 'source /root/backups/backup.settings; ssh $RSYNC_USER@$RSYNC_HOST du -sh backups/hetzner2/*'

# DECLARE VARIABLES
source /root/backups/backup.settings
stamp=`date +%Y%m%d`
backupDir_hetzner1="/usr/home/oseforum/tmp/backups_for_migration_to_hetzner2/oseforum_${stamp}"
backupDir_hetzner2="/var/tmp/backups_for_migration_from_hetzner1/oseforum_${stamp}"
backupFileName_db_hetzner1="mysqldump_oseforum.${stamp}.sql.bz2"
backupFileName_files_hetzner1="oseforum_files.${stamp}.tar.gz"
dbName_hetzner1='oseforum'
dbName_hetzner2='oseforum_db'
dbUser_hetzner2="oseforum_user"
dbPass_hetzner2="CHANGEME"
vhostDir_hetzner2="/var/www/html/forum.opensourceecology.org"
vanilla_backupdir_hetzner2="${vhostDir_hetzner2}/vanilla_docroot_backup.${stamp}"
docrootDir_hetzner2="${vhostDir_hetzner2}/htdocs"

# STEP 1: COPY FROM HETZNER1

mkdir -p ${backupDir_hetzner2}/{current,old}
mv ${backupDir_hetzner2}/current/* ${backupDir_hetzner2}/old/
scp -P 222 oseforum@dedi978.your-server.de:${backupDir_hetzner1}/current/* ${backupDir_hetzner2}/current/

# STEP 2: ADD DB

# create backup before we start changing the sql file
pushd ${backupDir_hetzner2}/current
cp ${backupFileName_db_hetzner1} ${backupFileName_db_hetzner1}.orig

# extract .sql.bz2 -> .sql
bzip2 -dc ${backupFileName_db_hetzner1} > db.sql

# verify the first 2 (non-comment) occurances of $dbName meet the naming convention of "<siteName>_db
vim db.sql

time nice mysql -uroot -p${mysqlPass} -sNe "DROP DATABASE IF EXISTS ${dbName_hetzner2};"
time nice mysql -uroot -p${mysqlPass} -sNe "CREATE DATABASE ${dbName_hetzner2}; USE ${dbName_hetzner2};"
time nice mysql -uroot -p${mysqlPass} < "db.sql"
time nice mysql -uroot -p${mysqlPass} -sNe "GRANT ALL ON ${dbName_hetzner2}.* TO '${dbUser_hetzner2}'@'localhost' IDENTIFIED BY '${dbPass_hetzner2}'; FLUSH PRIVILEGES;"

# STEP 3: Add backup of dynamic-content files

mv ${vhostDir_hetzner2}/* ${backupDir_hetzner2}/old/
time nice tar -xzvf ${backupFileName_files_hetzner1}

mkdir -p ${docrootDir_hetzner2}
time nice rsync -av --progress usr/www/users/oseforum/.* "${vanilla_backupdir_hetzner2}/"

# set ['Database'] Name/User/Password
vim "${vanilla_backupdir_hetzner2}/conf/config.php"

# set permissions
chown -R apache:apache "${vanilla_backupdir_hetzner2}"
find "${vanilla_backupdir_hetzner2}" -type d -exec chmod 0750 {} \;
find "${vanilla_backupdir_hetzner2}" -type f -exec chmod 0640 {} \;
chown apache:apache-admins "${vanilla_backupdir_hetzner2}/conf/config.php"
chmod 0440 "${vanilla_backupdir_hetzner2}/conf/config.php"

# STEP 4: Add static content files to docroot

# create static html content by scraping existing site
mkdir -p ${backupDir_hetzner2}/{wget}
pushd ${backupDir_hetzner2}/{wget}
time nice wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains "forum.opensourceecology.org" "http://forum.opensourceecology.org"

# copy static content to docroot
mkdir -p "${docrootDir_hetzner2}"
time nice rsync -av --progress wget/* "${docrootDir_hetzner2}/"
</pre>

[[Category: CHGs]]